Pilot Implementation/Formative Evaluation
PILOT IMPLEMENTATION: This is the point where you test and evaluate your training using pilot students. This is also known as a Formative Evaluation. Formative Evaluation is used to obtain data that can be used to revise the instruction to make it more efficient and effective. (Collect Data, Analyze Data, Revise Instruction)
What does the Pilot Implementation consist of?
One-To-One or Clinical Evaluation: The designer works with 3 or more learners individually to obtain data to revise the materials. The following three things are concentrated on during these sessions: Clarity: Is the message, or what is being presented, clear to individual target learners? Impact: What is the impact of the instruction on individual learner's attitudes and achievement of the objectives and goals? Feasibility: How feasible is the instruction given the available resources (time/context)?
Small-Group: A group of 8-20 learners, who are representative of the target population, study the materials on their own and are tested to collect the data required. Purpose: Determine the effectiveness of the changes made from the one-to-one evaluation and to identify any remaining problems the learner may have. Determine whether learners can use the instruction without interacting with the instructors (Self-paced Instruction).
Attitude Questionnaire: This is a questionnaire developed to gain the learners perceptions on a variety of subjects such as the course, environment, instructor, and materials to mention a few. This questionnaire should ideally be used with the small-group, the field trial, and also when the course is fully implemented. Purpose: To identify weaknesses and strengths in the implementation of instructional strategy through learner perceptions.
Example Questions:
- Was the instruction interesting?
- Did you understand what you were supposed to learn?
- Were the materials directly related to the objectives?
- Were sufficient practice exercises included?
- Were the practice exercises relevant?
- Did the test really measure your knowledge of the objectives?
- Did you receive sufficient feedback on your practice exercises?
- Did you feel confident when answering questions on the tests?
Field Trial: Often 30 learners are used to test the procedures required for the installation of the instruction in a situation as close to the "real world" as possible. Purpose: Determine the effectiveness of the changes made from the small-group evaluation and to see if it is possible to use the instruction in its intended setting. It is very important to locate and eliminate any remaining problem in the instruction. Data to be collected will be on learner achievement and attitudes; instructor procedures and attitudes; and resources such as time, cost, space, and equipment.
Implementation is the next step in the process. So what is Implementation anyway?
