Step 1:
Articulate the program outcomes that you plan to measure in your observation. For your first observation, pick 1-3 program outcomes to measure per observation cycle (it can be overwhelming to observe too many outcomes at once). Then, articulate the 3-5 metrics that can show you that participants are achieving the program outcomes.
Step 2:
Define the scale for each high-level program outcome. We recommend using a 5-point scale as a default (to provide room for change on the scale), and creating “anchors” for at least points 1, 3, and 5. If you feel strongly that raters should not be able to choose a mid-point, then use a four-point scale. Include a space by each rating where you will be able to write examples and notes that explain why the rater rated each item the way s/he did.
Step 3:
Describe the meaning of each rating in detail. Make sure your ratings make sense for your program – labels should be tailored for what you expect to be the low, mid, and high points for your participants.
