Methods in Program Evaluation
Session 7: In-Class Exercise Part A
Review the following article (ATTACHED):
Ager, A., Akesson, B., Stark, L., Flouri, E., Okot, B., McCollister, F., & Boothby, N. (2011). The impact of the school‐based Psychosocial Structured Activities (PSSA) program on conflict‐affected children in northern Uganda. Journal of child psychology and psychiatry, 52(11), 1124-1133.
Answer the following questions:
- What were the original program goals for the intervention?
- Sketch out a bare-bones logic model summarizing the inputs, activities, and outcomes. For the purposes of this exercise, skip the outputs and impacts.
- How did the authors measure and operationalize their primary outcome? What were the indicators that were used?
- What is the study design used?
- What are the threats to internal validity common in this design?
- Complete the table of participants.
| Baseline | Follow-up | % LTF | ||
| Children | Intervention | |||
| Comparison | ||||
| Parents | Intervention | |||
| Comparison |
Session 7: In-Class Exercise Part B
Read the case study:
WVI Child-Friendly Spaces (CFS) Intervention at Buramino Camp, Dolo Ado, Ethiopia:
Buramino refugee camp, located along the southern Somali-Ethiopian border, was selected in cooperation with World Vision Ethiopia as the first evaluation site. At the time of the evaluation, the Ethiopian Administration for Refugee and Returnee Affairs (ARRA), the main implementing partner of UNHCR, had organised five camps within two hours of the Dollo Ado transit centre to address the growing needs of Somali refugees who had migrated in response to drought and ongoing conflict in the region. In January 2012, World Vision Ethiopia’s first Child Learning Centre (subsequently referred to through this report as a CFS site) was established, offering a range of services to children aged six to 17 years.
The second CFS site started activities later that year, in May 2012. Within the wide continuum of CFS programming approaches, these two sites emphasised functional literacy and numeracy skills. Based on discussions with ARRA and parents of children attending these centres, psychosocial activities, such as drawing, singing, and recreational play, were included, but to a modest degree. On-site counselling and a feeding programme were also offered.
To help meet the overwhelming demand in the camp, each centre provided two three-hour sessions for children: a morning session for younger children aged six to 11 and an afternoon session for older children aged 12 to 17. The first CFS provided services for 784 children (500 aged between six and 11 and 284 aged between 12 and 17). The second CFS site provided services for 787 children (506 aged between six and 11 and 281 aged between 12 and 17).
Source: http://www.cpcnetwork.org/wp-content/uploads/2015/04/WV-CU-CFS-Ethiopia-Field-Study-Summary-Report.pdf
Answer the following questions:
- Develop one evaluation question
- Consider what study design would best answer your question
- Decide what data you need to collect
4.Recommend one random and one non-random sampling scheme. (circle your choices)
| Types of random sampling | Types of non-random sampling |
| Simple random sample Systematic Proportionate stratified Disproportionate stratified (Cluster sampling) | Quota sampling Extreme case sampling Maximum variation sampling Homogeneous sampling Typical case sampling Snowball sampling Criterion sampling |
5.Discuss the advantages and disadvantages of each scheme that you choose.
6.Why did you choose the random sampling scheme you did?
7.What aspect(s) of the evaluation question would this scheme help you answer?
8.Why did you choose the non-random sampling scheme?
9.What aspect(s) of the evaluation question would this scheme help you answer?