How does CCCD Affect Program Effectiveness and Sustainability? A Meta Review of Plan’s Evaluations Final Report Tosca Bruno-van Vijfeijken, Uwe Gneiting, and Hans Peter Schmitz Transnational NGO Initiative, Moynihan Institute of Global Affairs July 25, 2011 Table of Contents Executive summary......................................................................................................................... 1  Introduction..................................................................................................................................... 4  1.1. What is CCCD?.................................................................................................................... 5  1.2. Transnational NGO Initiative and Plan International .......................................................... 6  2. Research Question and Study Design ......................................................................................... 8  3. Research results I: Implementation of CCCD .......................................................................... 10  4. Research results II: Linkages between CCCD and program outcomes .................................... 14  4.1. Categorization of CCCD strategies.................................................................................... 14  4.2. Behavior Change................................................................................................................ 16  4.3. Access and Quality of Services.......................................................................................... 23  4.4. Sustainability...................................................................................................................... 29  4.5. Equity................................................................................................................................. 36  5. Internal factors and CCCD........................................................................................................ 41  6. Conclusions and recommendations........................................................................................... 44  References..................................................................................................................................... 47  Appendices.................................................................................................................................... 48  Acronyms and Abbreviations CBO: Community-based Organization CCCD: Child-Centered Community Development Approach CLTS: Community-led Total Sanitation CSP: Country Strategic Plan IMCI: Integrated Management of Childhood Illness INTRAC: International NGO Training and Research Centre OECD: Organization for Economic Co-operation and Development (T)NGO: (Transnational) Non-governmental Organization UNICEF: United Nations Children’s Fund WHO: World Health Organization Executive summary In 2003, Plan International adopted a Child-Centered Community Development (CCCD) approach as its international framework for its programmatic work. CCCD reframes some familiar tools of Plan’s work, including capacity-building and participation, while also expanding into new strategic areas, such as addressing national level policies and legislation and developing more holistic cross- level activities and partnerships. Plan’s shift to CCCD represents a significant shift in the organization’s approach to its development work. This study aims to contribute to a better understanding of how CCCD enhances Plan’s program effectiveness and sustainability by reviewing all available single-program, external evaluations completed between 2007 and 2010 in the areas of water and sanitation, education, and health. This study first assessed the implementation practices of CCCD and then evaluated the effects of CCCD on program outcomes and sustainability. It identified specific preferred targets and strategies used in CCCD programming. Plan primarily engages with communities and state institutions, while relatively less attention is focused on other civil society actors or linkages between local stakeholders. At the community level, strategies of capacity-building and participation are dominant. When interacting with government institutions, Plan also emphasizes capacity-building of front-line government personnel while advocacy strategies challenging governmental decision- making are much less prevalent. With regards to the effects of CCCD on program outcomes, this study defined four main program outcome dimensions (behavior change, service access and quality, sustainability, and equity) and placed its analytical focus on specific patterns and mechanisms linking CCCD strategies to these outcomes. Overall, the study found that CCCD did have a positive impact on program implementation although effects on behavior change and service access and quality have been more pronounced than those on sustainability or equity. Behavior change. Plan supports a variety of distinct behavior change processes aimed at changing the self-perception of community members, appealing to their self-interest, or taking advantage of peer pressure mechanisms. The effectiveness of these mechanisms is enhanced when community groups actively participate in program implementation using CCCD strategies. The main limitations 1 identified in this review point to gaps between awareness and practice, mostly when programs only include one-time interventions and when cultural barriers are high. Service access and quality. Plan’s work with community-based organizations (CBOs) was the most frequently mentioned CCCD-related strategy that enhanced program effectiveness in the area of improved service access and quality. In addition, Plan has seen some success in using community- level experiences to successfully influence policy decisions at the national level, thereby scaling up its programming. The main challenges to using community-based approaches in enhancing services include deficits in quality due to a lack of resources and capacity on the community level. Sustainability. The study found that Plan does not advocate or implement one single sustainability strategy. Instead, country programs either prioritize community capacity or state capacity as their main focus to achieve sustainability (depending on local context and program characteristics). In addition, Plan has had some success in enhancing the sustainability of its results by linking communities with state institutions, for example, by integrating community volunteers into state-run planning and implementation processes. The main challenges identified in this area are weak state capacity and political will to assume program responsibilities, the fragility of CBOs, and a certain lack of formal emphasis on sustainability strategies. Equity. The evaluations indicate inconsistency in Plan’s approach towards equity and show that equity concerns, while visible in program planning stages, frequently get lost during the implementation phase. Plan’s attention to equity is more consistent on gender issues and the rights of children more generally, while CCCD efforts are rarely directed at more distinct vulnerable groups, such as children with disabilities or migrant children. The most positive effects of CCCD emerge when it is implemented consistently across different levels using multiple strategic categories (i.e. participation, capacity building, advocacy, and governance and accountability.) This finding points to the complementary character of the different CCCD strategies in enhancing the effectiveness and sustainability of Plan’s programs. Plan’s comparative advantage in pursuing CCCD is most visible when it not only works with communities, but extends its efforts across a variety of levels to address development issues more effectively. This multi-level approach allows the organization to create linkages and synergies between different 2 actors and ties its experience from its community-level engagement to policy-making processes at the national and international levels. In further advancing its CCCD strategy, Plan faces two main challenges. First, CCCD implementation practices vary widely across country offices. While some variation due to different local and national circumstances is appropriate, enhancing the effectiveness and sustainability of Plan’s programs requires greater consistency in the application of CCCD. To do so, Plan internally should enhance shared understanding of CCCD by staff (1), provide specific guidelines for CCCD implementation within particular programs (2), clarify the relationship between CCCD and the sponsorship approach (3), and dedicate more resources to measuring community changes over time in order to better understand the effects of CCCD (4). Second, CCCD faces more inherent limitations associated with increasing reliance on local actors with limited resources and capacities. This study represents an initial attempt to systematize the relationship between CCCD and Plan’s program outcomes. The research team recommends building on the lessons learned from this project and to further invest in efforts to document and trace the effects of CCCD on Plan’s contribution to the improvement of children’s lives in its program countries. The results presented in this report represent hypotheses whose validity could be further strengthened with subsequent research systematically comparing cases where components of CCCD were applied with cases where no such intervention took place. Response from Plan International USA: Plan provided the Maxwell researchers with unrestricted access to our archives of third-party evaluations in order to gain new insights into the effectiveness of CCCD. We welcome the generally positive findings of this independent review of our field work since 2003. Plan also welcomes the insights provided by the study, which we intend to incorporate into our future programming. Plan International USA will share this study widely throughout the Global Plan Federation, which spans 60 countries and three-quarters of a billion dollars annually. We believe that the findings of this report will have relevance to other development actors working with children, communities and Rights-based Approaches to development more generally, and we welcome further dialogue. 3 Introduction 1 In 2003, Plan International adopted a Child-Centered Community Development (CCCD) approach as its international framework for its programmatic work. The decision to adopt CCCD reflected Plan’s changing approach to improving children’s lives around the world. CCCD reframes some familiar tools of Plan’s work, including capacity-building and participation, while also expanding into new strategic areas, such as addressing national level policies and legislation and developing more holistic cross-level activities and partnerships. Driven by Plan’s field-level experiences in implementing development projects, this strategic shift aimed at enhancing the effectiveness and sustainability of Plan’s program activities. Ten years after its formal adoption, CCCD has become widely adopted across Plan country offices and is reflected in recent program evaluations. Plan leadership has expressed interest in determining if and how the CCCD strategy is contributing to the achievement of the organization’s mission to end child poverty. In addition, donors increasingly demand evidence on the results of their investments, putting additional emphasis on the need to understand in what ways new strategies, such as CCCD, are affecting the effectiveness of Plan’s efforts. This report presents a review of program evaluations covering CCCD experiences and aims at identifying causal linkages between CCCD-driven program strategies and changes in effectiveness and/or sustainability of program results. Purpose: The purpose of this research and report was to provide a meta-analysis of existing program evaluations regarding mechanisms linking CCCD program strategies to program outcomes. Past Plan’s efforts to assess CCCD have predominantly revolved around evaluating the extent to which CCCD has been implemented in program countries. Underlying these assessments is the assumption that a well-implemented CCCD approach enhances the effectiveness and sustainability of Plan’s programs. Compared to earlier program strategies, CCCD offers a broadened and more complex tool set which directs Plan staff to interact differently with communities, to expand their engagement with government agencies, and to partner with domestic and transnational civil society 1 The research team thanks Tessie San Martin (CEO, Plan USA) and Justin Fugle (Senior Program Manager, Plan USA) for their initiative to conduct this study and their support and insights throughout the research process. 4 groups. This report offers initial insights as to how CCCD can affect social change processes and create a social and political environment more conducive to enhanced development outcomes. 1.1. What is CCCD? The shift towards CCCD as Plan’s strategic approach echoes broader trends in the international development field. On the one hand, CCCD is aligned with emerging principles on aid effectiveness established by the international development community, such as local ownership and alignment (OECD, 2005/08). On the other hand, CCCD reflects the inclusion of rights-based language by being defined as a “child rights approach to end child poverty” (Zuurmond, 2010). This utilization of a rights discourse reflects the diffusion of rights-based approaches (RBA) in the international development field over the past decade. Through CCCD, Plan’s organizational mission has remained the same, while the tactics and strategies of accomplishing this goal have been expanded and changed. The adoption of CCCD represents a significant departure from Plan’s previous approach to development work, which was characterized by individual support to sponsored children, direct provision of goods and services, and a welfare-based model of NGO interventions. Under CCCD, Plan redefined its role and responsibility in development processes and moved towards a facilitating role in an effort to enhance the ability of local stakeholders, including state actors, communities, and domestic civil society organizations, to create the changes necessary for sustained development progress. Plan has recently further solidified its CCCD strategy by defining specific program strategies and principles that embody the organization’s approach to its programmatic work. 5 Table 1. Plan’s CCCD principles and strategies CCCD principles CCCD strategies ¾ Children at the center ¾ Anchoring programs in the community ¾ Guided by international human rights ¾ Holding state actors accountable standards ¾ Strengthening the capacity of civil society ¾ Responsibility and accountability ¾ Engaging the corporate sector ¾ Inclusion and non-discrimination ¾ Advocating for policy change ¾ Gender equality ¾ Working in partnerships for greater impact ¾ Participation for greater impact 1.2. Transnational NGO Initiative and Plan International This report is the result of a collaborative evaluation between Plan International USA and the Transnational NGO (TNGO) Initiative at the Moynihan Institute of Global Affairs at Syracuse University, USA. The Transnational NGO Initiative is housed in the Maxwell School of Citizenship and Public Affairs. Plan USA sought the collaboration of the Transnational NGO Initiative with the aim of obtaining an external and rigorous examination of core assumptions about the positive effects CCCD would have on program outcomes. The researchers of the TNGO Initiative had previously undertaken a strategic evaluation of Plan Guatemala’s transition towards CCCD in 2009, which was published by Plan International in 2010 (Gneiting, et al., 2010). The TNGO Initiative thus has an intimate understanding of Plan International and the application of CCCD ‘on the ground,’ and the two organizations have developed relationships of trust. The TNGO Initiative advances the understanding of international NGOs as core players in global affairs through research, education, and practitioner engagement. In particular, the Initiative examines the governance, leadership and effectiveness challenges of TNGOs. The TNGO Initiative aims to produce positive feedback loops between its academic as well as applied research, its education work with students and its NGO practitioner engagement work. Our work with 6 practitioners – evaluation, other applied collaborative research and leadership preparation training – ensures that the concerns of practitioners are fed back into our research and teaching. In these efforts, the TNGO Initiative can draw on the interdisciplinary strengths of the Maxwell School, consistently rated by its peers as the premier public affairs school in the United States. On the research side, the academic as well as practitioner discourse around Rights-Based Approaches (RBA) in development, as well as organizational change requirements related to improved outcome accountability and learning needs of TNGOs are part of our agenda. The relationships of trust mentioned above were developed based on intensive involvement of Plan USA in the definition of the evaluation questions and identification of interview partners. The relationship also allowed the evaluation to take place in an environment of complete access to internal data and informants. Plan USA offered extensive feedback at interim and near-final stages of report writing. The researchers of the TNGO Initiative ensured the validity of the research questions and methodology, developed the coding scheme, selected and reviewed the evaluations, and conducted additional follow-up interviews to probe the content of selected evaluations. Due to this collaborative approach from design through final determination of the evaluation findings, the ownership of the evaluation outcomes will likely be strengthened. 7 2. Research Question and Study Design Question: The central research question underlying this report is ‘Through what processes and under what conditions does CCCD enhance the effectiveness and sustainability of Plan’s programs?’ The two analytical tasks of this project included: • to trace the causal mechanisms linking the implementation of CCCD strategies to changes in program outcomes; and • to identify internal and external factors affecting this causal process between strategy implementation and program outcome. Methodology: Three program areas where chosen for this study: water and sanitation, education, and health. Within these program areas, all available single-program, end-of-term evaluations since 2007 were reviewed according to the evidence on linkages between program strategies and outcomes. All evaluations were third-party, donor-funded evaluations that covered Plan’s programmatic work between 2003 and 2010. Evaluations of sponsorship programs were not included into the study. The final number of program evaluations that qualified to be included in the sample was 38 (see Appendix A). In addition, four global evaluations on education, child survival and overall program effectiveness were included in the review. Following an initial survey of the available evaluations, ten were selected for follow-up interviews with Plan staff involved in the program activities (see Appendix B). The research process began with the development of a coding scheme derived from Plan’s own operationalization of CCCD. The coding scheme divided recorded strategies by the main targets: communities, government, and civil society. Under each of the three group categories, information on core CCCD strategic categories including awareness/capacity building, participation, policy/legislation, and accountability/governance was collected and entered into a database. This initial content analysis provided the basis for assessing variation in the emphasis of CCCD implementation practices (prevalence of CCCD practices) and some initial ideas about possible mechanisms linking CCCD programming to outcomes. In a second step of the content analysis, a qualitative review of each program evaluation was performed. The focus here was on identifying specific mechanisms explaining the success or failure of specific CCCD-related practices in 8 enhancing program outcomes. We distinguished between short-term effectiveness (divided into behavior change and access/quality of services), long-term sustainability, and changes in equity. In a third step, we conducted interviews with Plan country offices to add additional insights to the written evaluations and draw on the general experience of Plan staff with CCCD. Limitations: A review based on past program activities and their evaluation cannot offer absolute confidence about the role of certain mechanisms linking CCCD strategies to program outcomes. First, although we limited our universe to evaluations completed after 2007, we found significant variation in the extent of reporting and implementation practices, especially in earlier evaluations. Since CCCD represents a broad mix of existing and new strategies and evaluators and staff may have had very different levels of familiarity with CCCD, it is important to be careful in attributing specific outcomes to a strategy that may not have been fully present in programming or evaluations. Second, although we reviewed all available evaluations in the three program areas, some outcomes of CCCD may be captured in evaluations not covered by this review, including those focused on ‘participation’ as a separate program area. Third, most of the data stems from single case studies (i.e. programs). Case studies are useful in generating ideas about possible explanations for successes and failures, but they lack qualities of a comparative analysis which would create greater confidence in the results. For example, some of the insights generated by this review should be further tested in subsequent studies comparing cases where components of CCCD were applied with cases where no such intervention took place. 9 3. Research results I: Implementation of CCCD To assess the relationship between CCCD strategies and program outcomes, we first determined how prevalent specific strategies were across program activities. We developed a coding scheme and recorded the frequency with which individual CCCD strategies were mentioned in the evaluations. The coding scheme was informed by the strategies and principles articulated in Plan’s CCCD framework as well as derived from an initial review of a few evaluations. The coding scheme separated the strategies according to the primary target of activity: • communities, • government, • civil society, and • cross-cutting strategies targeting more than one target/level. The findings presented in this section are based on the perceptions of the external evaluators which we added up to arrive at a frequency count of observed CCCD strategies. There is the possibility of an undercount since evaluators may not have been intimately familiar with CCCD or may have missed the presence of individual activities and strategies. Nevertheless, coding for the frequency of CCCD strategies allowed for establishing an initial overview regarding the prevalence of specific strategies in the implementation process by Plan program country offices. Table 2 shows significant variation in the frequency of strategies implemented towards each of the target groups (i.e. communities, state, civil society, and cross-cutting). CCCD strategies targeting communities are most frequently mentioned (46% of total number of strategies), closely followed by strategies targeting state actors (40%). Strategies targeting civil society (7%) and cross-cutting strategies (7%) are significantly less prevalent. This finding demonstrates that Plan continues to place a strong emphasis on its work with communities while at the same time has expanded its work targeting state actors. Greater frequency not only means that Plan targets communities and state actors in more projects but also that it applies a greater variety of strategies to each of these actors. 10 Table 2. Overall frequency of different CCCD strategies Overall frequency of community-level strategies 98 45.8% Overall frequency of state-level strategies 86 40.2% Overall frequency of civil society-level strategies 16 7.5% Overall frequency of cross-cutting strategies 14 6.5% In a next step, we assessed the frequency of specific strategies across all 42 evaluations. At the community level, the two most frequently mentioned strategies include efforts to promote the participation of community members and groups in the planning and implementation (mentioned in 74% of all evaluations) and efforts to increase the capacity of community members to contribute to program outcomes. The latter strategy encompasses the transfer of organizational skills to community-based organizations, CBOs (45%) or trainings on program-related issues, such as health practices (58%). Less frequent were strategies that aimed to raise the awareness of community members on issues related to human and child rights (34%) and issues of exclusion and discrimination (18%). At the state level, Plan’s predominant strategic avenue is to strengthen the capacity of government agencies that are involved in the delivery of services to communities. Two distinct strategies were most frequently cited (each one in 65% of all evaluations). First, Plan supports state agencies in delivering services to communities by offering technical assistance and complementary resource assistance. This can include a variety of activities, such as helping to equip government-run health centers or schools, supporting government staff in implementing community-based activities, or advising local governments. Second, Plan intervenes in service delivery systems by training government front-line staff, in particular health workers and teachers. To a lesser degree Plan engages in advocacy activities with state agencies at the local and national level. The most commonly mentioned advocacy strategy refers to Plan’s attempt to scale up certain methodologies and service delivery approaches, such as Community-led Total Sanitation (CLTS) or Integrated 11 Management of Childhood Illness (IMCI) (24%). 2 In most of these cases, Plan engages in evidence- based advocacy, i.e. it presents empirical evidence on the effectiveness of its approaches to policy- makers with the aim of scaling up similar methodologies nation-wide. Plan’s engagement in public awareness campaigns about the situation of children (18%) and efforts to affect funding allocation through advocacy (8%) were less frequently mentioned advocacy strategies. Lastly, more than one- third of the evaluations (37%) mentioned Plan’s effort to align its programs with existing government frameworks and policies. In sum, the data on state level strategies demonstrate Plan’s emphasis on a partnership approach with government agencies in an effort to improve the capacity of these agencies and to apply effective approaches in providing child-friendly services to mainly rural communities while placing less emphasis on more contentious advocacy efforts aimed at changing the behavior of state actors. At the civil society level, the evaluations offer less systematic evidence regarding Plan’s implementation of CCCD strategies. Plan’s engagement in local or national civil society advocacy alliances is mentioned in about a quarter of all evaluations (24%).While Plan’s interaction with local and other international NGOs appear regularly in the evaluations, most of them refer to contractual relationships with these NGOs and their role in providing services to communities. Only a smaller number of evaluations refer to efforts at establishing partnerships with local NGOs that aim to build the capacity of local NGOs to provide services independently of Plan or to participate in accountability and policy processes (19%). The relatively low frequency of Plan’s CCCD strategies targeting other civil society organizations suggests that Plan’s primary form of engagement with local NGOs continues to be in the form of implementation-focused, contractual relationships. This finding echoes previous assessments of Plan’s partnership strategy (INTRAC, 2009; Gneiting, et al. 2010). A last CCCD category assessed the frequency of Plan’s efforts to employ strategies that aim to link communities (including CBOs) with state actors. Instances of this strategy category include efforts to increase the ability of communities to participate in municipal or national-level planning and decision-making processes and to give feedback on the quality of services to state providers (16%). 2 IMCI was developed jointly by the World Health Organization (WHO) and the United Nations Children’s Fund (UNICEF) to improve the health care provided to sick children. 12 They also encompass efforts to increase the responsiveness of state agencies and the integration of CBOs into government-run service delivery systems (21%). While Plan places a strong emphasis on interacting with state and community actors, the lack of cross-cutting strategies indicates that Plan could expand its efforts to foster linkages between these two types of actors. Table 3. Frequency of CCCD strategies across evaluations COMMUNITY LEVEL Frequency Percentage Awareness Human rights 13 34.2% Gender and discrimination 7 18.4% Capacity Organizational strengthening of CBOs 17 44.7% Program-specific training of community members 22 57.9% Participation Community commitment to program objectives 5 13.2% Community participation in program implementation 28 73.7% Community mobilization 6 15.8% STATE LEVEL Capacity Technical support of state agencies to implement programs 25 65.8% Training of government front line staff 25 65.8% Advocacy Research and publication 6 15.8% Sensitization for community issues 7 18.4% Efforts to scale up Plan methodologies 9 23.7% Alignment with existing policies and programs 14 36.8% CIVIL SOCIETY LEVEL Partnerships with civil society actors in the area of service delivery 7 18.4% Alliances with civil society actors in the area of advocacy 9 23.7% CROSS-CUTTING STRATEGIES Increasing community access to state institutions 6 15.8% Increasing responsiveness and accountability of state institutions 8 21.1% 13 4. Research results II: Linkages between CCCD and program outcomes Plan adopted CCCD under the assumption that this strategic approach would enhance the effectiveness and sustainability of its programmatic work. Underlying this shift is the assumption that the program implementation process has an effect on the program outcome. This section reviews evidence related to the mechanisms linking strategies and outcomes. As mentioned above, due to the methodological limitations of this project, this study does not allow to conclusively determine whether CCCD is an effective strategy but instead to structure the evidence on how processes shape outcomes. Thus, this section focuses on tracing the causal mechanisms, through which CCCD can enhance program outcomes as well as the limits and external conditions for the effectiveness of this strategy. 4.1. Categorization of CCCD strategies We grouped Plan’s CCCD strategies into four categories, which each represent a distinct strategic approach through which CCCD aims to affect program outcomes (Table 4). These categories align with the dimensions of change outlined in Plan’s global effectiveness framework (Plan International, 2008). Their content and significance is briefly summarized below. Table 4. Relationship between CCCD and program outcomes Outcome dimensions Behavior Change Access and Quality Sustainability Equity CCCD strategies Participation Capacity and awareness Advocacy Governance and accountability 14 Participation The review above revealed that Plan implements two main participation strategies. First, Plan promotes the active participation of communities in program planning and implementation through collaboration with and support of CBOs. These groups fulfill a variety of program-related functions, such as service outreach (e.g. community health volunteers), management of water systems (e.g. water committees), school management councils, or peer-to-peer education (e.g. adolescent groups). Secondly, Plan promotes the participation of children in their schools, their communities and beyond by forming and strengthening the organization and training of children and youth groups and their formal participation in decision-making processes. Capacity and awareness In the area of capacity and awareness, Plan implements a range of different strategies targeting a variety of actors. Components of this strategic category include awareness raising activities on child rights and the relevance and purpose of certain programs and practices (mostly at the community level), technical knowledge transfer towards community volunteers and government front line staff, including program-related methodologies, such as teaching or health practices, and the strengthening of organizational and leadership skills of CBOs. This category also encompasses activities geared at supporting the availability of infrastructure and materials necessary for the delivery of services (e.g. school buildings, learning materials, health centers, etc.). Advocacy Plan’s engagement in advocacy and policy-related activities at the national level is an integral part of its CCCD strategy. The most prevalent strategies include the publication and dissemination of empirical evidence in order to invoke policy changes, efforts to include lessons from Plan pilots into national policy-making processes, alliances with other national and international NGOs to raise awareness of the national public on a certain issue, shadow reporting on the Convention on the Rights of the Child (CRC), efforts to influence budgeting decisions at the municipal level, and others. 15 Accountability and participatory governance A last set of strategies aims to increase the accountability of state actors by creating greater access for citizens to participate in planning and decision-making processes. Related to this goal are efforts that aim to improve the effective governance of service delivery systems by linking CBOs and agents with state-run delivery systems. Together with input from Plan USA, the research team defined four program outcome dimensions that are considered relevant for the three program areas under review and are accessible concepts for people not familiar with CCCD. The project identified behavior change and access and quality of services as its main dimensions to measure the effectiveness of programs. Furthermore, equity and sustainability were identified as relevant program outcome dimensions. Each of these categories is briefly introduced in the subsequent sections, which will summarize the effects of four CCCD strategy categories on four outcome areas. 4.2. Behavior Change Under CCCD, Plan defines its primary program outcome dimension as “…[behavior] changes among rights holders, duty bearers and civil society organizations, and the synergies among these social actors” (Plan International, 2008). Behavior change among all relevant stakeholders is seen as a precondition to foster an environment, in which child rights can be fulfilled. Interviews with Plan field program managers pointed to a greater focus on behavior change as one of the key distinguishing criteria of CCCD. This emphasis assumes that • program effectiveness is influenced by the behavior of individual community members; and • increasing community demand for the effective and equitable delivery of health, education and water services is a critical precondition for improving government-sponsored service delivery. This study finds significant evidence showing how CCCD strategies contribute to behavior change at the community level through a variety of causal pathways. The most relevant findings concern the different types of behavior change processes that Plan supports (1), and the role of community 16 participation in facilitating these processes (2). The review found less evidence for success in changing the behavior of state actors, primarily because strategies aimed at achieve this type of change are only rarely applied. Capacity and awareness Implementing activities targeted at raising the awareness and capacity of community members is one of the most prevalent strategic focus areas of Plan’s CCCD approach. The evaluations reviewed for this project point to three distinct causal mechanisms, through which these activities have had an effect on community-level behavior. • Identity change using rights discourse: Through CCCD, Plan is increasingly adopting a rights discourse in its interactions with communities. Although the prevalence of a rights discourse varies across program countries (it is most prevalent in Latin American countries), there are several instances of such a discourse, particularly on the rights of children, having a positive impact on the self-perception and self-esteem of community members. This changed discourse, in turn, affected behavior. For instance, raising the awareness of community members regarding the rights of children has been reported to contribute to a reduction in violence against children by teachers and a greater acceptance and support of children’s education by parents. 3 This process has been particularly beneficial in decreasing discrimination against girls (see section on CCCD and equity). Several evaluations also reported an increase in the self-esteem, confidence and level of mobilization of children and adolescents as a result of greater rights awareness. 4 While there is less consistent data on the broader effects of the mobilization of children for their rights fulfillment, there are immediate behavior effects in community settings. • Appealing to the benefits of programs: The second strategy observed to have an effect on the behavior of community members focuses on the process of communicating the benefits of program activities. This strategy is based on the recognition that a commitment by community members to program or project goals is necessary to ensure their ownership and participation in program activities. By appealing to the self-interest of community members, the likelihood of 3 Plan Ecuador/Education/2008; Plan Sierra Leone/Education/2010. 4 Plan Bolivia/Education/2010; Plan Nicaragua/Health/2008; Plan Sri Lanka/Health/2009. 17  changing their behavior is observed to increase. Several cases illustrate this strategy. For instance, Plan Kenya shared the experience of households with above average health indicators with other community members to demonstrate the positive effects of certain health- and nutrition-related practices. 5 Plan Bangladesh reported on the significant change in behavior of community members once they realize the potential health benefits of latrines and certain hygiene practices. 6 And Plan Ethiopia succeeded in significantly increasing school enrolment and graduation rates by placing emphasis on the value of education in its communication strategy with communities. 7 These three examples show that appealing to self-interest often leads to improved program outcomes. • Peer pressure: The third strategy through which CCCD is affecting the behavior of community members focuses on the creation of community-based ‘naming and shaming’ processes. Behavior change has also been achieved as a result of the creation of social sanctioning mechanisms on the community level. The most prominent example is the implementation of the CLTS methodology, which addresses the problem of open defecation through a combination of community mobilization and ‘naming and shaming’ as enforcement mechanism for behavior change. Other examples include actions by community health volunteers and groups, such as open demarcation of health practices by households 8 or social pressure mechanisms to change the child-related behavior of mothers. 9 Participation Participation of community members is the second major strategic approach of Plan’s CCCD framework that can positively affect behavior change. Several observations from the evaluations point to the relevance of community participation as a vehicle for the behavior change processes described above to take place. For instance, a comparative study between participatory and non- participatory approaches to community sanitation in Cambodia found that behavior change 5 Plan Kenya/Health/2009. 6 Interview with WASH Advisor, Plan Bangladesh; June 6, 2011. 7 Plan Ethiopia/Education/2009. 8 Plan Nepal/Health/2007. 9 Plan Kenya/Health/2009. 18 (measured in latrine use) among community members was significantly greater using a participatory strategy compared to approaches that solely focus on latrine subsidies. 10 The study showed that community-based communication strategies were responsible for a greater level of understanding of the health risks associated with hygiene and sanitation practices. Other evaluations reported how community participation in the form of community health volunteers has significantly increased the utilization of available health services due to their embeddedness and their ability to motivate mothers and peers. 11 Overall, Plan’s consistent support of community-based mechanisms to diffuse knowledge and raise awareness (e.g. peer-to-peer approaches, community volunteers doing community outreach, etc.) is seen to enhance the effectiveness of the behavior change processes. We found less evidence for the effects of child and adolescent participation on behavior change (other than their own). Nevertheless, three instances of the relationship between child participation and behavior change can be noted. Within community-based monitoring schemes, such as CLTS, children actively support the observation of behavior changes and can report deviations. Furthermore, their active participation increases their ability to shape the behavior of other family members, which is seen as more effective than targeting parents directly. 12 Child and adolescent participation can also contribute to the diffusion of knowledge on culturally sensitive topics, such as sexual and reproductive health. Due to the resistance of parents and the lack of institutionalized learning opportunities, peer-to-peer education on sexual and reproductive health topics is an effective path of adolescent participation to affect behavior change. Advocacy Plan is increasingly investing in advocacy activities targeting state actors at the national and local level. However, this study found little empirical evidence about the effect of Plan’s advocacy activity on the behavior of state actors. Instead, the majority of these activities appear to be focused on scaling up Plan methodologies and increasing the quality and access of communities to state services, assuming an existing willingness of political decision-makers to buy into Plan’s objectives. In most program countries, Plan appears to perceive the lack of effective government 10 Plan Cambodia/Water and Sanitation/2009. 11 Plan Nepal/Health/2007; Plan Kenya/Health/2009; Plan Uganda/Health/2009. 12 Interview with Plan Bangladesh WASH Advisor, June 6, 2011. 19 services in health, education and water to be rooted in the lack of government capacity (i.e. technical knowledge, staff, and financial resources), not in a lack of political willingness. As a result, much of its advocacy work takes a cooperative approach to interactions with state agencies by aligning its work with existing policies and programs, demonstrating the effectiveness of pilot projects and attempting to engage in partnerships for service delivery arrangements. While there are positive effects reported from this strategy with regards to improving the access and quality of services of communities (see section ‘Access and Quality of Services’), this strategy is less effective when government actors lack a commitment to support the development of poor communities. There is consistent evidence showing that a lack of state capacity is not the primary cause for inadequate access and quality of services. For instance, Plan Sri Lanka had to abandon its peer education project for sexual and reproductive health due to the resistance of education authorities to include these issues in school curricula. Attempts to support the creation of youth-friendly health centers failed due to the lack of support by the health ministry. In the case of Plan China, the evaluation demonstrates that lack of financial resources is not always the most relevant constraint for service provision. Instead, government support for education or water and sanitation projects to individual communities was found to be contingent on individual relationships between communities, Plan, and local political leadership. As a Plan China manager put it: “If we meet [government] leaders who show disapproval, there is no way for us to cooperate. Sometimes we have no other choice but to choose those old friends who approve our approach or the agencies that do not have a heavy load of work.” 13 Given the lack of behavior change strategies towards government actors, Plan is dependent on existing government policies and political will in assuring that its efforts will reach the poorest communities and that communities can participate in political decision-making. Plan’s CCCD strategy with regards to government relations appear to be most effective in countries with an existing commitment to child rights and community development and with a minimum level of existing service delivery systems. 13 Plan China/Education/2009; Plan China/Water and Sanitation/2010. 20 Accountability and governance Similarly to the findings regarding advocacy strategies in this section, Plan’s efforts to increase accountability and the participation of communities in governance arrangements is predominantly focused on program-related issues. As a result, the findings on the effectiveness of Plan’s accountability and governance strategies will be presented in the subsequent section on service access and quality. Challenges and limitations Besides tracing the causal linkages between Plan’s CCCD strategy and behavior change on the community level, this study also aimed to identify factors that intervene in the process and to offer possible answers to the question under what conditions CCCD is effective in contributing to behavior change. Given the complexity of this question and the limited data available, the following list is nowhere near exhaustive but rather points to recurring themes that were discovered during the review process. Although generally effective, the behavior change processes supported by Plan also exhibited some limitations. The most common limitations encountered were a gap between increases in awareness and knowledge and actual changes in community-level practices. Several evaluations found a discrepancy in the ability of community members to articulate the relevance of certain practices and their actual application.14 Three primary explanations for this gap were cited. First, cultural barriers to issues, including girls’ education, sexual and reproductive rights or certain health practices can foster community-based sanctioning mechanisms undermining the progression from awareness to practice. As a result, one-time awareness raising and capacity-building activities, as carried out by some programs, are ineffective in changing behavior of community members. 15 Overcoming cultural barriers requires more long-term activities related to training and monitoring (either through community groups or government institutions) in order to ensure the internalization of desired behavior changes. 14 Plan Bangladesh/Water and Sanitation/2007; Plan Zimbabwe/Water and Sanitation/2009; Plan Malawi/Education/2010; Plan Nepal/Health/2007; Plan Kenya/Health/2009; Plan Senegal/Health/2008. 15 Plan Zimbabwe/Water and Sanitation/2009; Plan Malawi/Education/2010; Plan Nepal/Health/2007. 21 A second reason is related to characteristics of CBOs which are frequently used as vehicles for community participation. Some evaluations recount significant challenges for community volunteers to reach the most remote households. 16 Others point to high turnover in CBO membership, which greatly reduces their organizational capacity to diffuse knowledge and monitor practices. 17 Finally, CBOs can face risks of elite dominance, which may also reduce their ability to reach the entire community and the most excluded households. 18 Plan’s relationship with communities represents a third factor shaping CCCD effects on community-level behavior. There is some evidence that Plan’s initial engagement with communities plays an important role in fostering the necessary commitment to effective behavior change. A Plan Bangladesh evaluation reporting a high level of project ownership and commitment described the initial communication process as follows: “Plan staff initially spends much time developing rapport with the community and local administration. Relationship building with stakeholders at all levels around their common interest in children’s welfare and futures, is pursued by Plan and ultimately contributes to the local ownership and commitments to the development work. The rapport-building time is not fixed and the staff takes their own initiative to make the thing happen within the political economy and power structures of the village.” 19 As a result of its sponsorship activities and long-term presence in communities, Plan is uniquely positioned to communicate early and sufficiently with community members about the purpose and benefits of a project. This also allows Plan to accompany communities in their adoption and internalization of practices beyond the end of individual projects, thereby deepening the capacity and mobilization gains of communities. 16 Plan Bolivia/Health/2007. 17 Plan El Salvador/Health/2007; Plan Pakistan/Health/2007. 18 Plan China/Water and Sanitation/2010; Plan Bangladesh/Water and Sanitation/2007. 19 Plan Bangladesh/Education/2008. 22 4.3. Access and Quality of Services Despite framing its primary program outcome as behavior change, Plan implements a variety of strategies that attempt to directly affect the enjoyment of rights among children. One relevant determinant of children’s fulfillment of rights is their equal access to basic services that allow them to survive and prosper. This outcome category is particularly relevant for water and sanitation, health, and education because two of the main measures of success include: • children are able to access goods or services, and • goods or services are provided in a manner that benefits children’s well-being. The evaluations reviewed offer a variety of examples showing how certain CCCD strategies have improved the access and quality of services for children. First, Plan’s consistent support of CBOs as central actors in service delivery schemes can often effectively supplement limited and weak government-run service delivery systems in developing countries. Second, Plan’s evidence-based advocacy strategy can lead to the wider adoption of child-friendly approaches and methodologies at the national level, thereby reaching a far greater number of children. Plan’s long-standing community involvement combined with an expanding engagement with government agencies puts the organization in a unique position to create synergies based on linking state agencies and community-based approaches to service delivery. Participation One of the most consistent themes throughout the evaluations has been the positive contributions that community participation has had on the management of service delivery systems. In all three program areas, Plan has successfully worked with a wide variety of CBOs (CBOs) in an effort to increase the access and quality of services. Some of the most frequently cited examples of CBOs include School Management Councils, Community Health Volunteers, Parents-Teacher Associations, and Community Water Committees. In addition to contributing to a greater sense of community ownership and understanding, CBOs take on a variety of functions in the management of services, including planning and situational analysis, monitoring activities (e.g. teacher 23 attendance, health and hygiene practices), assisting with actual service delivery (e.g. basic health treatments), or the mobilization of additional resources. These functions complement the most commonly cited deficits of government-run service delivery system, which in many program countries are characterized by a lack of community reach, a lack of human and financial resources, and a lack of quality control and feedback mechanisms. We found significant variation in the level of effects community participation has had on access and quality of services. The engagement of community health volunteers in child and maternal health programs was in many cases reported to be effective in increasing the access of community members to health services due to their ability to monitor practices, share information and to deliver basic services themselves. 20 Within water and sanitation, the results appear more mixed. While one evaluation (Bangladesh) reported almost 100% access to latrines as a result of implementing a community-led total sanitation (CLTS) approach21 , Plan Cambodia reported lower latrine access and quality of communities participating in CLTS compared to communities participating in latrine- subsidy programs. 22 Lastly, in education, community participation is reported to have contributed to greater community management of schools in several cases, including school planning and monitoring of teacher attendance. However, the data reviewed offered mixed evidence regarding actual increases in education quality as a result of community participation. 23 Capacity In an effort to contribute to better service quality, Plan has placed significant emphasis on strengthening the capacity of government front line staff. This strategy is particularly prevalent in Plan’s education projects, in which Plan is placing a strong emphasis on training teachers. While these efforts in many cases have led to a significant increase in teacher capacity, several evaluations found a gap between levels of knowledge and actual increases in education quality 24 . Similar to the factors affecting community-level behavior change, the evaluations cited two main variables for explaining the gap between teacher knowledge and practice: 1) high turnover of teachers and 2) lack 20 Plan Bolivia/Health/2007; Plan Kenya/Health/2009. 21 Plan Bangladesh/Water and Sanitation/2007. 22 Plan Cambodia/Water and Sanitation/2009. 23 Plan Bolivia/Education/2010; Plan Ghana/Education/2008. 24 Plan Sierra Leone/Education/2010; Plan Malawi/Education/2010. 24 of continuous and institutionalized training, monitoring, and supervision. 25 Other capacity-related strategies, such as the training of service administrators or local government representatives, are mentioned anecdotally but do not offer sufficient evidence to draw conclusions on the effectiveness of CCCD. Advocacy Advocacy at the national level has become a central strategy component under CCCD. Its increased implementation is based on the realization that community access to high quality services is influenced by decisions taken on the national level (e.g. policy frameworks, funding allocation, etc.). We find significant variation across program countries regarding the use of advocacy strategies. Evaluations from countries without a strong emphasis on national level advocacy repeatedly point to the need to engage on the national level for greater program effectiveness. For instance, evaluations from Sri Lanka and Nicaragua emphasized that the inclusion of topics related to sexual and reproductive health into school curricula required engaging more systematically policy makers at the national level. 26 An evaluation of Plan Malawi’s child survival program emphasized the need to target the central government for greater resource allocation in order to improve the quality of health services and drug availability in communities. 27 In the program countries that have implemented national level advocacy strategies, we find significant evidence for the potential of advocacy in improving the access and quality of services on the community level. The most commonly cited benefit of advocacy strategies regarding program effectiveness is their potential to scale-up effective program approaches and for Plan to reach a greater number of children. In most documented cases, this was achieved through evidence-based advocacy and the inclusion of community-based methodologies into government policy frameworks. For instance, the Ministry of Health in Bolivia adopted Plan’s community-level IMCI as a public policy and took over the training of community health workers. 28 Plan Nepal was able to scale-up its community-based newborn treatment by community health workers (which is viewed as 25 Plan Sierra Leone/Education/2010; Plan Ghana/Education/2008. 26 Plan Sri Lanka/Health/2009; Plan Nicaragua/Health/2008. 27 Plan Malawi/Health/2007. 28 Plan Bolivia/Health/2007. 25 the more effective approach to combat child mortality due to the high rate of home births), which has now become part of national policy. 29 Accountability and governance The last strategic component of CCCD affecting service access and quality concerns efforts by Plan to strengthen the governance of service delivery systems by linking CBOs and volunteers to state agencies. This strategy (albeit not formulated as such within Plan’s CCCD documents) is increasingly implemented by Plan across program countries. The evaluations revealed a range of expressions on how Plan has facilitated and attempted to institutionalize interactions between communities and the government in an effort to improve the quality and access of services in communities. One illustration refers to the participation of communities and CBOs in planning meetings and forums on the municipal or district level. For instance, Plan Pakistan supported the invigoration of district health forums, in which communities could directly interact with local authorities regarding their health needs. Similarly, Plan Bolivia supported the establishment of administrative information committees, which constitute a regular feedback mechanism on maternal and child health indicators between communities and local authorities. 30 Furthermore, several countries reported that they are gradually succeeding in integrating community volunteers into more formal service delivery system, thereby institutionalizing the links between communities and state agencies. As a result, these volunteers are no longer perceived as ‘Plan volunteers’ but instead are trained and supervised by the state agency directly. 31 Lastly, in the education area, evaluations reported that community-level school management groups are assuming monitoring functions and are able to share their observations with local government authorities in an effort to increase the quality of education service provided by local teachers. 32 There is insufficient evidence to conclude that greater links between communities and state agencies have led to an improvement in the quality and access of services. The vast majority of evaluations did not assess this link. Instead, most evaluations conceptualized the existence of a link between communities and state agencies as a sufficient outcome to demonstrate an increase in the ability of 29 Interview with Plan Nepal Health Program Manager; June 10, 2011. 30 Plan Bolivia/Health/2007. 31 Plan Bolivia/Health/2007; Plan Kenya/Health/2009. 32 Interview with Plan Malawi Education Program Manager; June 1, 2011; Plan Ghana/Education/2008. 26 communities to voice their demands vis-à-vis the government. Very few studies articulated if and how these linkages have an effect on program outcomes in terms of access and/or quality. Evidence from a small number of evaluations indicates that the primary mechanism for communities to hold governments accountable is political pressure and an increasing concern for political leaders to stay in power. 33 Another example points to a more simple change in awareness as a result of increased community-state interactions, which can induce authorities and service providers to increase their commitment to access and quality of services for communities and children. 34 Challenges and limitations Plan’s CCCD strategy constitutes a comprehensive effort to work with a variety of stakeholders in order to increase the access and quality of education, health, and water services for the communities. In addition to tracing the mechanisms through which this strategy has been effective, this study also identifies several factors that are likely to influence the effectiveness of Plan’s strategy. Community-based approaches to service delivery face a number of challenges to their effectiveness. First, a comparative evaluation of community-based versus state-based approaches to community sanitation in Cambodia found that while community-based approaches are more likely to increase the participation of community members, they are less effective in providing access to sanitation, in particular to the poorest. Second, the average quality of sanitation infrastructure was found to be lower in community-based programs. Although evaluations of CLTS have found that communities are resourceful in building latrines in the absence of outside funding, it has also been observed that the improvements in overall sanitation are oftentimes limited since CLTS for the most part focuses on community-level behavior change (i.e. open defecation) and does not offer solutions to problems of disease-spreading due to low quality latrines. 35 Similarly, an evaluation that compares state-run with community-managed schools in Guinea-Bissau found that education quality was generally higher in state-run schools due to greater number of teachers and their superior training. Third, limited resources were a major constraint to provide adequate materials for community-based 33 Plan Bangladesh/Water and Sanitation/2007. 34 Plan El Salvador/Water and Sanitation/2009; Plan Ghana/Education/2008. 35 Plan Cambodia/Water and Sanitation/2009. 27 schools. 36 Lastly, one of the most commonly cited factors necessary for an improvement in access to services during the project/program cycle remains the availability of infrastructure (e.g. school buildings, health centers, etc.), which is a factor that can be supported by communities through the provision of labor and materials but rarely is achieved without the support of government or Plan funding. 37 These limitations of community-based approaches to water, health and education are not new and were one of the main reasons why Plan has expanded its work to different levels and stakeholders. The review of the evaluations suggest that not the commitment and involvement of only the state but of a variety of stakeholders, including authorities on the local and the national level, other civil society organizations, and communities, is a key variable to effectively improve the access and quality of services. Considering this finding, Plan’s limited CCCD-based engagement with local civil society represents an important weakness. The evaluations demonstrate that the most effective programs might not be the ones where state agencies unilaterally assume the task of implementing services to communities but instead are cases, in which multiple stakeholders interact, pool resources and capacities and assume different tasks in diverse forms of governance arrangements. The evidence also points to several contextual factors. First, given Plan’s program-oriented advocacy approach, interactions with state agencies appear to be facilitated by the existence of national legal frameworks and policies, which Plan can support in their effective implementation. Second, the existence of decentralization policies, which give greater autonomy and power to district level authorities (Plan’s primary government counterpart in many program countries), have shown to facilitate Plan’s work on this level. Lastly, interviewees remarked that a basic level of state capacity in terms of institutional presence, availability of infrastructure and service provision, are a precondition in allowing Plan to assume its facilitating role under CCCD. 36 Plan Guinea-Bissau/Education/2010. 37 Plan Burkina Faso/Education/2008; Plan Sierra Leone/Education/2010; Plan Mali/Education/2010; Plan Ethiopia/Education/2009; Plan China/Water and Sanitation/2010. 28 4.4. Sustainability Another major motivation for Plan’s shift towards CCCD was the realization that its welfare-driven approach of directly providing goods and services to poor communities was not sustainable. Under CCCD, the goal of creating sustainable impact has been elevated to be a strategic priority of Plan’s work. Sustainability in this context implies that program outcomes are: • sustained beyond Plan’s engagement in a particular area; and • resilient to other changes in the local environment. Beyond using these two criteria, the review did not assess whether the strategies implemented, such as participation or capacity-building are sustainable in and of themselves, but whether their existence can account for changes in the perceived sustainability of other program outcomes, such as behavior change and access/quality of services. CCCD addresses the issue of sustainability using a variety of strategies. Plan primarily enhances sustainability with its focus on community participation and the institutionalization of linkages between communities and state agencies. This study also finds that sustainability remains an area of concern for Plan. Although local stakeholders do in most cases no longer rely on Plan’s direct material support and service delivery contributions, they often exhibit now a level of dependency on Plan’s technical expertise and facilitating role. Participation Community participation was the most frequently mentioned CCCD strategy mentioned in the context of sustainability. Several of the evaluations reviewed for this study acknowledged the significance of community ownership (as a result of participation) as relevant to the sustainability of program outcomes. Plan Nepal summarized this process as follows: “We are involving all the community members in the planning of our program. This increases the ownership in implementation and it helps the sustainability of the program.” 38 For instance, a comparative study on the sustainability of rural sanitation infrastructure projects in Timor Leste concluded that 38 Interview with Plan Nepal Health Program Manager; June 10, 2011. 29 participation of communities contributed to a significantly higher percentage of functioning water supply systems up to six years after construction compared to projects that did not actively involve communities. 39 The most common expressions of sustainability-related participation strategies were the engagement of communities in managing services (e.g. schools, community health centers) and the participation of communities in managing infrastructure systems (e.g. water supply systems). Less frequently, the evaluations pointed to the ability of CBOs to continue to shape local and national-level planning processes as a result of their participation in Plan projects. 40 The engagement of CBOs and volunteers in project management processes can contribute to overcoming the often observed drop in activity levels after the end of a project cycle since this approach relies on community resources for many of the day-to-day operations. Plan’s long-term presence in communities through sponsorship activities allows the organization to support the continuous participation of CBOs even after a particular project ends. In addition, established CBOs can serve as a starting point for future community development work (carried out by Plan or other development organization) since the organizational structure remains intact after the end of a particular project. Several evaluations pointed to the advantages of working with established groups since these groups already possess the capacity and organizational experience to manage projects and work with other actors. 41 The evaluations also reveal that Plan’s participation strategies are facing several sustainability challenges. The two most commonly cited limitations to community participation as a sustainability tool were: • reliance on volatile volunteer groups, and • lack of financial and other resources. 39 Plan Timor-Leste/Water and Sanitation/2009. 40 Plan Pakistan/Health/2007, Plan Ecuador/Education/2008. 41 Plan Zimbabwe/Water and Sanitation/2009; Plan Ecuador/Education/2008; Plan Cameroon/Health/2007; Plan Malawi/Health/2007. 30 With regards to community volunteers, several evaluations expressed concerns that participation among community volunteers would diminish after the completion of a project. 42 Since volunteers are generally not compensated by Plan, the economic incentives for community volunteers to participate are low. High personnel turnover among community-based volunteers is common. Plan’s primary response to this sustainability challenge is to institutionalize training mechanisms for CBOs through linking them with state agencies. The sustainability of this approach is analyzed in the subsequent section on governance. Lack of financial resources represents the second limitation of sustaining the participation of CBOs. Plan’s strategy of long-term community participation relies on the ability of CBOs to finance their activities through community contributions or securing other sources of funding. However, several evaluations recount difficulties of CBOs to secure further funding after the end of a project cycle. 43 Securing financial resources represents a major challenge to sustaining the effective contribution of community participation to program objectives. Capacity and awareness Strengthening the capacity and awareness of all stakeholders is a critical step towards reducing the reliance of communities on resources provided by Plan or its partners. Furthermore, capacity and awareness are a necessary condition enabling communities and other local stakeholders to independently participate in long-term development processes. With regards to changes in awareness at the community level, the observed changes are most visible in a reduced need for continuous involvement by Plan or other external actors (compared to strategies that only invest in infrastructure). For instance, cases are documented where parents continued to send their children to school and remain involved in school management because they better understood the benefits of education.44 As an evaluation from Plan China summarized: “The benefits and value of CCCD and school development planning were significant enough for them to continue to try to act and practice according to these tenets in the future, despite the inevitable and 42 Plan Pakistan/Health/2007; Plan Burkina Faso/Education/2008; Plan Bolivia/Health/2007; Plan Tanzania/Water and Sanitation/2007; Plan El Salvador/Health/2007. 43 Plan Guinea-Bissau/Education/2010; Plan Cambodia/Water and Sanitation/2009; Plan Pakistan/Water and Sanitation/2008; Plan Albania/Education/2008; Plan Malawi/Health/2010. 44 Plan Burkina Faso/Education/2008; Plan China/Education/2009. 31 expected withdrawal of Plan China’s support and the uncertainty of Government support.” 45 Similar experiences are reported at the government level where Plan’s diverse awareness raising activities has contributed to a greater visibility of certain issues, such as HIV/AIDS or sexual and reproductive rights, and influenced the agenda setting of public institutions. 46 The evaluation findings are less positive with regards to Plan’s success in strengthening the capacity of government staff to improve the delivery of services to communities in a sustainable way. The quality and sustainability of services is often compromised by a lack of sufficient training and the absence of institutionalized monitoring and supervisory mechanisms. 47 Evaluations also pointed to a number of external factors explaining these challenges. First, state agencies may lack interest in and understanding of Plan’s program approach and objectives, which results in a lack of commitment or even resistance to institutionalize capacity-building and training mechanisms for service delivery personnel. Second, state agencies may be willing to cooperate with Plan, but lack institutional capacity to effectively implement the required training or monitoring mechanisms. A number of evaluations specifically point to the high turnover of government front line staff and decision-makers, which represents a form of institutional instability negatively affecting sustainable improvements in the quality of service delivery systems. The reviewed data provided very limited evidence about the role of CCCD in increasing the capacity of local civil society organizations as a tool for greater sustainability. Most evaluations mentioning Plan’s collaboration with local NGOs made reference to their contractual relationship and their role in service provision, not their role in ‘upward’ policy dialogue and accountability processes. 48 Contractual relationships that involve a transfer of financial resources are less likely to be sustainable once Plan’s support ends. As one of the reports observed: “Most of the NGO partners did not indicate that they expected to continue to provide services if Plan were no longer funding them, nor did they show indications of trying to take more responsibility for programs.” 49 In other 45 Plan China/Education/2009. 46 Plan Ecuador/Health/2010. 47 Plan Ecuador/Health/2010; Plan Sri Lanka/Health/2009; Plan Malawi/Education/2010; Plan Mali/Education/2010; Plan Bangladesh/Water and Sanitation/2007; Plan Bolivia/Health/2010 Plan Pakistan/Water and Sanitation/2008; Plan Bolivia/Education/2010; Plan Tanzania/Water and Sanitation/2007; Plan Cameroon/Health/2007. 48 See for example Plan Bolivia/Health/2007; Plan Bangladesh/Education/2008; Plan Malawi/Health/2007. 49 Plan Nepal/Health/2007. 32 instances, Plan did engage in efforts to increase the organizational and technical capacity of local NGOs, but these efforts were limited and not part of a broader sustainability approach towards local partners. 50 Advocacy Advocacy strategies have the potential to contribute to the sustainability of Plan’s programs by institutionalizing effective service delivery approaches at a national policy level. Plan’s most common advocacy strategy focuses on scaling-up effective methodologies and community-based service delivery approaches through evidence-based lobbying. This strategy can extend successful program activities to areas where Plan is not active and it can contribute to greater sustainability of program outcomes since it supports the institutionalization of strategies, funding, planning and program implementation processes at the national level. There are also isolated cases where Plan has contributed to the creation of new policy frameworks focused on the well-being of children and adolescents. 51 In other cases, evaluations point to increased budget allocation towards child-focused policies at the local level. 52 At the same time, the evaluations also reveal that Plan is much less active in other areas with little evidence of changes in the legal environment or in funding allocations at the national level. Plan’s advocacy strategy is based on a partnership approach with state agencies, alignment with existing government policies and programs, and complementing technical and financial resources for improved policy design and implementation. The organization’s existing relationships with state agencies (particularly ministries and local level government) often ensure access to decision-making processes while its ties with communities afford Plan significant influence during implementation processes. Although this approach can improve design and implementation of existing programs, it relies on the good will of decision-makers to adopt policies for the benefit of poor, rural communities. As one Plan manager described this limitation: “We are facilitators of development, not duty bearers. We complement the efforts of duty bearers. Our objective is to enter in partnerships and support them. But oftentimes we do our part and the counterpart does not do its 50 Plan Cameroon/Health/2007; Plan Pakistan/Health/2007. 51 Plan Pakistan/Health/2010. 52 Plan Bolivia/Health/2010; see also Gneiting, et al. (2010). 33 part. “ 53 Plan’s reliance on cooperation with state agencies makes it less effective when political circumstances are less favorable, for example, when corruption or discrimination undermines government programs. More confrontational and mobilization-oriented advocacy strategies, currently not part of Plan’s strategic repertoire, would be required to address such political challenges. Accountability and governance Improving governance and accountability mechanisms by linking CBOs with state agencies is one of the most frequently cited sustainability strategy. To ensure that service delivery activities carried out by CBOs continue after Plan’s involvement ends, Plan works to integrate community-based mechanisms with existing state-level service delivery mechanisms. The most common expression of this strategy was found within its community health programs where Plan has worked to integrate community health workers within state systems and thereby formalizing their role in government- run service delivery programs. In other instances, Plan achieved the integration of CBOs in local planning and feedback processes, thereby creating increased opportunities for information exchange, shared planning and expressing demand for services. 54 While this strategy was implemented consistently across programs, its actual effect on sustainability appears less clear. In fact, a significant number of evaluations pointed to the limited success of strategies to promote the uptake of program activities by state agencies. The main challenges identified included a lack of state capacity and willingness to take over program components (1), dependence on Plan’s technical and financial resources for effective program implementation (2), and confusion about the division of responsibilities between state agencies and CBOs (3). 55 Challenges and limitations Within the scope of this study, analyzing the effect of CCCD on the sustainability of Plan’s programs is complicated by the lack of available data on activity levels after the end of Plan’s 53 Interview with Plan Ghana Learning Advisor, June 2, 2011. 54 Plan Ecuador/Education/2008; Plan Bolivia/Health/2007; Plan Bangladesh/Education/2008. 55 Plan Uganda/Health/2009; Plan Sierra Leone/Health/2010; Plan Ecuador/Health/2010; Plan Bangladesh/Water and Sanitation/2007; Plan Ethiopia/Education/2009. 34 programs. Still, there is sufficient evidence across the evaluations pointing towards specific sustainability challenges. Plan’s 2010 Program Guide emphasized the need for change at multiple levels and of multiple actors in order to create sustainable impact (Zuurmond, 2010). In practice, this project did not find a commonly applied sustainability strategy across programs. While some programs appear to emphasize the role of the state in taking over program activities after the end of Plan’s engagement, others rely on the capacity and awareness gains at the community level as primary sustainability strategy. A third strategy combines community-level and state-centric approaches and is based on the idea that community participation in program activities is best institutionalized through long- term linkage to state agencies. In the process, training and supervisory functions shift from Plan to the state and spaces for feedback mechanisms between communities and local authorities are created. It is apparent from the evaluations that sustainability strategies that prioritize the role of the state depend on factors beyond Plan’s control, in particular the level of institutional capacity and political will. The key necessary condition for sustainability is a favorable political environment within program countries. Of the factors that lie within Plan’s control, the evaluations are divided with regards to the strengths and weaknesses of a long-term presence in the communities. Some evaluations identified this presence as a positive factor because only a long-term engagement can ensure the institutionalization of sustainable linkages between communities and local government. In contrast, other evaluations emphasize a risk of dependency and over-reliance on Plan’s facilitating role. Plan’s extended presence and support of communities can negatively affect their ability and willingness to independently engage in development activities, such as community-level development planning, engage with local authorities, or raise funds from alternative sources. 56 It can also lead to a detachment of community volunteers from their communities since they become increasingly affiliated with Plan’s program activities. 57 Related to unintended consequences of Plan’s extended community presence, several evaluations also mentioned the lack of a formal exit strategy as a problem. In some cases, Plan has attempted to address such sustainability concerns by 56 Plan Ethiopia/Education/2009; Plan Ghana/Education/2008. 57 Plan Senegal/Health/2008; Plan Ghana/Education/2008; Plan Malawi/Health/2010. 35 supporting local partners to take over program functions a few months before the end of project activities. However, a number of evaluations show that this short-term approach is not adequate in ensuring sustaining programs beyond Plan’s own engagement. 58 4.5. Equity “Promoting an environment of social inclusion and protecting children from discrimination is […] a key principle of CCCD” (Zuurmond, 2010: 23). This principle emerged from the strong association between lack of rights and membership in a specific social group (e.g. children with disabilities, indigenous children). By placing greater emphasis on equity concerns, Plan can better identify and target the most disadvantaged groups and contribute to a reduction in inequality. In contrast to the program outcomes discussed above, equity effects are not consistently assessed in the evaluations reviewed. For example, changes in terms of behavior as well as access and quality of services are not consistently disaggregated by gender or other categories of inclusion. Instead, the majority of evaluations merely summarize the efforts by Plan to address issues of discrimination or marginalization. This review summarizes the limited information about equity effects contained in the evaluations, but more systematic follow-up studies are needed to increase the confidence in the results presented here. Plan’s emphasis in the area of equity focuses more generally on the rights of children and gender issues. The strategies most frequently identified were: • awareness raising campaigns at the community level, • community participation strategies aimed at reaching the most marginalized, and • advocacy efforts aimed at including the concerns of children and adolescents in national policy frameworks. The evaluations reveal only a weak link between CCCD and equity since strategies specifically designed to increase equity are less common in program activities. Some vulnerable groups, such as 58 Plan Indonesia/Water and Sanitation/2010; Plan Ethiopia/Education/2008; Plan Nepal/Health/2007. 36 children with disabilities, indigenous children, migrant/stateless children are not specifically targeted, despite being more vulnerable than other community members and issues of exclusion and discrimination below the community level have yet to gain systematic attention. Participation From an equity perspective, Plan perceives participation of marginalized groups as a means to increase their empowerment in order to shape decision-making processes. One of Plan’s most frequently cited efforts in this area focuses on the participation of female community members of all ages at the community level. Plan’s education program in Sierra Leone summarized this strategy as follows: “A major strategy was to make women more visible in community structures as a first step towards meaningful participation and empowerment […]. In these ways, it was intended that women’s attendance, participation and effect on decision making could improve.” 59 Some evaluations highlighted the problem that existing community groups are formed by male community elites and do not allow for the active participation of women. 60 Strengthening gender equity and female participation is more successful when Plan is involved early on in the formation of CBOs it collaborates with as part of its program activities. Besides the direct effects of participation of women, broad community participation was also cited as a means to better identify the most excluded community members, for example by using activities such as wealth rankings. Such efforts can increase compassion and concern for disadvantaged community members. But community-based approaches without a build-in concern for equity can also increase exclusion of marginalized groups, in particular the poorest members of communities. For instance, various evaluations reported that the poorest members of a community had less access to fee-based water programs 61 and latrines in community-led sanitation projects, 62 while also facing greater difficulties to participate in health-related activities. 63 In some cases, community-based volunteers had difficulties reaching the poorest and most remote families due a 59 Plan Sierra Leone/Education/2010. 60 Plan China/Water and Sanitation/2010; Plan Bangladesh/Water and Sanitation/2007. 61 Plan Pakistan/Water and Sanitation/2008. 62 Plan Cambodia/Water and Sanitation/2009. 63 Plan El Salvador/Health/2007. 37 lack of resources and time. “The geographic element of vulnerability may be the hardest to address in these programs dependent on community volunteers.” 64 Capacity With regards to issues of equity and inclusion, Plan’s predominant capacity-related strategy focuses on awareness campaigns regarding the rights of children and girls at the community level. These activities aim at changing the attitudes of parents towards girls’ education and thereby increase their access to educational opportunities. 65 Rights discourse regarding the rights of children in general not only has the potential to increase the confidence and self-esteem of children and adolescents, but can also lead to an increase in respect and inclusion afforded by other community members. 66 A second category of strategies refers to efforts by Plan to make the access and quality of schools more equitable. Examples of this strategy include Plan’s contributions to the development of materials specific to indigenous children and supporting the establishment of Alternative Schools, which are more easily accessible for “hard-to-reach” children. 67 Both of these observations are based on limited anecdotal evidence. Advocacy From an equity perspective, Plan has focused the vast majority of its advocacy efforts on promoting the rights of children and adolescents overall. Plan’s engagement at the national level can address issues of child discrimination in three complementary ways. First, Plan has directly lobbied policy makers for the creation and implementation of child- and youth-friendly policy frameworks in order to end the general neglect of such interests at the national levels. Plan has been able to contribute to the creation of these policies by participating in a variety of national level forums and working groups and also by using its ongoing interactions with policy makers. The primary advocacy tool used relies on presenting empirical evidence about effective service delivery approaches and offering technical and financial support for the design and implementation of child- and youth- 64 Plan Bolivia/Health/2007. 65 Plan Burkina Faso/Education/2008; Plan Mali/Education/2010. 66 Plan Ecuador/Education/2008. 67 Plan Ecuador/Education/2008. 38 friendly policies. 68 Second, Plan has participated in civil society campaigns and alliances, in which it promotes policy changes favoring children by partnering with multiple local stakeholders. 69 The Social Contract for Education Campaign in Ecuador is a good example for this type of national- level advocacy work. 70 Third, Plan has supported the mobilization and organization of children and adolescents at the national level, which raises public visibility of issues affecting those sections of society. While some evaluations clearly document the success of Plan’s advocacy efforts to reform policies, others recount failure to accomplish such goals primarily due to the resistance or lack of commitment of policy makers. This resistance is particularly visible in the area of sexual and reproductive rights. 71 Several evaluations concluded that Plan did not target the most relevant policy makers or did not employ the appropriate strategies in its advocacy approach. 72 Each of these claims about the success or failure of Plan’s advocacy efforts offer compelling starting points for more systematic studies, but the conclusions remain tentative until such strategies are more widely applied and evaluated. Accountability and governance The evaluations reviewed contain little specific evidence about the relationship between Plan’s accountability and governance strategies and their effects on equity concerns. Two possible mechanisms were identified. First, linking communities with state agencies can strengthen the participation of women and children in local planning processes as well as provide feedback on the quality of services. Second, Plan’s focus on community-based service delivery systems anchors these systems within communities, which can have positive effects on the accessibility of services for all community members. However, the concrete effects of participatory governance and accountability mechanisms on the inclusion and empowerment of vulnerable groups has received little attention in the evaluations. 68 Plan Bolivia/Health/2007; Plan Bangladesh/Education/2008; Plan Nepal/Health/2007. 69 Plan El Salvador/Health/2007; Plan Ghana/Education/2008; Plan Malawi/Health/2010. 70 Plan Ecuador/Education/2008. 71 Plan Sri Lanka/Health/2009; Plan Ecuador/Health/2010. 72 Plan Burkina Faso/Education/2008; Plan China/Education/2008; Plan Uganda/Health/2009; Plan Malawi/Education/2010. 39 Challenges and limitations With regards to equity, the limited implementation and neglect of this principle in Plan’s programs represents the main current limitation of CCCD. 73 Evaluations point to a gap between the recognition of issues of equity and inclusion in program designs and Country Strategic Plans (CSPs) and their actual application within projects and programs. 74 In cases where equity is addressed, Plan’s focus is predominantly on two issues: the rights of children and gender issues. Other groups subject to discrimination, such as children with special needs, indigenous populations, and the poorest members of communities, are much less frequently addressed. A second limitation revolves around Plan’s (and the evaluators) understanding of its approach to addressing issues of inclusion and non-discrimination. Of the cases that do assess effects on equity, several evaluations appear to perceive them as unintended or indirect consequences of Plan’s programming. For example, improvements in the access to education that benefit both girls and boys or the participation of both women and men in certain training activities are cited as instances of inclusion although no specific actions for inclusion were taken and underlying issues of discrimination (cultural, economic) were not addressed in these instances. Lastly, the survey of the evaluations shows no consistent strategic understanding of child participation as a means to increase their inclusion in development processes. While child participation is a central principle of CCCD, there is no standardized approach for implementing child participation across program areas and countries. Child participation is more frequently mentioned in the context of program planning and community level implementation, less so with regards to their role in addressing issues of exclusion and discrimination beyond the community level. One of the reasons for this finding may be the fact that national level child participation activities are evaluated as part of separate child participation and/or child protection programs. 73 Plan Sierra Leone/Education/2010; Plan Indonesia/Water and Sanitation/2009; Plan Burkina Faso/Education/2008; Plan Zimbabwe/Water and Sanitation/2009; Plan Nepal/Health/2007; Plan El Salvador/Health/2007; Plan Ghana/Education/2008. 74 Plan Bolivia/Health/2007. 40 5. Internal factors and CCCD One of the central objectives of this research was to identify factors inside and outside of Plan that enable or disable CCCD effects on program outcomes. While many of the factors addressed so far concern characteristics in the external environment, this study also identified a series of factors internal to Plan International which can explain the (lack of) linkages between CCCD and program outcomes. Situational analysis The adoption of CCCD has placed greater emphasis on the principle of targeting the most vulnerable populations and identifying root causes for their exclusion and marginalization. A situational analysis focused on identifying excluded and marginalized groups can enhance program design and increase chances of recognizing the most vulnerable groups early on. In programs where a rigorous situational analysis was conducted at the beginning of the CSP, it usually enhanced program outcomes. For instance, as a result of doing a situational analysis, Plan Ghana was better able to determine the causes behind low student achievement ratios, and was subsequently better able to address them. 75 Plan Bolivia conducted a situational analysis and found convincing evidence on the systematic difference between indigenous and non-indigenous children in terms of educational enrolment and achievement, which allowed the organization to more heavily focus its program design on supporting indigenous children. 76 Although situational analyses are apparently part of every new CSP, there was limited evidence that such findings actually impact program design in a consistent manner, for example by significantly expanding the reach to the most vulnerable groups. Program planning Program level planning represents a second important factor influencing the ability of CCCD to enhance program outcomes. Here, two observations related to participation and exit strategy stand 75 Plan Ghana/Education/2008. 76 Plan Bolivia/Education/2010. 41 out. First, while references to participatory approaches to program planning were rare, some examples provide evidence showing how local participation in program design and planning improved outcomes. In the case of Plan Ecuador, a bottom-up participatory planning process in the development of the country strategy (based on community and child consultation) increased the ownership of Plan’s CCCD strategy among Plan’s beneficiaries as well as partners. It also meant that Plan staff responded positively to the emphasis on national as well as local advocacy efforts, which resulted from the participation process. 77 Second, several evaluations pointed to a lack of a well-developed exit strategy. 78 As Plan’s role under CCCD evolves from implementer and provider of material support to facilitating community agency and capacity building, articulated criteria and triggers for an exit strategy become a necessary part of the ongoing dialogue with stakeholders. Monitoring, Evaluation and Learning systems A consistent finding across evaluations is the apparent lack of focus in Plan’s M&E system (PALS) on long-term outcomes in terms of CCCD categories, such as empowerment and advocacy. While PALS has gone part way in facilitating a better measurement of program outcomes informed by CCCD, it is not fully effective in capturing the long-term changes associated with CCCD efforts. 79 Furthermore, several evaluations remarked that PALS does not adequately assess accountability and sustainability improvements. Most frequently, PALS’ focus remains on measurement of service delivery and outreach activities as well as outputs. These measurements are often more quantitative in nature (e.g. changes in levels of awareness, knowledge and skills). There appears to exist a “…limited culture of defining activity outcomes, and analyzing results and changes after activity completion”, as an informant from Plan Burkina Faso indicated. 80 Regarding CCCD-related learning processes, Plan has gradually institutionalized CCCD as its strategic framework. Plan commissioned two global thematic studies that assessed the implementation and effectiveness of its programs from a CCCD perspective. Plan’s 2010 Program Guide defines CCCD principles and strategies and Plan’s Global Effectiveness Framework bases its dimensions of change on CCCD as its guiding framework. Nevertheless, this study failed to find 77 Plan Ecuador/Education/2008. 78 Plan Indonesia/Water and Sanitation/2009; Plan Ethiopia/Education/2009; Plan Nepal/Health/2007. 79 Plan Bangladesh/Water and Sanitation/2007. 80 Plan Burkina Faso/Education/2008. 42 evidence for the existence of program-specific guidelines on the implementation of CCCD. In fact, the most frequent suggestion by program managers was the expressed need for guides on when and how to apply CCCD in their programmatic work. Related, a variety of evaluators voiced criticism about the lack of baseline data, which complicated their ability to measure changes in program indicators. Staff level issues There is still significant variation in the level of understanding and commitment related CCCD among Plan staff. 81 This likely affects the extent of CCCD implementation and the potential of the strategy to improve program outcomes. Staff appears to not be rewarded for focusing on longer term outcomes and staff performance evaluations do not incorporate outcome and impact level achievements associated with the ambitious CCCD agenda. Plan is starting to address CCCD- related staff level issues by formalizing the creation of a so-called ‘CCCD Academy’, which aims to develop and roll out a curriculum for organization-wide staff training on CCCD. 82 Internal and downward accountability CCCD principles, such as participation and accountability, are also considered relevant for Plan’s internal processes and relations with communities. Based on the review of country-level and global evaluations, this study confirms earlier findings by the same authors (Gneiting et al, 2010) about the absence of internal reflection processes regarding how the CCCD approach affects Plan’s relationship with local communities. By developing stronger ‘downward accountability’ mechanisms towards local communities, Plan could recognize its own power towards them while allowing communities to apply participatory accountability mechanisms in their own interactions with Plan. 81 Interview with Plan Ghana Learning Advisor, June 2, 2011; see also Gneiting, et al. (2010) and Synthesis of Country Programme Progress Reports for FY10, H. Gallagher. 82 Plan Ecuador/Education/2008; see also Thematic Evaluation on Primary Education, 2009 and Thematic Evaluation on Child Survival, 2007. 43 6. Conclusions and recommendations Plan’s shift to CCCD represents a significant shift in the organization’s approach to its development work. In its design, CCCD takes a more comprehensive approach to development that recognizes the multi-dimensional nature of poverty and goes well beyond addressing immediate material needs. By acknowledging structural factors of exclusion and discrimination and addressing them at multiple levels, Plan has an enhanced ability to help shape environments that are conducive to the fulfillment of child rights. The adoption of CCCD also reflects a greater alignment with international standards of human rights and effective development aid, including the emphasis on local ownership of program activities and a harmonization with national development priorities (OECD, 2005/08). Putting CCCD in the context of Plan’s many years of experience in development work, the most innovative characteristic of this strategic shift is the greater focus on the role of governmental actors. While Plan has coordinated its efforts with government agencies for some time, this interaction has become more strategic under CCCD since it is based on the expectation of states’ responsibility for development activities in the medium- to long-term. Plan’s existing ties with local communities remain strong under CCCD and are now being complemented by a more multi-level approach to improving lives of children and their communities. Plan’s comparative advantage in applying CCCD is most pronounced when it works not only at the community but across a variety of levels to address development issues. This multi-level approach allows Plan to create linkages and synergies between local actors and tie its experience from its community-level engagement to policy-making processes at the national and international level. Regarding the relationship between CCCD and program effectiveness and sustainability, the review of Plan’s external evaluations has led to the identification of patterns that demonstrate how different CCCD strategies can affect different program outcomes. Overall, results appear more significant with regards to the effects of CCCD on the program outcome dimensions of behavior change and service access and quality than with regards to sustainability and equity. This study also finds that the most positive assessments were found within programs that implemented CCCD consistently across the different strategic categories (i.e. participation, capacity, advocacy, and governance and 44 accountability). This finding points to the complementary character of different CCCD strategies in enhancing the effectiveness and sustainability of Plan’s programs. Limitations to the effectiveness of CCCD are mostly due to two main factors: inadequate implementation and limitations related to the local context. Some of the strategies that are found to be less frequently implemented refer to actions targeting inequity (beyond gender issues and the rights of children in general), the strengthening of accountability mechanisms, and mobilization strategies beyond the community level. In addition, the research also discovered patterns of recurring limitations related to the external environment, most importantly: • limits to the effectiveness and sustainability of community-based approaches; • lack of institutional capacity and political will among governmental actors; and • institutional fragility of state agencies, lack of resources, and the politicization of programs and issues. Many of the external limitations to CCCD effectiveness could be addressed by using some of the currently less prevalent strategies of cross-level mobilization and a greater focus on the accountability of government actors. A related question concerns the significant variation in the ways program effectiveness and sustainability was measured throughout the evaluations. While some evaluations focused on changes in traditional development indicators, such as school enrolment or immunization levels, others measured program effectiveness in more process-oriented terms (i.e. indicators related to participation, empowerment, capacity, etc.). This variation in indicators did not only complicate the analysis of this study but also points to a certain lack of clarity with regards to how to measure the success of Plan’s work. Plan’s program effectiveness framework establishes that program outcomes are redefined under CCCD towards changes among local stakeholders, which in turn contribute to greater fulfillment of child rights. Nevertheless, we find a lack of consistent effort in assessing and tracing the links between CCCD outcomes and program impact (which is what this project aimed to accomplish). Instead, there appears to remain a certain gap between the two levels of analysis: Plan’s efforts to measure CCCD outcomes and donor-driven assessments of program impact. 45 A last question emerges from the observed variation in CCCD implementation across Plan’s program countries. This reflects a certain level of autonomy of program countries to adapt CCCD to the local context. While this pragmatic approach to the implementation of CCCD might enhance the suitability of CCCD in local contexts, it might also contribute to apparent lack of shared understanding of CCCD across program countries. From the interactions with Plan staff in a variety of program countries, it has become apparent that CCCD is understood by some as a participatory approach while others mention child centeredness as its primary feature and again others emphasize its focus on the state as duty bearer. Promoting a shared understanding of CCCD, while still allowing for local variation, should be a goal of the continued implementation of the program. The creation of the CCCD Academy is an important step in this regard. Recommendations Two main recommendations follow from this study. First, in order to take full advantage of the potential of CCCD, implementing the full scope of available CCCD strategies more consistently and across levels is recommended. Strategies less frequently used should be more fully integrated into future programming, including efforts to enhance inclusion, accountability, and broad-based mobilization. Second, this study recommends building on the lessons learned from this project and to further invest in efforts to document and trace the effects of CCCD on Plan’s contribution to improvement of child rights in its program countries. Due to the limited data, the results presented here can only be used to generate hypotheses about how CCCD affects program outcomes as well as possible intervening factors. Plan should endeavor to systematically and regularly collect data on the health of children and their communities over time and invest in future studies establishing a more controlled environment (e.g., before-after comparisons, use of control groups). Collecting this data and establishing a more reliable understanding about the strengths and weaknesses of CCCD will allow Plan to improve its programmatic work and also provide a valuable means of communication for interactions with donors and the public. 46 References Gneiting, Uwe, Tosca Bruno-van Vijfeijken, Hans Peter Schmitz, and Otto Valle. 2010. Rights- Based Approach to Development. Learning from Guatemala (a report commissioned by Plan International). Syracuse: Moynihan Institute of Global Affairs. INTRAC. 2009. Assessment of Plan International’s partnership strategy. London: INTRAC. OECD. 2005/08. The Paris Declaration on Aid Effectiveness and the Accra Agenda for Action. Paris: OECD. Plan International. 2008. Global Effectiveness Framework. Woking: Plan International. Zuurmond, Irko (ed). 2010. Promoting child rights to end child poverty. Woking: Plan International. 47 Appendices A. List of evaluations B. List of programs selected for follow-up interviews 48 Appendix A: List of evaluations Full Title Country / Region Program Area Year Evaluation of Decentralized Urban Total Sanitation Bangladesh / Asia Water and Sanitation 2007 Project of PSTC Thematic Evaluation on Child Survival Bolivia / Latin America Health 2007 Case Study Plan Bolivia Evaluation of Lipangalala Water Supply and Sanitation Tanzania / Africa Water and Sanitation 2007 Project- Ifakara Programme Unit, Plan-Tanzania Thematic Child Survival -- case study Cameroon / Africa Health 2007 Thematic Child Survival -- case study Malawi / Africa Health 2007 Thematic Evaluation on Child Survival Nepal / Asia Health 2007 Case Study Plan Nepal Thematic Evaluation on Child Survival Case Study Plan El Salvador / Latin America Health 2007 El Salvador Thematic Evaluation on Child Survival Case Study Plan Pakistan / Asia Health 2007 Pakistan School Improvement Program (SIP) End Term Egypt / Africa Education 2007 Evaluation Thematic Evaluation on Primary Education: Burkina Faso / Africa Education 2008 Plan Burkina Faso Case Study Evaluating the Government funded Rural Water Pakistan / Asia Water and Sanitation 2008 Supplies in District Chakwal Final Evaluation of the Plan / NNO Child Survival and Senegal / Africa Health 2008 Safe Motherhood Project in the Health District of Louga EVALUACION DEL PROYECTO “MAXIMIZAR EL Nicaragua / Latin America Health 2008 ACCESO A LOS SERVICIOS DE SALUD REPRODUCTIVA DE CALIDAD PARA ADOLESCENTES” Thematic Primary Education eval Ecuador case study Ecuador / Latin America Education 2008 Quality Education Program in Central and Eastern Albania Education 2008 Albania - School Improvement Project (SIP) 2006-2008 Thematic Evaluation on Primary Education Bangladesh / Asia Education 2008 Plan Bangladesh Case Study Thematic Evaluation on Primary Education Ghana / Africa Education 2008 Plan Ghana Case-study Adolescent Sexual and Reproductive Health Project - Sri Lanka / Asia Health 2009 Summative Evaluation Final Evaluation for Community Based Reproductive Uganda / Africa Health 2009 Health Services Project Thematic Evaluation on Primary Education - China / Asia Education 2009 Plan China Case Study 49 Adolescent Sexual and Reproductive Health Project - Sri Lanka Health 2009 Summative Evaluation Community-Led Total Sanitation (CLTS) in Cambodia Cambodia / Asia Water and Sanitation 2009 KIDCARE Final Evaluation Report September 2009 Kenya / Africa Health 2009 Indonesia / Asia Water and Sanitation 2009 End of Project Evaluation Water and Environmental Sanitation Program Plan International Indonesia Rehabilitation of Potable Water Sources in Chipinge and Zimbabwe / Africa Water and Sanitation 2009 Tsholotsho Districts Thematic Evaluation on Primary Education Ethopia / Africa Education 2009 Plan Ethiopia Case study EVALUACIIÓN EXTERNA,, “EDUCACIIÓN El Salvador / Latin America Water and Sanitation 2009 PARTIICIIPATIIVA EN HIIGIIENE Y SANEAMIIENTO Education Renewal Project Sierra Leone / Africa Education 2010 Final Evaluation Evaluación final del proyecto “Promoviendo el Ecuador / Latin America Health 2010 Cumplimiento, Ejercicio y Exigibilidad de los Derechos Sexuales y Reproductivos de Niños, Niñas y Adolescentes en Ecuador” A REPORT ON END OF TERM PROJECT Malawi Education 2010 EVALUATION OF PLAN MALAWI - LEARN WITHOUT FEAR PROJECT PLAN GUINEA-BISSAU Guinea-Bissau / Africa Education 2010 FINAL EVALUATION OF QUALITY BASIC EDUCATION CPO PLAN CHINA China / Asia Water and Sanitation 2010 FINAL EVALUATION ON COMMUNITY LED WATER AND ENVIRONMENTAL SANITATION PROGRAM IN 6 VILLAGES OF CHUNHUA COUNTY Final Evaluation of the Mother Child Health Care CPO Guinea-Bissau / Africa Health 2010 Final evaluation of the MDG / education for children Mali /Africa Education 2010 project «Improving the Quality of Basic Education» Evaluation of Reproductive Health Initiative with Pakistan /Asia Health 2010 Adolescents Informe de Evaluación Final “ MEJORES PRACTICAS Bolivia/Latin America Health 2010 EN EL CUIDADO DEL RECIEN NACIDO, NIÑOS, NIÑAS MENORES DE 5 AÑOS ” Indonesia /Asia Water and Sanitation 2010 End of Project Evaluation Water and Environmental Sanitation Program Plan International Indonesia 50 EVALUACIÓN FINAL PROYECTO “ESCUELAS DE Bolivia/Latin America Education 2010 CALIDAD. ESTRATEGIA DE NÚCLEOS CATALIZADORES” FINAL EVALUATION OF THE MATERNAL AND Malawi/Africa Health 2010 CHILD HEALTH PROJECT PLAN GUINEA-BISSAU Guinea-Bissau/Africa Water and Sanitation 2010 FINAL EVALUATION OF THE POTABLE WATER AND SANITATION CPO Appendix B: List of programs selected for follow-up interviews Program Country Program Area Interview Date Bangladesh Country Director May 6, 2011 Malawi Country Director May 12, 2011 Bolivia Education Program May 23, 2011 Pakistan Child Survival Program May 24, 2011 Cambodia Water and Sanitation Program May 26, 2011 Malawi Child Survival Program June 1, 2011 Ghana Education Program June 2, 2011 Indonesia Water and Sanitation Program June 6, 2011 Bangladesh Water and Sanitation Program June 6, 2011 Nepal Child Survival Program June 10, 2011 51