Monitoring And Evaluation

Tracking Our Progress: A Global Monitoring and Evaluation Framework for the UN DESD

by Daniella Tilbury

The attitudes of the Students of the Community College in Shaqra University towards the Decentralization of the Secondary School Leaving Certificate Examinations

by Dr. Ahmed Gumaa Siddiek

Work in Progress

Abstract: Contemporary education requires educational authorities to look for the ideal and keep pace with scientific... more Abstract: Contemporary education requires educational authorities to look for the ideal and keep pace with scientific and technological progress to achieve successful learning outcomes. Examinations are means of measurement and evaluation of the work carried out by the teaching force to promote the level of students' achievement, and determine the degree of assimilation and understanding of subjects taught to them. A test is an essential device to help achieve educational goals; it is also an effective tool to reflect the effectiveness of teaching, the success of the curricula and usefulness of the textbooks. The decentralization of secondary school certificate examinations in Saudi Arabia has brought about some complications at the local and international level. The impact on the credibility of the secondary school certificate Arabia as a criterion for learning outcomes and effective tool for measuring educational outcomes, has become a controversial issue. The decentralization resulted in the creation of another assessment mechanism (Qiyas), which led to more complications and dissatisfaction among students and parents as it ignores 12 years of student's achievements in high school. This research is an attempt to explore the opinions and attitudes of students about the effects of decentralization of the secondary school examinations and its relationship with their recent and previous learning achievements.

From Mapping Information Ecologies to Evaluating Media Interventions: An Expert Survey on Evaluating Media Interventions in Conflict Countries

by Iginio Gagliardone

The paper outlines the results of a survey of opinions about the role and practice of monitoring and evaluation of... more The paper outlines the results of a survey of opinions about the role and practice of monitoring and evaluation of media interventions and media assistance in conflict countries. It addresses a paradox that has characterized these types of interventions in recent years: the growing perception that media and communications serve as critical drivers of change in conflict environments, accompanied by an uncertainty about the efficacy of individual activities. This survey was initiated out of the belief that the first step towards better assessment is soliciting the opinions of a range of stakeholders involved in these programs: the donors that fund the projects, the implementers working in the field, and the methodologists often involved in their assessment.

How the Light Gets In - the value of imperfect systems of evaluation

by François Matarasso

Chapter in a book on cultural policy edited by Binnaz Ayata and published in Sweden by Studentlitteratur

This paper looks at some of the problems arising from the focus on targets and evaluation in British cultural policy.... more This paper looks at some of the problems arising from the focus on targets and evaluation in British cultural policy. It concludes by proposing some principles that might underlie a workable approach to evaluating cultural programmes.

Resilience in Practice

by Susan Upton

Co-authored with Maggie Ibrahim

Practical Action has produced a briefing paper called ‘Resilience in Practice’, which explains just how resilience is... more Practical Action has produced a briefing paper called ‘Resilience in Practice’, which explains just how resilience is being built into projects. The paper is made up of six case studies of projects in Peru, Nepal, Bangladesh, Sudan and Kenya that illustrate Practical Action’s work in building resilience into its programmes. They provide an evidence base for how Practical Action is turning resilience into practice and for including processes and resources that are essential for supporting learning, adaptation and experimentation.

The case studies reinforce the need for organisations to be proactive in reaching out to build partnerships and alliances with people and organisations operating outside of their specialist intervention areas. This means they also require an investment and challenge traditional way of working.

The briefing paper highlights that there are still a range of challenges that need to be overcome and that there are areas that need further research. The challenges include a lack of relevant climate data, appropriate tools and incentives in organisations to support integration of sectors, a lack of scenario planning methods and clear indicators of resilience on which to base planning, monitoring and evaluations.

Capturing Mobile Devices Interactions Minimizing the External Influence

by Ivan Pretel

The exponential evolution that Information and Communication Technologies (ICT) have suffered the last years has been... more The exponential evolution that Information and Communication Technologies (ICT) have suffered the last years has been one of the reasons why all type of sectors as industrial or educational have improved their productivity. One of the disadvantages of this fast evolution is that not all users have been used to this new technologies, basically because the way of doing a task has drastically changed. This problem is known as “increase of the technological or digital gap”. In order to reduce this technological gap we have designed, implemented and evaluated a new Virtual Learning Environment (VLE) based on digital pens. This solution consists of the integration of typical devices with a Learning Management System (LMS), concretely, Google Apps. A digital pen looks and feels like a normal ballpoint pen but thanks to it the handwriting can be captured, stored and sent by a safe way. In this case these data are sent to the LMS where students can manage and share notes by this platform and complete an existing document in the LMS with the digitalized information. The platform also offers a way to evaluate the students. Students can do several tests created by teachers previously. To carry out this activity, students will use a digital pen. Thus, the digitized information with the answers of the students will send to the platform where the tests are corrected immediately.

Forest At Work: Conservation And Sustainable Management Of The Former Finch Pruyn Lands

by Rachel Neugarten

M.S. Thesis, published by Cornell University 2010 (no access until 2015)

Major selloffs of industrial timberlands in the U.S. in the past two decades have prompted environmental concerns... more Major selloffs of industrial timberlands in the U.S. in the past two decades have prompted environmental concerns about fragmentation and conversion of forest lands, as well as social and economic concerns about the loss of traditional livelihoods in forestry and rural community decline. In an effort to maintain intact forests and the many ecological and socioeconomic values they provide, conservation organizations, public agencies, and local communities are investing in complex "working forest" land deals in which land and property rights are divided among multiple actors. These transactions represent large, expensive, and relatively untested experiments in integrating conservation, sustainable forest management, and economic development. As such, there is a need for critical assessment in order to evaluate outcomes, manage adaptively, and inform the design of future transactions. We reviewed existing definitions of sustainable forest management, as well as case studies of working forests, to evaluate how ecological and socioeconomic indicators are incorporated in forest management and policy. We also undertook an in-depth case study of a working forest transaction involving the former Finch Pruyn lands in New York State, to explore how this particular arrangement integrates international, regional, and local sustainability goals. We found that our current ability to learn from past experience with working forests is severely limited by a lack of integrated, iterative monitoring data. Monitoring programs tend to be short-term and stymied by small budgets, high staff turnover, and the complexity of the underlying socio-ecological systems. We also found that management objectives for the Finch Pruyn working forest reflect ecological criteria that are consistent with international standards for sustainable forest management, as well as many of the goals described by regional and local actors, such as providing new public recreation opportunities and maintaining some level of forest- related employment. Specific goals related to supporting local economic development were less well reflected in management objectives. Such goals, however, are arguably beyond the scope of a single land deal. In general, we found that working forest transactions have the potential to achieve multiple conservation and sustainability goals, as well as helping to reconcile long-held disputes over forest land management. However, our current ability to assess outcomes is limited by the relatively recent emergence of this strategy and a lack of empirical evidence, particularly related to socioeconomic outcomes. The lack of evidence could exacerbate existing disputes about the relationship between forest land use, ecological integrity, and socioeconomic well-being. Given the inherent complexity of the issues surrounding working forests, we recommend integrated ecological and socioeconomic monitoring to support adaptive management and to build on existing networks between conservation groups, environmental agencies, forest landowners, and local communities. By bringing critical attention to these large, complex experiments in forest conservation and sustainable management, we hope to inform efforts to simultaneously protect ecological integrity and meet the needs of current and future generations.

Integrating Ecological and Socioeconomic Monitoring of Working Forests

by Rachel Neugarten

Neugarten, R. A., S. A. Wolf, R. C. Stedman, and T. H. Tear. 2011. Integrating Ecological and Socioeconomic Monitoring of Working Forests. BioScience 61:631–637.

Large-scale sell-offs of industrial timberlands in the United States have prompted public and private investments in a... more Large-scale sell-offs of industrial timberlands in the United States have prompted public and private investments in a new class of “working forest” land deals, notable for their large size and complex divisions of property rights. These transactions have been pitched as “win-win-win” deals that provide social, economic, and ecological benefits. Despite hundreds of millions of dollars invested in these transactions, we found a paucity of evidence that their supposed benefits are being realized. Monitoring programs necessary to gather such evidence tend to be underfunded, short term, and focused on a limited set of indicators. The few projects with more comprehensive monitoring programs had long-term funding sources, formal mechanisms for incorporating data into subsequent management decisions, and combined multidisciplinary monitoring techniques. We propose that a relatively modest allocation of funds to monitoring could help assess—and hopefully improve—the effectiveness of current and future transactions, to see if the promise of “win-win-win” is actually delivered.

Evaluating the success of conservation actions in safeguarding tropical forest biodiversity

by Thomas Brooks

Brooks, T.M., Wright, S.J. & Sheil, D. (2009) Evaluating the success of conservation actions in safeguarding tropical forest biodiversity. Conservation Biology 23: 1448–1457.
DOI: 10.1111/j.1523-1739.2009.01334.x.

We reviewed the evidence on the extent and efficacy of conservation of tropical forest biodiversity for each of the... more We reviewed the evidence on the extent and efficacy of conservation of tropical forest biodiversity for each of the classes of conservation action defined by the new International Union for Conservation of Nature (IUCN) classification. Protected areas are the most tested conservation approach, and a number of studies show they are generally effective in slowing deforestation. There is some documentation of the extent of sustainable timber management in tropical forest, but little information on other landscape-conservation tactics. The extent and effectiveness of ex situ species conservation is quite well known. Forty-one tropical-forest species now survive only in captivity. Other single-species conservation actions are not as well documented. The potential of policy mechanisms, such as international conventions and provision of funds, to slow extinctions in tropical forests is considerable, but the effects of policy are difficult to measure. Finally, interventions to promote tropical conservation by supporting education and livelihoods, providing incentives, and furthering capacity building are all thought to be important, but their extent and effectiveness remain poorly known. For birds, the best studied taxon, the sum of such conservation actions has averted one-fifth of the extinctions that would otherwise have occurred over the last century. Clearly, tropical forest conservation works, but more is
needed, as is critical assessment of what works in what circumstances, if mass extinction is to be averted.

Report on the Social Audit Pilot 1998/99

by Garth Nowland-Foreman

COMMACT Aotearoa, Auckland (2000)

Evaluación del impacto ambiental de un proyecto de desarrollo integral de la pesca artesanal

by Carlo Tassara

Carlo Tassara, Stefano Marani, Hernando Sánchez Moreno y Alfonso Escobar Nieves
Publicado en el libro: Carlo Tassara (Editor). 1995 (Septiembre). Pesca artesanal, acuicultura y ambiente. Experiencias y perspectivas de desarrollo (pp. 457-484)
ECOE. Bogotá.
Septiembre 1995.

El texto concierne la evaluación de impacto ambiental de un proyecto de desarrollo de la pesca artesanal en Colombia.... more El texto concierne la evaluación de impacto ambiental de un proyecto de desarrollo de la pesca artesanal en Colombia. El ensayo arranca con la revisión de las características socioeconómicas y ecológicas de la región y el análisis de las actividades del proyecto y de sus impactos potenciales. En base a una atenta ponderación de los mismos y de sus posible mitigaciones, las conclusiones del ensayo descartan el relevante aumento de la producción pesquera y apuntan hacia la promoción del valor agregado vinculado con actividades como la comercialización, el procesamiento de los productos ícticos, los servicios a la pesca (reparación de redes y embarcaciones, compra compartida de las artes de pesca, etc.) y el turismo a pequeña escala.

La valutazione dei programmi di pesca artigianale

by Carlo Tassara

Carlo Tassara
Pubblicato nella rivista:
“Forum Valutazione” N° 2 (pp. 141-146). Ottobre 1991.
Franco Angeli. Milano.
Ottobre 1991

Lo scopo della scheda è quello di identificare alcune questioni relative al monitoraggio e alla valutazione dei... more Lo scopo della scheda è quello di identificare alcune questioni relative al monitoraggio e alla valutazione dei programmi di pesca artigianale, nella consapevolezza che, in alcuni casi, si tratta di problemi comuni anche ad altri settori.

Il monitoraggio dei progetti di sviluppo della pesca artigianale: l’esperienza di un progetto in Colombia

by Carlo Tassara

Carlo Tassara e Francesca Declich
Pubblicato nella rivista:
“Forum Valutazione” N° 3 (pp. 73-89). Maggio 1992.
Franco Angeli. Milano.
Maggio 1992

Il saggio delinea un quadro sintetico ma esaustivo dello stato dell'arte nel settore della pesca artigianale in... more Il saggio delinea un quadro sintetico ma esaustivo dello stato dell'arte nel settore della pesca artigianale in America Latina (en in particolare in Colombia), proponendo una sintesi della conoscenza progettuale e valutativa accumulata nella cooperazione allo sviluppo a livello settoriale e regionale.

Lo sviluppo integrale della pesca artigianale e il suo monitoraggio: un’esperienza di cooperazione in Colombia

by Carlo Tassara

Carlo Tassara e Francesca Declich
Pubblicato nella rivista:
“Forum Valutazione” N° 4 (pp. 107-130). Novembre 1992.
Franco Angeli. Milano.
Novembre 1992

Il saggio evidenzia gli aspetti teorici e operativi del monitoraggio dei progetti di sviluppo come processo di... more Il saggio evidenzia gli aspetti teorici e operativi del monitoraggio dei progetti di sviluppo come processo di apprendimento istituzionale.
Nello studio di caso presentato, riferito alla pesca artigianale in Colombia, gli autori mostrano come sia possibile ottenere una partecipazione sostanziale dello staff tecnico e dei beneficiari non solo alla raccolta delle informazioni, ma anche alla formulazione e ri-formulazione degli strumenti di rilevazione sulla base dell'esperienza acquisita e dei dati raccolti.

Ambiente e sviluppo. Attività e proposte delle ONG italiane su PEC e VIA

by Carlo Tassara

Carlo Tassara e Piergiorgio Menchini
Pubblicato nella rivista:
“Forum Valutazione” N° 4 (pp. 23-46). Novembre 1992.
Franco Angeli. Milano.
Novembre 1992

Dopo una introduzione che ripercorre l'itinerario di analisi, riflessione e azione delle ONG sull'intreccio tra... more Dopo una introduzione che ripercorre l'itinerario di analisi, riflessione e azione delle ONG sull'intreccio tra questioni ambientali e processi di sviluppo, il testo illustra alcune iniziative per l'elaborazione e l'adozione di una metodologia di Valutazione di Impatto Ambientale (VIA) da applicarsi nell'ambito della cooperazione allo sviluppo.

2007 Review of Kazakhstan UN Development Assistance Framework

by Johannes Chudoba

see also ppt under "talks"

Executive Summary
Kazakhstan has made impressive progress in recent years, and the UN Country Team has been... more Executive Summary
Kazakhstan has made impressive progress in recent years, and the UN Country Team has been working hard to support the Government and the population over the past three years. As significant challenges remain, Kazakhstan can benefit substantially from a further close engagement with the UN on the priorities defined in the UN Development Assistance Framework 2005-2009. The UN Country Team can boost the impact of
its work by more effectively using the UNDAF as a guiding structure for internal and external coordination, with clearly defined joint goals, indicators and mechanisms.
President Nazarbayev has summed up the achievements over the past decade as Kazakhstan no longer being “a third world country”; for the coming decade, he has defined a focus on “improving the quality of life,” which is at the core of UN programming. The first of the UN’s priority areas, human poverty, goes well beyond income poverty, as it arises from and reinforces the lack of access to decision-making, resources, services and opportunities. A substantial proportion of the population (40% or more) with per capita incomes only slightly above the subsistence minimum remains vulnerable. An issue that is easily overlooked is the serious disparity between different parts of the country. A particularly serious problem in Kazakhstan is also
the severe damage inflicted on the environment under Soviet rule.
The second priority area is basic social services. In the health sector, mortality statistics indicate that Kazakhstan still has a long way to go to attain international standards. The areas of reproductive health, HIV/AIDS, TB and maternal and child health are of particular concern to the UN. Progress on health indicators has been slow, so that the MDGs of reducing child and maternal mortality and combating major diseases (TB and HIV/AIDS) are unlikely to be met. Part of the health problems results from low access to safe water and sanitation, and from environmental pollution. In education, too, challenges remain, especially in terms of quality and uniform standards. Though Kazakhstan has formally achieved the MDGs of universal primary education and gender equality in primary and secondary education, a lack of schools and teachers, particularly in remote rural areas, and the quality of education require continued attention.
For Kazakhstan to be able to join the world’s 50 most competitive countries, and to ensure a better quality of life for all its citizens, key reforms and improvements in governance, the UN’s third priority area, will need to be more fully implemented. Among issues of concern are effective oversight of the executive branch and the protection and promotion of human rights, as outlined in the recently published Human Rights Baseline report.
This review confirms the relevance of the UN’s support to Kazakhstan and documents the contributions it has made to date. It also confirms that the UN has valuable contributions to make to Kazakhstan’s further development. It shows, however, that the current UNDAF remained a document rather than becoming a living process and consequently did not serve as effectively as it could have to strengthen strategic guidance,
coherence and coordination among UN agencies and their partners, and facilitate communication.
For the current UNDAF to really make a difference, it needs to be operationalised by following through on RC leadership, backed by concrete commitments from UNCT agencies, based on well defined indicators, facilitated by light, results-driven coordination, and presented through audience-specific communication (clear, accessible, jargon-free, and relevant).
For the next UNDAF process, the UNCT should seize the opportunity to freshly assess current and emerging needs, define matching UN strengths, identify greatest synergy and leverage potentials, and clarify specific roles, responsibilities, relationships and expected results – in a transparent, inclusive, non-formalistic
process.

Will the lessons be learned?: Reflections on local authority evaluations and the use of research evidence

by Jill Clark

This paper was published in the Journal: Evidence and Policy (2008) volume 4, Issue 2, Pages 255-268 and is co-authored by Jill Clark and Elaine Hall

Sure Start programmes are complex, community-based initiatives – fore-runners of the Children’s Centres Initiative -... more Sure Start programmes are complex, community-based initiatives – fore-runners of the Children’s Centres Initiative - which have been evaluated nationally and locally. Using an in-depth, retrospective case study of an evaluation of one local programme, the authors raise key issues pertinent to both practice and evaluation in the field, highlighting conflicts and dilemmas both within evaluation generally and, specifically relating to the evaluation of this programme. We illustrate the difficulties placed upon local evaluators by the lack of clear structures within which to work, and provide useful lessons as we move forward into the development and evaluations of new services for children and families.

A local evaluation of Sure Start – Leam Lane area

by Jill Clark

This report was co-authored by Clark J, Hall E, McCaughey C, Mroz M in 2006

This report brings together elements of work from a three year evaluation study conducted by staff in the Centre for... more This report brings together elements of work from a three year evaluation study conducted by staff in the Centre for Learning and Teaching, University of Newcastle upon Tyne. As part of the three-year evaluation study, we have provided several in-house reports which specifically focus on particular phases, or projects within the programme. These formative reports contained both findings and recommendations about all aspects of the projects, ranging from the suitability of the venue to the impact on the children, parents and staff involved. These reports were intended to be tools for programme staff to refer to, reflect on, and improve services. This document, however, intends to integrate and summarise the individual reports and provide a synthesis of the evidence and findings.

Exploring multi-professionalism and the changing roles of health professionals within Sure Start

by Jill Clark

Co-authored report with Elaine Hall

This document reports on one of these particular case studies - the exploration of multi-professional working within... more This document reports on one of these particular case studies - the exploration of multi-professional working within Sure Start, in particular the experience of health professionals. Background information is presented, with supporting references and research work, which underpins the context of multi-professional working, and the difficulties faced by professionals working in environments such as Sure Start programmes. In consultation with the programme co-ordinators, several key research questions were identified and the research process took place in three phases: reflection exercises, network diagrams and individual interviews. Data collected is presented, alongside analysis, results and discussion of the main themes which emerged. Conclusions include: • The ‘multi-professional’ knows who she is and: • Connection with key areas of professional practice and core beliefs are maintained • Existing knowledge and networks are used • Experiment and risk-taking are embraced • Combining new work, liaison and support is the main challenge • Moving innovative practice back in to mainstream is a priority. External barriers to multi-professional working are: • Lack of understanding about Sure Start • Uncertainty or cynicism about ‘mainstreaming’ • Caseloads of professional colleagues • Lack of overt and continuous structural support from management in the various ‘domains’.

Ghosts at the Feast: The role of research centres in supporting innovative practice in local authorities

by Jill Clark

Co-authored with Elaine Hall in the Journal: Studies in research: Evaluation, Impact and Training (2007) volume 2, p. 1-9.

This paper reports on the tensions of evaluation for project workers, managers and researchers. In the UK, a great... more This paper reports on the tensions of evaluation for project workers, managers and researchers. In the UK, a great deal of the innovative practice dedicated to improving the life chances of children and their families is developed by Local Authorities under the umbrella of project funding which includes an independent evaluation: apparently an ideal opportunity for researchers and practitioners to work collaboratively but, in our experience, there are key structural problems. We explore why it is that researchers fail to give formative feedback to local authorities and why they can feel that they are unwanted guests – ‘ghosts at the feast’.