Evaluations Sociological Studies

VISTAA has partnered with a number of organizations providing them professional monitoring, evaluation and other sociological services to assess the effectiveness, efficiency and equity of the projects they funded or implemented. Our clients are USAID, EU, World Bank, UNICEF, State Agencies and other organizations. The number and range of VISTAA’s M&E/Sociological Services are described below.

 

Qualitative Assessment of Services and Accountability of WUAs in Armenia, World Bank development” project, funded by UNICEF
Location Client/Donor Start End
Armenia The World Bank Group April 2017 June 2017

Description of Project:


The qualitative research aims to examine challenges to accountability and effectiveness of WUAs, as experienced and perceived by water users, compare these views to perceptions of water service providers (WUA management representatives) on the same set of issues, and facilitate the proposal of actions/ solutions at the WUA level (or beyond, if applicable) to improve effectiveness and satisfaction with irrigation water supply. The research will incorporate qualitative research methodology: focus group discussions (FGD), key informant interviews (KII), and a pilot community scorecard for water users and representatives of WUA administration.


In addition to understanding water users’ and service providers’ perspectives, this research aims to test and propose a regular mechanism for engagement with water users to incorporate their feedback, engage them in decision-making processes, and  provide opportunity for a continuous improvement of the quality, accountability,  and satisfaction with water services.
Description of services provided by VISTAA staff within the assignment:


The following is the description of activities to be implemented by the Consultant:

 

  • To review and finalize the methodology, samples, draft interview and discussion guides, and scorecard, discuss and agree finalized research instruments with World Bank team.
  •  to develop a methodology for outreach to relevant authorities to obtain their concurrence prior to beginning of research, a strategy for recruitment of respondents, and a detailed work plan with timeline for the research
  • To conduct training for moderators and pilot-test the methodology. At least 2 pilot FGDs and KIIs (or at least one per moderator) will be included in the pilot.
  • To conduct fieldwork in all sample communities according to the agreed work plan.
  • To submit a detailed outline of the report for comments and approval by the World Bank team

 

Final Evaluation of the EU-funded “Mitigating social consequences of labour migration and maximising migrants’ involvement in local development” project, funded by UNICEF
Location Client/Donor Start End
Armenia UNICEF August 2016 November 2016

Description of Project:

 

The purpose of this final evaluation is to assess the relevance, effectiveness, efficiency, impact and sustainability of interventions under the EU-funded “Mitigating social consequences of labour migration and maximizing migrants’ involvement in local development” project with a focus on social services responding to the needs of socially vulnerable and migrants’ families in Armenia.

 

The specific objectives of the evaluation aim to assess: a) whether the project has been successful in achieving its key outputs and outcomes established in the project action plan and logical framework, primarily related to their coherence and suitability in addressing the “social costs”; b) how far the project has been able to address the issues of sustainability of the interventions, including changes in policy and practice; c) project achievements for migrants’ families within context of overall reform of integrated Social Services and UNICEF’s work in this regard, as well as migration and development nexus.

 

Description of services provided by VISTAA staff within the assignment:

 

  1. Conducting the Evaluation of the project
  2. Design phase: Desk research, Development of Methodology and implementation plan of the evaluation
  3. Field phase (qualitative and quantitative): Data collection and analysis
  4. Reporting phase: Preparation of the final report and recommendations
  5. Final presentation: presenting the findings and recommendations to the Client and other stakeholders
Evaluation of the Family Support Services and Stakeholders Contribution to Related Services/Systems, UNICEF
Location Client/Donor Start End
Armenia UNICEF December 2014 November 2015

Description of Project: The purpose of this formative evaluation is to: 

 

  • assess the effectiveness of the Services/CP Systems Targeted by the Evaluation in responding to social and child protection needs of vulnerable families with children[5]
  • assess UNICEF’s support to the Services/CP Systems Targeted by the Evaluation since 2010.
  • The specific objectives of the evaluation are to assess the relevance, efficiency, effectiveness and sustainability of: integrated social centres, including territorial offices of social services;
  • family, women and child protection units (CPU), guardianship and trusteeship commissions (GTC);
  • cooperation between Services/CP Systems Targeted by the Evaluation and education, health facilities as well as police;

 

linkages and coherence between integrated social services, GTCs and CPUs in responding to the needs of vulnerable families with children.

 

Description of actual services provided by VISTAA staff within the assignment:

 

  1. Evaluation of UNCIEF funded Child Protection sector services
  2. Design phase: Desk research, Development of Methodology and implementation plan of the evaluation
  3. Field phase (qualitative and quantitative): Data collection and analysis
  4. Reporting phase: Preparation of the final report and recommendations
  5. Final presentation: presenting the findings and recommendations to the Client and other stakeholders
Mid-Term Programmatic Evaluation of the USDA Caucasus Agriculture Development Initiative in Armenia, USAID
Location Client/Donor Start End
Armenia USAID August, 2012 October, 2012

Narrative Description of the Project:
 
The U.S. Department of Agriculture Foreign Agricultural Service commissioned this study to evaluate its activities under the Caucasus Agricultural Development Initiative in Armenia (CADI) from FY 2011 to the present.  CADI projects are focused on six program areas supporting agriculture and agribusiness development in Armenia. This evaluation seeks to determine and describe the effectiveness of the CADI by defining the approaches used for each project, providing output and outcome results at the mid-term and analyzing those results to determine prospects for sustainability.  The evaluation also provides a description of the overall U.S. government assistance to the agriculture sector in Armenia, with an emphasis on defining USDA’s role in supporting this assistance framework.


 
Description of services provided by VISTAA staff within the assignment:
 
Advisors were expected to assist the program evaluation, visiting sites, conducting interviews with recipients and contribute written material to the final evaluation report. In addition the VISTAA M&E Specialist assisted the QED Senior Evaluator with the presentation of findings at the U.S. Embassy on September 7, 2012.

“Initiative on Soaring Food Prices (ISFP), Technical Cooperation Program (TCP), “Beneficiary satisfaction and impact assessment survey”
Location Client/Donor Start End
Armenia European Commission Delegation February, 2010 April, 2010

Narrative Description of the Project:

 

  • Assess the level of satisfaction of beneficiaries with the FAO ISFP projects and the impacts on their lives
  • Collect the views of project beneficiaries, implementing agencies and input suppliers on the major constraints or problems they faced with the projects and suggestions for improvements
  • Contribute to lessons learning process with the purpose of optimizing future emergency responses

 

Description of Services Provided by VISTAA Staff:

 

  • Conducted survey to assess the level of satisfaction of beneficiaries with the FAO ISFP projects and the impacts on their lives
  • Collected viewpoints of project beneficiaries, implementing agencies and input suppliers on the major constraints or problems they faced with the projects and suggestions for improvements
  • Prepared the Report with recommendation contributing to lessons learning process with the purpose of optimizing future emergency responses.
“Situation Analysis of Children in Armenia”, ECD. “LOT N° 7 Culture, Governance and Home Affairs”
Location Client/Donor Start End
Armenia European Commission Delegation November, 2009 October, 2010

Narrative Description of the Project:

 

The global objective of the assignment is to conduct a comprehensive and in-depth Situation Analysis of Children in Armenia including major issues and factors affecting the situation of children and the realization of their rights.

 

Specific objectives: 
 

  • Present comprehensive gender and geographic disaggregated data and trends on the status of indicators for children and measuring progress against agreed international standards, namely Millennium Development Goals, World Fit For Children (WFFC) Goals and Education for All (EFA) Goals
  • Identify and describe patterns of inequality, marginalization, vulnerability and exclusion
  • Formulate recommendations for the realization of children’s rights in the context of existing capacities policy and budgeting frameworks, legislation, capacity building efforts, service delivery systems and other strategies and interventions at different levels of society
  • Highlight knowledge and information gaps among children for further data collection, research and analysis
  • Provide a comparative analysis of the Convention on the Rights of the Child (CRC) and legislation in Armenia, extensively drawing on the CRC implementation guidelines to identify gaps for advocacy;
  • Identify, prioritise and provide recommendations for a set of key actions related to realization of children’s rights for further support by the European Commission

 

Description of Services Provided by VISTAA Staff:

 

The Situation Analysis will be conducted through coordination of the EC Delegation and UNICEF in Armenia and with full participation of the Government and relevant stakeholders in all stages of the process: planning, data collection, data analysis, reporting and dissemination. An Advisory Board will be established in order to ensure regular feedback to stakeholders at all stages of analysis development.

 

Various other stakeholders will be involved in the process of analysis elaboration such as technical experts at Ministry and Marz level, civil society organizations/NGOs, academia, private sector, donors, UN agencies, women’s groups, and young people in various forms of participation including interviews and meetings, workshops for causality analysis using the human rights based approach, and participation in focus group discussions.

 

The Situation Analysis should draw on all available child-related data, information and research and take stock of existing analytical processes and products, including electronic data sets in DevInfo.

 

The analysis will be underpinned by a human rights-based approach taking into account:

 

  • Causality analysis within a Conceptual Framework
  • Role-pattern analysis, examining the duty bearers and rights holders
  • Capacity gap analysis, examining the capacity of all those individuals and institutions responsible for Respecting, protecting and fulfilling children’s rights

 

The assessment of the realization of children’s rights will consider specific denials and violations of children’s rights, and situations that contravene human rights principles such as universality and non-discrimination. It will identify existing problem, place / location of occurrence, affected groups / individuals, frequency and volume of problems and existing or missing mechanisms to address those problems.

“Monitoring Civil Society Participation in Armenia’s Country Coordinating Mechanism (CCM) for the Global Fund”, OSI
Location Client/Donor Start End
Armenia European Commission Delegation January, 2009 July, 2009

Narrative Description of the Project:

 

The purpose of the study is to identify best practices being implemented by the Country Coordinating Mechanism (CCM) in order to ensure robust civil society involvement, document challenges that the CCM faces, and identify recommendations in order to strengthen the CCM.

 

Activities: To conduct research to monitor civil society participation in Armenia’s CCM for the Global Fund (GF) to Fight AIDS, Tuberculosis and Malaria. This CCM monitoring project is designed to evaluate how civil society can have greater impact as an advocate for improved Global Fund programming at the country level, principally through participation CCMs. The project is not focused solely on issues of civil society representation on CCMs, but more broadly on the ability of civil society to have a significant impact on improved Global Fund programming. Issues to be considered include: how civil society members are selected for CCM membership; whether they are able to be meaningfully involved in program design and oversight; to what degree they represent the perspectives of broader civil society and what supports are available to promote more effective civil society engagement in CCMs.

 

Description of Services Provided by VISTAA Staff:

 

Research was done by team of sociologists to review CCM documents relevant to civil society participation, including written procedures for the CCM, Terms of Reference for members, CCM minutes, and more general documents about civil society’s role in the GF structure. The research has addressed the following main questions: Is civil society able to have direct impact on improving the quality and effectiveness of Global Fund-supported programs? What are examples of civil society having this impact through CCMs, and what factors made this possible? What can be done to foster civil society’s role as an advocate for improved Global Fund programming through its involvement in the design, monitoring and evaluation of GF programs?

 

Report has been prepared, with recommendation contributing to a lessons learning process. It will help inform and improve the Global Fund to Fight AIDS, Tuberculosis and Malaria’s policies regarding country coordinating mechanisms.

Consumer Survey in All Sectors of Armenian Economy, including, Residential (urban and rural), Commercial, Industrial, and Agricultural Sectors
Location Client/Donor Start End
Armenia USAID Energy Efficiency Project November, 2005 March, 2006

Narrative Description of the Project:

 

The objectives of this Project are:

 

To identify the market potential and economic potential for energy efficiency and renewable energy technologies in all sectors of Armenian economy
AEAI will provide a set of strategic recommendations to US Agency for International Development and the Government of Armenia on implementation of energy efficiency measures and renewable energy technologies

 

Description of Services Provided by VISTAA Staff:

 

The following tasks were performed within the project:

 

  • Task 1: Review the questionnaires designed by the AEAI specialists and provide comments and suggestions on those. The contractor should provide detailed information on the techniques of determining the survey respondents according to the provided samples in each sector (enclosed in the Attachment A).
  • Task 2: Meet with AEAI specialists to discuss the survey conducting details and specifics for each sector. The AEAI specialists will instruct the survey conducting personnel on the specific issues related to the energy efficiency and renewable energy, and also the coding system for data entry in the SPSS software.
  • Task 3: Conduct a survey of residential, commercial, municipal and industrial sectors using the questionnaire and survey samples for each sector developed by the AEAI specialists.
  • Task 4: Provide AEAI with a detailed report on the survey process in electronic and print form / both in soft and hard copy.
  • Task 5: Provide AEAI with a database of survey responses inputted into SPSS software in a format suitable for further analysis. The Contractor should discuss the format of the database with AEAI specialists.