Independent evaluation of the profiling exercise implemented in Chihuahua State, Mexico, 2020-23 At Danish Refugee Council

Independent evaluation of the profiling exercise implemented in Chihuahua State, Mexico, 2020-23

1. Who is JIPS?

JIPS is an interagency service established in 2009 to bring governments, communities, humanitarian and development actors together to collaborate towards collective outcomes and durable solutions to displacement situations. It does so by supporting collaborative and responsible data processes including profiling, developing the capacities of governments and key stakeholders, and advancing global discourse towards sound global standards. Field-focused and committed to enhancing local ownership and capacity, JIPS is a globally recognised impartial broker that draws on extensive field experience in diverse displacement contexts and a unique combination of technical, soft and political skills to drive change in the contexts it supports and the global discussions it informs.

1. Purpose of the consultancy

The purpose of the consultancy is to conduct an independent evaluation of the relevance and effectiveness of the profiling exercise that was implemented in Chihuahua State, Mexico, between 2020 and 2023 with support from JIPS and led by a Technical Working Group (TWG) comprising 21 organisations from civil society, federal and state government, and the international community.[1]

By assessing the performance and intermediate outcomes directly attributable to the profiling process, activities, and outputs, the evaluation will provide valuable insights and lessons learned for JIPS and the TWG on the ways in which the exercise was able to contribute to transformative change at the state and federal levels.

The evaluation results will further yield recommendations to enhance the process and outcomes of such collaborative data collection exercises. This will help JIPS enhance its profiling approach and support services, while also providing timely learning for wider efforts towards more coordinated and joined-up efforts for high-quality data and concerted responses to internal displacement.

1. Background

Internal displacement – or forced internal displacement (FID) – has been documented in Mexico since at least the 1970s. However, it was not until April 2019 that the Government of Mexico officially recognized its occurrence. While efforts have been made to collect and systematize information on the phenomenon, for example through statistical approaches and case studies, there are still information gaps and opportunities to deepen the understanding of internal displacement in Mexico through tailored methodologies. Therefore, an inter-institutional Working Group (WG) carried out this “Ejercicio de Caracterización del Desplazamiento Interno en Chihuahua” (profiling exercise of internal displacement in Chihuahua State, hereinafter the “Exercise”), which is a collaborative effort that seeks to understand the FID caused by violence in that state; being the first effort of this nature in the country.

The Exercise consisted of:

1. A review of available secondary information;
2. A household survey;
3. Interviews with key informants to deepen understanding of the differential impact of displacement on the indigenous population;
4. Focus group discussions to validate and deepen the survey findings;
5. A collaborative process of analysis and interpretation of the results of the survey;
6. A regulatory, programmatic and public policy mapping that sought to identify opportunities to improve the response to displacement based on the results of the exercise.

Although the results of the survey are not statistically representative of all persons internally displaced in the state due to the non-probabilistic sample used[2], the exercise contributes to a better understanding of the characteristics of the population who participated in the interviews and focus groups, including the impacts induced by their displacement, their experiences, and their perspectives.

As the first exercise of its kind in Mexico, it has contributed to building information management capacities on forced internal displacement and represents an important precedent for the study of the phenomenon in the country. The findings of the exercise are presented in a report, two thematic documents and a storymap.

1. Detailed objectives

The independent evaluation will examine the relevance and effectiveness of the collaborative profiling exercise, including the key outputs generated between 2020-2023, and the intermediate outcomes directly attributable to the process, activities, and outputs. The evaluation will surface critical insights, enablers of success, lessons learned, and recommendations on how to enhance the profiling process and amplify the positive outcomes for profiling partners. More specifically, the evaluation will aim to examine the following aspects:

1. Relevance: To what extent is this project an appropriate/relevant approach to supporting the response to internal displacement, including in terms of better evidence and more targeted advocacy? How could it be improved? This should also consider whether the profiling exercise outputs (intermediary outputs as well as comprehensive report, online data story, 2 thematic briefs) were sufficiently tailored to different stakeholders’ needs and made results accessible and actionable.
2. Collaboration: How well was the collaborative approach managed and implemented? What were the short-term outcomes from the collaboration? Are there any longer-term outcomes from the collaboration? How did this approach serve to contribute to improvements in data systems? How did this collaboration strengthen strategic alliances among the Working Group members beyond this Exercise (i.e. response to displacements, incidence, etc.)? How else did it improve the response to internal displacement? The following aspects should be considered:

• The collaborative set-up and nature of the exercise, incl. considerations such as the relevance of the large and diverse number of stakeholders (state, federal, and international), their level of engagement, the effectiveness of inclusion means, the effectiveness of the approach tailored to the context and different stakeholders’ needs;
• The relevance and effectiveness of JIPS’ collaboration/facilitation, technical, and capacity development support.

1. Effectiveness: How effective has the project been at improving the response to internal displacement, including in terms of better evidence and more targeted advocacy? What could be done to improve that? Which target groups were reached, and were any missed? What other contributions did the profiling exercise make on aspects such as:

• Collaboration between actors on IDP data;
• Informing policies, strategies, and/or programmes on internal displacement of state and federal authorities;
• Strengthened advocacy on internal displacement in Chihuahua;
• Broader acknowledgement of the lived experiences of IDPs in the state and beyond.

1. Scope of work and methodology

The independent evaluation will encompass the following components:

• Desk review of relevant documents pertaining to profiling exercise, including its methodology and tools, operationalisation, and outputs.
• Qualitative data collection, such as throughkey informant interviews (KIIs) and focus group discussions (FGDs) with key stakeholders involved in the process, including the 21 members of the Technical Working Group incl. UNHCR Mexico, JIPS, CMDPDH, local and national government officials, as well as displacement-affected communities.
• 1 multi-stakeholder validation workshop to which at least all members of the TWG and JIPS will be invited to share back and validate the findings from the evaluation and to offer a forum to discuss and refine the key recommendations.

The consultant will work in close collaboration with JIPS and UNHCR Mexico, who will provide an initial contact list and relevant documents for the desk review; review the methodology approach; support the organisation and implementation of the validation workshop with the TWG; review the final report and evaluation brief; and support in any other ways as agreed. The design and dissemination of the final evaluation brief will be covered by JIPS and is not required from the consultant.

Relevant documents to be included in the evaluation:

• Materials from the scoping mission in 2019 and subsequent missions, incl. terms of reference (TORs), mission presentations, meeting notes and reports;
• Profiling exercise concept note, process and methodology documents;
• Technical Working Group TORs, meeting minutes, and presentations;
• Training materials;
• Related donor reporting;
• Key presentations
• Data collection and analysis tools;
• Profiling report;
• Online data story;
• 2 thematic briefs.

Foreseen limitations:

• Changes of staff in profiling partner organisations
• Difficulty to establish clear causality between efforts in gathering information, in this case the profiling exercise implemented in Chihuahua State, with changes in policy and programming.

1. Deliverables

The applicant is responsible for the satisfactory delivery of all below outputs:

• An inception report, including a description of the methodology approach for the evaluation (incl. interview guides and planned KIIs and FGDs)
• An in-person or hybrid validation workshop in Chihuahua or Mexico city with the TWG
• A final, comprehensive evaluation report (internal, in Spanish)
• A final evaluation brief (external, in Spanish and English, around 5 pages)
• A presentation at a global online event to present the findings to a larger audience beyond Mexico

1. Duration, timeline, payment

The total expected duration to complete the deliverables is 5 months, between April and August 2024. Please see Annex I for the intended detailed timeline.

The consultant will receive 20% of the total fee at the signature of the contract, 20% on the completion of the inception report, and the remaining 60% on satisfactory delivery of services at the completion of all deliverables. The consultants’ fee must include all taxes and other changes, including any VAT costs. Any specialized technical equipment needed to fulfil services should be included in the rate.

The financial proposal (bid form) must be in CHF.

1. Eligibility, qualification, and experience required

Eligibility: The consultant has the authorisation to work remotely. The Consultant must be available upon request during the timeframe of the contract between April and August 2024.

Qualifications, experience, and skills

• A degree (Masters or equivalent) in political science, social science, or any other related field;
• 5-7 years of prior experience conducting evaluations in the humanitarian and/or development field, including research project performance and outcome evaluations, evaluating activities and programs linked to forced displacement;
• Previous experience working with multiple stakeholders in evaluations, including communities, government officials, UN, NGOs and civil society;
• Working experience and expertise in the Mexican context.
• Proficiency in Spanish and English;

Desirable

• Previous experience working on large scale data collection exercises;

Required documents:

• CV
• A portfolio with relevant examples of previous work to showcase the consultant’s work and capabilities against this TOR

1. Technical supervision

The selected consultant will work with JIPS’ key focal points for this evaluation project: its Deputy Head, Corina Demottaz, corina.demottaz@jips.org, and its Senior Profiling Advisor, Pedro Mendes, pedro.mendes@jips.org. The consultant will further work with UNHCR Mexico focal points: Carolina Reyes, reyesros@unhcr.org. Regular meetings will be scheduled to support and guide the consultant.

1. Location and support

Remote and on-site work. The Consultant will provide their own computer and mobile telephone.

1. Travel

Field missions for the qualitative data collection and the multi-stakeholder validation workshop. Travel including visa and accommodation, are to be covered by the consultant.

1. Submission process

Please refer to the RFQ document.

1. Evaluation of bids

Please refer to the RFQ letter invitation.

Only those shortlisted will be contacted for an interview with the panel to ensure their understanding of the consultancy services.

JIPS reserves the right to negotiate based on the availability of the budget allocated to this consultancy.

1. ANNEX I: EVALUATION TIMELINE & DELIVERABLES

Activity

March

April

May

June

July

Deliverables

W1

W2

W3

W4

W1

W2

W3

W4

W1

W2

W3

W4

W1

W2

W3

W4

W1

W2

W3

W4

Onboarding of consultant + inception of evaluation needs & objectives

x

Inception report

Methodology design, finalization of workplan

x

x

Remote interviews

x

x

x

x

x

x

x

x

Analysis & report writing

x

x

x

x

x

x

1st draft comprehensive evaluation report for review

Revision of draft report

x

Final comprehensive report & presentation of results.

x

x

Advocacy brief

x

Draft evaluation brief for review

[1] The Technical Working Group included: Oficina del Alto Comisionado de las Naciones Unidas para los Refugiados (ACNUR), Alianza Sierra Madre, A.C. (ASMAC), Comisión para el Diálogo con los Pueblos Indígenas de México (CDPIM), Comisión Ejecutiva de Atención a Víctimas (CEAV), Comisión Ejecutiva de Atención a Víctimas del Estado de Chihuahua (CEAVE), Centro de Derechos Humanos de las Mujeres (CEDEHM), Comisión Mexicana de Defensa y Promoción de los Derechos Humanos (CMDPDH), Consejo Estatal de Población de Chihuahua (COESPO), Comisión Estatal de Vivienda, Suelo e Infraestructura del Estado de Chihuahua (COESVI), El Colegio de la Frontera Norte (COLEF), Comisión Mexicana de Ayuda a Refugiados (COMAR), Consejo Nacional de Población (CONAPO), Consultoría Técnica Comunitaria, A.C. (CONTEC), Estatal Desarrollo Integral de la Familia del Estado de Chihuahua (DIF), Foro para el Desarrollo Sustentable, A.C. (FORO), Instituto de Geografía para la Paz, A.C. (IGP-Geopa), Instituto Nacional de Estadística y Geografía (INEGI), Servicio Conjunto de Caracterización del Desplazamiento Interno (JIPS), Secretaría de Gobernación (SEGOB), Secretaría General de Gobierno del Estado de Chihuahua (SGG), Servicio Jesuita a Refugiados México (JRS), y Unidad de Política Migratoria, Registro e Identidad de Personas (UPMRIP).

[2] Para más detalles ver: https://www.inegi.org.mx/programas/ecadefi/2021/

CLICK TO APPLY