DCA RFP, Consultancy For The Endline Evaluation Of The Project At DanChurchAid

TERMS OF REFERENCE (ToR) FOR CONSULTANCY FOR THE ENDLINE EVALUATION OF THE PROJECT: Saving Lives and Enhancing Resilience and Sustainability of Livelihoods for Economic Recovery of the Poor with Support of Community-Managed Microfinance (CMMF) Scheme Support in Magwi County, Eastern Equatoria State.

Introduction and Context

DanChurchAid (DCA) is a Danish Humanitarian and Development Organization, which primarily works with both local and international NGO partners, and it’s a strong member of international networks/ alliances including churches. DCA is headquartered in Copenhagen and has offices in 19 countries worldwide. It seeks to assist the most marginalized populations through its three global goals – Save Lives, Build Resilient Communities, and Fight Extreme Inequalities with a cross-cutting focus on Gender Equality, Youth Engagement, and Environmental responsibility. DCA works with local communities and is increasingly involved in building the capacity of partners, communities, and local government institutions as well as its staff. DCA’s South Sudan’s office is based in Juba and implements Programmes through its partners in Jonglei, Upper Nile, Eastern Equatoria, and Central Equatoria states.

This Terms of Reference (ToR) for Consultancy describes DCA South Sudan’s plan to evaluate one of its projects titled: Community Managed Microfinance (CMMF) Project in South Sudan. The ToR outlines the project background and specific outcomes, objectives of the evaluation, methodology, and the expected deliverables. Guidance has also been provided for the competitive bidding and sourcing of the External Consultant.

Project Background

This project, “Saving lives and enhancing resilience and sustainability of livelihoods for the economic recovery of the poor with support of community-managed microfinance (CMMF) scheme support” in Magwi County, Eastern Equatoria State, is a three (3) year project. The project was implemented in the payams of Nimule, Pageri and Magwi, Magwi County. The project aimed at building a stable, resilient and sustainable livelihoods of 600 (480f/120m) individuals in 20 groups from the vulnerable communities of Magwi through building their capacity and provision of timely access to loans/credits facilities in community-formed microfinance schemes.

The Project Implementation Area

Magwi County in Eastern Equatoria State is where this project was implemented. The county went through a number of catastrophes in the recent past. The 2013 and 2016 wars for example caused very high displacement of people in Magwi into internally displaced people’s (IDP) camps and into refuge in neighboring countries very much destabilizing the sustainability of livelihoods (agriculture and small-scale businesses) of the rural and urban poor. However, with return of relative peace, the displaced population in IDP camps and from Uganda have been returning but with barely anything in hand to start life with. The women especially, as culture dictates, only meant to take care of children and are far from positively contributing to the family needs as majority do not have any means for generating income. The abrupt closure of the international border with Uganda in March 2020 due to COVID 19 too made the situation even worse for those who were starting to pick up their lives again. The closure of the border meant that the thriving small-scale cross border businesses came to an end and most CMMF groups formed which were a source for accessing capital facilities came to a halt resulting into diminished income. This proposed intervention is therefore planned to reactivate the lost opportunities and reorganize the poor especially the women by coming together to form CMMF groups and start saving to contribute to creating credit facilities which they will pick loans to start businesses, have income and contribute to support their household, improve household status as well as take their children to school.

Project Summary

Project name:

Saving lives and enhancing resilience and sustainability of livelihoods for economic recovery of the poor with support of community managed microfinance (CMMF) scheme support

Project goal:

CMMF group members provided With stable credit facilities to build resilient and sustainable livelihoods for Economy recovery.

Specific objectives:

1. To form and train 15 groups in CMMF philosophy that will have timely access to credit facilities and growth into viable clusters in three years
2. To build the capacity and knowledge of CMMF Members to enable the establishment of resilient and sustainable income generating activities (IGAs)/livelihoods for improved income generation, asset development and equity
3. To create a viable information management system that tracks, records progress and guides group members’ performance

Project results areas:

The project has two specific results. These include.

1. Result 1: Improved CMMF members’ knowledge and capacity in financial literacy and business management leading to sustainable growth in clusters
2. Result 2: Financial capacity and knowledge of CMMF members built and are able to manage resilient and sustainable income generating activities (IGAs)
3. Result 3: CMMF Cluster group model established to enhance provision of stable and increased portfolio of loans/credit facilities for the vulnerable poor to build resilient and sustainable livelihoods.

Target: 600 individuals (20 groups), IDPs, Returnees and host communities.

Locations: Magwi County (Nimule, Pageri and Magwi Payams)

Duration: 01/09/2021 – 31/12/2024

Implementing partner: SPEDP

Evaluation Purpose

The main purpose of this endline evaluation assesses the relevance, coherence, effectiveness, efficiency, sustainability, and potential impact of the project interventions. The evaluation will help to provide practical recommendations and document the lessons learned for adoption in other projects.

Specifically, four key evaluation themes will be covered during the evaluation process: project logic, participation, partnership, and sustainability. The Consultant will also assess cross-cutting themes, including disability inclusion, gender and social inclusion, accountability to the affected population.

The specific objectives of the evaluation will be:

1. To establish whether and to what extent the project design and interventions have been relevant to the needs of the target participants.
2. To assess the extent to which the project has achieved the planned outcomes and outputs.
3. To assess the extent to which the components of VSLA and Micro-finance interventions have been efficient.
4. To establish the indications of the impact of the project interventions on the target participants.
5. To assess if the benefits of the project are likely to continue beyond the project life span.
6. To assess the extent to which the project contributed to reducing vulnerability and reliance on humanitarian assistance.
7. To assess the extent of corrective measures identified and addressed in a timely manner during the implementation process.
8. To document lessons learnt and good practices that can be replicated in future projects.

Scope of the Evaluation

Geographically, the endline evaluation will cover the project location Magwi County (Nimule, Pageri and Magwi Payams) where SPEPD with the support of DCA implemented the project intervention. The evaluation will broadly assess the key achievements of all the expected outcomes as outlined in the project logframe. The evaluation will also assess the different project activities as in the logframe, with a strong emphasis on the following:

• Determining the extent to which the project outcomes and outputs have been achieved and whether there were unexpected outcomes
• Determining the relevance, effectiveness, coherence, efficiency, sustainability, and impact of the project
• Documenting the challenges, lessons learnt and key recommendations for improvement.

The endline evaluation will take place between November and December, with the Final Evaluation Report expected by 15th December 2024.

Users of the evaluation

DCA and the implementing partner will be the primary users of the evaluation findings for learning and decision-making). Moreover, the evaluation findings will provide appropriate measures to be taken and recommendations to improve future programming.

Evaluation Questions and Criteria

The endline evaluation will be guided by the Organization for Economic Co-operation and Development’s/Development Assistance Committee (OECD/DAC) and Active Learning Network for Accountability and Performance (ALNAP) criteria for evaluating humanitarian actions of relevance, effectiveness, efficiency, coherence, impact, and sustainability criteria)[1]. The following are the key evaluation questions:

Relevance

The relevance of the project will be assessed by looking into whether the humanitarian action is tailored to local needs, increasing ownership, accountability, and cost-effectiveness accordingly.

• Are interventions tailored to IDPs and the host community’s needs (men, women, PwDs, and other vulnerable population segments), settings and locations?
• To what extent did the project build local capacities and worked towards improving the resilience of communities and people affected by crisis human-made or disasters?
• Were the project participants involved in the process of developing, implementing and evaluating the project?
• How well was the project aligned with the Humanitarian Response Plan (HRP) of South Sudan?

Coherence

The extent to which the interventions of different actors are harmonized with each other promotes synergy, avoid gaps, duplication, and resource conflicts.

• To what extent has the project engaged in stakeholder coordination, including local authorities and clusters?
• Which factors have restricted coordination, and which factors have supported it? Are there any coordination success factors that can be transferred to other situations?
• How did the various components of the project work together to achieve the overall goals and objectives?
• Are the project activities and outputs logically connected and aligned with the project’s purpose and intended outcomes?
• To what extent did the project adapt and adjust its strategies in response to changing circumstances and new information while maintaining its overall coherence?
• Do the staff understand the approach to the project and how their individual roles contribute to its success?
• How would the communication and collaboration between DCA and SPEDP be rated?
• In what ways did DCA and SPEPD effectively share resources and support each other to achieve the project objectives?
• Has the partnership between SPEDP and DCA resulted in measurable benefits or outcomes for the parties involved? If so, what are the key examples?
• How effectively did DCA and SPEDP address any challenges or conflicts that might have arisen during the partnership? How satisfied are DCA and SPEDP with the level of transparency and trust?
• Have there been any missed opportunities or areas where DCA and SPEDP members could have worked more effectively together?
• What recommendations are proposed for strengthening and enhancing the partnership between DCA and SPEDP in the future?

Partnerships

The assessment and analysis of collaborative partnerships between organisations to achieve common goals

Effectiveness

Evaluation of coverage will involve determining who is supported by humanitarian action, and why.

What were the main reasons that the intervention provided or failed to provide major population groups with, proportionate to their needs?

• To what extent have the intended outcomes (and use of outputs) been achieved by the end of the project?
• Are the IDPs and host community participants being identified and assisted in a timely manner?
• What factors contributed to the achievement or non-achievement of planned project results? What are the main operational bottlenecks, what is causing them and how can they be addressed?
• To what extent are the targeted participants being reached by the project interventions? Do men, women and PwDs have equal access to the project interventions? How successful have DCA and SPEDP been reaching the most vulnerable groups in the most affected geographic area?
• To what extent has the affected population been properly targeted and reached by the project? If not reached, what were the impediments? What strategies can be used to improve targeting?
• Is the project covering an appropriate number of people in need?
• Is the project adequately responding to the changing humanitarian context?

Efficiency

This will assess the project output, both qualitative and quantitative, in relation to the inputs i.e., were inputs, staff, time, funding, equipment used in the best possible way to achieve outputs.

• To what extent were the components of VSLA and Micro Finance interventions efficient?
• Did the project leverage on other projects in the same area?
• How did the costs compare to other projects targeting similar outputs?
• Did the project adhere to established timelines and milestones?
• Were there any unexpected challenges or obstacles that affected the project efficiency?

Sustainability

The project sustainability will be assessed by establishing to what extent the net benefits of the intervention continue or are likely to continue?

• To what extent was the project able to connect short-term interventions to medium-term strategies and longer-term perspectives?
• Exit Strategy: To what extent did the project build the capacity of the local recipients and structures in contributing to the exit strategy? How feasible is the exit strategy considering the context and capacities?
• What is the likelihood of the continuation of positive project outcomes beyond the end of the project (both by primary stakeholders and duty bearers)?
• How does the project seek to safeguard sustainability?
• Are there any factors threatening the sustainability of project outcomes? How does the project seek to mitigate these risks?
• To what extent was the target group becoming more aware and resilient and their aid-dependence is considered less than before?

Impact

The project impact will be assessed by establishing to which extent the intervention has generated or is expected to generate significant positive or negative, intended, or unintended, higher-level effects?

• What evidence is there that the expected outcomes have been realized by the project?
• What have the achievements of the project been in relation to these outcomes and, to what extent have other contextual and operational factors played an influential role.
• What are the unintended positive and negative impacts of the implementation of the project? If it has, what measures have been and can be taken to eliminate or reduce the negative impacts?

Lessons learnt

The Consultant will document challenges, programmatic lessons learnt and key recommendation

• What are the key lessons learned?
• What are the recommendations for improvement of the project or for other similar programs elsewhere in South Sudan?
• What best practices are to be adopted for such projects in future?
• What mistakes should be avoided if the project were to be replicated?
• Is there any identifiable harm caused by the project either for participants or non-beneficiaries?

Evaluation Methodology

The evaluation will employ a participatory approach. The results of the evaluation and lessons learned will be used to improve future projects by DCA and SPEDP. The evaluation will adopt both quantitative and qualitative methods, using both primary and secondary data sources. Interviews will be conducted with the project participants, DCA and SPEDP staff and other key stakeholders. Document reviews will also be undertaken.

A quasi-experimental research methodology will be used to evaluate the impact of the VSLA and Microfinance on the project participants. The treatment group will include members of the VSLA, while the control group will consist of individuals or settings that did not participate in the intervention. This assessment will employ quantitative and qualitative data collection methods using primary and secondary data sources. This will involve interviews with the VSLA groups and partners, and a desk review of the activity implementation and monitoring reports. The logic model of the VSLA will be used to refine the assessment framework and criteria.

Specifically, a survey questionnaire targeting the project participants will be administered to gather data on knowledge and capacity in financial literacy and business management, their ability to build and manage resilient and sustainable income-generating activities (IGAs), and access to loans and credit facilities indicators among others

In terms of qualitative assessment, focus group discussions with the group members will be conducted to gather in-depth insights into their experiences and perceptions regarding the VSLA and Microfinancing model. Key informant interviews will also be conducted with the DCA partners and community leaders to document the key successes, challenges and lessons learned in the implementation process. Moreover, the most significant change stories of the specific households or groups to demonstrate the impact of the VSLA and microfinancing shall be conducted.

A range of existing information will be made available to the Consultant upon notification of the award. These will include Project proposal and logframe, monthly and quarterly reports, financial reports and other relevant documents.

As part of this assignment, the Consultant will apply different methodologies, including primary data collection and a review of existing resources. The primary data collection will include Key Informant Interviews (KII) with implementing partners (IP)- duty-bearers and key stakeholders; Most Significant Change (MSC) guide; beneficiary surveys; and substantiating photos and videos.

The Consultant shall present and discuss the findings, conclusions, and recommendations with DCA and SPEDP, reformulate them if necessary and identify key actors and methods to respond to these recommendations in future programming.

The Consultant will provide a detailed plan of the proposed methodologies in an inception report. The proposed methodology should include an evaluation matrix to reflect evaluation questions, judgement criteria, indicators linked to the judgement criteria, data sources, data collection and analysis plan. DCA and SPEDP shall review the methodology proposed by the Consultant and provide feedback before the evaluation process begins.

While maintaining independence, the evaluation will seek the views of all parties, including the affected population. The emphasis of analysis and learning will be in the targeted areas of the project, result achieved, and process adopted, as well as coordination and collaboration among partners.

Data privacy and protection

The Consultant is expected to sign an understanding to ensure protection of personal data collected during this assignment. The consultant in the (inception report) needs to elaborate on how participant data will be collected and protected; what equipments will be used to store the data, and how long this data will be stored. It is the Consultant’s responsibility to ensure that all staff involved are clear on the evaluation’s aim and purpose, as well as all project details.

Translators, if needed, must be organized by the Consultant and should strictly comply with the above measures. It is the role of the Consultant to ensure that interviewers are trained in confidentiality procedures. Interviewers need to be trained in obtaining verbal consent for interview participation. Staff should have experience in program monitoring, surveying, and data collection and effectively use technology-based analysis software to collect and analyze data. The Consultant will prepare all manuals, guides, and training material used to train data collectors.

All tools will be designed in English and approved in collaboration with DCA and SPEDP. The Consultant will be expected to translate all the tools into relevant languages in case needed. The main language of reports and tools will be English.

Management of the Evaluation Process

The DCA MEAL Manager and Programme Manager will be the main contact persons for the evaluation process. DCA will manage the contract and engagement with the Consultant in accordance with the terms of the contract. The Consultant should make the necessary arrangements and coordinate with DCA and SPEDP before fieldwork to ensure no issues arise during the data collection. Relevant contacts will be shared with the Consultant.

The summary of the roles and responsibilities are outlined below.

Evaluation Commissioner: Programme Manager. Commissions/authorizes the evaluation study, the main user of the evaluation results.

Evaluation Manager: MEAL Manager. Provides overall management of evaluation and technical support if needed. In particular, the MEAL Manager will provide technical support during the end line evaluation process to ensure that the evaluation is of the required quality and standard.

Consultant: Responsible for carrying out the evaluation as agreed upon in the ToR (and the Inception Report).

Logistical support: Head of Prolog. Ensures that the evaluation administration regarding the finances and procurement is compliant with the existing donor/organization’s regulations.

Field Coordination: DCA and SPEDP MEAL Teams: Day-to-day coordination and communication with the Consultant during field data collection

Expected Deliverables

The following deliverables are expected to be produced by the Consultant. All the deliverables must be submitted to DCA in soft copy. These include:

Inception Report (IR):

• The IR should set out the planned design and methodology to meet the above-mentioned objectives and to answer the evaluation questions. Furthermore, the overarching evaluation questions should be specified further in the IR
• The IR should also reflect the limits of the suggested design and methodology and explore the feasibility for answering the evaluation questions and reflect on the ToR, describe the overall approach of the evaluation and how data will be collected by providing an evaluation matrix, data collection tools such as questionnaires and interview guidelines as well as the evaluation schedule.
• The IR should follow a standard outline which will be provided to the Consultant(s) after contracting and needs the approval of the contracting party.

Data collection tools:

• The Consultant shall develop the data collection tools and have them approved by DCA before field data collection. The data collection tools shall be shared and approved together with the IR.

Draft Evaluation Report:

• The Consultant shall prepare the draft evaluation report with details of findings, recommendations and lessons learned for review by the DCA and SPEDP.

Validation of results:

• A virtual meeting or face-to-face to discuss the main evaluation findings, draft report with the DCA and the Implementing partner.

Final Evaluation Report:

• The Consultant will share the final evaluation report after incorporating the comments for DCA and SPEDP.

Evaluation Management Response Matrix:

• The Consultant, as part of the assignment shall elaborate a Management Response Matrix, listing the recommendations and the person responsible for each recommended action. The priority level for each recommendation shall be provided in the Evaluation Management Response Matrix provided by DCA.

Time frame/schedule

The duration of evaluation shall be 45 days, starting from 1st November 2024. The table below presents the tentative schedule prepared to guide the evaluation process.

1. Review of the draft IR by DCA and implementing Partner (7 days)
2. Review of the draft IR by DCA and implementing Partner (5 days)
3. Finalisation and approval of the IR (3 days)
4. Digitalisaten of data Collection Tools and Mobilisation of logistics (2 days)
5. Field data Collection (8 days)
6. Data Analysis and preparation of the draft Evaluation report (10 days)
7. Review of the draft evaluation report (5 days)
8. Preparation of the Final Evaluation Report (5 days)

Confidentiality

All documents and data acquired from documents during interviews and meetings are confidential and used solely for the evaluation. The deliverables and all material linked to the evaluation (produced by the Consultants or the organization itself) are confidential and remain the property of the contracting party.

Expertise of the Consultants

This assignment is open to Consultants with sound experience in the services outlined above.

The Consultant/team must meet the following requirements:

• Post-graduate qualifications in Sociology, Development/Humanitarian studies and a qualified team member with experience in assessing Livelihoods, VSLA, IGAs and Micro finance Programmes
• Proven experience of carrying out endline evaluations focusing on project implementation in Livelihoods, VSLA, IGAs and Micro finance in Returnees/IDP setting such as South Sudan
• Extensive experience in applying various evaluation methodologies. Relevant training and demonstrated expertise in participatory methods, quantitative and qualitative studies
• Minimum of 10 years of experience evaluating development and/or humanitarian programs implemented by International Aid Organizations.
• Knowledge and experience of South Sudan political and cultural contexts

Technical and Financial Offer

The Consultant shall be required to submit the following:

1. Technical and Financial offer
2. The technical proposal should among others detail the following:
   1. Consultants understanding of the assignment with specific attention to the objectives of the assignment and how the objectives will be measured
   2. Methodology: A clear description of the overall design and methodology of the endline evaluation, and clear work plan
   3. Proposed team with CVs of the Individual Consultant or Consulting team that will be involved in the assignment.
   4. At least 2 sample evaluation reports conducted in the last 12 months for assessing the quality of work of the Consultant.
3. The financial proposal shall include the following:
4. Proposed budget for the evaluation in United States Dollars (USD)
5. Professional consultancy fees and logistics costs (include all the professional fees, travel and data collection expenses, and clearly state the level of effort for each member of the proposed team)
6. Proof of professional registration, taxation and other relevant documents may be required

Consultant Evaluation Criteria

The following ranking criteria will be considered for the contract granting:

Technical Expertise of the Consultant

1. The number of the proposed team is adequate, with expertise in the key sectors to meet the requirements of the assignment (5 points).
2. Relevant academic and professional qualifications of the Consultant in conducting evaluations (include CVs) (5 points).
3. Consultant’s knowledge and experience in the context, or similar contexts with the ability to work in the geographical location of the assignment (5 points).
4. Demonstrated previous experience in similar evaluations by submitting 2 sample reports. A review of these reports demonstrates quality work (5 points).

Understanding of the assignment and Methodology

1. The technical proposal clearly describes the understanding of the ToR, responding to the objectives of this evaluation (10 points).
2. Demonstrated understanding of the project and the logic model (5 points).
3. Clear description of how the evaluation questions will be answered, including the evaluation matrix (10 points).
4. The proposed evaluation design is appropriate, with clear description to meet the requirements of the evaluation (10 points).
5. Sampling criteria is clearly elaborated and appropriate for the evaluation (10 points).
6. Clear description of the data collection, and analysis tools and procedures (5 points).
7. Mechanisms for quality and ethical procedures are well articulated (5 points).
8. Detailed work plan with realistic time estimates of each major segment of the work plan/milestones (10 points).

Financial Proposal

1. The budget is clear and realistic, aligned with the work plan (Include all the professional fees, travel and data collection expenses) – (15 points).

8. Payment Terms.

30% Payment will be made upon submission and approval of the Inception Report, and the remaining 70% upon submission and approval of the Final Evaluation Report with other key deliverables. The payment will be subject to local or national tax laws.

[1] For more details: https://www.alnap.org/help-library/evaluating-humaintarian-action-using-the-oecd-dac-criteria

How to apply

Application Process

If interested in applying for the aforementioned consultancy, please send your CV along with a written technical and financial proposal, outlining a plan and approach for executing the consultancy in line with the points mentioned in section 2 to viona@dca.dk with agym@dca.dk and kamm@dca.dk in cc.

The deadline for application is the 04-11-2024 @ 15:00 PM CAT. The subject of the application letter or e-mail should read Application for the Community Managed Microfinance (CMMF) DCASS Project.

CLICK HERE TO APPLY