Call for Proposals: Project Evaluation Lebanon At Caritas Germany

Project Title: “Sustainable strengthening of primary health care centers in Lebanon to secure the health care provision and strengthen the resilience of refugees and vulnerable Lebanese with non-communicable diseases or mental health diseases, 2020-2023.”

BACKGROUND

​​​The Syrian crisis has been going on for eleven years with few signs of abating. According to UNHCR, the estimated number of Syrian refugees in Lebanon is 1.5 million, of which around 850.700 are registered with UNHCR. Additionally, as a country of only 4.2 million people, Lebanon hosts large numbers of Palestinian and Iraqi refugees and has one of the highest per capita concentration of refugees in the world. The conflict in Syria has significantly impacted Lebanon’s social and economic growth, caused deepening poverty and humanitarian needs – both for refugees and host communities. It exacerbated pre-existing development constraints in the country that is ruled by corrupt sectarian elites. The nationwide social unre​st, the severe ongoing economic and financial crisis as well as the Covid 19-pandemic lead to a further deterioration of the living conditions of refugees and Lebanese.

​Non-communicable diseases (NCDs), that are results of lifestyle and environmental factors as well as mental health diseases (MHDs) caused by the experience of war, violence and economic hardship are widely spread among vulnerable Lebanese and refugees in Lebanon. These populations suffer from limited access to medical treatment. Since the onset of the Syrian crisis, Lebanon’s healthcare facilities have been overstretched by an increase in utilization, which challenges the absorption capacity of the health care centers as well as their financial sustainability. Beirut Port Explosion temporarily and the Covid-19 pandemic continuously put additional strain on the overburdened healthcare system.

​Against this background Caritas Germany (CG) and Caritas Lebanon (CL) implemented the project “Sustainable improvement of health care and treatment of chronically ill refugees and vulnerable Lebanese in Lebanon” from 2017 to 2020. The project focused on the standardization of CL’s Primary healthcare operations and services, and on increasing the health resilience of vulnerable refugees and Lebanese suffering from NCDs in 10 PHCCs operated by CL.

​​​The BMZ continuation project was also jointly designed by CL and CG and includes additionally a Mental Health (MH), a Psychosocial Support (PSS) and a Health Outreach component. Some of the MH Components have been part of a previous German Federal Foreign Office (GFFO) funded project. The project activities are taking place in the 10 primary health care centers (PHCCs) operated by CL, which are located all over Lebanon – while the specialized MH component is only implemented in 6 of these PHCCs. The project is implemented by CL with technical support by CG. The project is funded by the German Federal Ministry of Economic Cooperation and Development (BMZ).​​​​

CONTEXT

​​​Since July 2020, CG and CL are implementing a health project with the objective to strengthen primary health care centers to secure the health care provision and strengthen the resilience of refugees and vulnerable Lebanese. It focuses on people suffering from NCDs and as a new component, on people with MH diseases. The project aims to achieve three outcomes:

1. Standardization and capacitation of CL’s PHCCs to provide access to high-quality treatment of NCDs and MH services to refugee and vulnerable Lebanese populations.
2. Improving access to treatment and strengthening resilience with regard to NCDs for refugees and vulnerable Lebanese.
3. Improving access to treatment and strengthening resilience with regard to MH diseases for refugees and vulnerable Lebanese.​​

The target group considered for the evaluation are NCD and MH patients, as well as project staff. For Outcome 1 the target group is CL as an organisation and its health staff. This includes the head nurses, nurse assistants and psychologists that are CL staff and the physicians and psychiatrists that are paid per consultation. For Outcome 2 and 3 the target groups are 50% refugees (including Syrians, Iraqis, Palestinians and migrants), and 50% vulnerable Lebanese from host communities living in the catchment areas of CL PHCCs.

At the beginning of 2022, a mid-term evaluation of the MH component of the project took place in order to provide recommendations for further improvement of this component. The evaluation report and subsequent review of the implementation of the recommendations conducted one year later will be made available, but the main focus of this evaluation shall be on the NCD and capacity building component of the project.

Current State: Until end of July 2023 the following activities have been conducted contributing to the above-mentioned outcomes:

Outcome 1: Standardization and capacitation of CL’s PHCCs

Output 1.1: CL health personnel are trained on NCDs topics and treatments.

Health personnel was trained on

• common health conditions in elderly people (online 2021),
• kidney disease and anemia (2021),
• Third Annual Geriatric Skills Enhancement in Primary Care (GEM) workshop: “Infections in Older Adults: Early Recognition and Prevention” (2022, online by American University Beirut (AUB)),
• diet for NCD patients (2023),
• NCD screening and referral for PHCCs not included in the Ministry of Public Health (MoPH) network (2023).
• Respiratory diseases: to be conducted in August 2023.

Output 1.2: MH and non-MH staff receive trainings on MH topics.

CL non-MH staff received training on

• MH red flags identification in adults: the most common MH disorders in adults (Dec 2021-Jan 2022),
• SGBV safe identification and referral training by International Rescue Committee (IRC) (2022 and 2023),
• Problem solving (2023).

CL MH staff received training on

• CL psychologists received a training on the clinical management of suicide risk by Embrace (2021, refresher 2023),
• SGBV safe identification and referral training by (IRC) (2022 and 2023).

Output 1.3: Self-care workshops are implemented for healthcare staff.

• CL psychologists developed a self-care curriculum with 9 sessions (3 sessions per year),
• a base- and endline survey was developed on yearly basis (for each 3 sessions separately),
• the first 2 rounds of 3 sessions have been conducted in 8 locations (including staff of the 2 smaller PHCC, the mobile medical unit (MMU) and warehouse) in 2021 and 2022; current round is ongoing.

Output 1.4: CL SOPs (based on MoPH standards) are implemented in the PHCCs.

• Half yearly audits in 2021 and since 2022 yearly audits of the SOP implementation were conducted,
• audit criteria were developed by an external consultant who also conducted the 2 audits in 2021 and trained the CL healthcare quality officer to conduct the audit from 2022 onwards,
• all PHCC have improved their compliance with the SOP and until now, 9 out of the 10 PHCC meet over 80% of the SOP.

Output 1.5: CL PHCCs are integrated in the MoPH network and receive drugs from YMCA.

• 3 additional PHCCs (St. Michel, Sin el Fil, Rayfoun) were integrated into the MoPH network in December 2021. The PHCC in Saida was already integrated at the start of the project.
• In the beginning of 2022 staff received training from the MoPH on NCD and malnutrition screening and referral, administrative requirements and management of the PHENIX platform used by the MoPH for monitoring services in all integrated PHCCs.
• The process of receiving drugs from the network was delayed, mainly due to shortage of medication in the national stock. By the 2nd quarter of 2023, all 4 integrated PHCCs received acute and chronic medication from the network.

Outcome 2: Improving access to treatment and strengthening resilience with regard to NCDs

Output 2.1: Refugees and Vulnerable Lebanese are provided with subsidized health care services related to NCDs in CL PHCCs

• NCD patients received medical consultations, nutrition consultations, diagnostics and laboratory tests and chronic medication.
• Due to the economic crisis, patients’ financial contribution to medical consultations was removed from 2021 onwards.
• Vulnerability criteria were relaxed in 2022 allowing for more patients accessing services based on their economic vulnerability.
• The tool for measuring patients’ medical status was modified after a project review process in 2022.
• CL negotiated a standardized USD price for laboratory tests across all laboratories contracted in 2022.
• Due to the difficulties of refugees reaching the PHCCs, the use of MMUs for NCD related medical services was piloted at the end of 2022 and incorporated into the project in 2023. Three MMUs in the North, Akkar and Bekaa were then targeting specifically underserved areas with large refugee populations living in informal settlements in order to increase their access to healthcare services.

Output 2.2: Refugees and vulnerable Lebanese receive awareness sessions related to NCD topics.

• Awareness raising sessions were conducted in the PHCCs on diabetes, dyslipidemia, cardiovascular & prostatic diseases, osteoporosis, anemia, healthy diet for adults and elderly. Brochures were provided on all topics.
• The originally used monitoring tool (pre-/post-test) was replaced by a simpler tool after a project review process in 2022 revealed that the original tool was not very feasible in its application and did not yield the needed information.

Output 2.3. Health Outreach Workers (HOWs) are trained and conduct household visits for new patient identification and medical follow-up.

• Due to the Covid-19 pandemic (2019-2022), the HOW component could only be launched in the second half of 2021.
• HOWs received trainings on their role and responsibilities, and the health needs assessment tool to be used.
• HOWs did not conduct household visits during the first year of the project 2020-2021 but conducted phone calls instead to identify and refer potential patients.
• Referrals to the PHCCs were made, but less than 5% of referred patients actually visited the PHCCs. Main reasons cited by patients for not visiting the PHCCs were distance of facilities from residence, high transportation/fuel cost and unavailability of medication in the PHCC due to general medication shortages in the country in 2021.
• This led to a complete stop of the activity and modification of the HOW concept in 2022
• The activity was restarted again in 2023 with a new concept for the HOW who conduct health awareness sessions on NCD topics and provide info on CL services; and additional support for patients’ medical follow-up and vulnerability verification through CL’s medical social workers.

Outcome 3: Improving access to treatment and strengthening resilience with regard to mental health diseases.

​Output 3.1: Refugees and vulnerable Lebanese with mental illness are treated free of charge in CL’s PHCCs.

• CL psychologists provided psychological consultations and follow-up sessions ongoingly.
• In 2022, CL decided to shift all psychologists from a salary-based employment to consultation-based employment. The shift was not conducted for psychologists of this project. The previously 7 psychologist positions, covered with 40% of their salary under the project, were reduced to 4 positions with 70% salary coverage.
• Psychiatrists provided psychiatric consultations on needs basis in 7of the 10 PHCCs.
• Psychotropic medication was handed out on prescription to all patients who fulfilled the project’s vulnerability criteria.

Output 3.2: Refugees and vulnerable Lebanese receive MH awareness sessions on mental diseases, risk factors and prevention.

• Awareness raising sessions were conducted regularly in the PHCCs on MH problems related to the Covid-19 pandemic, MH problems of elderly patients, MH red flags in adults and general MH awareness.

Output 3.3: Refugees and vulnerable Lebanese participate in psycho-educational group sessions and activate their self-help potential.

• A concept for psycho-educational support groups for NCD patients with difficulties to adapt to and manage their illnesses was developed by psychologists under supervision of MH coordinator.
• 20 psycho-educational support groups with 8 sessions each for up to 12 NCD patients per group have been conducted partly live and partly online; for the online sessions, participants were provided with phone recharge cards. The concept was revised beginning of 2023 and number of sessions reduced to 5 based on beneficiary feedback.
• A concept for psycho-educational support groups for patients with MH problems was developed by psychologists under supervision of MH coordinator; first roll-out was in 2 PHCCs (Sin el Fil and St. Michel) in 2022, but with lesser success due to high stigmatization of MH diseases and high transportation costs.

Besides the PHCC staff, the project also financed 50% of the position of the MH Coordinator and the Healthcare Quality Coordinator. Furthermore, a budget top-up in 2021 provided funds for a Clinical Supervisor for psychologists and psychiatrists as well as a consultancy for staff-care.

OBJECTIVE OF THE EVALUATION

This (final) external evaluation is intended to provide CG and CL with evidence on the effectiveness of the aforementioned project and facilitate learning to support strengthening of future health programming. The evaluation will specifically aim at achieving the following two overall objectives:

A) Assess the extent to which the project was delivered against its objectives and expected results and inform final project reporting.

B) Draw key lessons from the project and incorporate them in recommendations that will help to enhance the design and implementation of CL’s health programming for the future.

The results will be used as lessons learnt within CG and CL but will also be presented to the BMZ, the back-donor of this project.

SCOPE OF THE EVALUATION

The scope of work comprises the period from July 1, 2020, until December 31, 2023, of the implementation of the BMZ funded project “Sustainable strengthening of primary health care centers in Lebanon to secure the health care provision and strengthen the resilience of refugees and vulnerable Lebanese with non-communicable diseases or mental health diseases, 2020-2023”. It will in principle review the full project period except of the last 3 months (from Jan-March 2024: no cost extension).

The project activities are taking place within the PHCCs of CL, which are spread throughout all of Lebanon, as well as 3 areas in the North, Akkar and Bekaa, where the MMUs are active. The evaluation will cover at least 50% of the PHCCS, one MMU and the Headquarters (HQ) of CL.

The target group to be evaluated will be the patients who received consultations free of charge, diagnostics and laboratory tests, medication, awareness sessions as well as psycho-educative group sessions. In addition, the staff of the PHCCs, MMUs, who have benefitted from capacity building measures and self-care activities, will be part of the evaluation.

EVALUATION CRITERIA AND KEY EVALUATION QUESTIONS

This evaluation will focus on four evaluation criteria of the OECD-DAC. These criteria are relevance, effectiveness, efficiency and impact (“impact” & outcome).

The specific objectives to be addressed by this evaluation are expected to be further refined in consultation with the evaluator during the inception phase of the evaluation process. The following key questions are thus indicative:

Relevance:

• To what extent did the objectives and activities of the NCD and MH services meet the overall priorities and needs of patients and contribute to their better well-being?
• Which activities were most relevant to the needs and problems of the different target groups (vulnerable Lebanese, Refugees, CL healthcare staff)? Would other options have been better suited to people’s needs in this context?
• To what extent did the objectives and activities of the training and capacity building components address the priorities and needs of staff and contribute to a better service delivery?
• How well was the ability to respond to changed circumstances (Covid-19, Beirut Blast, economic crisis incl. electricity and fuel shortages …)?

Effectiveness:

• To what extent have the objectives been reached? What factors (both at design and implementation levels) facilitated or inhibited the meeting of objectives and results?
• To what extent is the organizational set up, project management system contributing to or challenging the successful implementation of the project? How and how far do relevant stakeholders participate in the project’s management?
• Are the project activities adequate to realize the objectives? Are best practices followed and lessons learnt incorporated?
• What steps need to be taken to increase the effectiveness and quality of the NCD and MH services (on organizational level)?
• Are the procedures and the criteria for beneficiary selection adequate? How can these procedures be improved?
• Were the MEAL tools developed appropriate in design and in usage?

Efficiency:

• Were services provided in time and impacts achieved within an appropriate time period?
• Were the financial resources and other inputs efficiently used to achieve results?

Impact:

• What real difference has the project made to the beneficiaries?
• Are there any unintended outcomes (positive and negative)?
• Can changes in the health status of the target population be attributed to the project?
• What is the assessment of the project’s contribution to human and institutional capacity building in terms of knowledge and competencies?

APPROACH AND METHODOLOGY

The Evaluator is expected to perform the evaluation in a strictly methodical manner to produce verifiable information and make recommendations that are sufficiently valid and reliably based on dependable data and analysis, which are accessible to the client. The evaluation will adopt a participatory, transparent, and solution-oriented approach using internal and external stakeholders. The evaluation process, as well as the results should be gender-sensitive and human-rights-based. The evaluator will use mixed methods that are able to appropriately address the primary evaluation questions, as well as properly support its derivative conclusion and recommendation.

The evaluation methods to be used may include, but are not limited to:

• Review of project documentation (Desk Research): An array of major documents that should be studied (e.g. project documents, monitoring reports, final reports, as well as statistical data, concepts, ToR and JDs, MEAL tools and SOPs that were developed etc.). Existing project documents and reports are to be shared with the evaluator in order to facilitate the realization of the tasks. The evaluator is invited to request additional documents that may be needed for the completion of the evaluation.
• Interviews with all the key informants and key players: (semi-)structured interviews with beneficiaries are a must and should be supported by interview protocols and the list of respondents (e.g. direct project beneficiaries, local staff, etc.).

The selection criteria should be based on gender, age, beneficiaries/non-beneficiaries, and geographical spread in the project region. The exact sample size should be proposed by the evaluator and discussed together with Caritas ​Lebanon​ and Caritas Germany before the start of the evaluation.

• Direct observation during field visits: Caritas ​Lebanon​ will organize field visits at different project sites.
• Focus group discussion with beneficiaries and local partner staff (if feasible within the timeframe).
• Standardized Survey and questionnaires: to quantify the result and deliver a statistical analysis.
• Participatory methods: If feasible, methods that allow impact measurement and reflection for illiterate people, e.g. ranking, scoring, rich picture, narratives, stories, timelines etc. must be considered to learn about the beneficiaries’ perspectives.

DELIVERABLES AND TIMEFRAME

Deliverables for the evaluation are:

• Inception Report:

An inception report will be submitted by the evaluating consultant to demonstrate his understanding and planning of the evaluation, which will be reviewed and discussed in cooperation with Caritas Lebanon and Caritas Germany.

The inception report should include an evaluation matrix (including the final evaluation questions and indicators); the overall evaluation design and methodology with a detailed description of the data collection methods and data analysis techniques, as well as a proposed timeframe for the activities and deliverables. The evaluator is free to suggest additional methodological framework for the evaluation.

The inception report of the evaluation should not exceed 5 pages and follow a predefined structure.

• Workshops/ Meeting:

The evaluator is free to select the type of workshops (e.g. kick-off/mid-term/validation workshop) during the conduction of the evaluation.

The Kick-off workshop provides an opportunity to discuss organizational processes, methodology, data collection instruments, timetable, milestones, reporting dates, task and responsibilities. Moreover, the mid-term workshop is useful to discuss problems that may arise in the course of an evaluation and allow a joint development of strategies for solutions.

During a validation workshop, the evaluator will present the findings collected in the field to Caritas Lebanon and Caritas Germany. The workshop will be organized in order to discuss and validate findings, lessons learned, and recommendations proposed by the evaluator. Stakeholders are invited to make recommendations for amending the review.

• Draft Report:

The consultant is expected to submit the primary findings resulting from his evaluation, as well as his initial recommendations to the desk officer of Caritas Germany. The Draft Report should be presented after the field work has been concluded and should incorporate comments supplied by Caritas Germany and its partner organizations.

• Final Report:

The final report serves to illustrate the relevant evidence corresponding with the evaluation issues, questions and criteria listed in the Terms of Reference. The final report should provide an executive summary in English. The Final report should take all aspects reviewed during kick off and/or validation workshop into consideration and is subject to approval by Caritas Lebanon and Caritas Germany.

The proposed report should not exceed 30 pages (excluding appendices). All documents and tools are to be written in English language.

TIMEFRAME FOR DELIVERABLES

The actual evaluation work is to be carried out from 01.01.2024 to 29.02.2024, resulting in a total of 20 days.

The timeline for the activities consists of the following phases:

Preparation Phase: 4 days

• Analysis of relevant project documents, as well as further research
• Preparation of inception report and exchange with Caritas Lebanon and Caritas Germany
• Kick-off Workshop involving Caritas Lebanon and Caritas Germany

Field Phase: 12 days (including travel days)

• Briefing with Caritas Lebanon, Caritas Germany and other relevant actors in the health sector
• Field visits
• Validation workshop involving Caritas Lebanon and Caritas Germany

Synthesis Phase: 4 days

• Reporting including management response workshop
• Draft evaluation report
• Produce final report including revision as per feedback

The key products expected for the evaluation are the following (dates may be redefined with the evaluator if necessary):

• Inception Report: 08.01.2024
• Draft Report: 05.02.2024
• Final Report: 19.02.2024

The evaluator should present a detailed work plan and timeframe of all activities including the relevant resource allocation.

ROLES AND RESPONSIBILITIES

Caritas Lebanon is responsible for organizing and facilitating the logistics to and in Lebanon. Caritas Lebanon and Caritas Germany will provide access to all relevant project documents.

The consultant will be working under and reporting to the Caritas Germany desk officer and the project coordinator/Head of health department of Caritas Lebanon during the evaluation phase. They will be organizing field visits and provide a car with driver. Accommodation is self-organized.

GUIDING PRINCIPLES AND DONOR CONCEPTS

The consultant is to conduct the evaluation in accordance with the principles outlined in the “Caritas Internationalis management standards” document, the “Caritas Code of conduct”, as well as the “Guidelines on Combating Fraud and Corruption in the Project Work of Caritas Germany”.

The evaluator must take all required steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of the people and the communities of which they are members, as well as to ensure that the evaluation is technically accurate, reliable, and legitimate, and conducted in a transparent and impartial manner. Moreover, the evaluation should ideally contribute to organizational learning and accountability.

PROFESSIONAL QUALIFICATION AND EXPERIENCE

The evaluation consultant should meet with the following, required skills and competence profile:

• Evaluator should be a reliable and effective project manager with extensive experience in conducting evaluations and a proven record in delivering professional results. (Consultant is required to send samples of relevant evaluation) (Required)
• In-depth knowledge and experience in the public health sector; (Required)
• Knowledge of strategic and operational management of humanitarian operations, as well as proven ability to provide strategic and practical recommendation to key stakeholders (Optional)
• Operational experience in emergency response program, particularly trough participatory evaluation processes (Optional)
• Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and prepare well-written reports in a timely manner (Required)
• Experience in qualitative and/or quantitative data collection and data analysis techniques (Required)
• Strong expertise in using participatory tools (Optional)
• Familiarity with transparency and downward accountability processes and their values (Optional)
• Fluency in written and spoken English, ideally in conjunction with good skills in spoken French and/or Arabic (Required)
• Country and regional experience in Lebanon/Middle East (Optional)
• Good understanding and appropriate sensitivity in regard to different cultures and traditions (Required)
• Minimum qualification of a master´s degree or equivalent combination of education and relevant work experience (Optional)

TERMS OF PAYMENT

The evaluator is to receive payment from Caritas Germany in three instalments:

• 1st Payment: 30 % upon signing of the contract*;*
• 2nd Payment: 30 % upon the submission of the first draft of the evaluation report;
• Final Payment: 40 % upon the submission of the final evaluation report, subject to its acceptance by Caritas

Caritas Germany will not settle any payment unless the consultancy institution accomplished all the tasks in a timely fashion. The basis for payment scheduling is to be determined during contract negotiations.

How to apply

Interested candidates should submit their application material by 23.10.2023.

The application itself should include the following components:

• Curriculum Vitae (CV) including 3 professional references, and indication of daily rates
• Cover letter transparently summarizing relevant experience in public health as well as practical experience in planning and conducting project evaluations
• Track record of conducted studies, research, publication and references
• A technical proposal for the evaluation, including the proposed evaluation methodology and work plan
• Financial proposal underlying the evaluation, including proposed fee for a maximum of 20 working days
• A sample of recent writing (report or similar) relevant to the terms of reference stated above

The proposal should be addressed to:

Miriam Schwenk

Desk Officer Lebanon

Caritas Germany

Karlstraße 40,

79104 Freiburg

Phone: + 49 (0) 761 200 291

Email: miriam.schwenk@caritas.de

Homepage: www.caritas-international.de