The role of Monitoring and Evaluation in assessing progress of Operational Research in the EAPHLNP Study Sites in Kenya

Abstract

ntroduction: Studies on performance of operational research (OR) projects have outlined the various enabling factors leading to favorable research outcomes. This descriptive study aimed at documenting the performance of OR activities based on monitoring and evaluation (M&E) indicators as well as highlighting lessons learned.

Methodology: An M&E framework was developed for the OR studies in three thematic areas; Tuberculosis (TB), malaria and enterics as well as for components of capacity strengthening and administrative. This was done by KEMRI OR Secretariat in consultation with the East Central Southern African – Health Community (ECSA-HC) Secretariat, and the principal investigators of each thematic area from East Africa Community partner states (namely Burundi, Rwanda, Tanzania and Uganda. The framework included outcome indicators which were defined for each study, target values defined in accordance with approved protocols. Reporting interval was set at quarterly per year. TB studies had 8 reporting indicators, Enterics had 5 reporting indicators, malaria had 11 indicators, while administrative and capacity building had 15 reporting indicators. The framework was then adopted by the region. In Kenya, the initial roll-out of the research in all three thematic areas was done in February 2013. The first quarter of M&E was conducted in the study sites, as defined in the editorial of this journal, in June 2013 while the second “quarter” was carried out in June 2014.

Findings: The findings presented here covers between February 2013 and June 2014.Generally, there was little progress in all the three thematic areas. During the first evaluation, the number of enrolled persons presumed to have TB at the satellite facilities were 185 persons equivalent to 6.2% of the expected target number (3,000 persons). During the same period, non-satellite sites enrolled 124 persons presumed to have TB (8.2%) of the expected 1,520 persons. In the second evaluation, enrollment at the satellite sites was at 13.3% compared to 1.2% in the non-satellite sites. This represents a two-fold percentage increase in the in the satellite sites compared to non-satellite sites. Using ZN indicator, there were differences in the number of actual TB cases detected in satellite and non-satellite sites compared with the target values. Number of TB – cases detected using ZN at satellite sites rose from 7.8% (target of 784 cases) to 11.0% (target of 1725 cases). In the non-satellite sites, there was a decline from 3.1% (target of 508 cases) to 1.1% (target of 1118 cases). In the Enteric Study, there was a marginal decline in the number of patients recruited from 21.6% (target of 1440 patients) as at the first evaluation to 17.2% (target of 1800 patients) in the second evaluation. For malaria study, a total of 333 patients had been enrolled against a targeted of 300 patients into the study representing an over enrollment of 111%. For the administrative indicators, the OR Secretariat had over attained in three target areas namely publication and sharing of OR findings in country and regional bulletin (1067%) and holding of OR-Technical working group meetings (133%).

Lessons Learned: Several reasons contributed to low performance in achieving indicators in both TB and enteric studies. These include high staff turnover particularly in the non-satellite sites, high workload and breakdown in communication among sites personnel regarding participation in research activities. In the satellite sites where study interventions were provided, better performance in achieving indicators was attributed to improved capacity in personnel and other non-financial motivational aspects such site exchange visits, refresher courses and frequent attendance of project workshops and meetings. On-site involvement of scientists from KEMRI to assist in patient recruitment, specimen collection and shipment alongside with study site staff resulted to over-attainment of performance indicators as demonstrated in the malaria study. The observed lag time between the various M&E field visits by OR team could have partially contributed to the missed opportunities of identification and correction of any deviations from the project protocols. Changes in leadership especially at the project top management at the Ministry of Health affected the overall performance of OR activities mainly due to delays in disbursement of funds as well as delays in obtaining no-objection to incur expenditure on essential activities that were not originally in the approved annual workplans.

Conclusion& Recommendation: Generally, it was established that several factors, some of which could have been augmented if the M&E exercise was conducted in accordance with the framework, affected the achievement of study indicators.. M&E component is thus a critical activity especially in tracking research progress, and should be conducted consistently and within the stipulated timeline. This will subsequently provide opportunities of early identification and correction of any deviations from the protocol.