Using ODK for RBF structural quality checklists in Zimbabwe

Over the last decades, the health system in Zimbabwe has been confronted with a double crisis: an economic collapse and the AIDS pandemic. This destabilized the health system at its core. And overall life expectancy dropped from 61 years old in the 1990s to 48 years old in 2005, before climbing back up to 58 years old in 2012.

To address the consequences of these external shocks, the government set up different instruments, including a Results-Based-Financing (RBF) and staff retention scheme. Currently, BlueSquare works closely with Cordaid to improve the Ministry of Health’s RBF data management system.

Shifting a complex structural quality checklist to mobile devices

As in most RBF systems, health care providers in Zimbabwe are paid, in part, based on the quality of their services. To measure quality of care, the government uses structural checklists (find examples here) and a mix of district teams and external evaluators to conduct quarterly assessments.

However, meaningful assessment of a facility’s (clinic or hospital) quality of health care services is complex. The Ministry of Health and Child Care in Zimbabwe’s (MoHCC) checklist for quality measurement directly reflects this complexity. While it aggregates all inputs into a final, unique score per facility per period, it actually contains more than 400 different data elements in a single 24-page paper document.

The data collected through this process is paramount for RBF, not only because it’s a part of the payment formula for health care facilities, but also because of it’s high value for broader health system monitoring and evaluation.

For the last few years, this quality of care evaluation process has taken place on paper, leading to errors, delays in information transfers, duplication of efforts, and more importantly, the loss of valuable data within the structural checklists, as only a final score was digitized.

• Seeking to accelerate the entire process and make high-value data more readily available, together with Cordaid, we have worked to shift data collection to mobile devices. Our aim was to ensure that:
• Initial input was collected directly in a digitized version, using an offline, tablet-based form.
• Granular data collected at all facilities could then be uploaded, in as much details as possible, to the DHIS2 (a RBF data management tool and national data warehouse).

Using ODK for complex quality checklists

In implementing these changes, a more extensive list of features was developed:

• Managers are now able to define questions, including the scoring computation
• The form can be downloaded to a verifiers’ tablet and be completed offline at the health facility
• The evaluation results (scoring) are available immediately (allowing the facility manager can see them before the verifier leaves)
• When the verifier has an internet connection again, they can upload the results to DHIS2 as data elements (possibly one per question)

While the technical design for this was relatively straightforward, had we built everything from scratch it would have required the development of a full custom Android mobile application. Meaning, significant costs and time.

This is not only a problem of money, but also of turnaround as Cordaid wanted to start using the updates as soon as possible, which would have been difficult with a full custom solution. For this reason, we looked into existing packages that could accelerate this development.

Open Data Kit (ODK) is a free and open-source set of tools, which helps organizations author, field, and manage mobile data collection solutions. BlueSquare already uses ODK for RBF data collection in a number of countries (e.g. Kyrgyzstan, Democratic Republic of Congo or Haiti), but we didn’t know if the tool would support the complexities of Zimbabwe’s quality checklist.

The XLSForm standard is a way of describing forms that use an Excel specific format. While a bit technical, it’s more like Excel formulas than code. In other words: it does not require software developers to create and manage, which is a strength.

Here is a simple example:

Each line represents a single question with its type (e.g. date, integer, choice), its name (what will be recorded) and its label (what’s shown on the screen).

For the “select” questions, you just have to list the possible choices with their names in a separate sheet:

A form like this can then be shown using the ODK Collect Android application:

The checklist challenges we identified for ODK/XLSForm were:

• Conditional scoring capability (e.g. a question with ten possible items, you get a score of 0 if you tick less than 5, 2 if you tick 6 or 7, 4 for 8 or 9 and the maximum of 6 for 10 items ticked)
• Showing intermediary results (total score of all questions in the first section)
• Ability to bypass some questions (if they are not relevant, updating the maximum score in consequence)

To our relative surprise, ODK was more than up to the task and we were able to fill in the representative sets without any big issues.

Next Step : linking ODK to DHIS2

While the offline capability makes the technical solutions more complex, existing (open source) solutions like ODK, allow for simple and fast deployment of mobile data collection tools. Using ODK, when compared to the previous paper collection methods, has already been a giant step forward. Now all the data is now available in an Excel format and meaningful statistics can be easily extracted.

Our next step for the program is to integrate the ODK collected data with the DHIS2 platform. This integration however, requires additional software development and time. Stay tuned to our next tech blog posts for updates!

Written by Martin Van Aken, CTO of BlueSquare. Find him on LinkedIn here and Twitter here.