Deep in the heart of the county of champions, in the scenic and partly remote wards of Arror, Tambach, Kayego and Kaptarakwa, the activities for the citizen-led household census in Elgeyo Marakwet County are underway, targeting data collection in 101,000 households. The training of enumerators, data collection and data entry activities have been taking place over the past month, and we are looking forward to analysing the data so far received.
A good Programme Manager will tell you that much of their work involves balancing different dynamics that occur during a project, both positive and negative. Despite the great success we have had with the project, a few challenges faced us that we had to overcome. The data collection exercise kicked off on 10th December 2020, 4 days later than expected due to logistical challenges. The new way of working post-COVID19 has meant that there is less face time and all planning and follow up of activities have to be undertaken remotely. So what are some of the things that we have learnt, and how have we mitigated the challenges we have faced?
Data Collection
To date, there are two ways which are normally used to incentivize enumerators: 1) a fixed efficiency wage (i.e. x amount per week) with close supervision and 2) a clear piece rate based on delivery of work with close supervision. The fixed efficiency wage approach values quality as it requires that the enumerator values the accuracy of the work leading to more care and caution during data collection. Additionally, this may hamper the incentive for speed. Hence the piece rate – that payment will be made based on the scope of work undertaken. Whilst it incentivizes the speed, the quality checks may occur too late after field work has been undertaken – and going back to the field is too expensive.
In Elgeyo Marakwet, the choice of working with Community Health Volunteers (CHVs) in Elgeyo was a no-brainer to generate the data. If we cater for the skills factor – indeed, CHVs are not traditionally enumerators – are we assuming that these volunteers are vested enough to do a good job of data collection? Or do they need other ‘incentives’ besides transport facilitation?
Well the data has told it all; we find a mix between good and extremely bad data. We therefore have had 2 levels of checks. First, in the field, we do a random scan of 10% of the filled questionnaires in order to take note of the data collected, which has led to having to redo a number of interviews. Secondly, because the data digitization is happening almost concurrently with data collection, this has enabled the second check; picking up on glaring issues early in the process. We have used SMS to reach out to CHVs and exhort them to fill in the questionnaires steadily and sturdily. Using serialization, we track the issuance and return of questionnaires and so it is easy to trace who was assigned which particular batch of questionnaires.
In general, more data leads to better results; but we have also remained conscious of the fact that garbage in is garbage out. Simply having a lot of (dirty) data does not help; it can actually muddy the waters and make decision-making more difficult for the county.
Nevertheless, we must appreciate when it comes to data in governance, buy-in comes slowly at multiple levels. With time, it will become apparent to CHVs that it is through collecting quality data that data dividends accrue – even though this is not apparent in the near term.
Data digitization
Luckily, we started with the end in mind by adopting a robust data digitization tool that has enabled near real time analysis on questions of interest; as well as metadata on the process itself, essentially, how fast and efficient the data entry process is. We are therefore able to track which data has been captured erroneously and request recapture.
Both the piece rate and efficiency wage approaches have some common elements to consider and are applicable when working with data clerks. However, we must always ‘verify’ the quality of the data capture. To this end, we have used a hybrid approach – where we have both the piece rate but also non-monetary daily prizes that ensure that data clerks don’t lose sight of quality as they ‘chase’ speed.
Training Training Training!
We have learnt that you cannot train enumerators and data clerks enough! To be able to ‘trust but verify’, iteration will be important – backstopping the process of data collection and capture ensures quality is built into the entire process. Design of such a project must ensure flexibility in the process – responding to concerns of the data clerks and sometimes even the enumerators in the field – ensuring that quality and quantity are not mutually exclusive.
The data collection task is being undertaken in partnership with the Elgeyo Marakwet County Government, Open Governance Institute, with support of GIZ Kenya’s Good Governance Programme.
