Evaluating health worker performance in Benin using the simulated client method with real children

Implementation Science

PubMedID: 23043671

Rowe AK, Onikpo F, Lama M, Deming MS. Evaluating health worker performance in Benin using the simulated client method with real children. Implement Sci. 2013;795.
Background: The simulated client (SC) method for evaluating health worker performance utilizessurveyors who pose as patients to make surreptitious observations during consultations.Compared to conspicuous observation (CO) by surveyors, which is commonly done indeveloping countries, SC data better reflect usual health worker practices. This information isimportant because CO can cause performance to be better than usual. Despite this advantageof SCs, the method's full potential has not been realized for evaluating performance forpediatric illnesses because real children have not been utilized as SCs. Previous SC studiesused scenarios of ill children that were not actually brought to health workers. During a trialthat evaluated a quality improvement intervention in Benin (the Integrated Management ofChildhood Illness [IMCI] strategy), we conducted an SC survey with adult caretakers assurveyors and real children to evaluate the feasibility of this approach and used the results toassess the validity of CO. Methods: We conducted an SC survey and a CO survey (one right after the other) of health workers inthe same 55 health facilities. A detailed description of the SC survey process was produced.Results of the two surveys were compared for 27 performance indicators using logisticregression modeling. Results: SC and CO surveyors observed 54 and 185 consultations, respectively. No serious problemsoccurred during the SC survey. Performance levels measured by CO were moderately higherthan those measured by SCs (median CO - SC difference = 16.4 percentage-points). Surveydifferences were sometimes much greater for IMCI-trained health workers (mediandifference = 29.7 percentage-points) than for workers without IMCI training (mediandifference = 3.1 percentage-points). Conclusion: SC surveys can be done safely with real children if appropriate precautions are taken. CO canintroduce moderately large positive biases, and these biases might be greater for healthworkers exposed to quality improvement interventions.Trial numberhttp://clinicaltrials.gov Identifier NCT00510679.