Extent of data-informed decision-making among sub-district level public health administrators: A cross-sectional study of medical officers in India

Study design

A quantitative approach with a descriptive study design comprising a cross-sectional survey was adopted, and this study was a part of a larger research to determine determinants of data use for evidence-based decision-making at a sub-district level in Haryana.^[13]

Study area

Six districts of Haryana were selected based on their maternal and child health indicators, two each from good, medium, and poor indicators.^[14] The period of study spanned from December 2020 to May 2023.

Study population and sample

The study population consisted of all 151 MO-ICs positioned at sub-district health care centers (PHCs) in rural and urban locations. The medical officers who were “in-charges” were included, and those who were not “in-charges” or were posted at community health centers, district/sub-district hospitals, or any other district office were excluded. Data were collected from 120 MO-ICs who were available at their respective facilities. The difference between the anticipated and actual sample was due to vacant positions or MO-ICs being absent in two consecutive visits.

Study variable and its definition

The variable of interest – Data Utilization Score (DUS) was a composite score calculated to gauge the extent of data usage among MO-ICs.

Data collection procedure and study tool

An interview schedule that was broadly based on the Performance of Routine Information System Management (PRISM) tool,^[10] and was used in a previous study^[15] in three Indian states was adapted, pre-tested, and administered to 120 MO-ICs at their PHCs. Trained investigators gathered data through interviews conducted at the PHCs where the MO-ICs were posted, and each interview lasted for about 45–60 minutes.

Two types of assessment were conducted – subjective and objective. In the subjective assessment, the MO-ICs evaluated their own level of data usage across various segments, while interviewers conducted an objective evaluation of their data use. Several sets of questions were posed for subjectively determining data use in categories – (a) decision-making and skills to use data, (b) use of data in routine decision-making, (c) use of data for program monitoring, (d) use of data for PHC management, (e) use of data formats, and (h) access and use of data sources.

To have a neutral and objective perspective of data utilization, an objective assessment of MO-ICs was conducted, which comprised gauging the skills of MO-ICs for data utilization, verifying and validating recalled indicators with documented records, and examining the health facility for evidence of data use. The approach designed to capture this information included (a) skill test, (b) recall of indicators used in management meetings, (c) recall of full antenatal care (ANC) and full immunization rate of their PHC area, (d) recall of program indicators, and (e) latest data displayed in the facility. The skill test comprised questions on the graphical interpretation of pie charts and bar graphs, along with questions related to epidemiological incidence and prevalence rates. Program indicators included maternal health, child health, tuberculosis, malaria, and family planning indicators. Facilities were observed for display of the latest PHC data on indicators, services provided, performance of the facility, outpatient department coverage and geographical area covered in the form of graphs, tables, or charts.

Data analysis-DUS generation

To measure the extent of data utilization for evidence-based decision-making by MO-ICs at the PHC level, a DUS was calculated, encompassing both subjective and objective dimensions. Methodical steps were followed in constructing the score, and the categories of variables used in both dimensions were interchangeable.^[16] The aggregation approach was compensatory, and a simple sum was calculated. Normalization was done by ensuring that all variables were unitless, had positive polarity, and the variables were pre-coded and equally weighted. The final score, which explained the magnitude of data use, was obtained by adding the subjective and objective scores. Internal consistencies of the DUS and its dimensions were tested using Cronbach’s alpha, aiming for α>0.60 as desirable for psychometric scales, as our instrument/interview schedule was multi-item.

The extent of data utilization by MO-ICs was ascertained by transforming the score into a percentage for ease of comprehension, and a description of the dimensions/subdimensions of the score was provided. Furthermore, the characteristics of DUS were established, and normality was tested both by statistics (skewness, kurtosis, Kolmogorov– Smirnov test, and Shapiro–Wilk test) and plots (histograms and normal Quantile-Quantile plots).

Descriptive statistics

Out of the 120 MO-ICs interviewed, 75% were deputed at rural and 25% at urban facilities. About 80% of MO-ICs were doctors of medicine and surgery, and the rest were dental surgeons. Years of experience in the current job and in the health system were 4.8 and 7.6 years, respectively. The details are illustrated in Table 1.

DUS characteristics

The DUS had a mean of 55.25 with a 13.35 standard deviation. The median was 55.5, and the mode was 58. The minimum and maximum values were 7.50 and 88, respectively, with a range of 80.5. The internal consistency of DUS, DUS subjective, and DUS objective was 0.75, 0.81, and 0.63 initially; two of the items were excluded from the DUS objective, and the internal consistency therefore increased to 0.78, 0.81, and 0.67 for DUS, DUS subjective, and DUS objective, respectively. The DUS was normal, as explicitly shown by the results of statistical tests and visual inspection of plots [Figures 1 and 2]. As illustrated in Table 2, the analysis revealed that the score was approximately normally distributed in terms of skewness and kurtosis, as the z-value was between +1.96 and −1.96. In terms of the Shapiro–Wilk test and the Kolmogorov–Smirnov test, DUS was normally distributed as p value > 0.05.

Close

Figure 1:

Normality plot: Histogram of data utilization score.

Export to PPT

Close

Figure 2:

Normality plot: Quantile-Quantile plot of data utilization score (observed value vs. expected normal).

Export to PPT

Extent of data utilization

The least possible score was 0, and the maximum was 92. The average value of DUS attained through converting the average score-mean (55.25) in percentage was 60.1%. The range of scores is depicted in Figure 3. Out of the 120 MOICs interviewed, 72% of MO-ICs had DUS more than 50%; however, out of these, only 17% of MO-ICs could score above 75 %. Out of the 27% MO-ICs scoring below 50%, only 1% scored below 25%.

Close

Figure 3:

Extent of data use ascertained from the data utilization score (n = 120). MOICs: Medical Officers-In-Charge.

Export to PPT

DUS-Subjective description

Results under the various categories of subjective scoring are depicted in Figure 4 and Table 3. The majority of the MOICs agreed that decisions were based on evidence/facts, and they were involved in decision-making. Data utilization was during management meetings by MO-ICs and their subordinates for monitoring targets. In terms of their own assessment of the skills to utilize data, 82% of them perceived that they had the required skills to use data and information for decision-making.

Close

Figure 4:

Percentage of Medical Officers - In Charge (n = 120) using data for decision-making under various categories (a) decision-making and skills to use data, (b) use of data in routine decision-making, (c) use of data in program monitoring, and (d) use of data for primary health center management.

Export to PPT

For routine decisions, an average of 65% MO-ICs used data mainly for managing supplies and medicines, making plans, and monitoring key objectives and indicators. Data were also utilized for monitoring programs related to maternal health, child health and family planning. For managing their PHCs, data were primarily used for making reports and plans, and monitoring program outputs.

Access and use of data sources was explored, and routine health information systems were more accessible and useful to an average of 73% of MO-ICs, as compared to published research and survey reports [Table 3]. In addition, hard copies of data and summary reports were perceived to be most useful.

DUS-Objective description

The performance of MO-ICs in skill test and recall of indicators is displayed in Figure 5. The average score in the skill test was 65%. MO-ICs scored relatively high while correctly interpreting and analyzing graphs that were in the form of pie charts and bar graphs; however, for the calculation of epidemiological rates, the correct response was lower.

Close

Figure 5:

Performance of Medical Officers-In-Charge in (a) skill test (b) recall of indicators (n = 120).

Export to PPT

In recalling indicators, the highest correct recall was for indicators used in management meetings, with 75% of MO-ICs recalling one to three indicators, followed by 42% recalling basic indicators such as full ANC and immunization rate, and only 18% recalling program-related indicators. In reviewing the latest data displayed in facilities, 61% of PHCs had up-to-date tables on their walls, and 65% had up-to-date graphs or charts displayed.

Main findings

While discharging their duties as administrators, MO-ICs were involved in data-informed decision-making and were confident that they had the skills to do the same. The average DUS was 60.1%, and 72% of MO-ICs scored above 50%. Data in routine decisions were used while making decisions related to medical supply/drug management, formulation of plans, and monitoring of indicators, while less data were used for human resources management. Data usage was also high for monitoring various maternal health, child health and family planning programs. For the management of PHCs, data were used for reports and plan preparation, monitoring program output, monitoring own work, and the work of subordinates. Relatively less data usage was for budgeting and program strategy, which is comprehensible as these activities are mostly conducted at the block/district level. Routine health information systems, hard copies of data, and summary reports were the most useful sources and formats.

A mismatch of self-reported and actual skills was noticed, while 82% of MO-ICs claimed to have the skills to use and interpret data, the skill test results showed that only an average of 65% of MO-ICs had actual data use and interpretation skills. Skills for graphical interpretation were more than the skills for epidemiological rates interpretation. Recall of indicators was highest for those indicators regularly used by MO-ICs in management meetings, followed by basic indicators, and least for program indicators. The latest graphs and charts were displayed in two-thirds of the facilities.

Comparison with previous studies

Our study is novel in a way that it highlights the level of data utilization for data-informed decisions at the sub-district level of the public healthcare network, for which there is limited evidence not only in India but also globally. Furthermore, it is highly relevant as there is limited literature focusing on data use by MO-ICs placed at the PHCs. Studies have been conducted to assess data use by health functionaries at all levels of the public health system in India;^[15,17] however, the sample size at the PHC level was just 24; hence, we assume our findings to be concrete considering our sample size. Our study revealed a promising finding where MO-ICs were involved and interested in factual decision-making, and the anticipated skill level pointed toward their motivation. Our findings were encouraging as they were different than earlier research wherein the lack of interest and resistance of MOICs was cited as a roadblock toward data use for decision-making.^[18]

Study findings affirmed the relevance of data in program support and not just for record-keeping, hence indicating that evidence/facts were used for decision-making. Data were used by the MO-ICs and the team attached to them for planning and monitoring their targets, signifying a culture of data usage that had percolated to the sub-district level. The present study counters the findings of earlier studies, which asserted that there was a “data poor culture” and data were not used at all levels of decision-making.^[8]

We highlighted numerous decisions made by MO-ICs for the routine functioning of PHCs and sub-centers, based on routinely available data. Our study revealed the usefulness and accessibility of routine health information systems, which are contrary to findings of previous studies wherein access to actual data and inability to perceive the value of data were cited as a challenge.^[5] Findings resonate with previous studies and affirm that different data formats, such as hard copies and summary reports, serve as key drivers of information use.^[19] In addition, aligned with the national guidelines, the facilities had the latest data displayed on the premises.^[9]

Study findings revealed that routine data were being used to monitor maternal health, child health, and family planning programs, and they resonate with studies emphasizing that the routine Health Management Information System (HMIS) provides data in the form of indicators required to manage these programs.^[8,18] Previous research has indicated that information systems are designed to support MO-ICs in decision-making and performance monitoring roles, and our findings support this, as data at the sub-district level was used for monitoring program output, daily work, and report and plan preparation.^[20,21]

Management meetings, an organizational framework of the public health system of India, are a pivotal platform wherein discussions revolve around reviewing performance, setting targets, continued learning, planning, and strategizing with the support of district-level data.^[13,22] Our study revealed a substantial use of data and indicators during management meetings and pointed toward system readiness and a culture of information in Haryana, which is an improvement from a similar previous assessment wherein the “culture of information” was lacking.^[15]

The striking revelation of a mismatch between self-reported data interpretation and use skills with the tested skill aligns with the findings of previous studies, which concluded that a lack of skills to analyze data is the reason for limited data use.^[7,15,23,24] Similarly, limited recall of almost all categories of indicators was a concerning finding, as these indicators are important performance monitoring and decision support tools.^[8,15,21]

Policy relevance

National level guidelines lay emphasis on the use of information for program monitoring and management of PHC activities by the administrators – MO-ICs placed at the PHC level, and our study reveals an evident increase in the extent of data utilization for decision-making at the sub-district level from 49.9% to 60.1% over a decade in India in comparison to an earlier study conducted in three Indian states – Rajasthan, Maharashtra, and Uttar Pradesh.^[15,25]

The timing of our study, coinciding with approximately 15 years of HMIS establishment and implementation in India, highlights increased utilization of data for data-informed decisions and suggests a shift toward a data-driven culture fostered by government-implemented HMIS.^[26,27] While establishing a digitized web-based information system may not lead to a striking culture of data use, but an enhanced data use certainly is evident through a slow and steady transformation process, as per the findings of our study and previous research.^[28]

The launch and implementation of HMIS have paved the way for data culture; however, our findings do highlight gaps in processes rather than input issues.^[8,21] Our findings are anticipated to feed into program and policy initiatives to consider addressing the potential gaps to improve data usage. One of these is the mismatch between self-reported and actual skills, which can be addressed through targeted skill-building training – a measure shown to improve the competencies of medical officers.^[20,29-31] Focus should also be laid on the utilization of more indicators during management meetings, as this leads to data usage for decision-making.^[4,8,20] The findings point toward readiness for data-informed decisions at the sub-district level and a data-driven decision-making culture that needs to be tapped for the long-term performance and impact of public health programs.

Future research could explore data-use practices among various health functionaries positioned at different levels and geographies, providing a broader systems-level perspective on evidence-based decision-making practices.

Strengths and limitations

A core strength of our study was the comprehensive assessment of data use by including both subjective and objective assessments, utilizing an exhaustive data collection instrument.

We have two limitations. First, our sampling technique was non-probability, and data were collected from 80% of the anticipated sample, which may affect the generalizability of the results. Second, we limited our scope to the Haryana state, and future research can explore other states and regions of India.