HealthDataNavigator Assess available data on different performance domains across various settings

Data Comparability

Why is data comparabilty important for performance measurement ? 

Fundamentally we are interested in observing variation in performance measures where different types of variation can be identified:

  • individual-level variation: age, sex, co-morbidities
  • producer/provider level variation
  • other types of variation: random/residual

It is obvious that challenges in data availability and comparability issues are numerous in international comparisons. For that reason, we have listed a number of recommended actions for users and researchers dealing with international comparisons.

Recommended actions for data comparability

Check and review definitions,  sources and generation processes of data

Secondary data, usually obtained from administrative databases are prone to the following:

  • many relevant (clinical) variables may be missing from the (register-based) data
  • information may be recorded poorly, secondary diagnoses may be missing
  • background (co-morbidities, prior use of services) and follow-up information (outcomes) must be identified with record linkage, personal identification codes needed for linkage
  • recorded events may be "incompatible", e.g. reimbursements vs. discharges

The diagnosis coding practices differ in hospitals within countries and also across countries (even the use of ICD-10 classification). Read more

Consider appropriate risk adjustment and statistical modelling

When comparing regions, hospitals and years, patient-associated variation must be accounted for. Information on risk factors can be based on data recorded during the recent encounters or from the patient's medical history data. Read more

Consider issues in cost comparison

International comparisons of health expenditure and health prices must be based on a common currency. Purchasing power parities (PPP) are rates of currency conversion constructed to account for such price differences. Generally, the reported PPPs adjust for price differences at the level of the total GDP, not sub-aggregates of the GDP, such as health expenditures. However, cross-country differences in health care prices are not necessarily consistent with differences in prices in general. Read more

Measure total episodes of care

When records in different registers are deterministically linkable for all individuals on the basis of personal identification code, it is possible to follow patients to track their use of services – not only in specialized health care, but also in the sectors closely related (care of the elderly, primary inpatient care, purchases of prescribed drugs). The utilization of population-level health care registers with an episode-of-care approach offers versatile system- and producer-level performance measurements such as comparing outcomes and quality. Read more

Measure and compare outcomes (effectiveness) and quality

Clinical quality measures derived from routine data sets (patient administration systems) and relate to the process and outcomes of clinical care. The variation across indicators may reflect differences in the delivery and quality of health care. Face validity (professional consensus) and content validity (inclusion of available evidence) are important issues. Read more

Improving data comparability and solutions in recent projects

Many of the common comparability issues have been recognized, considered and solved in various ways in these recent international projects listed. Methodologies used in two of them are presented below.

Improving comparability in effectiveness assessment - methodology used in EUROHOPE/PERFECT

Relevance Adjusted factors Examples of needed data in the case of CHD Used in cross-country comparisons
1 Patients lifestyle/habits Smoking, obesity, exercise no
2 Genotype effects Genetic risk scores (GRSs) based on  single-nucleotide polymorphisms (SNPs) no
3 Severity of the disease(s) STEMI/non-STEMI, lab-tests sometimes
4 Co-morbidities, age and sex History of illness, medication yes
5 Adherence to treatment and rehabilitation/lifestyle changes Compliance in secondary prevention no

 

Improving comparability in cost efficiency assessment - methodology used in Nordic hospital comparisons

Relevance Adjusted factors Examples of needed data in the case of CHD Used in cross-country comparisons
1 Output case-mix ICD-10 dx, procedures, comorbidity, age, sex, acute/elective  yes
2 Severity of the disease(s) EMR based clinical data (e.g. lab)  no
3 Quality of output HSMR, readmissions, adverse outcomes during the stay  sometimes
4 Adherence to treatment and rehabilitation Patients' contribution in the 'co-production'  no
5 Differences in coding quality  Use of secondary dx  no
6 Cost standardisation Input price level  yes
7 Producer unit type standardisaton  Teaching vs. nonteaching hospital  yes

 

icon plus References