Using Propensity Score methods for estimating real-world treatment effects in the presence of measurement error and sparse outcome data

Abstract

The real-world treatment effect of a novel treatment can be estimated by analysing routinely collected patient data, in the form of Electronic Health Records (EHR). Unlike a Randomised Clinical Trial, the treatment allocation in this type of data is not randomised and there may be systematic differences between the treatment groups. Propensity Score (PS) (Rosenbaum & Rubin, 1983) methods are commonly used to correct for these differences and hence reduce the bias in the treatment effect estimate. The combined impact on the treatment effect estimate of two common issues in EHRs were investigated; covariate measurement error and sparse outcome data.

A comparison was made between the performance of four PS methods: 3:1 PS matching; Inverse Probability Treatment Weighting (IPTW) for the Average Treatment Effect (ATE); IPTW for the Average Treatment Effect on the Treated (ATT); and PS stratification. Simulation experiments were run, based on a data extract from The Health Improvement Network holding UK primary care data. The impact of measurement error and outcome prevalence were investigated for different scenarios to represent real-life situations. For each simulation, PS conditioning was applied to the data to address treatment allocation bias before using Cox proportional hazards regression to estimate the treatment effect on the time-to-event outcome.

In data with higher outcome prevalence, covariate measurement error had little effect on the treatment effect estimate. For data with sparse outcomes, ≤ 1%, higher negative measurement error (corresponding to under-recording) produced treatment effect estimates with little bias, but lower precision. PS Stratification was the recommended method for estimating ATE with lower bias and higher precision over the measurement error range and over the outcome prevalence range. IPTW for ATT was the recommended method for estimating ATT with higher precision in all scenarios and lower bias, particularly when there was lower outcome prevalence.