Pharmacoepidemiol Drug Saf. 2015 Sep;24(9):943-50.

Diagnostic accuracy of algorithms to identify hepatitis C status, AIDS status, alcohol consumption and illicit drug use among patients living with HIV in an administrative healthcare database.


Durand M1, Wang Y2, Venne F3, Lelorier J3, Tremblay CL4, Abrahamowicz M2.

  • 1Department of Internal Medicine, Centre Hospitalier de l’Unvisersité de Montréal, Montréal, Canada.
  • 2Department of Epidemiology and Biostatistics, McGill University, Montréal, Canada.
  • 3Department of Medicine, Université de Montréal, Montréal, Canada.
  • 4Department of Microbiology, CHUM Research Center, Montréal, Canada.


Abstract :

Purpose: To develop and evaluate diagnostic algorithms for AIDS, hepatitis C status, alcohol abuse and illicit drug use in the administrative healthcare database of the Province of Quebec, Canada (Régie de l’assurance-maladie du Québec – RAMQ).

Methods: We selected HIV positive patients contributing to both the RAMQ database and a local clinical database, which was used as gold standard. We developed algorithms to identify the diagnoses of interest in RAMQ using data from hospital discharge summaries, medical and pharmaceutical claims databases. We estimated and compared sensitivity, specificity, positive predictive and negative predictive values, and area under receiver operating curve for each algorithm.

Results: 420 patients contributed to both databases. Prevalence of conditions of interest in the clinical database were as follows: AIDS 233(55%), hepatitis C infection 105(25%), alcohol abuse 106(25%), illicit drug use 144(34%) and intravenous drug use 107(25%). Sensitivity to detect, AIDS, hepatitis C, alcohol abuse, illicit drug use and intravenous drug use was 46%[95%CI: 39-53], 26%[18-35], 50%[37-57], 64%[55-72] and 70%[61-79], respectively. Specificity to detect these conditions was 91%[86-95], 97%[94-98], 92%[88-95], 95%[92-97], and 90%[87-93], respectively. Area under ROC curve varied from 0.62[0.57-0.65] for hepatitis C to 0.80[0.76-0.85] for intravenous drug use. Test characteristics varied according to available follow-up time in RAMQ: sensitivity increased and specificity decreased with longer follow-up times.

Conclusions: Sensitivity was low to detect AIDS, alcohol abuse, illicit drug use and especially hepatitis C in RAMQ. Researchers must be aware of the potential for residual confounding, and must consider additional methods to control for confounding.

KEYWORDS: HIV AIDS; RAMQ database; administrative healthcare database; diagnostic accuracy; epidemiology; pharmacoepidemiology; sensitivity; specificity

PMID: 26114918


Supplementary material:

Not all research questions can be answered by prospective cohorts or clinical trials. The use of other research tools, such as administrative health care databases, offers an alternative that is often less costly and faster to address emerging clinical questions. Those databases are particularly well suited to study rare side effects of drugs, uncommon medical conditions and conditions with a delayed onset.

The databases of the Régie de l’Assurance Maladie du Québec (RAMQ) contains data on all medical fee for service billing, dispensed prescriptions drugs and discharge summaries for all acute care hospitalisations for about 40% of the population of Québec, Canada.

Using RAMQ data to study HIV infection and its complications offers several advantages: First, its large size and long follow-up allows studying of rare or delayed complications of HIV-infection, which are of increasing concern to patients living with HIV. Second, this database offers the possibility to select HIV-uninfected controls as a comparison group, a feature often missing from HIV databases. Finally, data from RAMQ represents a “real life” situation, often closer to reality than the well-controlled environments of clinical trials.

Yet in all administrative healthcare databases, diagnostic codes have less than 100% sensitivity and specificity, leading to residual confounding. Databases must be validated for variables that are important for the study of HIV/AIDS.

In this work, we selected four variables that are important potential confounders for most studies of HIV. Using a local clinical database as a gold standard, we extracted the status of AIDS, Hepatitis C, alcohol abuse and illicit drug use. We then linked the local database to RAMQ to determine the sensitivity, specificity, positive predictive value and negative predictive value of diagnostic algorithms developed in RAMQ. Briefly, 4 different algorithms were tested for each of the variables, to establish the best way to define these important covariates. The best algorithm was selected using the maximal area under receiver operating curve (ROC.)

We found that sensitivity of administrative data was low, ranging from 26% to detect hepatitis C co-infection to 70% to detect intravenous drug use. However, specificity was generally high, ranging from 90% for intravenous drug use to 97% for hepatitis C co-infection. What this means is that if an individual is infected with hepatitis C virus, there is only 26% chance of picking up this status using RAMQ data. However, if RAMQ does identify his as having hepatitis C, there are 97% chances that indeed, this patient has hepatitis C.

This work is important to the future use of RAMQ data to study HIV, as knowing the test characteristics of algorithms used to define important covariates allows to quantify the potential for residual confounding, and thus to best adjust for it through novel statistical methods. Here, low sensitivities indicate a high potential for residual confounding, of which researchers must be aware when using RAMQ data.


Acknowledgements: This work has been made possible by research grant from the Canadian Association for AIDS research (CANFAR, grant number 023501) and the Canadian Institutes of Health Research (CIHR grant number MOP 81275). Dr Durand received a post-doctoral fellowship from The CIHR HIV Clinical Trial Network and a clinical researcher fellowship from Fonds de Recherche du Québec-Santé.


Corresponding author:

Dre Madeleine Durand, MD MSc FRCPC


850, rue Saint-Denis, Montréal, (Québec) H2X 0A9


Multiselect Ultimate Query Plugin by InoPlugs Web Design Vienna | Webdesign Wien and Juwelier SchönmannMultiselect Ultimate Query Plugin by InoPlugs Web Design Vienna | Webdesign Wien and Juwelier Schönmann