Regular ArticleAdjudication of bleeding outcomes in an international thromboprophylaxis trial in critical illness
Introduction
The measurement of bleeding outcomes in clinical trials is prone to random and systematic error. Even with objective definitions of major and minor events, the rigorous assessment of bleeding is complex because of difficulty quantifying the volume of blood loss, and the need for population-specific criteria for major bleeding. Measuring bleeding in critically ill patients is particularly challenging because bleeding is a common occurrence in the intensive care unit (ICU), is often due to invasive procedures, and frequently occurs at multiple anatomical sites [1].
The PROphylaxis for ThromboEmbolism in Critical Care Trial (PROTECT; NCT00182143) [2] was an international randomized, double-blind and concealed trial that compared the low molecular weight heparin dalteparin to unfractionated heparin (UFH) for thromboprophylaxis in 3,764 medical-surgical ICU patients. The primary outcome was proximal leg deep vein thrombosis (DVT) diagnosed with twice weekly leg ultrasounds. A secondary endpoint was bleeding, which was the primary safety outcome. Other secondary endpoints were DVT at other sites (e.g., upper limb, distal leg DVT), pulmonary embolism, and any venous thromboembolism. The methods [3] and results [2] have been previously described. The main findings of the PROTECT trial were that rates of DVT and bleeding were not different between groups, but use of dalteparin was associated with a significantly lower rate of pulmonary embolism.
To accurately capture both the number and severity of bleeding episodes in PROTECT, every reported bleeding event was independently adjudicated in duplicate by a central adjudication committee. The objective of this study was to describe the methods, results, and outcomes of the adjudication process for bleeding outcomes in PROTECT.
Section snippets
Measurement of bleeding in PROTECT
We previously developed HEME (HEmorrhage MEasurement in the ICU), a comprehensive validated bleeding measurement tool that is highly reliable and specific for medical-surgical patients who are admitted to the ICU [1]. Using the HEME tool, all bleeding events in PROTECT were prospectively measured by dedicated trained research coordinators at each participating site. Bleeding severity was established by physiologic and anatomic parameters and the need for therapeutic interventions. Bleeding was
Calibration exercise
For the first 45 bleeding events which occurred in 40 patients, crude agreement between pairs of adjudicators ranged from 86.7 – 93.3% and all 4 adjudicators agreed on the assessment of bleeding in 82.2% events. Overall kappa for all 4 adjudicators was 0.81. We resolved all disagreements. As a result of discussions during the calibration phase, we 1) addressed methodological issues (e.g., handling of recurrent bleeds), 2) added a category (e.g., ‘no bleed’), 3) expanded a category (e.g., a
Discussion
Our strategy of assessing bleeding using a central adjudication committee demonstrated satisfactory performance characteristics in the context of a large randomized controlled trial in critically ill patients. We used a rigorously developed ICU-specific bleeding assessment tool that has construct validity and high intra-rater reliability [1]. We conducted a calibration exercise with all adjudicators and documented excellent agreement, then assessed all bleeds for the remainder of the trial by
Conflict of interest statement
None of the authors has any conflict of interest to disclose.
Acknowledgements
Funding Support: The PROTECT trial was funded by the Canadian Institutes of Health Research, The Heart & Stroke Foundation of Canada and the Australian and New Zealand College of Anesthetists Research Foundation. Study drug was provided by Pfizer, Inc; Esai, Inc. provided study drug for the United States. DM Arnold holds a New Investigator Award from the Canadian Institutes for Health Research. F Lauzier is a recipient of a Research Career Award from the Fonds de Recherche du Québec-Santé. D
References (22)
- et al.
A multifaceted strategy to reduce inappropriate use of frozen plasma transfusions in the intensive care unit
J Crit Care
(Dec 2011) - et al.
Prophylaxis of Thromboembolism in Critical Care (PROTECT) Trial: a pilot study
J Crit Care
(Dec 2005) - et al.
Definition of major bleeding in clinical investigations of antihemostatic medicinal products in non-surgical patients
J Thromb Haemost
(Apr 2005) - et al.
Definition of major bleeding in clinical investigations of antihemostatic medicinal products in surgical patients
J Thromb Haemost
(Jan 2010) - et al.
Outcome assessment for clinical trials: how many adjudicators do we need? Canadian Lung Oncology Group
Control Clin Trials
(Feb 1997) - et al.
Adjudicating ventilator-associated pneumonia in a randomized trial of critically ill patients
J Crit Care
(Dec 1998) - et al.
Comparing classifications of death in the Mode Selection Trial: agreement and disagreement among site investigators and a clinical events committee
Contemp Clin Trials
(Jun 2006) - et al.
Misreporting of myocardial infarction end points: results of adjudication by a central clinical events committee in the PARAGON-B trial. Second Platelet IIb/IIIa Antagonist for the Reduction of Acute Coronary Syndrome Events in a Global Organization Network Trial
Am Heart J
(Feb 2002) - et al.
Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
J Clin Epidemiol
(Jan 2011) - et al.
Bleeding during critical illness: A prospective cohort study using a new measurement tool
Clin Invest Med
(2007)
Cited by (24)
Risk of incident bleeding after acute kidney injury: A retrospective cohort study
2020, Journal of Critical CareCitation Excerpt :The primary outcome was the incidence of de novo bleeding events after ICU admission, which was independently adjudicated by two investigators (A.D. and J.B.). The bleeding episodes were characterized using standardized definitions from the HEmorrhage MEasurement (HEME) in the ICU tool, a validated bleeding measurement tool in critically ill patients [19,20]. Only major bleeding episodes were considered in this study, defined by any of the 5 following criteria: 1) life threatening bleeding with hypovolemic shock; 2) life threatening bleeding into a critical site (e.g. intracranial or pericardial); 3) other critical bleeding (e.g., epidural, intraocular or intra-articular); 4) bleeding requiring an invasive procedure (e.g., surgery, embolization); or 5) clinically important bleeding requiring transfusion of 2 or more units of packed red blood cells, or associated with a decrease in systolic blood pressure of at least 20 mmHg or an increase in heart rate of at least 20 beats/min [19,20].
The association between platelet transfusions and bleeding in critically ill patients with thrombocytopenia
2017, Research and Practice in Thrombosis and HaemostasisEpidemiology of Bleeding in Critically Ill Children
2017, Journal of PediatricsCitation Excerpt :It was not always clear from the medical records whether the transfusion was in response to the bleed. Therapy provided would need to be separated from physiologic derangements to provide a more accurate assessment of the severity of a bleed.16 We may have underestimated the incidence of minor bleeds.
Platelet Transfusion Practices in the ICU: Data From a Large Transfusion Registry
2016, ChestCitation Excerpt :Our findings have clinically important implications on transfusion medicine practice. Limitations were the retrospective design, which did not allow systematic assessments of bleeding55,56; thus, our definition of ineffectual platelet transfusions did not consider cessation of bleeding, but rather an inadequate platelet count increase, which is often used to evaluate platelet transfusion success in practice. The patients were mostly undergoing surgery, with a high prevalence of cardiac and respiratory diagnoses, which explains the high prevalence of transfusions with other blood products, the relatively short length of stay in the ICU, and the low overall mortality.
Investigator-Reported Bleeding Versus Post Hoc Adjudication of Bleeding: Lessons From the CHAMPION PHOENIX Trial
2016, Journal of the American College of CardiologyThrombocytopenia in critically ill patients receiving thromboprophylaxis: Frequency, risk factors, and outcomes
2013, ChestCitation Excerpt :Bleeding was considered minor if it was overt but did not meet the criteria for major bleeding. Two independent, blinded investigators adjudicated bleeding events using a validated tool9 and found excellent agreement.20 Patients were evaluated for HIT with the serotonin release assay (SRA) if their platelet count decreased to ≤ 50 × 109/L, if there was an otherwise unexplained platelet count decrease to ≤ 50% of the patient's baseline (defined as the first platelet count after enrollment as long as the count was > 100 × 109/L), if they developed a symptomatic or asymptomatic thrombotic event, or if HIT was otherwise clinically suspected.