Elsevier

Thrombosis Research

Volume 131, Issue 3, March 2013, Pages 204-209
Thrombosis Research

Regular Article
Adjudication of bleeding outcomes in an international thromboprophylaxis trial in critical illness

https://doi.org/10.1016/j.thromres.2012.12.005Get rights and content

Abstract

Introduction

Measuring bleeding in critical care trials is challenging. We determined the reliability of adjudicated bleeding assessments in a large thromboprophylaxis trial in the intensive care unit (ICU).

Materials and Methods

PROphylaxis for ThromboEmbolism in Critical Care Trial (PROTECT) was an international randomized controlled trial that compared dalteparin to unfractionated heparin for the prevention of deep vein thrombosis in the ICU. Daily bleeding data were collected prospectively using a validated tool. Bleeds were adjudicated in duplicate by 2 of 4 members comprising a central adjudication committee. Bleeds were stratified by severity and study drug, then randomly assigned to adjudicator pairs. Adjudicators were blinded to treatment allocation, study centre and peer-assessments. We calculated agreement on bleeding severity and examined the effect of adjudication on overall trial results.

Results

In PROTECT, 491 patients had bleeding events including 208 with major bleeding and 283 with minor bleeding only. Of 491 patients, 446 were adjudicated in duplicate: 182 with major, 250 with minor and 14 with no bleeding. After adjudication, 52 of 244 bleeds were downgraded to minor; whereas only 15 of 244 were upgraded to major. Overall agreement among adjudicators was excellent (crude agreement = 86.3%; kappa = 0.76). Hazard ratios for major or any bleeding with dalteparin or unfractionated heparin were similar when analyzed using non-adjudicated events.

Conclusions

Major bleeds were sometimes over-called by research coordinators in a large ICU thromboprohylaxis trial. Adjudicator agreement was excellent. Central adjudication allowed reliable bleeding assessment and enhanced the rigor and validity of this major safety outcome.

Introduction

The measurement of bleeding outcomes in clinical trials is prone to random and systematic error. Even with objective definitions of major and minor events, the rigorous assessment of bleeding is complex because of difficulty quantifying the volume of blood loss, and the need for population-specific criteria for major bleeding. Measuring bleeding in critically ill patients is particularly challenging because bleeding is a common occurrence in the intensive care unit (ICU), is often due to invasive procedures, and frequently occurs at multiple anatomical sites [1].

The PROphylaxis for ThromboEmbolism in Critical Care Trial (PROTECT; NCT00182143) [2] was an international randomized, double-blind and concealed trial that compared the low molecular weight heparin dalteparin to unfractionated heparin (UFH) for thromboprophylaxis in 3,764 medical-surgical ICU patients. The primary outcome was proximal leg deep vein thrombosis (DVT) diagnosed with twice weekly leg ultrasounds. A secondary endpoint was bleeding, which was the primary safety outcome. Other secondary endpoints were DVT at other sites (e.g., upper limb, distal leg DVT), pulmonary embolism, and any venous thromboembolism. The methods [3] and results [2] have been previously described. The main findings of the PROTECT trial were that rates of DVT and bleeding were not different between groups, but use of dalteparin was associated with a significantly lower rate of pulmonary embolism.

To accurately capture both the number and severity of bleeding episodes in PROTECT, every reported bleeding event was independently adjudicated in duplicate by a central adjudication committee. The objective of this study was to describe the methods, results, and outcomes of the adjudication process for bleeding outcomes in PROTECT.

Section snippets

Measurement of bleeding in PROTECT

We previously developed HEME (HEmorrhage MEasurement in the ICU), a comprehensive validated bleeding measurement tool that is highly reliable and specific for medical-surgical patients who are admitted to the ICU [1]. Using the HEME tool, all bleeding events in PROTECT were prospectively measured by dedicated trained research coordinators at each participating site. Bleeding severity was established by physiologic and anatomic parameters and the need for therapeutic interventions. Bleeding was

Calibration exercise

For the first 45 bleeding events which occurred in 40 patients, crude agreement between pairs of adjudicators ranged from 86.7 – 93.3% and all 4 adjudicators agreed on the assessment of bleeding in 82.2% events. Overall kappa for all 4 adjudicators was 0.81. We resolved all disagreements. As a result of discussions during the calibration phase, we 1) addressed methodological issues (e.g., handling of recurrent bleeds), 2) added a category (e.g., ‘no bleed’), 3) expanded a category (e.g., a

Discussion

Our strategy of assessing bleeding using a central adjudication committee demonstrated satisfactory performance characteristics in the context of a large randomized controlled trial in critically ill patients. We used a rigorously developed ICU-specific bleeding assessment tool that has construct validity and high intra-rater reliability [1]. We conducted a calibration exercise with all adjudicators and documented excellent agreement, then assessed all bleeds for the remainder of the trial by

Conflict of interest statement

None of the authors has any conflict of interest to disclose.

Acknowledgements

Funding Support: The PROTECT trial was funded by the Canadian Institutes of Health Research, The Heart & Stroke Foundation of Canada and the Australian and New Zealand College of Anesthetists Research Foundation. Study drug was provided by Pfizer, Inc; Esai, Inc. provided study drug for the United States. DM Arnold holds a New Investigator Award from the Canadian Institutes for Health Research. F Lauzier is a recipient of a Research Career Award from the Fonds de Recherche du Québec-Santé. D

References (22)

  • PROTECT Investigators. Writing Committee: Cook D, Meade M, Guyatt G, Walter S, Heels-Ansdell D, Warkentin TE, et al....
  • Cited by (24)

    • Risk of incident bleeding after acute kidney injury: A retrospective cohort study

      2020, Journal of Critical Care
      Citation Excerpt :

      The primary outcome was the incidence of de novo bleeding events after ICU admission, which was independently adjudicated by two investigators (A.D. and J.B.). The bleeding episodes were characterized using standardized definitions from the HEmorrhage MEasurement (HEME) in the ICU tool, a validated bleeding measurement tool in critically ill patients [19,20]. Only major bleeding episodes were considered in this study, defined by any of the 5 following criteria: 1) life threatening bleeding with hypovolemic shock; 2) life threatening bleeding into a critical site (e.g. intracranial or pericardial); 3) other critical bleeding (e.g., epidural, intraocular or intra-articular); 4) bleeding requiring an invasive procedure (e.g., surgery, embolization); or 5) clinically important bleeding requiring transfusion of 2 or more units of packed red blood cells, or associated with a decrease in systolic blood pressure of at least 20 mmHg or an increase in heart rate of at least 20 beats/min [19,20].

    • Epidemiology of Bleeding in Critically Ill Children

      2017, Journal of Pediatrics
      Citation Excerpt :

      It was not always clear from the medical records whether the transfusion was in response to the bleed. Therapy provided would need to be separated from physiologic derangements to provide a more accurate assessment of the severity of a bleed.16 We may have underestimated the incidence of minor bleeds.

    • Platelet Transfusion Practices in the ICU: Data From a Large Transfusion Registry

      2016, Chest
      Citation Excerpt :

      Our findings have clinically important implications on transfusion medicine practice. Limitations were the retrospective design, which did not allow systematic assessments of bleeding55,56; thus, our definition of ineffectual platelet transfusions did not consider cessation of bleeding, but rather an inadequate platelet count increase, which is often used to evaluate platelet transfusion success in practice. The patients were mostly undergoing surgery, with a high prevalence of cardiac and respiratory diagnoses, which explains the high prevalence of transfusions with other blood products, the relatively short length of stay in the ICU, and the low overall mortality.

    • Thrombocytopenia in critically ill patients receiving thromboprophylaxis: Frequency, risk factors, and outcomes

      2013, Chest
      Citation Excerpt :

      Bleeding was considered minor if it was overt but did not meet the criteria for major bleeding. Two independent, blinded investigators adjudicated bleeding events using a validated tool9 and found excellent agreement.20 Patients were evaluated for HIT with the serotonin release assay (SRA) if their platelet count decreased to ≤ 50 × 109/L, if there was an otherwise unexplained platelet count decrease to ≤ 50% of the patient's baseline (defined as the first platelet count after enrollment as long as the count was > 100 × 109/L), if they developed a symptomatic or asymptomatic thrombotic event, or if HIT was otherwise clinically suspected.

    View all citing articles on Scopus
    View full text