Objectives Point-of-care ultrasound (POCUS) diagnostic accuracy research has significant variation in blinding practices. This study characterises the blinding practices during acute POCUS research to determine whether research methodology adequately reflects POCUS use in routine clinical practice.
Design, settings and participants A search for POCUS diagnostic accuracy studies published in Emergency Medicine, Anaesthesia and Critical Care journals from January 2016 to January 2020 was performed. Studies were included if they were primary diagnostic accuracy studies. The study year, journal impact factor, population, hospital area, body region, study design, blinding of the POCUS interpreter to clinical information, whether the person performing the POCUS scan was the same person interpreting the scan, and whether the study reported incremental diagnostic yield were extracted in duplicate by two authors. Descriptive statistics were provided and prespecified subgroup analysis was performed.
Main outcome measures The primary outcome was the number of studies that blinded the POCUS interpreter to at least some part of the clinical information. Secondary outcomes included whether the person performing the POCUS scan was the same person interpreting it and whether the study reported incremental diagnostic yield.
Results 520 abstracts were screened with 97 studies included. The POCUS interpreter was blinded to clinical information in 37 studies (38.1%), not blinded in 34 studies (35.1%) and not reported in 26 studies (26.8%). The POCUS interpreter was the same person obtaining the images in 72 studies (74.2%), different in 14 studies (14.4%) and not reported in 11 studies (11.3%). Only four studies (4.1%) reported incremental diagnostic yield for POCUS. Inter-rater reliability was moderate (k=0.64). Subgroup analysis based on impact factor, body region, hospital area, patient population and study design did not show significant differences after completing pairwise comparisons.
Conclusions Although blinding the POCUS interpreter to clinical information may be done in a perceived attempt to limit bias, this may result in accuracy estimates that do not reflect routine clinical practice. Similarly, having a different clinician perform and interpret the POCUS scan significantly limits generalisability to practice as it does not truly reflect ‘point-of-care’ ultrasound at all. Reporting incremental diagnostic yield from implementing POCUS into a diagnostic pathway better reflects the value of POCUS; however, this methodology was infrequently used.
Trial registration number The study protocol was registered on Open Science Framework (https://osf.io/h5fe7/).
- critical care
- emergency medicine
Data availability statement
Data are available in a public, open access repository. All data are freely available on Open Science Framework (OSF) at https://osf.io/h5fe7/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Contributors KW, RB, RRU, JB, TAM, MYW, RT and MDFM contributed substantially to the study design, data analysis and interpretation, and the writing of the manuscript. KW and RP are cofirst authors owing to equal, significant, contributions to all aspects of the study. RP had full access to all study data and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.