Article Text


Assessing the impact of bibliographical support on the quality of medical care in patients admitted to an internal medicine service: a prospective clinical, open, randomised two-arm parallel study
  1. Matteo Mario Pastori1,
  2. Manuela Sarti2,
  3. Marco Pons3,
  4. Fabrizio Barazzoni2
  1. 1Dipartimento di Scienze della Salute, Università degli Studi di Genova (D.S.Sal.), Genova, Italy
  2. 2Ente Ospedaliero Cantonale (EOC) Head Office, Bellinzona, Switzerland
  3. 3Department of Internal Medicine, Ente Ospedaliero Cantonale (EOC), Regional Hospital of Lugano, Lugano, Switzerland
  1. Correspondence to Dr Manuela Sarti
    Ente Ospedaliero Cantonale (EOC) Head Office, viale Officina 3 Bellinzona, 6500 Switzerland; manuela.sarti{at}
  2. MMP and MS contributed equally.

Statistics from


EBM is the integration of the best scientific evidence with clinical expertise and patient values. The practice of medicine, based on scientific evidence, requires the application of the best available evidence in the process of individual patient care.1

However, during the diagnostic procedures and treatments chosen by the physician, it is essential to take into account the individual patient with his/her experiences and cultural values,1–4 although it is not clear which the best way to involve him/her is.5 ,6

Application of the best current knowledge to decisions in individual patients1 is the key to evidence-based medicine practice. Several studies have shown that results regarding patient health and treatment chosen by the physician are strongly associated with use of the scientific literature. In particular, research studies suggest that library services professionally provided have an impact on health outcomes for patients and may contribute to saving time required for healthcare professionals.1 ,7

Other studies point out the association between health literacy (defined as ‘the degree to which individuals have the capacity to obtain, process and understand basic health information and the services required to make basic health decisions’) and health outcomes. They also show how to improve the skills of physicians; for example, rather than screening patients for health literacy, they could routinely use evidence-based ‘teach back’ communication techniques.5 ,6 ,8–12 Efforts to improve health-related outcomes for patients with health literacy have been described as any level of interaction with healthcare professionals and healthcare systems, as well as within a wider community.6 These aspects are to be taken into account when practising evidence-based medicine for the best approach to medical decision-making. The concept of EBM, as described above, is to translate the need of physicians for information into answerable questions to track down the best information used to answer those questions.13 ,14

Research of relevant scientific information through direct consultation of a medical specialist or textbooks or journals or online resources (eg, Up-To-Date, EBM journal, Cochrane Library, Clinical Evidence, Ovid, MEDLINE/PubMed, Translating Research Into Practice ‘TRIP’ Database) is a different way to answer the questions that occur during patient visits.15 Some resources are more reliable and some are easily available or faster, such as Up-To-Date, an electronic textbook, but all of them use evidence-based principles to assess the validity of the information.16

Greater ease in finding such information could allow physicians to answer questions with high-quality bibliographical evidence, thus providing the best care for the patient.7 Most doctors, however, do not use bibliographical data to answer questions that arise daily.

It is probably due to limited time to search, lack of training in clinical assessment of information and low expectations to find a relevant and accurate answer to questions.17

These difficulties have been highlighted in several studies where, for example, daily activities of physicians were analysed and using online evidence-based resources (eg, TRIP Database, InfoRetriever, DynaMed, Clinical Evidence), the ability to answer questions designed during the patient visit was assessed. Practicing physicians who are inexperienced in the use of online evidence-based resources answered a proportion of their clinical questions that was comparable to the reports by more-experienced researchers; however, the time required to find answers limit the practical use of these databases during patient care time.16 ,17

An improvement in the use of evidence-based resources that are more accessible at the point-of-care and recommendations to ask questions in a format that can be directly answered with evidence (eg, ‘PICO’ format ensures that the question includes information about the patient, the intervention, the relevant comparison and the outcome of interest) could lead doctors to change their professional attitude to use bibliographical research in an effective and practical way in patient care.17 ,18

The ideal information source will be directly relevant, contain valid information and can be accessed with a minimum amount of work for physicians. For example, evidence made quickly available to clinicians on a busy medical inpatient service, using an ‘evidence cart’, increased the extent to which evidence was sought and incorporated into patient care decisions.13 ,19 This cart contains multiple sources of evidence (eg, Best Evidence, JAMA Rational Clinical Examination series, the Cochrane Library, MEDLINE) and compilations of the best evidence found in response to clinical questions asked by medical staff, critically appraised topics (CATs) and the Redbook.13

Another important study by Izcovich and colleagues20–22 assessed the impact of bibliographic assistance on clinically important outcomes for hospitalised patients. The results did not reach statistical significance, but the bibliographic assistance may have an impact on a subgroup of (31) patients who received hand-delivered information. This intervention seemed to decrease the rate of transfer to an intensive care unit and in-hospital mortality.22

Another clinical study by Banks et al23 showed that a presentation of a case at the ‘morning report’ (MR), along with grand rounds, followed by the timely dissemination of the results of an online literature review, led to a reduction in the length-of-hospital stay and in total hospitalisation charges, compared with controls. MR, in association with a computerised literature research guided by librarians, was an effective way to introduce evidence-based medicine into patient care practices.

Taking account of the previous studies, the purpose of our study was to assess the impact of the best daily bibliographical support on the quality of medical care in patients admitted to an internal medicine service in a non-university hospital in Ticino Canton (Switzerland) and thereby improving online evidence-based research.

Materials and methods

This prospective, open, randomised two-arm parallel study was carried out by gathering up clinical questions asked by physicians in the internal medicine service of the Regional Hospital of Lugano (Ente Ospedaliero Cantonale, EOC, Switzerland), during care and treatment of inpatients.

The trial was registered in the EOC Registry (registration number: 14-055), which is the official database of the EOC clinical trials, in accordance with the new Swiss Federal Act on research involving humans (HRA), and entered into force on 1 January 2014.

From November 2012 to February 2013, patients hospitalised in internal medicine wards that generated diagnostic and treatment questions were randomly assigned to two groups for the trial: a so-called ‘intervention’ group (supported by bibliographical research) and a ‘control’ group (not supported by bibliographical research). Randomisation took place with a 1–1 ratio.

Primary end points were patient transfer to an intensive care unit (ICU), hospital readmission at 30 days after discharge and hospital death. The secondary end points were the length-of-hospital stay and the transfer to other institutions.


During the study, an attending physician in public health gathered up focused questions concerning health and patient care during the daily morning report in the internal medicine service (table 1).

Table 1

Characteristics of the investigated participants (intention-to treat population)

These questions were asked specifically by clinical physicians involved in the morning report: hospitalists, residents and fellows who reformulated them, according to the PICO model (Patient/Population, Intervention, Comparison and Outcome) (table 2) defined in the EBM practice,24 ,25 in order to gather useful keywords for the bibliographical research.

Table 2

Examples of questions and application of PICO model

The literature research was carried out in the morning after the clinical round, only for individuals belonging to the ‘intervention’ group, using pre-filtered EBM resources, such as: Best Evidence, the Cochrane Library, which focused primarily on systematic reviews of controlled trials of therapeutic interventions; Clinical Evidence, a high-quality international database of thoroughly developed systematic overviews assessing the benefits and harms of treatments; MEDLINE/PubMed, an attractive database to find medical information because of its relatively comprehensive coverage of medical journals and ready accessibility; OVID (Technology's EBM reviews) and the electronic textbook UpToDate.

Up-To-Date was used only when prefiltered EBM resources, such as Best Evidence and the Cochrane Library, were unlikely to be helpful. If a research did not yet provide a satisfactory answer to a focused clinical question, it was addressed to MEDLINE/PubMed.

The attending physician answered within 12 h after the question was asked.

The answers, structured according to the CAT model (box 1),26 with recommendations and the gathered publications, were emailed by the attending physician to the whole medical team taking part in the morning report, including physicians treating the patients who had raised the questions.22

Box 1

Critically Appraised Topic (CAT) model

Model CAT

Summary of a single page, structured according to the following default location:

  1. Topic title

  2. Structured clinical question

  3. Research strategy

  4. Quality of evidence found

  5. Summarised from examined articles (such as a table with statistics)

  6. Conclusions/comments

In the same email, the doctor was asked to specify if the responses were taken into account in the final clinical decision, using a yes or no question.

The same medical team treated the control and intervention groups.

Statistical analysis


A ‘randomised block’ was used to balance the number of patients in the ‘intervention group’ and ‘control group’ and to have a random distribution of cases in advance, using specific software.27

A randomisation list was prepared before starting the study in order to achieve a sequence of balanced assignments. Allocation concealment was guaranteed by a central allocation and medical care staff were informed about the allocation of patient, only after the dedicated physician had concluded the bibliographic research.

Data analysis

The participant's age was calculated by taking the difference in years between the date of admission and the date of birth. All means are given with ±SD. The association between categorical variables was examined by means of a χ2 test, relative risk (RR) and 95% CI. Numerical variables were compared by means of the Student t test. Days of hospitalisation were logarithmically transformed (ln) to achieve normal distribution. All tests were two-tailed and considered significant if p<0.05. Data were analysed on the intention-to-treat population (participants assigned to the two groups according to the randomisation procedure) and on the per-protocol population (participants assigned to the two groups according to the application of bibliographical research). All statistical analyses were performed in SPSS V.20.0 for Windows.


Patient disposition and demographic characteristics

Eight hundred and sixty-six patients were hospitalised in internal medicine wards of Hospital of Lugano from November 2012 to February 2013.

A total of 201 participants generated questions and, divided in ‘intervention’ (n=101) and ‘control groups’ (n=100), were included in the study in the aforementioned period of time. The most frequent diagnoses of the patients were pneumonia (20%), cardiovascular (17%), infectious (11%) and urinary tract diseases (9%). In the ‘intervention group’, bibliographical results were obtained for 98 participants (missing answers in 3 participants). The clinical team approved bibliographical results for 90.8% of these participants (89/98). Bibliographical results were therefore applied to 88.1% (89/101) of ‘the intention-to-treat population’.

There were no differences between the two groups with respect to age, sex or type of questions, neither in the intention-to-treat population (table 1) nor in the per-protocol population (data not shown).

Results of primary and secondary end points: intention-to-treat population

The receipt of the bibliographical research showed to have a significant protective effect on transfer to ICU (RR=0.30; 95% CI 0.10 to 0.90; χ2=5.3, p=0.02), with 4% (4/101) of transfers to ICU in the intervention group compared to 13% (13/100) in the control group.

No death was observed in the study, therefore we could not point out any difference between intervention and control groups on this parameter.

Hospital readmissions were influenced by the bibliographical research (RR=0.42; 95% CI 0.17 to 1.0; χ2=3.36, p=0.05), with 5.9% (6/101) of rehospitalisations in the intervention group compared to 14% (14/100) in the control group.

However, hospitalisation length in days (t=–0.18, degrees of freedom (df)=199, p=0.85) and transfer to other institutions (RR=1.20; 95% CI 0.75 to 1.94; χ2=0.60, p=0.4) did not differ between intervention and control groups (table 3).

Table 3

Outcomes in intervention and control groups (intention-to-treat population)

Results of primary and secondary end points: per-protocol population

The receipt of the bibliographical research did not show a significant protective effect on transfer to ICU (RR=0.38; 95% CI 0.13 to 1.14; χ2=3.24, p=0.07), with 4.5% (4/89) of transfers to ICU in the intervention group compared to 11.6% (13/112) in the control group.

Hospital readmissions were not significantly influenced by the bibliographical research (RR=0.42; 95% CI 0.16 to 1.11; χ2=3.34, p=0.06), with 5.6% (5/89) of rehospitalisations in the intervention group compared to 13.4% (15/112) in the control group.

Hospitalisation length in days (t=–0.69, df=199, p=0.5) and transfer to other institutions (RR=1.12; 95% CI 0.69 to 1.79; χ2=0.21, p=0.6) did not differ between intervention and control groups.


This study confirmed the feasibility of bibliographical assistance in daily medical practice in an internal medicine service. In particular, we demonstrated that it was very useful and effective for patient care to have a dedicated physician that daily sends the bibliographical research results by email to the clinical team within 12 h after asking the focused question. Using this procedure, the prompted questions for the patients in the intervention arm were satisfactorily answered and a high percentage of these answers (90.8%) were approved by physicians.

Unlike Sackett's study,13 we measured transfer to an intensive care unit, hospital death, the length-of-hospital stay in days and the occurrence of repeated hospital admissions. The bibliographical research assistance based on an intention-to-treat population showed to have a protective effect on the transfers to ICU and at the limits of statistical significance, also on 30-day hospital readmissions.

We carried out the study during 3 months in an internal medicine ward where heterogeneity of pathologies is known to be high. As a result, some events, not taken into account in our study, could have affected the outcome. We collected information on age, sex and type of questions and the analyses on these factors allowed us to exclude a confounding effect related to these factors. However, information on other factors (eg, primary diagnosis) is unfortunately incomplete. This could be a limit if compared with the study of Banks et al,23 because in that study the cases were matched with controls according to the matching criteria of patient's age, identical primary diagnosis and secondary diagnosis (within 3 additional diagnoses), using International Classification of Diseases (ICD-9) codes.

In any case, our study is a randomised trial that used a block randomisation procedure that produced balanced study arms (intervention and control groups). Other randomisation methods (eg, group assignation by flipping a coin) have important limits for this type of study with a small/moderate sample size.28 ,29 In large trials (>200) simple randomisation can be trusted to generate similar numbers of participants among groups.30 However, randomisation results could be problematic in relatively small sample size clinical trials (n≤200), such as ours, resulting in an unequal number of participants among groups.

The prior trial by Izcovich et al22 included all patients admitted to the general internal medicine ward of the hospital for a total of 809 patients, of which 407 were randomised to the search-supported arm (intervention arm) and 402 were randomised to the control arm, but only 151 prompted at least one question: 78 randomised to the intervention group and 73 randomised to the control arm. The patients were randomly assigned to an intervention group or a control group by flipping a coin at the time of admission.

However, independently of the choice of a randomisation procedure, we believe that the right time for randomisation is when the admitted patients generate questions. In our study, we used this timing to sample only participants from the population on which the study was based (ie, patients hospitalised in the internal medicine ward and who generated questions on medical care). Precisely 201 of all patients that were hospitalised in internal medicine wards of our hospital (866) generated questions and at that time we did the randomisation of these patients. In our opinion, we applied a more suitable procedure for an evidence-based study.

We acknowledge that our study may be underpowered and this can be inferred from the large CI’s of the calculated RRs. If the follow-up period had been longer, with a study carried out for more than 3 months, the evidence for the impact of bibliographic assistance on clinical outcomes would probably have been higher in the per-protocol population analysis as well.

However, we actually demonstrated the impact of bibliographic assistance, based on focused questions and a suitable answering system, on clinical outcomes. In particular, we found a significant reduction in patient transfers to ICU in the intention-to-treat population despite a short follow-up time, from November 2012 to February 2013.

On the other hand, our analysis on the per-protocol population is not conclusive probably because of the limited power of this study, and further studies with long-term follow-up data are warranted to confirm the usefulness of bibliographical assistance on daily medical practice.


The author would like to acknowledge Roberto Grilli, MD, Director of the ‘Regional Health and Social Agency’, Emilia-Romagna, Italy, for his critical review of the manuscript and important suggestions and Paola Gandolfi-Decristophoris, PhD, Clinical Trial Unit EOC, for statistical consultancy.


View Abstract


  • Contributors MMP contributed to drafting of the clinical protocol and to the practical implementation of the study in an internal medicine service of the hospital. MS and FB contributed to drafting of the clinical protocol and analysed the trial results. MP contributed to the practical implementation of the study in an internal medicine service of the hospital.

  • Competing interests None.

  • Ethics approval Ticino Canton Ethics Committee (Switzerland).

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.