Research Critique

This is a critique I did for my nursing research class (not a formal paper, but posted on our class bulletin board):

Probably one of the most controversial research reports of all times in the great BSN versus ADN entry level debate among nursing academia is the Aiken study. This one research study was the major catalyst for the Institute of Medicine (IOM) calling for a ratio of 60% BSN and 40% ADN/ diploma in the North Carolina nursing workforce.

Study Design: quantitative, observational study with cross-section design

Objective: To examine whether the proportion of hospital RNs educated at the baccalaureate level or higher is associated with risk-adjusted mortality and failure to rescue (deaths in surgical patients with serious complications).

Purpose: "We tested whether hospitals with higher proportions of direct-care RNs educated at the baccalaureate level or above have lower risk-adjusted mortality rates and lower rates of failure to rescue (deaths in patients with serious complications). We also examined whether the educational backgrounds of hospital RNs are a predictor of patient mortality beyond factors such as nurse staffing and experience. These findings offer insights into the potential benefits of a more highly educated nurse workforce."

Conclusion: In hospitals with higher proportions of nurses educated at the baccalaureate level or higher, surgical patients experienced lower mortality and failure-to-rescue rates.

"Our findings indicate that surgical patients cared for in hospitals in which higher proportions of direct-care RNs held bachelor's degrees experienced a substantial survival advantage over those treated in hospitals in which fewer staff nurses had BSN or higher degrees. Similarly, surgical patients experiencing serious complications during hospitalization were significantly more likely to survive in hospitals with a higher proportion of nurses with baccalaureate education."

"Nursing education policy reports published in the past decade concluded that the United States has an imbalance in the educational preparation of its nurse workforce with too few RNs with BSN and higher degrees. Our findings provide sobering evidence that this imbalance may be harming patients."

Sampling plan:
Cross-sectional analyses of outcomes data for 232,342 general, orthopedic, and vascular surgery patients discharged from 168 non-federal adult general Pennsylvania hospitals between April 1, 1998, and November 30, 1999, linked to administrative and survey data providing information on educational composition, staffing, and other characteristics.

The authors analyzed outcomes data derived from hospital discharge abstracts that were merged with information on the characteristics of the treating hospitals, including unique data obtained from surveys of hospital nurses. The institutional review board of the University of Pennsylvania approved the study protocol.

Voluntary, anonymous survey: Questionnaires sent to 50% random sample of RNs residing in Pennsylvania and on the rolls of the Pennsylvania Board of Nursing; surveys completed by 10,184 nurses, a 52% response rate (which averaged more than 60 nurses per hospital).

Random selection of participants with 52% response rate.
Very large achieved sample: 10,184
Researchers indicated no response bias from the relatively low 52% response rate achieved sample. "Demographic characteristics of the respondents paralleled those of Pennsylvania hospital nurses in the National Sample Survey of Registered Nurses" in terms of average ages, working full-time, and having a BSN degree (30% and 31% respectively).

Descriptive statistics (means, SDs, percentages, chi-square, and F tests) were used to compare groups of hospitals that varied in terms of their educational composition on hospital characteristics, including nurse experience and nurse staffing, and patient characteristics.

Logistic regression models were used to estimate the effects of a 10% increase in the proportion of nurses who had a bachelor's or master's degree on patient mortality and failure to rescue, and the estimate the effects of nurse staffing, nurse experience, and surgeon board certification.

The probabilities of poor outcomes were calculated for patient in hospitals assuming that 20%, 40%, and 60% of the hospital RNs held bachelor's or master's degrees and under various patient-to-nurse ratios.

Possible confounding variables and biases:
Selection bias: BSN, MSN, nurses with doctorates all grouped together; ADN's and diploma nurses grouped together (Pennsylvania has one of the highest populations of diploma nurses in the US).

This factor alone seriously compromises the study's generalizability to a target population of just ADNs and just BSNs and produces flawed methodology for the study conclusions.

It was not known how many nurses in the sample were originally ADN nurses who went back to get their BSN's (the BSN sample likely has nurses with more education and experience than the ADN's).

Need to factor out the experience variable.

Aiken noted the study found nursing experience was not associated with lower mortality rates of patients. "It's the educational level that's important and experience alone cannot achieve the same level of benefit for the patient as education plus experience," she said.

This is flawed methodology, because many BSN nurses were originally ADN nurses who went back to get their BSN's. The opposite is not true- BSN's don't go back to get their ADN. For an accurate comparison, need to compare strictly new ADN graduates with BSN graduates and patient outcomes.

Another possible confounding variable:

Patients admitted to a high tech and/or teaching hospital by virtue of the type of hospital have a better chance of survival. These hospitals often employ higher percentages of BSN and higher degree nurses than smaller outlying hospitals.

Are the researchers adequately controlling for patient acuity level?
The article noted control of acuity level, but no explanation was given for how this was done.

Nurse patient ratios:
BSN/ MSN nurses tended to work in the high tech/ teaching hospitals and to have lower ratios of nurse to patient 4:1 as compared with the ADN at 8:1.


This highlights the danger of institutions making practice recommendations on the basis of the findings of one, unreplicated study, especially if the study involved has serious biases and flaws in methodology.

References

Aiken, L.H., Clarke, S.P., Cheung, R.B., Sloane, D.M., & Silber, J.H. (2003). Educational levels of hospital nurses and surgical patient mortality. Journal of the American Medical Association (JAMA), 12, 1617-1623.

Bernier, S. (2003). N-OADN update: JAMA Article “Educational Levels of Hospital Nurses and Surgical Patient Mortality."

North Carolina Institute of Medicine (IOM). (2004, March). Executive summary: Final report of the Task Force on the North Carolina Nursing Workforce.