Soundness of scientific research studies

Course Overview

One of the hallmarks of a profession is the unique body of knowledge that supports its practice behaviors. That knowledge was methodically developed and critically appraised. While only a relatively small proportion of health science and professionals might have the interest or opportunity to undertake a full blown research project, ALL health professionals have the charge to be able to critically review scientific evidence and apply it to solving problems & answering questions in their everyday health care settings.

Perhaps some of you have studied research in your AA or AS programs or maybe even created and presented Table Clinics or Poster Sessions at your professional meetings. Whether you have or have not implemented formalized research projects, it is certainly true that we all conduct “mini research studies” hour-after-hour, day-after-day in our professional settings.

Professional standards in health disciplines today call for the use of evidence-based practice. The most common source of evidence comes from research. One of the most reliable sources of evidence-based research information is published study results found in peer reviewed journals.

As professionals in the field you will often need to evaluate professional literature to identify current evidence, best practices and relevance to your discipline. The ability to find valid, reliable published literature and determine its applicability to your needs are skills that are required in many job positions in the health and human services fields, and thus are skills that are critical to your professional success.

While “becoming” a professional researcher is generally a specific career goal, critically analyzing the soundness of scientific research studies and using what we find to guide our practice behaviors is a standard that is expected of each of us as Baccalaureate health care professionals.

To be able to critically analyze the scientific literature and be a good “consumer” of evidence-based literature, it is necessary to have a basic knowledge of the research process and the elements of research studies.



You have learned much during the first few weeks of this course and this Analysis assignment is an opportunity for you to pull all of your learning together.  To be successful on this assignment, apply everything that you have learned from your readings, the discussion forums, and the module information pages.


For additional help, download and review page in Module 3 that identifies the Major Elements of a Research Article.


1.  Problem Statement/Purpose Statement

a. What is the complete Title of the research experiment that you selected in Module 2?

b. Follow the same five criteria directions you did in the Module 1 “Problems Everywhere” discussion forum and write a problem statement for this research experiment. (Hint: citations and references needed; use the readings to inform/guide/support your responses throughout this paper).

c.   Using the definition for a research “purpose”, what do you think the purpose of this research experiment is?

d.    In your opinion is this experiment important to our health care system in this country?  Why?  Why not?

2.   Research Question/Hypotheses

a. Using your textbook and course content define the term hypothesis and write one for the research study you are analyzing.


b. Using your textbook (hint: citations and references needed) and course content define the term simple hypothesis and write one for the research study you are analyzing. Then define the following terms and use your simple hypothesis to write a directional and a null hypothesis. 

Simple hypothesis:


Directional hypothesis:


Null hypothesis:


3.   Identification and Operational Definitions of Study Variables

a.   Name the independent and dependent variable/s in your study?



b.    Concept of Operationalization: In an experimental research study the researchers are hoping to find that applying the IV will change the DV.  In other words, suppose that researchers are hoping to find that after applying fluoride to teeth (IV) those teeth will be strengthened (DV).  If so, after applying the fluoride to the teeth (IV) the researchers would have to examine the teeth (DV) for any changes in strength.

SO, with that example in mind, after the IV is applied or used in your study, howdid the researchers going to collect data on their DV(exactly how did they determine if the IV had changed/had an impact on the DV; exactly how did they know that the DV had been impacted or changed) Please explain thoroughly.


4.   Literature Review

Before researchers begin their experiment they review the literature to see what previous experiments might have taken place using the same or similar variables; nobody wants to reinvent a wheel. Literature Reviews are relatively short and are always located between the abstract and the methods sections of a research article. Locate the Literature Review in your research study.  It will be below the abstract, before methods and either:

  • under a section called Introduction
  • OR Literature Review
  • OR there will be no specific heading.

Note that it is NOT in the Abstract.

a.   Where, SPECIFICALLY, is the Literature Review in your article? (Page number?  Column? Paragraph/s?)

b.   Read carefully through the Literature Review section in your experiment and name the specific variables that are being addressed in the Literature review section only

c. After reviewing the course content on primary and secondary sources search the reference page of your article for 1 primary source OR 1 secondary source that you are certain about just from reading the title in the citationTell me the number of the citation and whether that citation is primary or secondary.  Use the readings/course content to provide a rationale for your selection (i.e., use the buzz words that are posted on the content page to help you).  Remember:  make your decision based ONLY on the title.

d.   At the beginning or end of the Literature Review and before the Methods section the researchers summarize what they found in earlier studies on the variables of interest.  Researchers use this summary to point out a GAP in the literature.  A gap is something that is still not understood, something that these researchers hope to understand after conducting their study. What GAP did these researchers point out in your study?  Please explain thoroughly.


5.  Study Design

a. Using your understanding of the multiple research study designs from your textbook and the content pages what research design was used for this study(i.e., provide a definition of the type of design that you identify in your research article)?

b. In your opinion, was this design appropriate for this study?  Why or why not? Base this on evidence from your readings and course content.

c. Were threats to internal validity (extraneous variables) controlled in this study?  How? Give specific examples.

d. Think of another extraneous variable that should/could be controlled in this study?  Why? (i.e., how do you think this variable might impact the DV or the outcome of the study?)

e.   How would YOU have controlled for this extraneous variable?




6.  Protection of Human Subjects in Research

a. Do the researchers indicate that they took steps to protect the human subjects that were a part of their study?  Identify what steps they took and how they describe their application to their research study.  Explain why this is necessary/required (I need to “hear” that you understand the meaning, importance and ethical necessity of these measures in your response.


7.Population, Sample, and Setting

a. Who/what is the POPULATION for this study?

NOTE: No matter what the researchers themselves tell you, do not be fooled into thinking that the sample is the population!  This is a common misconception in research studies!  Remember, the population is the complete set of persons or things possessing the characteristics that the researchers want to study/want to infer their results to.  At the end of the study (in the Discussion, Implications, Limitations sections) researchers discuss who their results are being inferred to or generalized to:  all of the smokers?  The overweight?  The grieving? These are often much different than the limited group they sampled:  i.e., a military base, hospitals in another country, etc. It is a very good idea to read these sections  before determining WHO or WHAT the population is for this studyYou might be very surprised!

b. Specifically, how was the SAMPLE selected from this population?  Exactly how did they select the actual participants – what was the process? (i.e., were there inclusion and exclusion criteria; how did they recruit the sample and determine who would become part of the study, or who would be excluded?)

c. How large was the sample?  Tell me the exact number that actually participated in the study #! (just give me ONE number here)

d. In your opinion does the sample “represent” the population for this study? (i.e., is the sample similar to the overall population that the researchers might want to infer the results to?)  How so? Explain your answer.

e. Where did this study take place:  Laboratory?  Clinic?   School?  Other?


8.    Data collection INSTRUMENT

Data are collected so that researchers can measure the impact that the IV has had on the DV. Instruments are used to collect these data. Are the instruments surveys of some type? Are they scales? Or telephones?  Questionnaires? Interviews?  Lab tests?  Other? Review carefully – remember instruments used may be more than things like thermometers, scales, etc.

a.   WHAT specific instrument/s was/were used to COLLECT DATA in this study? (I am looking for the instrument/s, not the data.). For example, what did the researchers use to measure/collect data for the study outcome (DV)?

b.   How did the researchers describe their data collection INSTRUMENT/s and how they used it/them? Be specific.

c.  Using definitions and descriptions of RELIABILITY presented on the content pages and the inter/intra observer reliability posting forum to begin your response, describe what steps were taken to assure that the data collected in this study are reliable?  (i.e., Were the observers calibrated?  Did they Standardize their procedures?  Was their data collection instrument calibrated? Standardized?  Tell me everything you see that suggests that these researchers tried to collect reliable data. Explain your answer thoroughly.


9.  Data analysis procedures.       Remember****: data analysis procedures are different than data collection instruments/tools!

a.   Define descriptive statistics and identify/name the DESCRIPTIVE statistics used to organize, describe and summarize data in this study.  Review the content pages on descriptive statistics, the posting forum on Statistics in the media AND Topics 47, 49, 50, 54, 55, 56 of your textbook before answering this criterion and be sure to incorporate the information into your response.

b.   Define inferential statistics and identify/name the INFERENTIAL statistics used to infer results to the population. Review the content pages on inferential statistics, Topics 47, 51, 58, 59, 60, 61 in your textbook AND the Statistics in the Media posting forum before answering this criterion.

c.   Google the name of ONE of the Inferential statistical tests mentioned in your research study.   What does your source say about this inferential test?   Why do you think your authors chose this inferential test for their study (think about the Levels of Measurement for example)?  Please explain. Please provide a link to your findings and do not forget to cite the information and write a reference page citation on this source, as well.


10.  Conclusions/implications/limitations

a. In light of this research experiment’s methods what do you think is the major strength of this study?  Please support your response thoroughly. This question refers to the study methods, not the purpose or outcome of the study.

b.   In YOUR opinion what is/are the study’s major limitation/s?  Please support your response thoroughly. Again, this question is related to the study design and methods – not the outcomes for the study.

c.   Please look at the Acknowledgment section, or any area of your study that acknowledges funding sources.  Review adjectives for the term BIAS (click your thesaurus in WORD). Define bias and indicate whether or not the funding source/s listed in this research study suggest that the study’s findings maybe biased.  Please defend your answer thoroughly and let me hear that you understand the research concept of bias in your response!




The new engl and jour nal of medicine

original article

A Communication Strategy and Brochure  for Relatives of Patients Dying in the ICU

Alexandre Lautrette, M.D., Michael Darmon, M.D., Bruno Megarbane, M.D., Ph.D.,

Luc Marie Joly, M.D., Sylvie Chevret, M.D., Ph.D., Christophe Adrie, M.D., Ph.D., Didier Barnoud, M.D., Gérard Bleichner, M.D., Cédric Bruel, M.D.,

Gérald Choukroun, M.D., J. Randall Curtis, M.D., M.P.H., Fabienne Fieux, M.D.,

Richard Galliot, M.D., Maité Garrouste-Orgeas, M.D., Hugues Georges, M.D.,

Dany Goldgran-Toledano, M.D., Mercé Jourdain, M.D., Ph.D., Georges Loubert, M.D., Jean Reignier, M.D., Fayçal Saidi, M.D., Bertrand Souweine, M.D., Ph.D.,

François Vincent, M.D., Nancy Kentish Barnes, Ph.D., Frédéric Pochard, M.D., Ph.D., Benoit Schlemmer, M.D., and Elie Azoulay, M.D., Ph.D.



There is a need for close communication with relatives of patients dying in the intensive care unit (ICU). We evaluated a format that included a proactive end-of-life conference and a brochure to see whether it could lessen the effects of bereavement.


Family members of 126 patients dying in 22 ICUs in France were randomly assigned to the intervention format or to the customary end-of-life conference. Participants were interviewed by telephone 90 days after the death with the use of the Impact of Event Scale (IES; scores range from 0, indicating no symptoms, to 75, indicating severe symptoms related to post-traumatic stress disorder [PTSD]) and the Hospital Anxiety and Depression Scale (HADS; subscale scores range from 0, indicating no distress, to 21, indicating maximum distress).


Participants in the intervention group had longer conferences than those in the control group (median, 30 minutes [interquartile range, 19 to 45] vs. 20 minutes [interquartile range, 15 to 30]; P<0.001) and spent more of the time talking (median, 14 minutes [interquartile range, 8 to 20] vs. 5 minutes [interquartile range, 5 to 10]). On day 90, the 56 participants in the intervention group who responded to the telephone interview had a significantly lower median IES score than the 52 participants in the control group (27 vs. 39, P = 0.02) and a lower prevalence of PTSD-related symptoms (45% vs. 69%, P = 0.01). The median HADS score was also lower in the intervention group (11, vs. 17 in the control group; P = 0.004), and symptoms of both anxiety and depression were less prevalent (anxiety, 45% vs. 67%; P = 0.02; depression, 29% vs. 56%; P = 0.003).


Providing relatives of patients who are dying in the ICU with a brochure on bereavement and using a proactive communication strategy that includes longer conferences and more time for family members to talk may lessen the burden of bereavement. ( number, NCT00331877.)

n engl j med 356;5       february 1, 2007

From the Saint-Louis Hospital and Paris 7 University, Assistance Publique — Hôpitaux de Paris (A.L., M.D., S.C., F.F., N.K.B., F.P., B.S., E.A.), Lariboisière Hospital (B.M.), Bichat-Claude Bernard Hospital

(C.B.), and Cochin Hospital and Paris 5 René Descartes University (G.C.) — all in Paris; Rouen University Hospital, Rouen (L.M.J.), Saint-Denis Hospital, Saint-Denis (C.A.), University Hospital, Grenoble (D.B.), Argenteuil Hospital, Argenteuil (G.B.),

Pontoise Hospital, Pontoise (R.G.), SaintJoseph Hospital, Saint-Joseph (M.G.-O.); Tourcoing Hospital, Tourcoing (H.G.), Gonesse Hospital, Gonesse (D.G.-T.), Salengro Hospital, Lille (M.J.), Raymond Poincaré Hospital, University of Versailles, Versailles (G.L.), La Roche sur Yon Hospital, La Roche sur Yon (J.R.), Nemours Hospital, Nemours (F.S.), Clermont-Ferrand Hospital, Clermont-Ferrand (B.S.), and Avicenne Hospital, Avicenne (F.V.) — all in France; and the School of Medicine, University of Washington, Seattle (J.R.C.). Address reprint requests to Dr. Azoulay at Service de Réanimation Médicale, Hôpital Saint-Louis, 1 Ave. Claude Vellefaux, 75010 Paris, France, or at elie.azoulay@

Last Updated on June 30, 2021