No 72: How To Read A Paper Template
A quick guide for clinicians on systematically reviewing a scientific paper
There are hundreds of articles, webpages and books on how to read a paper.
However, most of them are really wordy. So this post will quickly summarise the important details and then provide a template that can be filled in.
Unfortunately, even with the step by step template, you may still need to spend an hour reading and analysing the paper. There is no shortcut to actually putting in the effort and brain power.
How to Review a Paper by Dereck Alderson
Power Calculations explained by the EMJ
https://www.gigacalculator.com/calculators/power-sample-size-calculator.php
If you have read this far, then you probably don’t need to be told that learning how to “systematically” review a paper is very important and probably one of the most useful skills for a competent scientist or medical professional to develop.
Why is it important? Because understanding the flaws, biases, spin and downright fabrications written in “scientific” publications is the essence of evidence based medicine and understanding if what you are reading is relevant to you and applicable to your patients.
Do you trust this paper enough to change the treatment of your patient?
The quickest way to summarise a research question is:
PICO = The patient or population, intervention, comparison, and outcome.
The below template has been made so that you can copy+paste this into a word document, work through a paper and then easily present it to your supervisor or journal club. Good luck.
The Quick Guide to How to Read a Paper
1. Find a paper
2. Skim the abstract
3. If it seems relevant to you then skim the article
4. The first skim involves noting the stated aim, study design, primary outcome and the major points in the discussion.
5. Review each of the tables or graphs – these should be stand alone and understandable without reading the text.
6. Thorough review
7. Read each step of the methods, note the specifics of the study design and statistical method. A highlighter comes in very handy!
8. Does it make sense? Any obvious flaws or biases?
9. Is the study population similar to your patient population?
10. Read the results, are they statistically significant? Are they clinically significant?
11. If you want to know more about the subject then read the introduction and discussion.
12. Do you agree with the authors analysis and conclusion?
13. Was the study designed well, did it produce rigorous results and are they applicable?
*** Top tip: if you are doing a “lit review” and looking for a relevant paper then go to PubMed, set up your search term, got to “display options” and click “abstract” and “50 results”, then “Ctrl+F” so that you can search all of the papers for a key word, then scroll down looking at the highlighted words and you can easily see which abstracts are most relevant to you.
The Quick Template
· Title
· Authors and affiliates:
· Journal citation/link:
· Your quick PICO summary –
· Relevance – for easy reference rate the paper here 1/5
Case report
Case series
Expert opinion
Case-control or cohort study or cross-sectional study
Literature review
Non-randomised controlled trial / Natural experiment
Randomised controlled trial (RCT)
Double blind RCT (individual or cluster randomisation?)
Systematic review with meta-analysis (not always more useful than a well done RCT)
· Why was the study done?
· Is it scientifically credible?
· Is this a novel subject?
· Is it a worthwhile subject?
· Was the study protocol published prior to being conducted?
· Was the study registered as an RCT or systematic review prior to being conducted?
· Did the study follow a standardised protocol? PRISMA, CONSORT etc?
· What is the primary aim?
· What are the secondary aims?
· What was the null hypothesis?
· If the study was a trial, was it designed to show equivalence to gold standard, improvement over placebo or improvement over gold standard?
· If it was an early phase study was it designed to show safety or efficacy or feasibility?
· Who was studied?
· What were the selection criteria / eligibility criteria?
· If its an observational study, was the exposure clearly stated?
· Was the control group well defined?
· How were subjects recruited to the study?
· Are these subjects representative of the general population?
· Was the study setting clearly described?
· Were patients involved in the study design?
· How many subjects were involved?
· What was the Power of the study?
· Does the power calculation look reliable? Under-powered or over-powered?
· How was the study done?
· Were subjects randomised?
· Was the randomisation concealed and successful?
· What type of randomisation was used? Individual or cluster or none?
· Was analysis based on randomisation and intention to treat?
· Was the study blinded? No, single or double blinded?
· What interventions were involved?
· Was the control and intervention group treated similarly?
· Was the control group given gold standard treatment?
· Was compliance measured?
· How long was the study?
· How long was the recruitment phase?
· How long was the follow up?
· Was this time adequate for the study?
· Was the study stopped early?
· What outcome measures were used?
· Was the primary outcome measure appropriate and clinically relevant?
· Were the outcome measures clearly defined and reproducible?
· Were benefits, harms and costs stated?
· Was outcome assessment blinded?
· If it is an RCT, have the standardised CONSORT reporting measures been stated?
· What statistical measures were used to analyse the outcomes?
· What measures of precision were used?
· Were numerators and denominators clear?
· Was there sub-group analysis? Was this pre-planned?
· Mean, median, mode, ANOVA, regression, t-test, chi-squared etc
· Were the P-value, confidence interval or hazard ration statistically significant?
· Were tables and graphs clear?
· Is the demographics table clear?
· Were subjects lost to follow up? Was this comparable between the control and intervention groups?
· Was data missing?
· If the result was not significant, is this because the study was under-powered?
· What did the authors state the study limitations were?
· Are there any other limitations they have not included?
· Are there any confusing aspects about the study?
· Have you spotted any methodological or analytical errors?
· Are their references accurate and correct?
· What is the significance of the study?
· Does the study have external validity to the “general population” or is it only relevant to a niche group?
· Was there any conflict of interest?
· Was funding declared?
Slightly different standardised questions need to be applied to different study types such as: systematic reviews, lab studies, prognosis studies, diagnostic studies
Systematic Reviews
· Most of the above questions still apply.
· Was the search criteria clearly defined?
· Was study inclusion criteria clear and appropriate?
· Was the search strategy comprehensive and complete?
· Were the included papers graded for bias and validity?
· What were the exposures and comparisons?
· Were randomised controlled trials included?
· Have the biases been cleared explored for each paper?
· How was confounding dealt with?
· Were the studies comparable?
· Were there similar populations and similar outcome measures?
· Could useful summary effects estimates be calculated?
· Is there a box plot?
· Were the studies adequately powered?
· Were effect estimates consistent?
· Is there publication bias?
· Has a Forest plot been done?