Table 3 Modified downs and black checklist (based on (MacLehose et al. 2000))
Item | Criteria | Score |
|---|---|---|
Reporting | ||
1 | Is the hypothesis/aim/objective of the study clearly described? | Yes = 1 No = 0 |
2 | Are the main outcomes to be measured clearly described in the Introduction or Methods section? If the primary outcomes are first mentioned in the Results section, the question should be answered no. | Yes = 1 No = 0 |
3 | Are the characteristics of the patients included in the study clearly described? In cohort studies and trials, inclusion or exclusion criteria should be given. In case-control studies, a case-definition and the source for controls should be given. | Yes = 1 No = 0 |
4 | Are the interventions of interest clearly described? Treatments and placebo (where relevant) that are to be compared should be clearly described. | Yes = 1 No = 0 |
5 | Are the distributions of principal confounders in each group of subjects to be compared clearly described? A list of principal confounders is provided. | Yes = 1 No = 0 |
6 | Are the main findings of the study clearly described? Simple outcome data (including denominators and numerators) should be reported for all significant findings so that the reader can check the major analyses and conclusions. (This question does not cover statistical tests, which are considered below). | Yes = 1 No = 0 |
7 | Does the study provide estimates of the random variability in the data for the primary outcomes? In non-normally distributed data the inter-quartile range of results should be reported. In normally distributed data, the standard error, standard deviation or confidence intervals should be reported. If the data distribution is not described, it must be assumed that the estimates used were appropriate, and the question should be answered yes. | Yes = 1 No = 0 |
8 | Have all significant adverse events that may be a consequence of the intervention been reported? This should be answered yes if the study demonstrates a comprehensive attempt to measure adverse events. (A list of possible adverse events is provided). | Yes = 1 No = 0 |
9 | Have the characteristics of patients lost to follow-up been described? This should be answered yes where there were no losses to follow-up or where losses to follow-up were so small that findings would be unaffected by their inclusion. This should be answered no, where a study does not report the number of patients lost to follow-up. | Yes = 1 No = 0 |
10 | Have actual probability values been reported (e.g. 0.035 rather than < 0.05) for the primary outcomes except where the probability value is less than 0.001? | Yes = 1 No = 0 |
External validity | ||
11 | Were the subjects asked to participate in the study representative of the entire population from which they were recruited? The study must identify the source population for patients and describe how the patients were selected. Patients would represent the entire source population, an unselected sample of consecutive patients, or a random sample. Random sampling is only feasible where a list of all members of the relevant population exists. A study does not report the proportion of the source population from which the patients are derived; the question should be answered as unable to determine. | Yes = 1 No = 0 Unable to Determine = 0 |
12 | Were those subjects who were prepared to participate representative of the entire population from which they were recruited? The proportion of those asked who agreed should be stated. Validation that the sample was representative would include demonstrating that the main confounding factors' distribution was the same in the study sample and the source population. | Yes = 1 No = 0 Unable to Determine = 0 |
13 | Were the staff, places, and facilities where the patients were treated representative of most patients' treatment? For the question to be answered yes, the study should demonstrate that the intervention was representative of that in use in the source population. The question should be answered no if, for example, the intervention was undertaken in a specialist center unrepresentative of the hospitals most of the source population would attend. | Yes = 1 No = 0 Unable to Determine = 0 |
Internal validity–bias | ||
14 | Was an attempt made to blind study subjects to the intervention they have received? For studies where the patients would have no way of knowing which intervention they received, this should be answered yes. | Yes = 1 No = 0 Unable to Determine = 0 |
15 | Was an attempt made to blind those measuring the primary outcomes of the intervention? | Yes = 1 No = 0 Unable to Determine = 0 |
16 | If any of the study results were based on “data dredging,” was this made clear? Any analyses that had not been planned at the outset of the study should be indicated. If no retrospective unplanned subgroup analyses were reported, then answer yes. | Yes = 1 No = 0 Unable to Determine = 0 |
17 | In trials and cohort studies, do the analyses adjust for different lengths of patients’ follow-up, or in case-control studies, is the period between the intervention and outcome the same for cases and controls? Where follow-up was the same for all study patients, the answer should yes. If different follow-up lengths were adjusted for, for example, survival analysis, the answer should be yes. Studies where differences in follow-up are ignored, should be answered no. | Yes = 1 No = 0 Unable to Determine = 0 |
18 | Were the statistical tests used to assess the primary outcomes appropriate? The statistical techniques used must be appropriate to the data. For example, nonparametric methods should be used for small sample sizes. Where little statistical analysis has been undertaken but no evidence of bias, the question should be answered yes. If the data distribution (normal or not) is not described, it must be assumed that the estimates used were appropriate, and the question should be answered yes. | Yes = 1 No = 0 Unable to Determine = 0 |
19 | Was compliance with the intervention/s reliable? There was non-compliance with the allocated treatment or contamination of one group. The question should be answered no. For studies where the effect of any misclassification was likely to bias any association to the null, the question should be answered yes. | Yes = 1 No = 0 Unable to Determine = 0 |
20 | Were the primary outcome measures used accurately (valid and reliable)? For studies where the outcome measures are clearly described, the question should be answered yes. For studies that refer to other work or demonstrate that the outcome measures are accurate, the question should be answered yes. | Yes = 1 No = 0 Unable to Determine = 0 |
Internal validity–confounding (selection bias) | ||
21 | Were the patients in different intervention groups (trials and cohort studies), or were the cases and controls (case-control studies) recruited from the same population? For example, patients for all comparison groups should be selected from the same hospital. The question should be answered, unable to determine for cohort and case-control studies where there is no information concerning the source of patients included in the study. | Yes = 1 No = 0 Unable to Determine = 0 |
22 | Were study subjects in different intervention groups (trials and cohort studies), or were the cases and controls (case-control studies) recruited over the same period? For a study that does not specify the period over which patients were recruited, the question should be answered as unable to determine. | Yes = 1 No = 0 Unable to Determine = 0 |
23 | Were study subjects randomized to intervention groups? Studies that state that subjects were randomized should be answered yes except where randomization would not ensure random allocation. For example, the alternate allocation would score no because it is predictable. | Yes = 1 No = 0 Unable to Determine = 0 |
24 | Was the randomized intervention assignment concealed from both patients and health care staff until recruitment was complete and irrevocable? All non-randomized studies should be answered no. If the assignment was concealed from patients but not from staff, it should be answered no. | Yes = 1 No = 0 Unable to Determine = 0 |
25 | Was there an adequate adjustment for confounding in the analyses from which the main findings were drawn? This question should be answered no for trials if: the main conclusions of the study were based on analyses of treatment rather than an intention to treat; the distribution of known confounders in the different treatment groups was not described, or the distribution of known confounders differed between the treatment groups but was not taken into account in the analyses. In non-randomized studies, if the effect of the main confounders was not investigated or confounding was demonstrated. Still, no adjustment was made in the final analyses. The question should be answered as no. | Yes = 1 No = 0 Unable to Determine = 0 |
26 | Were losses of patients to follow-up taken into account? If the numbers of patients lost to follow-up are not reported, the question should be answered as unable to determine. If the proportion lost to follow-up was too small to affect the main findings, the question should be answered yes. | Yes = 1 No = 0 Unable to Determine = 0 |
Power | ||
27a | Did the study have sufficient power to detect a clinically significant effect where the probability value for a difference is due to chance is less than 5%? Sample sizes have been calculated to detect a difference of x% and y%. | Yes 1 No 0 Unable to Determine 0 |