Sophisticated research design in chiropractic and manipulative therapy; “what you learn depends on how you ask.” Part A. Quantitative research: size does matter



Published May 25, 2016
Lyndon Amorin-Woods


Many chiropractors remain sceptical of evidence-based practice (EBP) and some may view it as an attack on the profession which they feel must be resisted. A counter-argument is centred on the primacy afforded quantitative methodology as epitomised by the randomised controlled trial (RCT). This defensive posture may be mitigated by recognising the role complex research has played in the legitimisation of the profession. The pre-eminence of the randomised controlled trial (RCT), considered by many as the gold-standard of evidence, has led some authors to go so far as to functionally disregard all evidence that is not an RCT. However, it is readily apparent the RCT is not always the most appropriate study design to gather evidence, especially in the CAM health sector. This paper discusses the role of sophisticated design in quantitative chiropractic research, presenting examples sequentially through the traditional quantitative hierarchy and concludes that optimal methodology depends on the research question. Research design must allow for the various dimensions of the (chiropractic) clinical encounter, and may be sophisticated at all levels, but must above all, be contextual. The ‘best available’ or most relevant evidence depends on what one needs for a specific purpose. A critical caution is the proviso that care must be exercised not to draw inappropriate conclusions such as causation from descriptive studies.
Abstract 243 | PDF Downloads 213



Evidence-Based Practice, Quantitative Evaluation, Research Design

1. Ioannidis J. Why most published research fi ndings are false. PLoS Med. 2005;2(8):e124.
2. Goldenberg M. Iconoclast or Creed? Objectivism, pragmatism, and the hierarchy of evidence. Perspectives in Biology and Medicine. 2009;52(2):168-87.
3. Creswell J. Qualitative inquiry and research design. Choosing among five approaches. 2nd ed: Sage Publications, Thousand Oaks.; 2007.
4. Tschanz D. The Arab Roots of European Medicine. Saudi Aramco World. 1997;48(3):20-31.
5. Ibn Sīnā (Avicenna). The Canon of Medicine (The Qanun) Persia. 1025.
6. Awofeso N. What’s New About the “New Public Health”? Am J Public Health. 2004(94):705-9.
7. Huber V. The Unification of the Globe by Disease? The International Sanitary Conferences on Cholera 1851-1894. The Historical Journal. 2006;49(2):453-76.
8. Barahona C. Randomised Control Trials for the Impact Evaluation of Development Initiatives: A Statistician’s Point of View. ILAC Working Paper 13, Rome, Italy: Institutional Learning and Change Initiative. 2010.
9. Fisher RA. Statistical Methods for Research Workers. Edinburgh and London: Oliver & Boyd; 1932.
10. Medical Research Council. A Framework for development and evaluation of RCTs for complex interventions to improve health. London: Medical Research Council. 2000.
11. Medical Research Council. Developing and evaluating complex interventions: new guidance. London: Medical Research Council. 2008.
12. Medical Research Council. Streptomycin treatment of pulmonary tuberculosis. BMJ (Clinical research ed). 1948;2:769-82.
13. Bronfort G, Haas M, Evans R, Leiniger B, Triano J. Effectiveness of Manual Therapies: The UK Evidence Report. Chiropr Osteopat. 2010;18:3.
14. Haneline M, Meeker WC., editors. Introduction to Public Health For Chiropractors: Jones And Bartlett Publishers; 2011.
15. Villanueva-Russell Y. Evidence-Based Medicine and its Implications for the Profession of Chiropractic. Social science and medicine. 2005;60:545-61.
16. Esteb W. RCT and the Cult of Scientism 2010. Available from:
17. Benson K. HA. A comparison of observational studies and randomized, controlled trials. New England Journal of Medicine. 2000;342(25):1878-86.
18. Concato J., Nirav-Shah, Horwitz RI. Randomized, controlled trials, observational studies and the hierarchy of research designs. The New England journal of medicine. 2000;342(25 ):1887-92.
19. Rosner A. Fables or foibles: inherent problems with RCTs. J Manip Physiol Ther. 2003;26(7):460-7.
20. Rosner A. Evidence-Based Medicine: Changing with the Tides 2009. Available from:[2/04/2011.
21. Kent C. An Analysis of the General Chiropractic Council's Policy on Claims Made for the Vertebral Subluxation Complex. Foundation for Vertebral Subluxation. 2010(June 15):1-8.
22. SUNY Downstate Medical Center. Evidence Based Medicine Course. A Guide to Research Methods: The Evidence Pyramid: Medical Research Library of Brooklyn; 2001. Available from:
23. Health Sciences Library. Evidence-Based Practice Tools Seattle.WA: University of Washington; 2011. Available from:
24. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401-6.
25. Oxford Centre for Evidence-based Medicine: University of Oxford; 2011. Available from:
26. Ebell M, Siwek J, Weiss B, Woolf S, Susman J, Ewigman B, et al. Strength of Recommendation Taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature. Am Fam Physician. 2004;69:549-57.
27. Eddy DM. Evidence-Based Medicine: A Unified Approach. Health Affairs. 2005;24(1):9-17.
28. Guyatt G, Oxman A, Kunz R, Falck-Ytter Y, Vist G, Liberati A, et al. Rating quality of evidence and strength of recommendations: Going from evidence to recommendations. BMJ. 2008;336(7652):1049-51.
29. Schunemann H, Best D, Vist G, Oxman A. Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations. CMAJ. 2003;169:677-80.
30. Parascandola M, Weed DL. Causation in epidemiology. Journal of Epidemiology and Community Health. 2001;55(12):905-12.
31. Tonelli M. In defense of expert opinion. Acad Med. 1999;74(11):1187-92.
32. Enkin MW, Jadad AR. Using anecdotal information in evidence-based health care: Heresy or necessity? Annals of Oncology. 1998;9(9):963-6.
33. Lomas J, Enkin M, Anderson G, et al. Opinion leaders vs. audit and feedback to implement practice guidelines. JAMA. 1991;265:2202-7.
34. Vignais P, Vignais P. Discovering Life, Manufacturing Life: How the experimental method shaped life sciences. Berlin: Springer; 2010.
35. Rothman SS. Lessons from the living cell: the culture of science and the limits of reductionism. New York: McGraw-Hill; 2002.
36. Song XJ, Gan Q, Cao J-L, Wang Z-B, Rupert RL. Spinal manipulation reduces pain and hyperalgesia after lumbar intervertebral foramen inflammation in the rat. J Manip Physiol Ther. 2006;29(1):5-13.
37. Gordis L. Epidemiology. 4th ed. Philadelphia, PA: Saunders (Elsevier); 2008.
38. Jordan W, Anand J. Pulmonary embolism. Lancet. 1961 Nov 18;278(7212):1146-7.
39. Centers for Disease Control (CDC). Pneumocystis pneumonia--Los Angeles. MMWR Morb Mortal Wkly Rep. 1981 Jun 5;30(21):250-2.
40. Lafond D, Champagne A, Cadieux R, Descarreaux M. Rehabilitation program for traumatic chronic cervical pain associated with unsteadiness: a single case study. Chiropractic & Osteopathy. 2008;16(1):15.
41. BenEliyahu DJ. Magnetic resonance imaging and clinical follow-up: study of 27 patients receiving chiropractic care for cervical and lumbar disc herniations. Journal of manipulative and physiological therapeutics. 1996;19(9):597-606.
42. Smith M, Carber L. Chiropractic Health Care in Health Professional Shortage Areas in the United States. American Journal of Public Health. 2002;92(12):2001-9.
43. Cassidy JD, Boyle E, Cote P, He Y, Hogg-Johnson S, Silver F. Risk of vertebrobasilar stroke and chiropractic care: results of a population-based case-control and case-crossover study. Spine. 2008;33:S176 - S83.
44. Rochon P, Gurwitz J, Sykora K, Mamdani M, Streiner D, Garfinkel S, et al. Reader's guide to critical appraisal of cohort studies: 1. Role and design. BMJ (Clinical research ed). 2005;330(7496):895-7.
45. Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet. 2002;359(248-52).
46. Miller JE, Benfield K. Adverse Effects of Spinal Manipulative Therapy in Children Younger Than 3 Years: A Retrospective Study in a Chiropractic Teaching Clinic. J Manip Physiol Ther. 2008;31(6):419-23.
47. Euser A, Zoccali C, Jager K, Dekker F. Cohort Studies: Prospective versus Retrospective. Nephron Clin Pract. 2009;113:c214-c7.
48. Cifuentes M, Willetts J, Wasiak R. Health Maintenance Care in Work-Related Low Back Pain and Its Association With Disability Recurrence. Journal of Occupational and Environmental Medicine. 2011;53(4):396-404 10.1097/JOM.0b013e31820f3863.
49. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Br Med J. 2010; 340:c869.
50. CONSORT GROUP. The CONSORT Statement 2010. Available from:
51. Bishop PB, Quon JA, Fisher CG, Dvorak MFS. The Chiropractic Hospital-based Interventions Research Outcomes (CHIRO) Study: a randomized controlled trial on the effectiveness of clinical practice guidelines in the medical and chiropractic management of patients with acute mechanical low back pain. Spine J. 2010;10(12):1055-64.
52. Centre for Evidence Based Medicine. What is EBM? 2009. Available from:
53. Green S, Higgins JP, Alderson P, Clarke M, Mulrow CD, Oxman AD. Part 1 Cochrane Reviews: The Cochrane Collaboration; 2011 [updated 2011]. Available from:
54. Moher D., Liberati A., Tetzlaff J., Altman DG., The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009;6(7):e1000097.
55. Hawk C., Khorsan R., Lisi AJ., Ferrance RJ., Evans MW. Chiropractic care for nonmusculoskeletal conditions: a systematic review with implications for whole systems research. J Altern Complement Med. 2007;13(5):491-512.
56. Goertz M, Thorson D, Bonsell J, Bonte B, Campbel lR, Haake B, et al. Institute for Clinical Systems Improvement. Adult Acute and Subacute Low Back Pain. Updated. ICSI. November 2012.
57. Sackett DL., Rosenberg WMC., Muir Gray JA., Haynes RB., Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ (Clinical research ed). 1996;312(7023):71-2.
58. Diamond J, Robinson JA. Natural Experiments of History. Cambridge, MA: Harvard University Press; 2010.
59. Manchikanti L, Boswell M, Giordano J. Evidence-Based Interventional Pain Management: Principles, Problems, Potential and Applications. Pain Physician. 2007;10:329-56.
60. Amorin-Woods LG, Beck RW, Parkin-Smith GF, Lougheed J, Bremner AP. Adherence to clinical practice guidelines among three primary contact professions: a best evidence synthesis of the literature for the management of acute and subacute low back pain. J Can Chiropr Assoc 2014 (Sept);58(3):220-37.