PhD Proposal

Name of applicant: Austin Plunkett

Host institution: Queen Mary University London

Title of research proposal: Quality improvement by Distribution Of Computer-AssisTed Evidence Synopses

Short title: Quality aDvOCATES

Abstract

Musculoskeletal (MSK) conditions are the largest contributor to the burden of disability in the UK, but uptake of evidence by manual therapists who see patients with these conditions remains limited. Policy makers in government and professional bodies, as well as patient representatives, expect these professions to demonstrate evidence-informed practice. However, tight budgetary constraints, diverse working practices, and the complexity of primary care make it challenging to improve the uptake of evidence in these professions.

This demands innovative solutions in the production and dissemination of research evidence to reduce this well-recognised “research-to-practice gap” and to streamline the research-to-practice pipeline. To do this we propose a PhD project to:

  1. Explore, through reviewing the literature, clinician beliefs and attitudes towards evidence to understand barriers and enhancers to receipt and application of information (scoping review).
  2. Explore new and novel ways of encouraging the adoption of evidence informed practice in particular reference to behaviour change theories (scoping review).
  3. Develop IT solutions (including machine learning approaches) to target and source information relevant to osteopathic practice and care.
  4. Based on 1) and 2) develop and devise materials and strategies suited to, and acceptable for, dissemination and uptake by clinicians (feasibility study).
  5. Evaluate, with a sample of osteopaths: receipt and application to practice and impact on patient care (prospective cohort study).

This project has the following aims:

  1. to understand the value that manual therapists and their patients place on priority research topics;
  2. to design, implement and evaluate a programme to increase this perceived value through dissemination via peer advocates;
  3. to overcome bottlenecks in the discovery of relevant research and production of evidence synopses that are agreeable to clinicians.

Ultimately, this will streamline the research-to-practice pipeline and increase the evidential component in interventions delivered by manual therapists, thereby improving patient care and patient perceived benefit.

This project will establish a growing database of evidence synopses on priority clinical topics, and a network of evidence advocates within the osteopathic profession.

Introduction

Background and overview of the literature

The ‘research-to-practice gap’ describes the well-recognised struggle to put evidence into effective practice. Glasziou and Haynes (2005) use a ‘pipeline’ analogy and describe evidence ‘leaking’ at all stages along this pipeline. An estimated 85% of research is squandered (Chalmers and Glasziou, 2009) making ‘research waste’ a priority problem (Chalmers, Bracke, et al., 2014). Although the clinical encounter is complex and dynamic (Lau, Stevenson, et al., 2015) the pipeline model enables identification of ‘leakage points’ that may be targeted. Specifically, the student will target the following:

Bottlenecks

The pace at which primary research is generated outstrips the capacity for reviewers to summarise it (Tsafnat, Glasziou, et al., 2014; Olah and Carter, 2017). It is recognised that reviews need to be prioritised and streamlined if health-care professionals are to maintain an up-to-date evidence base (Bastian, Glasziou, et al., 2010). Despite this, reviews are often out of date due to the resources required to produce them and the high associated cost (Lau, Stevenson, et al., 2015). Innovation is required to overcome these issues (Bastian, Glasziou, et al., 2010). Computers can increase the efficiency of processing papers for review (Chalmers, Bracke, et al., 2014); specifically, machine learning may accelerate the discovery and categorisation of relevant evidence (Tsafnat, Glasziou, et al., 2014; O’Mara-Eves, Thomas, et al., 2015; Mo, Kontonatsios, et al., 2015). Searching databases, “sifting” abstracts, dedpulicating papers, organising papers into topics, and summarising research may all be accelerated, thereby helping to address this bottleneck.

Using machine learning to enable behaviour change strategies is a novel approach in the Allied Health domain. However, the approach of combining human and machine efforts into efficient workflows is being spearheaded by several authorities (Michie, West, et al., 2018; Michie, Thomas, et al., 2017; Thomas, Noel-Storr, et al., 2017). The student will explore whether the time and resources required to review clinically-pertinent literature can be significantly reduced.

Clinician awareness: impact and implementation

“[A]ctionable, accessible, and trustworthy information” can help clinicians bring research into their decision making processes (NIHR Dissemination Centre, 2016). However, passive dissemination alone has only minimal influence on professional behaviour (Feder, Eccles, et al., 1999; Côté, Durand, et al., 2009). Instead, influencing leaders within a profession, and members of professional communities and organisations, may be key to impacting on practitioner beliefs (Aarons, Green, et al., 2016; McCaughan, Thompson, et al., 2002; Kasiri-Martino and Bright, 2016; Dannapfel, Peolsson, et al., 2013). Identification and recruitment of advocates can increase the incorporation of evidence into clinical practise (Stevans, Bise, et al., 2015; Rowley, 2012) through improved “social validation” i.e. acceptance of change within a peer group (Glasziou and Haynes, 2005) and strong “social congruence” i.e. similarity of experiences and practices (Lockspeiser, O’Sullivan, et al., 2018). Recognition of clinician involvement in research activities encourages a positive attitude towards learning, and the development of a research culture (Royal College of Physicians, 2018).

Any attempt to bridge the gap between evidence-based care and clinician practises should measure impact and reference appropriate theoretical justification (Vachon, Désorcy, et al., 2013).

Clinician beliefs and attitudes towards evidence

While primary care may be improved by clinicians’ reading of evidence summaries (Schneider, Evans, et al., 2015), clinician attitudes are key to successful introduction (Côté, Durand, et al., 2009). Manual therapists may not follow advice in summaries or guidelines for various reasons, including:

Presentation of research in a format that clinicians find agreeable can help overcome these barriers (Wallace, Byrne, et al., 2014; McCaughan, Thompson, et al., 2002) and brief overviews may be more palatable than lengthier formats (Grimshaw, Eccles, et al., 2012). There are numerous potential candidate formats, since summary formats and rapid methodologies are under development by many organisations (Tricco, Zarin, et al., 2015). Consequently the student will conduct a systematic narrative review of the evidence to determine which methodology may address clinician-related barriers. Systematic narrative reviews combine a systematic search with a narrative critical analysis, aiming to mitigate bias in the selection of papers while permitting a narrative approach for producing insights and recommendations (Weed, 2005; Snilstveit, Oliver, et al., 2012).

Role of musculoskeletal therapists and the burden of musculoskeletal disease

People with musculoskeletal (MSK) conditions seek treatment from manual therapists including physiotherapists and osteopaths, who are state-regulated Allied Health professionals. MSK conditions are the largest contributor to the burden of disability in the UK (Arthritis Research UK, 2016) costing the UK economy an annual 8.9 million lost work days (Health and Safety Executive, 2016) estimated at £7.4 billion per year (Chartered Society of Physiotherapy, 2012). MSK disease may be the second most common cause of GP visits, accounting for over 20% of their workload (Arthritis Research UK, 2009).

This burden emphasises the importance of finding efficiencies in the evidence-to-practice pipeline, particularly regarding manual therapies.

Method and design

This will be a mixed methods project combining systematic scoping reviews to inform the design, modelling and testing of a complex intervention (a prototype tool for improving database searches, using machine learning techniques with a dissemination strategy) to promote evidence informed practice.

Phase I: Investigation

  1. Explore, through reviewing the literature, clinician beliefs and attitudes towards evidence to understand barriers and enhancers to receipt and application of information (Scoping review).
  2. Explore new and novel ways of encouraging the adoption of evidence informed practice in particular reference to behaviour change theories (scoping review).

Phase II: Development

  1. Develop IT solutions (including machine learning approaches) to target and source information relevant to osteopathic practice and care. This will probably take the form of a pilot system for conducting semi-automated literature searches and topic analysis. This will feature elements of machine learning, including query expansion techniques using word2vec (a neural network that maps words from a given corpus into a numerical ‘vector space’) and unsupervised topic allocation using latent Dirichlet allocation (a Bayesian approach to modelling the probability that any topic represents a given document), or similar approaches.

Phase III: Model and finalise complex intervention

  1. Develop conceptual models based on phase I and II for potential strategies for dissemination and utilisation of research in practice.
  2. Develop and devise material and strategies suitable and acceptable for dissemination and uptake by clinicians (feasibility study).

Phase IV: Implement and evaluate the intervention

  1. Evaluate, on a sample of osteopaths: receipt and application to practice and impact on patient care (prospective cohort study).
  2. Assess the impact of intervention on clinical practice, through reassessment of clinician attitudes towards evidence based practice and analysis of patient experience and care.

Surveys will aim to recruit in the region of 50-100 clinicians and 50-100 patients. Qualitative interviews will be conducted with 15-30 patients and clinicians, or until data saturation. Malterud, Siersma, et al. (2018) indicate that focused, specific, theory-driven qualitative analysis increases the “information power” of such sample sizes. They emphasise “sample specificity… established theory… data quality, and variability of relevant events” as important factors in understanding the “contribution of new knowledge from the analysis” (ibid).

Time period

This project will be achieved over a 3 year period in-line with the QMUL regulations set for a 3 year full-time PhD programme.

Table 1: Gantt chart showing key milestones

Final output

This research project will take a pragmatic approach, and some elements will be put into immediate practice. The information collected will enable the manual therapy Allied Health Professions to identify mismatches between the expectations of researchers, clinicians and patients regarding evidence-informed practice. This will provide valuable detail for regulators, policy makers and health care commissioners who aim to improve the rate at which evidence is adopted into practice. It will also highlight areas where further research would be beneficial.

It is hoped that this project will provide a template for others who are aiming to improve the efficiency of secondary research, and the impact of research on clinical practice.

Ultimately, this approach will enable the student to create a library of regularly updated synopses for manual therapy Allied Health Professions, and a growing network of clinicians who will advocate within their professions. This approach will be readily adaptable for use by other health-care professions, and generalisable across other practice settings including NHS and private clinics.

Student learning requirements

The student’s learning requirements are summarised here:

Methodologies

Machine learning

QI programme development

Research management

The student will be based within the Complex Intervention and Social Practice in Health Care (CISPHC) Unit located in the Centre for Primary care and Public Health (CPCPH). CPCPH is multidisciplinary, bringing together academics from a range of backgrounds including general practice, medical ethics, nursing, statistics, public health, sociology and psychology, and involved in research, teaching and service development. The Centre was rated fourth in the UK in health services research in the most recent research assessment exercise, with a high proportion of research output rated as internationally leading or of internationally excellent quality. The student would have the benefit of working in this highly specialised team who come from different backgrounds with many areas of expertise.

Within this vibrant and growing Centre, the student will have an exciting and supportive environment for learning and conducting research. There is an active weekly seminar programme where students are encouraged to present, and we have available a range of academic experts and clinicians with experience of dealing with chronic pain who will be able to advise and help the student. We have an established relationship with the Persistent Pain Service at Mile End Hospital and have worked with them closely for the past 4 years. Over the past 3 the Blizard Institute where the Centre for Primary Care and Public Health is based, has established a Graduate Studies Programme and support system with informal PhD student lunches and seminars to ensure high standards of supervision and encourage student interaction and mutual learning.

Timeframe

Start: November 2018

Finish: October 2021

9 month review: August 2019

18 month M Phil upgrade viva: May 2019

30 month viva: May 2020

Governance

Supervisors will comply with national and School of Medicine and Dentistry policies for best practice regarding supervision, feedback, assessment, student representation, progress and review.

Ethical considerations

Ethics approval requirements have been assessed by the Queen Mary University of London (QMUL) Research Ethics team and using the Medical Research Council Health Research Authority study research decision tool (Medical Research Council, 2017). NHS Research Ethics Committee approval is not required. The following ethical considerations are noted:

Despite participants being self-selecting, there is a small risk that poor clinical practise might be identified among participants. As part of the programme, all participants will be encouraged to reflect upon their practice and will be supported through regular meetings, existing professional support materials including regulatory guidance. Participants will also be encouraged to present their synopses to their peer-groups. Due to the anonymisation measures described above, it will not be possible to identify clinicians who demonstrate poor practises. However, through the repeated use of the attitude survey described in Phase IV, changes in attitudes towards evidence-informed practice will be evident. This information will be fed back to the profession, including to the regulator, the professional members’ organisation, and the educational institutions.

The student will retain editorial control of all summaries. Only summaries that are deemed of acceptable quality will be made public by the student. Advocate participants may be encouraged to publicise their involvement, but formal outputs will only be made public after this review process.

The advocate handbook produced in Phase III will help ensure the process is well-controlled.

Supervisor meetings

A schedule of meetings will be drawn up by the supervisors and student. Following medical school policy, the student and their principal supervisor will meet at least monthly but in the first year of the studentship we plan weekly hour-long meetings between the student and supervisors. The student and supervisors will keep written, agreed, records of all meetings. Supervisors will be available between meetings by e-mail, and for informal meetings as necessary. The student can expect to receive feedback on short written pieces of work within a week and feedback on longer pieces of work (e.g. thesis chapters) within two weeks.

Training and development

The principal supervisor will be responsible for discussing training needs with the student at regular intervals throughout the project and drawing up a training plan together with the student which ensures that they complete the 70 hours per annum transferable skills training recommended by QMUL’s training guide, and as part of the QMUL skills-based points system (Researcher Development Department, 2018). The principal supervisor will also take responsibility for identifying suitable training. Training will include the induction programme run by the School of Medicine and Dentistry, informal training within CPCPH and formal training inside and outside the School.

QMUL has a strong reputation for innovation through inter-disciplinary and cross-faculty collaboration. The Research Excellence Framework 2014 rated QMUL 5th in the UK for “world-leading or internationally excellent” research outputs, and HEFCE 2013 noted that QMUL has the highest rate for timely PhD completion of any UK university.

Research methods

The CPCPH has expertise in qualitative research methods which are taught in seminars and as part of bachelor- and masters-level courses (School of Medicine and Dentistry, 2018). The student could attend modules from these courses to re-familiarise themselves with research approaches and gain new knowledge. This will prepare the student for mixed-methods approaches and for qualitative data analysis including thematic analysis.

Statistical training

Statistical training will be given on the Research methods course but where necessary we will provide specialist training with one or two of our own statisticians from the centre for health sciences. Additionally the Royal Statistical Society run a variety of training courses throughout the year that may be of use.

Systematic narrative and critical literature reviewing

Within CPCPH we have considerable expertise in different aspects of conducting reviews of methodology, including search strategies, data extraction, issues with identifying methodology from papers, and presenting results. The student will receive appropriate training from CPCPH staff and if staff are unavailable we will organise attendance on one of the many courses available in this field.

Machine learning and computational training

The Faculty of Science and Engineering provides expertise and teaching in machine learning and related fields, and The Blizard Institute, where the CPCPH is based, has relationships with this faculty through the programmes including Queen Mary Innovation and the “CANBUILD - deconstructing cancer” project. The student will be able to access modules relating to the computational element of this PhD, contributing to the existing reputation the Blizard Institute has for interdisciplinary research.

General training

The university runs excellent courses for the software applications they will need to be proficient in, for example Endnote, Microsoft Word and PowerPoint. We can also include the PhD student in our M.Sc. for Global Health and Innovation modules such as: introductions to the library and searching electronic data bases. Additionally, there are excellent short course in using statistical software packages. This will also help foster a working team spirit within the CPCPH.

Monitoring progress

The School of Medicine and Dentistry policy will be followed, with a formal progress report and oral examination at 9 months (student progress and plans and a short critical appraisal assessed by supervisors and the Graduate Studies Committee) and transfer from MPhil to PhD within 18 months (via 10,000 word report and viva by two assessors, one external to the Centre). There will be a further report to supervisors at 30 months. Transfer to writing-up status will occur at 36 months. It will be made clear that the student should aim to submit their thesis at about 39 months to enable their PhD award within 4 years of commencing their studentship. At each milestone the student’s personal development plan will be monitored to ensure that they have received the recommended 70 hours per annum transferable skills training.

Finances and funding

Projected costs

Item Cost
PhD fees for three years £15,000
Computing resource for machine learning £2,500
Website and data hosting £2,000
TOTAL £19,500

The student is currently a QMUL employee. Living expenses totalling £54,000 over three years will be provided to the student via their salary as part of a research grant, cost centre code GPPH1F9R.

Supervision

Supervisor 1: Dr Dawn Carnes [email protected]

Supervisor 2: Dr Carol Fawkes [email protected]

Supervisor 3: Dr Anthony Constantinou [email protected]

References

Aarons, G., Green, A., Trott, E., Willging, C., Torres, E., Ehrhart, M., and Roesch, S. (2016) The Roles of System and Organizational Leadership in System-Wide Evidence-Based Intervention Sustainment: A Mixed-Method Study. Administration and Policy in Mental Health and Mental Health Services Research, 1–8. [online] http://link.springer.com/article/10.1007\%2Fs10488-016-0751-4.

Arthritis Research UK (2009) Musculoskeletal Matters: what do general practitioners see?, Keele University (Arthritis Research UK National Primary Care Centre). [online] http://www.keele.ac.uk/media/keeleuniversity/ri/primarycare/bulletins/MusculoskeletalMatters1.pdf.

Bastian, H., Glasziou, P., and Chalmers, I. (2010) Seventy-Five Trials and Eleven Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med, 7(9), e1000326. [online] http://dx.doi.org/10.1371/journal.pmed.1000326.

Bauer, M., Damschroder, L., Hagerdorn, H., Smith, J., and Kilbourne, A. (2015) An introduction to implementation science for the non-specialist. BMC Psychology, 3(1), 32. [online] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4573926/.

Bussières, A. E., Zoubi, F. A., Stuber, K., French, S. D., Boruff, J., Corrigan, J., and Thomas, A. (2016) Evidence-based practice, research utilization, and knowledge translation in chiropractic: a scoping review. BMC Complementary and Alternative Medicine, 16(216). [online] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4944433/.

Chalmers, I., Bracke, M., Djulbegovic, B., Garattini, S., Grant, J., Gulmezoglu, A., Howells, D., Ioannidis, J., and Oliver, S. (2014) How to increase value and reduce research waste when research priorities are set. The Lancet, 383(9912), 156–165. [online] http://www.sciencedirect.com/science/article/pii/S0140673613622291.

Chalmers, I. and Glasziou, P. (2009) Avoidable waste in the production and reporting of research evidence. The Lancet, 374(9683), 86–89. [online] http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(09)60329-9/fulltext.

Chartered Society of Physiotherapy (2012) Health and Social Care Bill – Lords Report Stage briefing: Impact of NHS reforms on musculoskeletal physiotherapy, Chartered Society of Physiotherapy. [online] http://www.csp.org.uk/sites/files/csp/secure/msk_focused_hscbill_lords_briefing_feb2012.pdf.

Côté, A., Durand, M., Tousignant, M., and Poitras, S. (2009) Physiotherapists and use of low back pain guidelines: a qualitative study of the barriers and facilitators. Journal of Occupational Rehabilitation, 19(1), 94–105. [online] http://www.ncbi.nlm.nih.gov/pubmed/19219536.

Dannapfel, P., Peolsson, A., and Nilsen, P. (2013) What supports physiotherapists’ use of research in clinical practice? A qualitative study in Sweden. Implementation Science, 8(31). [online] https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-8-31.

Evans, D. (2007) Changing the practice of osteopaths, chiropractors and musculoskeletal physiotherapists, in relation to the management of low back pain. [online] http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.484899.

Evans, D. (2013) Osteopathic principles: more harm than good? International Journal of Osteopathic Medicine, 16(1), 46–53. [online] http://bit.ly/osteopathic-principles.

Feder, G., Eccles, M., Grol, R., Griffiths, C., and Grimshaw, J. (1999) Using clinical guidelines. BMJ, 317, 728. [online] http://www.bmj.com/content/318/7185/728.full.

Glasziou, P. and Haynes, B. (2005) The paths from research to improved health outcomes. Evidence Based Nursing, 8, 36–38. [online] http://ebn.bmj.com/content/8/2/36.full.

Greenhalgh, T., Hughes, J., Humphrey, C., Rogers, S., Swinglehurst, D., and Martin, P. (2002) A comparative case study of two models of a clinical informaticist service. The BMJ, 324(524). [online] http://www.bmj.com/content/324/7336/524.

Grimshaw, J., Eccles, M., Lavis, J., Hill, S., and Squires, J. (2012) Knowledge translation of research findings. Implementation Science, 7(50). [online] https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-7-50.

Haynes, B. and Haines, A. (1998) Barriers and bridges to evidence based clinical practice. BMJ, 317(7153), 273–276. [online] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1113594/.

Health and Safety Executive (2016) Working days lost, Health and Safety Executive. [online] http://www.hse.gov.uk/statistics/dayslost.htm.

Kasiri-Martino, H. and Bright, P. (2016) Osteopathic educators’ attitudes towards osteopathic principles and their application in clinical practice: A qualitative inquiry. Manual Therapy, 21, 233–240. [online] http://www.ncbi.nlm.nih.gov/pubmed/26427824.

Lau, R., Stevenson, F., Ong, B. N., Dziedzic, K., Treweek, S., Eldridge, S., Everitt, H., Kennedy, A., Qureshi, N., Rogers, A., Peacock, R., and Murray, E. (2015) Achieving change in primary care—effectiveness of strategies for improving implementation of complex interventions: systematic review of reviews. BMJ Open, 5(e009993). [online] http://bmjopen.bmj.com/content/5/12/e009993.full.

Lockspeiser, T., O’Sullivan, P., Teherani, A., and Muller, J. (2018) Understanding the experience of being taught by peers: the value of social and cognitive congruence. Advances in Health Science Education, 13, 361–372. [online] http://link.springer.com/article/10.1007/s10459-006-9049-8.

Malterud, K., Siersma, V. D., and Guassora, A. D. (2018) Sample Size in Qualitative Interview Studies: Guided by Information Power. Qualitative Health Research, 26(13), 1753–1760. [online] http://journals.sagepub.com/doi/10.1177/1049732315617444.

McCaughan, D., Thompson, C., Cullum, N., Sheldon, T., and Thompson, D. (2002) Acute care nurses’ perceptions of barriers to using research information in clinical decision-making. Journal of Advanced Nursing, 39(1), 46–60. [online] http://dx.doi.org/10.1046/j.1365-2648.2002.02241.x.

Meats, E., Brassey, J., Heneghan, C., and Glasziou, P. (2007) Using the Turning Research Into Practice (TRIP) database: how do clinicians really search? Journal of the Medical Library Association, 156–63. [online] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1852632/.

Medical Research Council (2017) Do I need NHS REC approval? An online tool to help decide whether a study requires approval from an NHS Research Ethics Committee, NHS Health Research Authority. [online] http://www.hra-decisiontools.org.uk/research/.

Michie, S., Thomas, J., Johnston, M., Aonghusa, P. M., Shawe-Taylor, J., Kelly, M., Deleris, L., Finnerty, A., Marques, M., Norris, E., O’Mara-Eves, A., and West, R. (2017) The Human Behaviour-Change Project: harnessing the power of artificial intelligence and machine learning for evidence synthesis and interpretation. Implementation Science, 12. [online] https://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0641-5.

Michie, S., West, R., Johnston, M., Thomas, J., MacAonghusa, P., Kelly, M., and Shawe-Taylor, J. (2018) The Human Behaviour-Change Project: harnessing computer science to advance behavioural science, University College London. [online] https://osf.io/4msfn/.

Mo, Y., Kontonatsios, G., and Ananiadou, S. (2015) Supporting systematic reviews using LDA-based document representations. Systematic Reviews, 4(172). [online] https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-015-0117-0.

NIHR Dissemination Centre (2016) About the NIHR Dissemination Centre, National Institute for Health Research. [online] http://www.dc.nihr.ac.uk/about/.

Olah, C. and Carter, S. (2017) Research Debt, [online] http://distill.pub/2017/research-debt.

O’Mara-Eves, A., Thomas, J., McNaught, J., and Miwa, M. (2015) Using text mining for study identification in systematic reviews: a systematic review of current approaches. Systematic Reviews, 4(5). [online] https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/2046-4053-4-5.

Researcher Development Department (2018) Development and training for postgraduate research (PhD) students, Queen Mary University of London. [online] https://academicdevelopment.qmul.ac.uk/researcher-development/postgraduate-research-students/.

Rowley, E. (2012) Protocol for a qualitative study exploring the roles of “Diffusion Fellows” in bridging the research to practice gap in the Nottinghamshire, Derbyshire and Lincolnshire Collaboration for Leadership in Applied Health Research and Care (CLAHRC-NDL). BMJ Open, 2(1), e000604. [online] https://bmjopen.bmj.com/content/2/1/e000604.

Royal College of Physicians (2018) Research for all. Sharing good practice in research management., NHS Research and Development Forum. [online] https://www.rcplondon.ac.uk/projects/outputs/research-all-sharing-good-practice-research-management.

Schneider, M., Evans, R., Haas, M., Leach, M., Hawk, C., Long, C., Cramer, G., Walters, O., Vihstadt, C., and Terhorst, L. (2015) US chiropractors’ attitudes, skills and use of evidence-based practice: A cross-sectional national survey. Chiropractic and Manual Therapies, 23(16). [online] http://chiromt.biomedcentral.com/articles/10.1186/s12998-015-0060-0.

School of Medicine and Dentistry (2018) Intercalated BSc in Medical Education, Queen Mary University of London. [online] http://www.smd.qmul.ac.uk/undergraduate/courses/intercalated-degrees/medical-education/.

Silva, T. da, Costa, L. M., Garcia, A., and Costa, L. P. (2015) What do physical therapists think about evidence-based practice? A systematic review. Manual Therapy, 20(3), 388–401. [online] http://www.sciencedirect.com/science/article/pii/S1356689X1400191X.

Snilstveit, B., Oliver, S., and Vojtkova, M. (2012) narrative approaches to systematic review and synthesis of evidence for international developmental policy and practice. Journal of Developmental Effectiveness, 4(3), 409–429. [online] http://www.tandfonline.com/doi/pdf/10.1080/19439342.2012.710641.

Stevans, J., Bise, C., McGee, J., Miller, D., Rockar, P., and Delitto, A. (2015) Evidence-Based Practice Implementation: Case Report of the Evolution of a Quality Improvement Program in a Multicenter Physical Therapy Organization. Physical Therapy, 95(4), 1–12. [online] http://ptjournal.apta.org/content/early/2015/02/11/ptj.20130541.short.

Thomas, J., Noel-Storr, A., Marshall, I., Wallace, B., McDonald, S., Mavergames, C., Glasziou, P., Shemilt, I., Synnot, A., Turner, T., and Elliott, J. (2017) Living systematic reviews: 2. Combining human and machine effort. Journal of Clinical Epidemiology, 91, 31–37. [online] https://www.sciencedirect.com/science/article/pii/S0895435617306042.

Tricco, A., Zarin, W., Antony, J., Hutton, B., Moher, D., Sherifali, D., and Straus, S. E. (2015) An international survey and modified Delphi approach revealed numerous rapid review methods. Journal of Clinical Epidemiology, 70, 61–67. [online] http://www.jclinepi.com/article/S0895-4356(15)00388-1/abstract.

Tsafnat, G., Dunn, A., Glasziou, P., and Coiera, E. (2014) The automation of systematic reviews. British Medical Journal, 346(f139). [online] http://www.bmj.com/content/346/bmj.f139.

Tsafnat, G., Glasziou, P., Choong, M., Dunn, A., Galgani, F., and Coiera, E. (2014) Systematic review automation technologies. Systematic Reviews, 3(74). [online] http://systematicreviewsjournal.biomedcentral.com/articles/10.1186/2046-4053-3-74.

Vachon, B., Désorcy, B., Camirand, M., Rodrigue, J., Quesnel, L., Guimond, C., Labelle, M., Fournier, J., and Grimshaw, J. (2013) Engaging primary care practitioners in quality improvement: making explicit the program theory of an interprofessional education intervention. BMC Health Services Research, 13(106). [online] http://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-13-106.

Vogel, S., Mars, T., Keeping, S., Barton, T., Marlin, N., Froud, R., Eldridge, S., Underwood, M., and Pincus, T. (2013) Clinical Risk Osteopathy and Management Scientific Report. [online] http://bit.ly/croam-report.

Wallace, J., Byrne, C., and Clarke, M. (2014) Improving the uptake of systematic reviews: a systematic review of intervention effectiveness and relevance. The BMJ, 4, e005834. [online] http://bmjopen.bmj.com/content/4/10/e005834.full.

Weed, D. (2005) Weight of Evidence: A Review of Concept and Methods. Risk Analysis, 25(6), 1545–1557. [online] http://onlinelibrary.wiley.com/doi/10.1111/j.1539-6924.2005.00699.x/full.