Name of applicant: Austin Plunkett
Host institution: Queen Mary University London
Title of research proposal: Quality improvement by Distribution Of Computer-AssisTed Evidence Synopses
Short title: Quality aDvOCATES
Musculoskeletal (MSK) conditions are the largest contributor to the burden of disability in the UK, but uptake of evidence by manual therapists who see patients with these conditions remains limited. This PhD proposes a programme to favourably influence clinician attitudes towards evidence, thereby increasing the impact of research on clinical practise.
State regulated allied health professions including osteopathy and physiotherapy are predominantly sought by people with MSK conditions. Policy makers in government and professional bodies, as well as patient representatives, expect these professions to demonstrate evidence-informed practice. However, tight budgetary constraints, diverse working practices, and the complexity of primary care make it challenging to improve the uptake of evidence in these professions.
This problem is mirrored by the need for research institutions to demonstrate impact while reducing costs. This demands innovative solutions in the production and dissemination of research.
The well-recognised "research-to-practice gap" indicates the persistence of these twin challenges.
This project has the following aims:
- to understand the value that manual therapists and their patients place on priority research topics;
- to design, implement and evaluate a programme to increase this perceived value through dissemination via peer advocates;
- to overcome bottlenecks in the discovery of relevant research and production of evidence synopses that are agreeable to clinicians.
The objectives are: to investigate factors influencing the value placed on research, and to streamline the research-to-practice pipeline. Ultimately, this will increase the evidential component in interventions delivered by manual therapists, thereby improving patient care and patient perceived benefit.
This project will establish a growing database of evidence synopses on priority clinical topics, and a network of evidence advocates within the osteopathic profession.
Background and overview of the literature
The 'research-to-practice gap' describes the well-recognised struggle to put evidence into effective practice. Glasziou and Haynes (2005) use a 'pipeline' analogy and describe evidence 'leaking' at all stages along this pipeline. An estimated 85% of research is squandered (Chalmers and Glasziou 2009) making 'research waste' a priority problem (Chalmers et al 2014). Although the clinical encounter is complex and dynamic (Lau et al 2015) the pipeline model enables identification of 'leakage points' that may be targeted. Specifically, the student will target the following:
- the 'bottleneck' of summarising evidence (Tsafnat et al 2014)
- clinician awareness of evidence (Vogel et al 2013; daSilva et al 2015)
- the beliefs and attitudes of key influencers and their peers (Meats et al 2007; Aarons et al 2016).
The pace at which primary research is generated outstrips the capacity for reviewers to summarise it (Tsafnat et al 2014; Olah and Carter 2017). It is recognised that reviews need to be prioritised and streamlined if health-care professionals are to maintain an up-to-date evidence base (Bastian, Glasziou and Chalmers 2010). Despite this, reviews are often out of date due to the resources required to produce them and the high associated cost (Lau et al 2015). Innovation is required to overcome these issues (Bastian, Glasziou and Chalmers 2010).
The student will explore whether the time and resources required in reviewing literature can be significantly reduced. Computers can increase the efficiency of preparation for reviews (Chalmers et al 2014); specifically, machine learning may provide a means of accelerating initial stages by assisting reviewers in finding and categorising relevant research (Tsafnat et al 2014; O'Mara-Eaves et al 2015; Mo, Kontonatsios and Ananiadou 2015). Searching databases, "sifting" abstracts, organising papers into topics, and deduplicating papers may all be accelerated, thereby helping to address this bottleneck.
Clinician awareness: impact and implementation
"[A]ctionable, accessible, and trustworthy information" can help clinicians bring research into their decision making processes (NIHR Dissemination Centre 2016). However, passive dissemination alone has only minimal influence on professional behaviour (Feder et al 1999; Côté et al 2009). Instead, influencing leaders within a profession and professional organisations may be one key to impacting on practitioner beliefs (Aarons et al 2016; McCaughan et al 2002; Kasiri-Martino and Bright 2015; Dannapfel, Peolsson and Nilsen 2013). Identification and recruitment of advocates can increase the incorporation of evidence into clinical practise (Stevans et al 2015) through improved "social validation" i.e. acceptance of change within a peer group (Glasziou and Haynes 2005) and strong "social congruence" i.e. similarity of experiences and practices (Lockspeiser et al 2008). Recognition of clinician involvement in research activities encourages a positive attitude towards learning, and the development of a research culture (Royal College of Physicians 2017).
Any attempt to bridge the gap between evidence-based care and clinician practises should measure impact and reference appropriate theoretical justification (Vachon et al 2013).
Clinician beliefs and attitudes towards evidence
While primary care may be improved by clinicians' reading of evidence summaries (Schneider et al 2015), clinician attitudes are key to successful introduction (Côté et al 2009). Manual therapists may not follow advice in summaries or guidelines for various reasons, including:
- the size and complexity of research (Haynes and Haines 1998)
- practical issues including time (Bussières et al 2016)
- individual and cultural attitudes towards research (Dannapfel, Peolsson and Nilsen 2013)
- beliefs around professional identity (Evans 2007, 2013; Kasiri-Martino and Bright 2015)
- misalignment of research with clinician priorities and routines (Bauer et al 2015)
- reluctance or perceived inability to find and interpret research (Schneider et al 2015, Greenhalgh et al 2002)
Presentation of research in a format that clinicians find agreeable can help overcome these barriers (Wallace, Byrne and Clarke 2014, McCaughan et al 2002) and brief overviews may be more palatable than lengthier formats (Grimshaw et al 2012). There are numerous potential candidate formats, since summary formats and rapid methodologies are under development by many organisations (Tricco et al 2015). Consequently the student will conduct a systematic narrative review of the evidence to determine which methodology may address clinician-related barriers. Systematic narrative reviews combine a systematic search with a narrative critical analysis, aiming to mitigate bias in the selection of papers while permitting a narrative approach for producing insights and recommendations (Weed 2005, Snilstveit, Oliver and Vojtkova 2012).
Role of musculoskeletal therapists and the burden of musculoskeletal disease
People with musculoskeletal (MSK) conditions seek treatment from manual therapists including physiotherapists and osteopaths, who are state-regulated primary care allied health professionals. MSK conditions are the largest contributor to the burden of disability in the UK (Arthritis Research UK 2016) costing the UK economy an estimated £7.4 billion per year (Chartered Society of Physiotherapy 2012). MSK disease may be the second most common cause of GP visits, accounting for over 20% of their workload (Arthritis Research UK 2009).
This burden emphasises the importance of finding efficiencies in the evidence-to-practice pipeline, particularly regarding manual therapies.
Method and design
This will be a mixed methods project. The student will be required to learn the methodologies for, and to conduct, a systematic narrative review, a critical literature review, and a thematic analysis. Project management and group leadership skills will be required to lead teams of volunteer advocates that the student will recruit. The student will also be required to implement a prototype tool for improving database searches, using machine learning techniques.
This design is based on the 'plan, do, study, act' (PDSA) approach to quality improvement (Taylor et al 2013; NHS Institute for Innovation and Improvement 2008) which is an appropriate technique for learning how to effectively implement best practices (Kaplan et al 2010). The design is intended to address recommendations that "targeted messaging" and "short summaries" of evidence significantly improve the uptake of evidence (Wallace, Byrne and Clarke 2014). In place of the suggested "educational visits", the student aims to address issues of awareness and peer attitudes by recruiting and guiding teams of advocates.
Coordinate an expert panel to generate an initial list of research topics that are priorities in manual therapies. This will build on a previous Delphi consensus study that determined priorities in osteopathy (Rushton et al 2014).
Critical literature review of best-practices in quality improvement (QI) and impact assessment in PDSA programmes. This will be used to determine the specific approach outlined in Phases III and IV.
Systematic narrative review of "short-form" evidence synopsis methodologies. State-of-the-art literature review of behaviour-change models. These will influence management of volunteers in Phases III and IV, and interpretation of changes in their attitudes towards evidence-informed practice.
Implement and run a pilot system for conducting semi-automated literature searches and topic analysis. This will feature elements of machine learning, including query expansion techniques using word2vec (a neural network that maps words from a given corpus into a numerical 'vector space') and unsupervised topic allocation using latent Dirichlet allocation (a Bayesian approach to modelling the probability that any topic represents a given document), or similar approaches.
Recruit volunteer advocates and agree distribution of topics among recruits. For practical reasons the total number of recruits will be no more than 10 per cycle. Recruitment of volunteers is a common first step in a PDSA approach (Morelli 2016).
Assess the value advocates place on research into their given topic. This will likely use the "evidence based practice attitudes scale" (Aarons 2004), which several studies suggest is generalisable across professions (Keyser, Harrington and Ahn 2016), or the "evidence based practice attitude and utilization survey" (Leach and Gillham 2008) which was developed for complementary and alternative medicine practitioners. In addition, one or more of the following approaches may included:
- reflective diaries kept by volunteers;
- interviews with volunteers focusing on specific patient presentations relating to their chosen topic;
- focus group discussions;
- patient interviews.
Guide recruits in the production of short-form synopses. Manual therapists are required to demonstrate "continuous professional development" (CPD), and recruit involvement in this programme can count toward their CPD requirements. Guidance will be delivered using e-learning technologies such as video-conferencing and web-casts.
Assess the impact of synopses on advocates' clinical practice, through reassessment of attitudes towards evidence based practice and analysis of diaries and discussions from Phase III.
The student will produce a mixed-methods analysis of changes in clinician attitudes towards research, including thematic analysis of the data collected through diaries, discussions and interviews.
Review lessons learned from phases III and IV in order to refine the approach. Repeat the cycle of recruitment, attitude assessment, synopsis production and reassessment from phases III and IV. PDSA takes a cyclical approach, and Phase V represents the "act" in this cycle.
This research project will take a pragmatic approach, and some elements will be put into immediate practice. The information collected will enable the manual therapy Allied Health Professions to identify mismatches between the expectations of researchers, clinicians and patients regarding evidence-informed practice. This will provide valuable detail for regulators, policy makers and health care commissioners who aim to improve the rate at which evidence is adopted into practice. It will also highlight areas where further research would be beneficial.
It is hoped that this project will provide a template for others who are aiming to improve the efficiency of secondary research, and the impact of research on clinical practice.
Ultimately, this approach will enable the student to create a library of regularly updated synopses for manual therapy Allied Health Professions, and a growing network of clinicians who will advocate within their professions. This approach will be readily adaptable for use by other health-care professions, and generalisable across other practice settings including NHS and private clinics.
Student learning requirements
The student's learning requirements are summarised here:
- Systematic narrative review methodology.
- Critical literature review methodology.
- Qualitative data analysis methodology, such as thematic analysis.
- Query expansion techniques, using word2vec or similar.
- Methods for unsupervised topic classification, using latent Dirichlet allocation or similar.
- Statistical methods for evaluating performance of the topic classifier.
QI programme development
- Basic QI programme development skills.
- Group leadership and management skills.
- Patient interview techniques.
- Group coordination and motivation skills.
The student will be based within the Complex Intervention and Social Practice in Health Care (CISPHC) Unit located in the Centre for Primary care and Public Health (CPCPH). CPCPH is multidisciplinary, bringing together academics from a range of backgrounds including general practice, medical ethics, nursing, statistics, public health, sociology and psychology, and involved in research, teaching and service development. The Centre was rated fourth in the UK in health services research in the most recent research assessment exercise, with a high proportion of research output rated as internationally leading or of internationally excellent quality. The student would have the benefit of working in this highly specialised team who come from different backgrounds with many areas of expertise.
Within this vibrant and growing Centre, the student will have an exciting and supportive environment for learning and conducting research. There is an active weekly seminar programme where students are encouraged to present, and we have available a range of academic experts and clinicians with experience of dealing with chronic pain who will be able to advise and help the student. We have an established relationship with the Persistent Pain Service at Mile End Hospital and have worked with them closely for the past 4 years. Over the past 3 the Blizard Institute where the Centre for Primary Care and Public Health is based, has established a Graduate Studies Programme and support system with informal PhD student lunches and seminars to ensure high standards of supervision and encourage student interaction and mutual learning.
Start: May 2018
Finish: April 2020
9 month review: February 2019
18 month M Phil upgrade viva: October 2019
Supervisors will comply with national and School of Medicine and Dentistry policies for best practice regarding supervision, feedback, assessment, student representation, progress and review.
Ethics approval requirements have been assessed by the Queen Mary University London Reseearch Ethics team and using the Medical Research Council Health Research Authority study research decision tool (Medical Research Council 2017). NHS Research Ethics Comittee approval is not required. The following ethical considerations are noted:
- This proposal will recruit from private practitioners, not from NHS practitioners.
- Demographic data will not allow identification of any participants, since questions will be few and very broad.
- k-anonymity calculation can be conducted to ensure demographic data is non-identifiable.
- Participants are free to cease involvement at any time, and will not be bound by any form of contract.
- Commitment and distress levels among participants will be monitored.
- The quality of summaries will be carefully monitored, and they will be edited by the student.
Despite participants being self-selecting, there is a small risk that poor clinical practice might be identified among participants. As part of the programme, all participants will be encouraged to reflect upon their practice and will be supported through regular meetings, existing professional support materials including regulatory guidance. Participants will also be encouraged to present their synopses to their peer-groups. Due to the anonymisation measures described above, it will not be possible to identify individuals. However, through the repeated use of the attitude survey described in Phase III, changes in attitudes towards evidence-informed practice will be evident. This information will be fed back to the profession, including to the regulator, the professional members' organisation, and the educational instutions.
The student will retain editorial control of all summaries. Only summaries that are deemed of acceptable quality will be made public.
A detailed handbook providing clear guidance will be created and distributed to all participants, ensuring the process is well controlled.
A schedule of meetings will be drawn up by the supervisors and student. Following medical school policy, the student and their principal supervisor will meet at least monthly but in the first year of the studentship we plan weekly hour-long meetings between the student and supervisors. The student and supervisors will keep written, agreed, records of all meetings. Supervisors will be available between meetings by e-mail, and for informal meetings as necessary. The student can expect to receive feedback on short written pieces of work within a week and feedback on longer pieces of work (e.g. thesis chapters) within two weeks.
Training and development
The principal supervisor will be responsible for discussing training needs with the student at regular intervals throughout the project and drawing up a training plan together with the student which ensures that they complete the 70 hours per annum transferable skills training recommended by Queen Mary University London's training guide. The principal supervisor will also take responsibility for identifying suitable training. Training will include the induction programme run by the School of Medicine and Dentistry, informal training within CPCPH and formal training inside and outside the School.
School of Medicine and Dentistry policy will be followed, with a formal progress report at 9 months (student progress and plans and a short critical appraisal assessed by supervisors and the Graduate Studies Committee) and transfer from MPhil to PhD within 18 months (via 10,000 word report and viva by two assessors, one external to the Centre). There will be a further report to supervisors at 30 months. Transfer to writing-up status will occur at 36 months. It will be made clear that the student should aim to submit their thesis at about 39 months to enable their PhD award within 4 years of commencing their studentship. At each milestone the student’s personal development plan will be monitored to ensure that they have received the recommended 70 hours per annum transferable skills training.
Training and development
Queen Mary University of London (QMUL) has a strong reputation for innovation through inter-disciplinary and cross-faculty collaboration. The Research Excellence Framework 2014 rated QMUL 5th in the UK for "world-leading or internationally excellent" research outputs, and HEFCE 2013 noted that QMUL has the highest rate for timely PhD completion of any UK university.
Pending assessment the student may need training in the following:
At the CPCPH we run a masters course in Global health and Innovation, which includes a research methods and advanced research methods modules. The student could either attend one or both modules to re-familiarise themselves with research approaches and gain new knowledge. This will prepare the student for mixed-methods approaches and for qualitative data analysis including thematic analysis.
Statistical training will be given on the Research methods course but where necessary we will provide specialist training with one or two of our own statisticians from the centre for health sciences. Additionally the Royal Statistical Society run a variety of training courses throughout the year that may be of use.
Systematic narrative and critical literature reviewing
Within CPCPH we have considerable expertise in different aspects of conducting reviews of methodology, including search strategies, data extraction, issues with identifying methodology from papers, and presenting results. The student will receive appropriate training from CPCPH staff and if staff are unavailable we will organise attendence on one of the many courses available in this field.
Machine learning and computational training
The Faculty of Science and Engineering provides expertise and teaching in machine learning and related fields, and The Blizard Institute, where the CPCPH is based, has relationships with this faculty through the programmes including Queen Mary Innovation and the "CANBUILD - deconstructing cancer" project. The student will be able to access modules relating to the computational element of this PhD, contributing to the existing reputation the Blizard Institute has for interdisciplinary research.
The university run excellent courses for the software applications they will need to be proficient in, for example Endnote, Microsoft Word and PowerPoint. We can also include the PhD student in our M.Sc. for Global Health and Innovation modules such as: introductions to the library and searching electronic data bases. Additionally, there are excellent short course in using statistical software packages. Other courses that we would expect the student to attend are Problem Based Learning (PBL) training to contribute to not only their own training but that of others. This will also help foster a working team spirit within the CPCPH.
Finances and funding
PhD fees: To be paid by the National Council for Osteopathic Research (NCOR).
PhD stipend: To be paid by NCOR.
|Computing resource for machine learning||£2,500|
|Website and data hosting||£2,000|
Projected costs will be approximately £4,500 for the whole project.