Remember to login before you can view the full description of a study, submit a study, or post topics to the discussion forum.
Added: June 8, 2021
Updated: September 24, 2021
To test how process within Identification and Prioritisation can be improved.
Added: June 8, 2021
Updated: June 8, 2021
Trials are one of the best ways of testing treatments but they can be expensive and time consuming. The amount of data collected has a big influence on both cost and time.
We aim to understand how much time trial teams spend collecting the most important trial data (called primary outcomes) compared to the other data they collect (secondary outcomes). Outcomes are things like pain, blood pressure, or weight. Small-scale work suggests that trial teams spend most of their time on the less important outcomes. Our proposed large-scale work will find out whether this is correct. We also want to understand the time taken to collect core outcome sets–an agreed minimum amount of information–compared with trials that do not use them to see if they improve efficiency, or worsen it.
Once we have the above, we will speak with trial teams and others involved in trials to understand what will help them to plan and fund their work more efficiently and also to develop guidance trial teams can use in the future. We hope our results will make it more likely that time isn’t given to less important outcomes at the expense of the most important.
Added: March 30, 2021
Updated: October 22, 2021
Peer review is an integral part of decision-making processes to effectively allocate funding. However, concerns are consistently being raised about the bias, burden and reliability of peer review. In response, the NIHR Push the Pace-II interview project was conducted in 2016, to increase knowledge about the peer review process from perspectives of applicants, peer reviewers, funding committee members and NIHR staff. Although three themes associated with strengths, challenges and improvements to peer review were presented, further insights from this data could be elicited by using an alternative approach to the analysis.
This study will use an inductive (data-driven) thematic analysis approach to re-analyse anonymised interview data collected as part of the Push the Pace-II work to generate new themes and knowledge about peer review processes from the perspective of all key stakeholders who would be affected by any changes to decision-making practices (e.g. What are the motivations to peer review grant proposals? What are the expectations of reviewers when conducting peer reviews?). Understanding stakeholder expectation about the peer review process will provide essential information about the consequences of modifying and changing the peer review processes, for example impacts to reviewer recruitment and retention, or how peer review is implemented within the decision-making practice.
Added: March 30, 2021
Updated: October 10, 2023
Objective: This study investigated the content, quality and value of feedback given to applicants who applied to one of four research programmes in the UK funded (or jointly funded) by the National Institute for Health Research (NIHR).
Design/setting: Document analysis and an online survey.
Participants: NIHR applicant feedback documents comprised written feedback from Stage 1 and Stage 2 funding committees and external peer reviewers, and NIHR applicants.
Methods: A mixed-methods phased approach was conducted. Phase 1 examined 114 feedback documents and developed a conceptual framework of the key components of feedback using content analysis. Phase 2 was an online survey completed by 113 NIHR applicants. Frequencies of responses to closed questions were calculated. Perceptions of quality and value of feedback were identified using content analysis of open-text responses.
Results: In phase 1, a conceptual framework was developed with seven overarching categories: ‘Study structure and quality’; ‘Team and infrastructure’; ‘Acceptability to patients and professionals’; ‘Study justification and design’; ‘Risks and contingencies’; ‘Outputs’; ‘Value for money’. A higher frequency of feedback was provided at Stage 2 and for successful applications across the majority of components. In phase 2, frequency data showed that opinion on feedback was dependent on funding outcome. Content analysis revealed four main themes: ‘Committee transparency’; ‘Content validity and reliability’; ‘Additional support’; Recognition of effort and constraints’.
Conclusions: This study provides key insights and understanding into the quality, content, and value of feedback provided to NIHR applicants. The study identified key areas for improvement that can arise in NIHR funding applications, as well as in the feedback given to applicants that are applicable to other funding organisations. These findings could be used to inform funding application guidance documents to help researchers strengthen their applications and used more widely by other funders to inform their feedback processes.
Added: March 30, 2021
Updated: October 11, 2023
Throughout the funding lifecycle, researchers are required to complete a number of administrative activities, from gaining permissions from HEIs to submit an application to completing regular funder/sponsor monitoring reports on research progress, for funding organisations, Higher Education Institutions (HEIs), research co-ordinating bodies (e.g., the Clinical Research Network) and data platforms (e.g., Researchfish). However, there has been growing concerns about the value of these administrative activities and the bureaucracy in research and HEIs and whether these additional administrative activities are appropriate and proportionate or create unnecessary burden. As yet, it is unclear which activities/processes take up time and effort and the level of effort/burden that could be regarded as ‘avoidable’ or ‘unnecessary’ from a researcher perspective. It is therefore essential to explore and gain an understanding on the process, perceived effort or burden experienced by researchers as a consequence of completing these activities in the funding lifecycle.
To do this, we will carry out a two-part mixed methods study exploring researcher perspectives using an online survey (study 1) and follow-up interviews (study 2) with researchers who have completed the application submission and are fulfilling or have fulfilled monitoring requirements for funding from UK-based funding organisations. We will conduct quantitative analysis on closed-question responses and content and thematic analysis on open-text responses from an online survey and interview transcripts.
Through this study we will be able to identify activities in the process that are commonly considered to be an unnecessary burden or duplication across organisations and identify opportunities to enhance communication with researchers on the reasons for these processes and activities.
Added: March 30, 2021
Updated: September 16, 2022
Introduction: Allocation of research funds relies on peer review to support funding decisions. However, the process can be susceptible to biases and inefficiencies, resulting in unnecessary burden for stakeholders. To effect change and inform robust research, the field needs evidence on what interventions to enhance Peer Review and Decision-Making (PRDM) work, for whom across different funders and research contexts and how.
Methods: Peer-reviewed and grey literature on interventions in PRDM was sourced from a electronic databases and funder websites. Realist synthesis of interventions that generated outcomes for stakeholders used contexts-mechanisms-outcomes (CMO) analysis to identify links between common drivers for change in PRDM (contexts), stakeholder-specific interventions and reactions (mechanisms) and their outcomes.
Results: 95 publications and 36 web sources on recent funder interventions were included. Fifty-nine publications provided CMO links, which were aggregated into 10 high-level CMO Configurations showing what drives interventions to enhance PRDM. Key drivers were: ensuring that research delivers social benefit, minimising unnecessary stakeholder burden, promoting innovative research, improving identification and management of reviewers, improving the quality of reviews and increasing patient and public involvement in decision-making. Interventions have delivered largely positive outcomes (e.g., reviewer selection tools saved administrative time; shorter applications minimised researcher burden; and dedicated funding streams for early career researchers promoted innovation). Today, funders remain focussed on promoting research innovation, interdisciplinary collaboration and early-career researchers.
Conclusions: Realistic options to enhance PRDM for funders involve interventions that introduce incremental changes to the process, reducing burden. Innovative PRDM mechanisms to promote innovation are emerging.
Added: March 30, 2021
Updated: June 26, 2023
Innovations in decision-making practice for allocation of funds in health research are emerging; however, it is not clear to what extent these are used. This study aims to better understand current decision-making practices for the allocation of research funding from the perspective of UK and international health funders. An online survey (active March-April 2019) was distributed by email to UK and international health and health-related funding organisations (e.g., biomedical and social), and was publicised on social media. The survey collected information about decision-making approaches for research funding allocation, and covered assessment criteria, current and past practices, and considerations for improvements or future practice. A mixed methods analysis provided descriptive statistics (frequencies and percentages of responses) and an inductive thematic framework of key experiences. Thirty-one responses were analysed, representing government-funded organisations and charities in the health sector from the UK, Europe and Australia. Four themes were extracted and provided a narrative framework. 1. The most reported decision-making approaches were external peer review, triage, and face-to-face committee meetings; 2. Key values underpinned decision-making processes. These included transparency and gaining perspectives from reviewers with different expertise (e.g., scientific, patient and public); 3. Cross-cutting challenges of the decision-making processes faced by funders included bias, burden and external limitations; 4. Evidence of variations and innovations from the most reported decision-making approaches, including proportionate peer review, number of decision-points, virtual committee meetings and sandpits (interactive workshop). Broadly similar decision-making processes were used by all funders in this survey. Findings indicated a preference for funders to adapt current decision-making processes rather than using more innovative approaches: however, there is a need for more flexibility in decision-making and support to applicants. Funders indicated the need for information and empirical evidence on innovations which would help to inform decision-making in research fund allocation.
Added: March 29, 2021
Updated: March 29, 2021
Background
By 2011, the Health Technology Assessment (HTA) programme had published the results of over 100 trials with another 220 in progress. The aim of the project was to develop and pilot ‘metadata’ on clinical trials funded by the HTA programme.
Objectives
The aim of the project was to develop and pilot questions describing clinical trials funded by the HTA programme in terms of it meeting the needs of the NHS with scientifically robust studies. The objectives were to develop relevant classification systems and definitions for use in answering relevant questions and to assess their utility.
Data sources
Published monographs and internal HTA documents.
Review methods
A database was developed, ‘populated’ using retrospective data and used to answer questions under six prespecified themes. Questions were screened for feasibility in terms of data availability and/or ease of extraction. Answers were assessed by the authors in terms of completeness, success of the classification system used and resources required. Each question was scored to be retained, amended or dropped.
Results
One hundred and twenty-five randomised trials were included in the database from 109 monographs. Neither the International Standard Randomised Controlled Trial Number nor the term ‘randomised trial’ in the title proved a reliable way of identifying randomised trials. Only limited data were available on how the trials aimed to meet the needs of the NHS. Most trials were shown to follow their protocols but updates were often necessary as hardly any trials recruited as planned. Details were often lacking on planned statistical analyses, but we did not have access to the relevant statistical plans. Almost all the trials reported on cost-effectiveness, often in terms of both the primary outcome and quality-adjusted life-years. The cost of trials was shown to depend on the number of centres and the duration of the trial. Of the 78 questions explored, 61 were well answered, 33 fully with 28 requiring amendment were the analysis updated. The other 17 could not be answered with readily available data.
Limitations
The study was limited by being confined to 125 randomised trials by one funder.
Conclusions
Metadata on randomised controlled trials can be expanded to include aspects of design, performance, results and costs. The HTA programme should continue and extend the work reported here.
Added: March 29, 2021
Updated: March 29, 2021
Objectives To assess the value of pilot and feasibility studies to randomised controlled trials (RCTs) funded by the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme. To explore the methodological components of pilot/feasibility studies and how they inform full RCTs.
Study design Cross-sectional study.
Setting Both groups included NIHR HTA programme funded studies in the period 1 January 2010–31 December 2014 (decision date). Group 1: stand-alone pilot/feasibility studies published in the HTA Journal or accepted for publication. Group 2: all funded RCT applications funded by the HTA programme, including reference to an internal and/or external pilot/feasibility study. The methodological components were assessed using an adapted framework from a previous study.
Main outcome measures The proportion of stand-alone pilot and feasibility studies which recommended proceeding to full trial and what study elements were assessed. The proportion of ‘HTA funded’ trials which used internal and external pilot and feasibility studies to inform the design of the trial.
Results Group 1 identified 15 stand-alone pilot/feasibility studies. Study elements most commonly assessed were testing recruitment (100% in both groups), feasibility (83%, 100%) and suggestions for further study/investigation (83%, 100%). Group 2 identified 161 ‘HTA funded’ applications: 59 cited an external pilot/feasibility study where testing recruitment (50%, 73%) and feasibility (42%, 73%) were the most commonly reported study elements: 92 reported an internal pilot/feasibility study where testing recruitment (93%, 100%) and feasibility (44%, 92%) were the most common study elements reported.
Conclusions ‘HTA funded’ research which includes pilot and feasibility studies assesses a variety of study elements. Pilot and feasibility studies serve an important role when determining the most appropriate trial design. However, how they are reported and in what context requires caution when interpreting the findings and delivering a definitive trial.
Added: March 29, 2021
Updated: July 9, 2024
Funding organisations rely on a number of strategies to make recommendations for the allocation of research funding fair and transparent. A panel of experts, also referred to as funding committees, are often considered integral to the decision-making process for allocating research funding. It is therefore important to understand the role of funding committees, in terms of the social interactions and dynamics and how a common understanding is reached by the committee members. However, there is limited research investigating the role of members and the social interactions and processes which cannot be explored through survey or interview methods alone.
As a result of COVID-19, the NIHR chose to conduct some funding committee meetings completely virtual. It is from this unprecedented global health situation that the proposed research was adapted to focus on the social dynamics within technological contexts to understand online communication and interactions. Originating from hospitality and tourism research, netnography is an established method for conducting social media research. It is therefore our intention to conduct a netnographic study on virtual funding committee practices to understand how members participate (e.g. social interactions), insight into using online forms of communication (e.g. cultural changes), and the benefits, challenges and barriers to using online platforms (e.g. to explore future considerations).
The aims of the study are to examine and explore NIHR’s virtual funding committee meetings in terms of the formal (processes, technology, and resources) and informal (social interactions, social dynamics, perceptions, attitudes and expectations), implicit and explicit decision-making practices. A range of methods and approaches will be used to gather data that will be in text (e.g. documents, transcripts, data spreadsheets) and non-text data (e.g. videos, audio). By following the principles of netnography, all three types of data operations will be included in the study:
- Investigative: online social media data (recordings of funding committee meetings)
- Interactive: interviews with funding committee members and secretariat, and an online survey with all funding committee members
- Immersive: to assist the netnographic study an immersive journal will be kept to reflect, analyse and guide research process and decision-making
As far as we know this is the first netnographic study to explore social systems of shared meaning and the cultural understandings of virtual funding committee practice.