Abstract
Background
By 2011, the Health Technology Assessment (HTA) programme had published the results of over 100 trials with another 220 in progress. The aim of the project was to develop and pilot ‘metadata’ on clinical trials funded by the HTA programme.
Objectives
The aim of the project was to develop and pilot questions describing clinical trials funded by the HTA programme in terms of it meeting the needs of the NHS with scientifically robust studies. The objectives were to develop relevant classification systems and definitions for use in answering relevant questions and to assess their utility.
Data sources
Published monographs and internal HTA documents.
Review methods
A database was developed, ‘populated’ using retrospective data and used to answer questions under six prespecified themes. Questions were screened for feasibility in terms of data availability and/or ease of extraction. Answers were assessed by the authors in terms of completeness, success of the classification system used and resources required. Each question was scored to be retained, amended or dropped.
Results
One hundred and twenty-five randomised trials were included in the database from 109 monographs. Neither the International Standard Randomised Controlled Trial Number nor the term ‘randomised trial’ in the title proved a reliable way of identifying randomised trials. Only limited data were available on how the trials aimed to meet the needs of the NHS. Most trials were shown to follow their protocols but updates were often necessary as hardly any trials recruited as planned. Details were often lacking on planned statistical analyses, but we did not have access to the relevant statistical plans. Almost all the trials reported on cost-effectiveness, often in terms of both the primary outcome and quality-adjusted life-years. The cost of trials was shown to depend on the number of centres and the duration of the trial. Of the 78 questions explored, 61 were well answered, 33 fully with 28 requiring amendment were the analysis updated. The other 17 could not be answered with readily available data.
Limitations
The study was limited by being confined to 125 randomised trials by one funder.
Conclusions
Metadata on randomised controlled trials can be expanded to include aspects of design, performance, results and costs. The HTA programme should continue and extend the work reported here.
Aim
The aim of the project was to develop and pilot ‘metadata’ on clinical trials funded by the HTA programme. In exploring how to extend the metadata held in existing clinical trial registries, we considered two options. We could either aim to specify a comprehensive data set capable of answering all potential questions, or design a data set to answer particular questions. We pursued the latter option, starting with a set of themes and related questions that might plausibly be answered by such metadata.
We explored questions under six themes using classification systems in answering particular questions. Some classification systems were simple (yes/no) and some complex (16 headings for the European Medicines Agency guideline on handling missing data). Questions that had to do with whether or not analyses were as planned required not only classification of the planned and actual analyses, but also of their (dis)agreement. Data sources comprised both published and unpublished documents. Published sources were largely those in the HTA journal monograph series but also study protocols (most but not all published on the HTA website since 2006). Key unpublished sources included final application forms as well as vignettes, commissioning briefs and project protocol change forms.
The project explored the extent to which metadata could provide standardised data which would be useful not only in managing that portfolio but also in enabling assessment of the conduct, analysis and cost of those trials. Such assessment of the trials would require high-quality data, which had been subject to explicit quality assurance.
The four project objectives, as stated in the final application funded by the HTA programme, were:
to develop, pilot and validate metadata definitions and classification systems to answer specified questions within six themes
to extract data under these headings from published RCTs funded by the HTA programme
to analyse these data to answer specific questions grouped by theme
to consider further development and uses of the data set, including refinements of the metadata headings for their application to ongoing and future HTA trials.
The protocol stated that:
Metadata would provide standardised data about the portfolio of HTA trials. These data would enable assessment of questions such as how well the trials were conducted, and the extent to which their results were as expected. Some limited metadata are already publicly available; their extension as proposed here will require appropriate data headings (or classification systems), some of which would be developed in this project.
It also stated that:
The provision of such data would enable performance of the trial portfolio to be monitored over time. Such data would also indicate foci for improvement and help assess the contribution of the ‘needs-led’ and ‘value added’ scientific inputs. To the extent that similar data could be collated for other trials, these could be compared with the HTA trials.
Intended Impact of the Study
Inform the development of standardised data extraction and coding for NETSCC research programmes. Led to the development and implementation of Study Level Extended Dataset.
Project Lead
Project Collaborators
Louise Stanton
Ruairidh Milne
Andrew Cook
David Turner
Peter Davidson
Keywords
URL
Research Area(s)
Other Areas: Standardised data about the portfolio of HTA trials. Assessment of how well the trials were conducted, and the extent to results were as expected.
Study Design(s)
Data Collection Method(s)
Participant(s)
Other Participants: HTA funded clinical trials