On 18 November 2011, the European Medicines Agency assembled for the first time some 50 experts in subgroup analysis from Europe and the United States, in a one-day scientific workshop. The experts included representatives from regulatory authorities, a health-technology-assessment agency, academia and pharmaceutical industry, and discussed standards and methodology for the planning, conducting and reporting of subgroup analysis.
Subgroup analyses are an important tool in the assessment of confirmatory trials in marketing authorisation applications, and appropriate statistical methods are key for a sound clinical interpretation of subgroup results. Subgroup analyses are used for assessment of internal consistency, to try to identify patient groups with the most favourable benefit-risk profile or to try to rescue trials that ‘fail’ based on the full analysis. Subgroups may be pre-specified in the trial protocol, based on demographic, genomic or disease characteristics (e.g. sub-entities of a disease that are widely recognised within the medical community) or may materialise based on a need or desire to further explore study results.
Interpreting subgroup analyses presents particular methodological challenges, whereas not exploring subgroups because of these challenges would be an unsatisfactory solution as it would place excessive reliance on assumptions (e.g. homogeneity of response to treatment) that cannot be substantiated. Therefore, the Committee for Medicinal Products for Human Use (CHMP) is currently preparing an Agency guideline on this topic, setting out important methodological considerations and assessment strategies and discussing some example datasets.
The aim of the workshop was to present the scope and content of the proposed guideline to internal and external experts, and to receive feedback for further reflection. The workshop covered pre-planned subgroup analysis in confirmatory clinical trials as well as exploratory subgroup analysis, either pre-planned or resulting from the need for further exploration of study results guided by clinical or pharmacological reasoning.
During the workshop, experts discussed the role of subgroup analyses in the assessment of confirmatory clinical trials, the need for pre-specification, the available methodology for reliable subgroup findings to be incorporated in the benefit/risk assessment, and consequences of incorporating subgroup findings in the decision making procedure.
Examples were presented by industry, academic and regulatory experts during the meeting and lively discussion ensued. The main conclusions of the meeting were:
1. With regard to the design of trials:
- Despite all efforts to achieve homogeneity, clinical trial populations are frequently heterogeneous and regulators have a responsibility to examine the benefit/risk balance not only in the average patient but also in subsets of patients defined by biological or clinical characteristics besides traditional demographics such as gender, age, and ethnicity. Therefore, plans to explore benefits and risks in biologically defined and potentially clinically relevant subgroups should be incorporated into every development programme.
- Early dialogue between industry and regulators (and possibly health technology assessment bodies, which may have different preferences in the interpretation of subgroup analyses) about the most important subgroups to be accounted for in the design of the trial was considered highly valuable and will guide assessment.
2. With regard to the analysis of trial results:
- Statistical methods to investigate the consistency of effects across the entire target population were discussed (tools are not limited to interaction tests), as well as methods to deal with multiplicity issues to control the probability of false positive or false negative findings. Heterogeneous results across subsets will require a further level of scrutiny in the assessment phase. A quantitative definition of internal consistency does not exist.
- Post-hoc subgroup analyses to improve decision making should be interpreted with particular caution but should not be ruled out a priori; these analyses may convey very important information about the right target population. However, they pose difficult challenges for the assessment as the level of evidence of results from post-hoc analyses can hardly be statistically quantified. Whilst relevant to all subgroup investigations, biological plausibility and replication across studies, e.g. in the same therapeutic class, seem to be factors of particular importance to accept or reject post-hoc analyses. It was also agreed that this point needs refinement and could be discussed in detail in the upcoming guideline.
The participants of the workshop agreed that ultimately it is essential for the benefit of patients that subgroup analyses are based on rigorous methodology, balanced with pharmacological and clinical plausibility, such that conclusions are guided by the overall strength of evidence.
The conclusions of the workshop will be considered when drafting the guideline on subgroup analysis. The draft guideline is expected to be released for consultation in 2012 and will be published on the Agency's website.
The Agency will continue its public dialogue on subgroup analysis with stakeholders, including academia, regulators and pharmaceutical industry.