European Statistical Meeting on Estimands

28 September, 2015, 8 AM-4 30 PM

VENUE GSK at Stevenage Gunnels Wood Road Stevenage Hertfordshire SG1 2NY United Kingdom

Are you estimating what you think you are?
De-mystifying Estimands

Monday 28th September 2015, Hertfordshire

Please click below to see the speaker slides from the meeting:

Rob Hemmings
AnnKristinLeuchs
Chrissie Fletcher
Frank Bretz
James Carpenter
JamesRoger


Controversy and confusion exist on the definition and selection of appropriate estimands in the clinical trial context.

As a result the ICH Steering Committee endorsed a final Concept Paper in October 2014 with the goal of developing a new regulatory guidance, suggested to be an Addendum to ICH E9. The aim of the addendum is to promote harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. A working group sponsored by ICH has been established to develop the addendum and is due to report findings in December 2015.

This meeting will provide a forum to hear about the latest developments and discussions on this topic.  Speakers include members of the ICH working group as well as representatives from academia, European regulatory bodies and industry. The day will end with a panel discussion including all speakers.

Presenters include:

Rob Hemmings
Statistics and Pharmacokinetics Unit Manager, MHRA, UK

Ann-Kristen Leuchs
Federal Institute for Drugs and Medical Devices, Bonn, Germany

Chrissie Fletcher
 Executive Director, Global Biostatistical Science, Amgen Ltd, UK

Frank Bretz
Global Head Statistical Methodology group, Novartis, Basel, Switzerland

James Carpenter
Professor, London School of Hygiene and Tropical Medicine, UK

James Roger
Director, LiveData (UK) Ltd

Abstracts

Rob Hemmings (Statistics and Pharmacokinetics Unit Manager, MHRA, UK)

Estimands at ICH

After a number of years discussing the problem of ‘missing data’, solutions arose and became widespread practice, but these solutions seemed to violate the ITT principle as it is described in ICH E9.  Attention turned to identifying a precise answer to the question ‘what should we seek to estimate to demonstrate drug effects in a clinical trial’?  The breadth of this question called for an answer on a global scale and hence a concept paper was drafted, and eventually adopted, for a group to draft an addendum to ICH E9.  The talk will provide some more background on how the discussions came to be, the ICH process, the future plans of the group for developing the guidance and some initial thoughts from one regulator on what might be achieved and where further research work still needs to be done.

Frank Bretz (Global Head Statistical Methodology group, Novartis, Basel, Switzerland)

Estimands and their role in clinical trials

Defining the primary objective of a clinical trial in the presence of non-compliance or non-adherence to the assigned treatment is crucial for the choice of design, the statistical analysis and the interpretation of the results. At first glance this seems obvious, however, primary objectives stated in clinical trial protocols often fail to give a precise definition of the measure of intervention effect. The impact of potential confounding, e.g. due to non-compliance, missing data, treatment switching / discontinuation or intake of rescue medication, is frequently not taken into account when defining the intervention effect of interest. The need for a structured framework to specify the primary estimand (i.e. “what is to be estimated”) was highlighted in the context of missing data in the National Academy of Science document "The Prevention and Treatment of Missing Data in Clinical Trials"(2010). However, the need for clearly defined estimands applies to a broader setting. In this talk we will discuss the need for this framework, using real and hypothetical examples.

James R Carpenter (Professor LSHTM and MRC Clinical Trials Unit)

Estimands, Randomisation and Sensitivity Analysis

The increased focus on estimands - not just for sensitivity analysis, but for the primary analysis - is a welcome development, because it helps clarify the assumptions necessary for the analysis, and which of them can be verified.

However, it does raise a number of questions: -

1. How tightly should the estimand be defined? In other words, when is a sensitivity analysis actually changing the estimand?

2. What is the role of randomisation based inference?

3. To what extent should we worry if the sensitivity analysis violates the assumptions of the primary analysis?

I will argue that the way we answer these questions lead us to two different paradigms:

(a) Follow the established approach, where the primary analysis does not model deviations, and explore how robust inferences are to different assumptions about post-deviation behaviour

(b) Adopt a more formal causal approach to the primary analysis, in which the deviation process is explicitly accounted for.

If we shift to (b), we are moving decisively away from randomisation based inference. If we want to do this, it should be intentional, not accidental!

Plamen Kozlovski (Associate Global Program Medical Director at Novartis Pharma AG)

Estimands in Diabetes Trials – what are different stakeholders interested in?

Diabetes is a progressive disease which is associated with serious complications resulting from chronic exposure to elevated blood glucose levels and other metabolic abnormalities. Good blood glucose control assessed by glycated haemoglobin (HbA1C) is a key goal of treatment, as it has been demonstrated that near-normalization in HbA1c can prevent or delay the diabetic complications. This goal should be achieved without deterioration of the quality of life. However, when it comes to decision-making on priorities, the different parties involved in diabetes care – patients, health professionals, regulatory agencies, payers and pharmaceutical companies - may have different perspectives and needs. These differences in perspectives may imply different estimands – this may lead to challenges when designing a clinical development programme that aims to address the needs of all stakeholders.

Chrissie Fletcher (Executive Director, Global Biostatistical Science, Amgen Ltd, UK):

The importance of the proposed ICH E9 addendum on estimands and sensitivity analyses to the Pharmaceutical Industry

In recent years, there have been different perspectives emerging between regulators and Industry in terms of what estimand and measure of the intervention effect is of primary interest to be estimated for confirmatory trials supporting Marketing authorisations.  Choices made in how to deal with non-adherence issues, such as missing data or taking rescue medication, in the study design and planned analyses influences what is actually being estimated in the clinical trial.   In addition, sensitivity analyses conducted to support the conclusions from confirmatory trials can sometimes be misaligned with the estimand of primary interest, leading to difficulties in interpretation of results from confirmatory trials.  The results of a recent survey conducted by the ICH E9 working group who are developing the addendum will be presented highlighting current practices relating to choices of primary estimands, techniques for preventing and minimising missing data, and common sensitivity analyses used to support primary analyses.  Challenges with current practice in these areas and recommendations for overcoming these challenges will be discussed.   A summary of key topics debated in the ICH E9 working group will be provided including considerations for how the addendum will align with the current ICH E9 guidance. 

James Roger (Director, LiveData (UK) Ltd)

Whether to use MMRM as primary estimand.

There are two main impacts of early withdrawal on study results; first the potential selection bias caused by those withdrawing being different from the remaining subjects, and second the fact that subjects may receive alternative treatments after withdrawal. The most common method for handling early withdrawal in clinical studies is MMRM or some other form of missing at random (MAR) based analysis. The motivation behind MMRM is to solve the first issue while addressing an on-treatment question, i.e. what happens if a typical patient completes their assigned treatment. It does this by conditioning on the previous observations and other covariates that may inform on both missingness and outcome. So what is the scientific question of interest, i.e. the estimand that it targets? This cannot be answered without considering the second aforementioned issue which is related to treatment switching or modification after withdrawal.

If the design of the study, in terms of treatment after withdrawal from randomized treatment, matches the estimand, then collection of data after treatment withdrawal allows direct analysis. Then later absolute study termination after switching (truly missing data) can be handled via a modified MMRM approach. But when the design of the study after treatment withdrawal does not match the estimand any analysis must depend upon additional unverifiable information or assumptions. This is a whole new area of potential statistical research. Several of the more recent proposals ignore the first issue of selection bias. Any coherent approach must address both issues.

Ann-Kristen Leuchs (Federal Institute for Drugs and Medical Devices, Bonn, Germany)

Estimands in clinical trials and how they influence trial planning: a regulatory view

Estimands, which are precise definitions of that which is being estimated, are currently discussed in clinical research, especially in cases where post-randomization events such as non-adherence to treatment complicate the interpretation of trial results. Considering suitable estimands addressing the desired objectives and satisfying regulatory requirements is only the first step in appropriately planning clinical trials. The choice of estimand has consequences for various other factors to be considered during any trial’s planning phase. After deciding on the primary estimand, a trial design that enables addressing this estimand should be discussed. This, for example, includes considering measures to enhance retrieval of data or adherence to treatment. Following the specification of an appropriate design, an adequate primary analysis method that directly addresses the chosen estimand is to be selected including the use of retrieved data and missing data handling. The robustness of the primary analysis to deviations from its assumptions should be addressed in a final step by defining sensitivity analyses still addressing the primary estimand but using a different set of assumptions (internal validity). Moreover, sensitivity analyses aiming at alternative estimands can be considered to address the robustness with respect to the generalizability of results (external validity). This process is proposed as a suitable way to incorporate estimands in clinical development and illustrated with relevant examples from clinical practice and regulatory experience.

Registration Costs 

Fee includes lunch & refreshments

* event is co-organised by PSI and
located in the UK

For information regarding the scientific content, contact:
Carly Barnett 
Tel: +44 208 990 3781
Carly.m.barnett@gsk.com

Egbert Biesheuvel 
Tel: +31 6 46869728
Egbert.biesheuvel@danone.com