PSI ToxSIG Webinar: Beyond the looking glass - Interpreting animal welfare & behaviour by monitoring & assessing mice activity data
Analysing continuously collected locomotive activity data to interpret mice welfare and behaviour.
Subgroup analysis is routinely conducted in drug development, in various settings; one key aspect is the regulatory requirement to demonstrate consistency of treatment effect across a pre-defined set of subgroups (e.g., ICHE5, E9, E17). This is performed as a risk-benefit assessment, aiming to identify the right patient population to treat - and, here, that set of subgroups is agreed with regulators prior to the trial conduct. Another key aspect is subgroup selection, where the aim is to estimate the effect in the most promising subpopulation (typically for planning another trial). The latter can either be done with respect to the same fixed set of pre-specified subgroups as mentioned earlier, or in a data driven fashion (e.g., biomarker subgroup detection).
There are well-known inherent statistical difficulties with all the above; with consistency, due to limited data in the subgroups, there is a high risk of false positives (random highs) as well as a low power to detect true differential effects (since trials are seldomly sized for it). In the subgroup selection setting, it is of key importance to provide an honest estimate discounted for the number of subgroups inspected, in order to not overstate the real effect. Even in a consistency assessment setting, there might well be a certain tendency to focus on the most deviating subgroup results, hence possibly introducing a bias although not formally a 'selection' problem.
The PSI Subgroup SIG submitted a White Paper in May 2018 on some of these aspects, containing an overview of the inherent problems, recommendations for the planning stage, a novel permutation based approach for assessing expected deviations under a null assumption, and some simulation based conclusions where various methods were compared. Due to the complexity not all available methods were initially studied (e.g., the Bayesian ones) and further work is being conducted. The aim is to provide an updated document later when these methods have been developed and evaluated.
Scope since 2018: the SIG is not only devoted to questions related to fixed (pre-specified) sets of subgroups, but also to the developments in personalized medicine, where lots of progress has been made in recent years on data-driven subgroup-detection/ML methodologies (causal inference, individual treatment effects and individual treatment regimes). The SIG aims to provide as much guidance and clarity as possible on inherent issues and approaches to the problems encountered in the these areas, and to promote cross industry research collaborations.
Some examples of topics discussed are: permutation-based approaches for assessing expected ranges and for generating NULL data/t1e control; Shrinkage methods of various kinds; Model averaging approaches; Bootstrap bias reduction, novel Graphical methods, general principles to consider in ML Subgroup Detection, ML in dose-response settings, ITR, causal inference.
Björn Bornkamp, Aaron Dane, Paulo Eusebi, Christine Fletcher, Ilya Lipkovich, Henrik Loft, Brian Millen, Heiko Goette, Necdet Gunsoy, Tom Parke, Arne Ring, Gerd Rosenkranz, Bohdana Ratitch, Kostas Sechidis, Amy Spencer, David Svensson and Marius Thomas.
The SIG is currently lead by David Svensson.
(Up to May 2018, the lead was Aaron Dane).
How to get in touch: firstname.lastname@example.org