Researchers working with the ESRC Centre for Care have developed an open-source Python library that helps scientists determine whether their findings remain convincing across different, but equally defensible, ways of analysing data.
What is RobustiPy?
RobustiPy enables researchers to explore an analytical “multiverse”: the range of reasonable decisions that can arise when defining outcomes, selecting variables, specifying models and evaluating results. Instead of presenting evidence from a single preferred analysis, users can examine the wider pattern across many plausible alternatives and identify which conclusions remain stable.
The work, by Centre for Care researchers Daniel Valdenegro, Jiani Yan, Duiyi Dai and Associate Professor Charles Rahal, is described in a new paper in Patterns.
RobustiPy brings together large-scale specification searches, bootstrap-based uncertainty estimation, model selection and averaging, out-of-sample evaluation, joint summaries and explainable AI within one reproducible workflow. It is designed to help researchers understand not only what a particular model finds, but how much that finding depends on the reasonable analytical choices made along the way.
The complete source code is available through the RobustiPy GitHub repository. It is released under the GNU General Public License v3.0, allowing researchers and developers to inspect, use, modify and redistribute the software under the terms of the licence. The package is also documented, available through the Python Package Index and archived for reproducibility.
How can it be applied?
The software has applications across the health and social sciences, but it is particularly relevant to social care research. Evidence about care often draws on complex surveys, longitudinal studies and administrative records. Researchers may have several valid measures of wellbeing, care needs or service use, alongside many plausible decisions about which personal, household and local circumstances should be considered.
Consider a hypothetical evaluation of a new local-authority programme offering respite care, advice and practical support to unpaid carers. Researchers might want to know whether access to the programme is associated with improved wellbeing and a greater ability to remain in paid employment.
There would be several defensible ways to conduct the analysis. Carer wellbeing could be measured through mental health, life satisfaction, loneliness or reported stress. Researchers might choose to account for the carer’s age, income and health; the intensity and duration of caring; the needs of the person receiving care; or differences between local areas. They might also need to decide how to analyse repeated observations of the same carers over time.
A conventional study might report one selected combination of these choices. RobustiPy would allow the research team to define the reasonable alternatives in advance and examine how consistently the result holds across them.
The analysis might show that access to the support programme is associated with better wellbeing across nearly every defensible outcome measure and model. That would provide a stronger and more transparent basis for confidence than one result alone. Alternatively, it might reveal that the association is concentrated among carers providing particularly intensive care, or appears for mental health but not employment outcomes. That would still be valuable evidence, helping policymakers understand who may benefit, where uncertainty remains and what future research should examine.
Social care is too important for our confidence to rest on whichever sensible model happened to be run first… RobustiPy gives the defensible alternatives a fair hearing—and, mercifully, does not ask anyone to read several thousand regression tables. It does not replace scientific judgement; it gives that judgement a clearer audit trail.
Associate Professor Charles Rahal
The paper demonstrates RobustiPy through five simulations and ten empirical examples spanning economics, sociology, psychology and medicine. One application uses longitudinal data to examine local-authority adult social care spending and carers’ subjective wellbeing in England, illustrating the package’s relevance to questions about care policy and population wellbeing.
The tool nevertheless extends well beyond social care. Wherever researchers face multiple defensible ways to operationalise a concept, select variables or evaluate a quantitative model, RobustiPy offers a structured way to examine the consequences of those choices.
RobustiPy is not intended to replace substantive expertise. Researchers must still decide which models, measurements and assumptions belong in a defensible analytical multiverse. Running many poorly justified models does not make an analysis reliable. Instead, the software makes researchers’ decisions more explicit and reveals how much they matter for the final conclusion.
By making analytical uncertainty visible, measurable and easier to communicate, RobustiPy can help social care researchers present a fuller account of the evidence—not only whether an association was found, but how reliably it survives across the reasonable choices involved in finding it.
The project aligns closely with the Centre for Care’s commitment to producing accessible evidence that can improve how care is experienced and provided.
About the author
Charlie is an Associate Professor in Data Science and Informatics at the University of Oxford. He is a member of the Senior Management Board at the Leverhulme Centre for Demographic Science, a Co-Investigator at the ESRC Centre for Care, and a steering group representative at Reproducible Research Oxford (as well as being a Local Network Lead for the UK Reproducibility Network).
