On Predicting Probability of Technical and Regulatory Success With AI
Insights from The American Society of Clinical Oncology (ASCO) Annual Meeting 2024
Hi! This is BiopharmaTrend’s weekly newsletter, ‘Where Tech Meets Bio,’ where we talk about technologies, breakthroughs, and great companies moving the biopharma and medtech industries forward.
If you've received it, then you either subscribed or someone forwarded it to you. If the latter is the case, subscribe by pressing this button:
Webinar Invite: How Causal Inference Can Increase the Probability of Success in Clinical Trials
The mantra “Correlation does not equal causation” is well known. However, causal inference, a new set of statistical techniques, can identify causal relationships and even derive experimental results and counterfactuals from historical data.
In this webinar, the expert speakers from GSK and Intelligencia AI will introduce causal inference and apply it to ten years of oncological clinical trial data, seeking opportunities to optimize trial design decision-making.
What if there was a proven process to identify trade-offs that could save precious resources when designing trials with the greatest probability of success?
Speakers will present results from studying patient selection biomarkers, which focus a trial on a subset of patients, possibly increasing the trial’s PoS. Conversely, this focus can reduce the patient population for which regulators approve a drug. To help companies consider this trade-off, the analysis provides unbiased, precise estimates of how much the focus increases a trial’s PoS. This analysis can help identify opportunities for underutilized biomarkers that can significantly impact PoS or where they are overutilized and their use offers little benefit in terms of PoS, unnecessarily reducing the target population.
The findings shared will empower sponsors to understand the impact of their choices better, de-risk pipelines and make more informed decisions regarding their clinical development strategies.
Attendees will learn:
How to use causal inference to understand the impact of clinical design decisions, specifically focusing on biomarkers inclusion criterion and clinical trial success
The importance of combining quality data with the proper methodologies for more informed decisions
How to apply state-of-the-art methodologies to clinical trials to uncover underlying causal relationships that can optimize decision-making
The correlation between biomarkers and the probability of technical and regulatory success
Register for this webinar today to learn how causal inference can revolutionize clinical trial design decision-making.
Now, let’s get to this week’s topic!
Insights from ASCO 2024: What AI-Driven PTRS Assessments Unveil
(This guest post is authored by: Andreas Dimakakos, PhD, MBA, Director of Scientific Insights at Intelligencia AI and Elena Hilario Martinez, MSc, Customer Success Associate at Intelligencia AI).
The American Society of Clinical Oncology (ASCO) Annual Meeting ranks as one of the largest and most impactful oncology conferences. The content published during the annual meeting can change established practices, and many of the trends identified reflect the current or future industry landscape in oncology.
In this blog, we’ll showcase trends from ASCO 2023 vs. 2024, highlight our time-stamped pre- and post-ASCO predictions, and discuss the business implications this data and analysis have on de-risking clinical development and augmenting decision-making. This is particularly relevant for professionals responsible for identifying in- and out-licensing opportunities, assessing the acquisition potential and driving portfolio strategy in pharmaceutical companies.
A Comparative Analysis: ASCO 2023 vs 2024
Data always has a story to tell, and we can learn a lot from comparing data year over year, identifying trends and gathering actionable insights. We ran a comparative analysis between ASCO 2023 and ASCO 2024 abstracts.
Here are some of the inferred observations.
Based on the number of high-impact abstracts, different indications pulled the majority of the conference’s interest. At ASCO 2024, the focus on bladder, liver and prostate cancer significantly increased. On the other hand, the focus on myeloma, renal and colorectal cancer decreased compared to 2023.
Lung cancer, the leading cause of cancer-related deaths worldwide, retained the highest indication-specific focus.
Bladder, liver and prostate cancer only had a limited number of approvals in the past five years, which signifies an unmet need and justifies an increased industry interest.
Myeloma, renal cancer and colorectal cancer, on the other hand, have received numerous approvals (21 for myeloma, 9 for renal and 10 for colorectal since 2020) in the past few years, so market saturation is higher.
Going deeper into the data, specifically on the targets of the assets described in the abstracts, interest in Kirsten rat sarcoma virus (KRAS), vascular endothelial growth factor (VEGF) and hepatocyte growth factor (HGF) rose significantly in ASCO 2024.
Due to some recent approvals, KRAS has proven to be a very promising target in non-small cell lung cancer (NSCLC). Therefore, the industry has shifted its focus towards producing more KRAS-targeting drugs.
On the other hand, the epidermal growth factor receptor (EGFR) showed a significant reduction in focus. EGFR has mostly been targeted in NSCLC, and in the past few years, interest has shifted towards other mutations, like ALK, TP53 or the previously mentioned KRAS.
The heat map below (Figure 3) demonstrates that antibody-drug conjugates (ADCs) have been at the cutting edge of oncology research for the past few years. They gained significant traction between 2023 and 2024 (a 10% increase in Phase 3 ADC-focused trials) when several high-profile acquisitions of ADC-focused companies occurred. Conversely, there was a significant drop in monoclonal antibody (mAb) focus, showing signs of saturation and the start of a paradigm shift towards more targeted therapies.
ASCO Predictions at the Program Level
Identifying industry trends and keeping your finger on the pulse is undoubtedly a requirement for decision-making, but on its own, it leaves significant room for uncertainty and interpretation. It might be clear that a specific modality or target is trending. However, there will still be questions about the ideal indication, trial phase(s) and actual trial design. Information provided at the clinical program level significantly reduces that uncertainty.
By our definition, a program (also known as clinical pipeline or drug pipeline) is the clinical development of a drug (or a set of drugs in case of combination therapies) by a pharmaceutical company (alone or in collaboration with other partners) for an indication. A program consists of a set of clinical trials with the ultimate goal of marketing approval. Each program has unique and specific parameters that can justify a separate regulatory approval. Specifically, the definition of a clinical program is one of unique drug(s), drug dosage, mode of administration, adjuvant state, indication, sponsor, disease severity (e.g., stage of disease), line of treatment and biomarker information used as inclusion criteria.
Intelligencia AI’s solutions aim to reduce uncertainty and de-risk decision-making in pharma and financial market industries by providing an accurate probability of technical and regulatory success (PTRS) for each active clinical program in the entire oncology therapeutic area. The predictive performance of our algorithms in forecasting FDA approvals for Phase 2 oncology programs boasts a prospectively validated accuracy of 83%*and 90% when measured retrospectively.
To showcase the accuracy and robustness of our leading AI-driven PTRS assessments, we identified high-impact ASCO 2024 abstracts, measured the PTRS before and immediately after abstract publication, and time-stamped our predictions.
Let’s take the IBI343 clinical program in pancreatic cancer as an example. Before ASCO 2024, it was in the top PTRS quartile compared to peer programs (same phase and indication); the announcement of positive outcomes (40% ORR) boosted its PTRS by 37%.
Similarly, in the case of Abemaciclib in human epidermal growth factor receptor 2 negative (HER2-) breast cancer, the announcement of positive outcomes (primary endpoint met with PFS of 5.6 months) boosted its PTRS by 24%
Our PTRS algorithm’s accurate prediction of positive outcomes presented at ASCO 2024 highlights its reliability and precision. The high initial PTRS scores for these programs, even before the conference, demonstrate the algorithm’s strong predictive capability. Plus, the post-ASCO increase in PTRS scores for these programs confirms the algorithm’s sensitivity and ability to provide timely updates. This level of precision is crucial for high-stakes decisions such as go/no-go determinations, investment choices, and acquisition strategies.
The sponsor’s portfolio strategists could use this information to prioritize funding for this program. In the case of a positive prediction, this might include starting to prepare for subsequent trial(s) early to gain a significant time-to-market advantage. In the case of negative predictions, they might discontinue the program as early as possible and divert funds to more promising assets in the pipeline.
A Look at Our Data and Methodology
Two different datasets were leveraged for the analysis. The first used in the comparative analysis captured the entirety of ASCO abstracts corresponding to industry-led, interventional, FDA-track trials. For the second data set for the PTRS analysis, only Late Breaking Abstracts (LBAs) and those that drew the most attention by being showcased in specific ASCO sessions or pharma news sources were included. As per ASCO, LBAs include original research studies that highlight novel and high-impact research with practice-changing implications.
Our methodology provides a PTRS assessment for each clinical program. This significantly differs from other solution providers, who generally provide it at the drug-indication level. Clinical program-level assessments allow users to compare combinations, monotherapies, and even different administration modes in the same drug-indication. Combining that with Intelligencia AI’s capability to provide PTRS explainability and evolution over time results in an unprecedented level of data granularity that can be leveraged to de-risk decision-making further.
One limitation of the comparative analysis is that only two ASCO meetings were considered. However, the number of abstracts is significant enough to provide a reasonable level of reliability. As data from subsequent meetings are incorporated, the validity of the insights derived will further improve.
Applicability of ASCO Insights and AI-Driven PTRS Assessments
Our focus with this analysis differs from the traditional approach of simply highlighting relevant abstracts. We aim to generate valuable insights that optimize the pharma industry’s search and evaluation practices and portfolio strategy.
Access to the abstracts is public information, but what sets us apart is how we capture and utilize the relevant data. Our dedicated team of therapeutic area experts meticulously analyzes and assesses the abstracts. Combined with our proprietary data-capturing tools, which harmonize the information using custom ontologies, timely updates and expert QA throughout, we deliver a level of insights that allows us to predict the success or failure of individual programs with high accuracy and distinguish us from the competition.
In addition to identifying specific programs that show the most promise, either for investment or competitive intelligence reasons, search and evaluation professionals and portfolio strategists can test their strategic approach and assumptions against this combination of insights and predictions. With our time-stamped PTRS assessments and detailed documentation of their evolution, organizations can compare past strategic decisions with the predictions available at that time.
Understanding which trends and programs are currently most critical allows for the potential to pursue (or not) new clinical trials or continue (or end) current ones while maximizing the likelihood of successful transitions or approvals.
Conclusion
ASCO serves as an accurate representation of the industry’s direction. Due to the methodical process and deep granularity in our data, we’re uniquely positioned to help pharmaceutical companies dive even deeper. Combining that extensive knowledge with a robust, validated and time-stamped PTRS decreases the uncertainty indecision-making based on powerful AI and expertly curated clinical data. Using our approach, search and evaluation professionals can identify promising assets and test their strategies and decisions against past and current data. At the same time, portfolio strategists can prioritize the most promising assets among their pipelines.
**Based on Intelligencia AI predictions as of May 2022 for oncology programs with definite announcements in 2023.