Benchmarks of success in radiotherapy vs systemic therapy: National Clinical Trials Network (NCTN) randomized controlled trials sponsored by the National Cancer Institute (NCI) (2025)

Search for other works by this author on:

Article history

Abstract

Introduction

Randomized controlled trials generate the highest level of evidence in medicine but are resource intensive and take years to complete. To address these challenges in oncology, the National Cancer Institute (NCI) sponsored National Cancer Clinical Trials Network (NCTN) groups was founded with the goal of providing government funding and infrastructure to conduct trials that answer practice-changing questions for cancer care. Randomized trials conducted by the NCTN are estimated to have generated an additional 14.2 million life-years for patients with cancer.1

Reliance on the NCTN differs by oncological subspecialty. When considering systemic therapies, funding for drug trials has shifted from the NCI to pharmaceutical companies, which sponsored 89% of phase 3 randomized control trials conducted between 2010 and 2020.2 In contrast, most studies testing novel radiation therapy (RT) techniques are not industry sponsored.3 Thus, the specialty of radiation oncology depends almost entirely on the NCTN to conduct large, multicenter trials. Compared with industry, however, NCTN resources are more limited, and not all trials proposed to their respective steering committees can be funded. Even for studies that are approved, trade-offs in study design must be made to optimize feasibility, scientific rigor, and projected impact.

We previously reported on differences in primary endpoints by treatment modality among NCTN cooperative group randomized trials, specifically in gastrointestinal (GI) cancers.4 Our findings generated additional questions with regard to study design—endpoints, hypothesized effect sizes, noninferiority margins, statistical power, and sample size—in NCTN trials across disease sites. As such, we performed a comprehensive evaluation, the first to our knowledge, of active cooperative group randomized trials comparing differences in design between trials of RT vs systemic therapy. Based on our findings, we raise critical questions and provide recommendations for oncology studies across disease sites and treatment modalities.

Methods

Study identification

We used the Clinical Trials Support Unit (CTSU) website to identify all randomized trials across the NCTN cooperative groups, including ECOG-ACRIN, SWOG, the Alliance for Clinical Trials in Oncology (Alliance), NRG Oncology, and the Children’s Oncology Group (COG).5 The CTSU is a government-sponsored service designed to facilitate access to NCI-funded clinical trials by streamlining information and support services. The website lists all trials that are approved, accruing, or have completed enrollment but are still following patients per protocol. As such, the CTSU provides a comprehensive view of all active NCTN cooperative group trials. We captured all trials as of December 31, 2023.

Predefined study inclusion criteria were as follows: (1) disease site with more than 5 RT trials (because we wished to compare study design between trials of RT and systemic therapy, the disease sites with few [<5; n = 5] or 0 [n = 12] RT trials were excluded), (2) randomized trial, (3) comparison of systemic therapy or RT in at least 1 treatment arm (ie, studies assessing only surgical interventions [n = 1] or lifestyle [diet or exercise (n = 1)] were excluded) (Figure1).

Data collected

The following data points were obtained for each protocol available on the CTSU site: protocol number, ClinicalTrial.gov identifier, lead cooperative group, study status, title, trial phase, activation date, disease site, and planned accrual number. Studies were first characterized as RT vs systemic therapy trials. Radiation therapy trials included those assessing RT vs no RT or trials comparing different RT regimens (ie, different doses or modalities, such as proton vs photon therapy). Systemic therapy trials assessed different systemic regimens or compared systemic therapy with no therapy.

Next, trials were divided into the following 4 categories based on the nature of the trial question: (1) treatment intensification (ie, adding systemic therapy or RT to standard of care), (2) treatment deintensification (ie, decreasing or omitting systemic therapy or RT from standard of care), (3) comparison of dosing or timing (ie, trials comparing different dosing or timing schedules of systemic therapy or RT), and (4) comparison of agents or modalities (ie, trials comparing different systemic agents or RT modalities, such as proton vs photon). Each study protocol was reviewed to collect additional details regarding statistical design, including primary endpoints, hypothesis testing type (superiority vs noninferiority vs equivalence), patient disease population (early [stage I-III] vs late [stage IV] or recurrent), number of treatment arms, randomization ratio (ie, 1:1, 2:1), hypothesized effect size, statistical power, and statistical significance level (ie, ɑ). Trials for patients with recurrent cancers were grouped with late-stage cancers.

Analyses

Endpoint definitions

A study’s primary endpoint was determined based on the statistical design section of each protocol. Endpoints were categorized as time to event if they were defined as duration of time until occurrence of a specific outcome event, such as median overall survival or progression-free survival (PFS). In contrast, a binary endpoint was one with 2 clear choices, such as pathologic complete response (achieved vs not at surgery) or clinical complete response (achieve or not after neoadjuvant therapy). All non–overall survival endpoints were further categorized as event related vs symptom based. Event-related endpoints include composite endpoints for cancer recurrence, including PFS, disease-free survival (DFS), event-free survival or recurrence-free survival (RFS), while symptom-based endpoints capture quality of life (QOL) or specific treatment-related toxicity. Adjudication of unclear endpoints or design was undertaken by all authors with the advice of a senior statistician (Q.S.). Primary endpoints are defined in Table1.

Summary statistics

We first described the number of trials testing RT or systemic therapy across tumor types, stratified by trial phase (phase 2 vs 3) and disease stage (early vs late). To avoid counting a trial more than once, seamless phase 1/2 trials were considered phase 2 and the corresponding phase 2 endpoint and design were analyzed. Similarly, seamless phase 2/3 trials were considered phase 3, and the corresponding phase 3 endpoint and design were analyzed. Trials with separate randomizations and statistical designs dependent on enrollment criteria, however, were considered 2 studies (eg, NRG GI008, which assessed patients with positive vs negative circulating tumor DNA [ctDNA] separately).

Within each trial phase and disease stage, we first evaluated the following design parameters by treatment modality: (1) primary or co-primary endpoint (overall survival vs non–overall survival); (2) primary hypothesis testing method categorized as comparison between arms testing superiority, comparison between arms testing noninferiority or equivalence, or comparing with historical controls within each treatment arm; (3) effect size, defined as targeted improvements for superiority trials and noninferiority margin for noninferiority trials; (4) statistical power; and (5) statistical significance level. Next, we compared primary endpoints (overall survival vs non–overall survival) and primary hypotheses testing method by modality, stratified by trial question (eg, treatment intensification, treatment deintensification). Results are descriptive, with reporting of proportions, medians, and ranges for each study parameter. Because our sampling schema was restricted to NCTN studies of tumor types with sufficient available trials, they may not represent a true “statistical sample”; as such, comparisons are descriptive.

Results

A total of 551 NCTN cooperative group trials were included on the CTSU website as of December 31, 2023. The 5 disease categories with the greatest number of RT studies were breast, GI, lung, central nervous system (CNS), and prostate, yielding a total of 142 systemic therapy and 44/186 (23.7%) RT trials. The most common study question category for systemic trials was treatment intensification, comprising 107 (75.4%) of studies. For RT, treatment intensification was also the most common (n = 16 [36.4%]), however, closely followed by trials comparing dose or timing (n = 11 [25%]) and deintensification (n = 8 [18.2%]). Most RT trials were through NRG Oncology (n = 28 [63.6%]), while 20% to 30% of trials of systemic therapy were conducted through each of the following cooperative groups: Alliance, ECOG-ACRIN, SWOG, and NRG Oncology. Characteristics of the studies are shown in Table2.

Overall survival as a primary endpoint

Among all disease sites, 59.1% of RT trials had a primary endpoint of overall survival vs 26.8% of systemic trials. The difference varied by disease site and was most pronounced among lung cancer trials, where overall survival was the primary endpoint in 100% of RT studies vs 29.6% in randomized trials of systemic therapy (Figure2). Among breast cancer trials, few studies used overall survival as a primary endpoint overall, but proportionally this number was greater among RT trials (12.5% vs 5.6%). In contrast, there were no differences for trials in prostate and CNS cancers by treatment modality (Figure2). Similar differences were found when the analyses were restricted to phase 3 trials (56.7% vs 32.7%) (Figure2), but the differences were more pronounced in phase 2 trials, where overall survival was the primary endpoint in a greater proportion of RT trials and fewer systemic trials (71.4% vs 13.6%) (Figure2).

Among trials for late-stage cancer, overall survival was the primary endpoint for 100% of RT trials in breast, GI, lung, and prostate cancer. In contrast, the proportion of systemic trials with overall survival as the primary endpoint for late-stage cancer was one-third or less in breast (20%), GI (31.8%), and lung cancers (23.1%) and highest in prostate cancer, at 66.6%. Among CNS cancers, 50% of RT studies and 40% of systemic therapy studies had overall survival as the primary endpoint. For studies of localized cancers (stage I-III), overall survival was the primary endpoint in 51.5% vs 22.8% of RT vs systemic therapy trials, again with disparities by disease site (Figure2).

Choice of primary endpoint varied by nature of the trial question. Among trials assessing treatment intensification, overall survival was more common in RT trials than in systemic trials (81% vs 27%). For trials of treatment deintensification, overall survival was never a primary endpoint for systemic trials vs a small minority (12.5%) of RT trials. For trials comparing dosing or timing of radiation or systemic agents, overall survival was the primary endpoint in approximately 50% of trials in both modalities, and for studies comparing different drugs or radiation modalities, overall survival was the primary endpoint in two-thirds of RT trials and one-third of systemic trials (Figure S1).

Types of non–overall survival primary endpoints

The most common non–overall survival primary endpoints were event-defined surrogate endpoints, including DFS, PFS, event-free survival, and response rate (Figure3). Symptom-based primary endpoints were the minority across all trial modalities but nevertheless represented 33.9% of RT trials that did not have overall survival as a primary endpoint vs 1.9% of systemic studies. Examples of symptom-based primary endpoints include GI/genitourinary toxicity score (NRG GU003, NRG GU005), cognition (NRG BN005), and hearing loss (ACNS2031).

Noninferiority trials

The proportion of trials that used noninferiority or equivalence design was 31.2% for RT and 6.3% for systemic therapy (Table S1). The difference was greatest for trials in breast cancer, where 75% of all RT trials used a noninferiority or equivalence design. The choice of noninferiority design was concordant with trial question across modalities: Among studies assessing treatment deintensification, 100% were noninferiority for systemic trials and RT trials (Table S3).

The noninferiority margins, expressed by hazard ratios (HRs) comparing overall survival endpoint, ranged from 1.15 to 2.5 (median = 1.37) in RT trials and 1.15 to 1.7 in systemic trials (median = 1.25) (Table S1).

Target effect size

Among superiority trials for RT, the target hazard ratio for trials with overall survival as the primary endpoint ranged from 0.53 to 0.85 (median = 0.7 [IQR = 0.65-0.76]) and for non–overall survival event–defined endpoints, 0.45 to 0.65 (median = 0.56 [IQR = 0.5-0.6]). For systemic trials, hazard ratios for trials with a primary endpoint of overall survival ranged from 0.50 to 0.8 (median = 0.7 [IQR = 0.66-0.75]) and for non–overall survival event-defined endpoints, 0.40 to 0.81 (median = 0.65 [IQR = 0.6-0.7]). Figure4 shows data for hazard ratio, stratified by disease site.

Study ɑ and power

Most phase 2 (94.1%) and phase 3 (75.6%) trials used a 1-sided ɑ. Therefore, for purposes of comparison, trials with 2-sided ɑ were divided by 2.

For phase 2 trials, the median 1-sided ɑ was .1 (IQR = .1-.1), and the median power was .85 (IQR = .8-.9). In phase 3 trials, the median 1-sided ɑ was .05 (IQR = .025-.05), and the median power was .85 (IQR = .8-.9). There were outliers across disease sites, however, in trial phase and modality assessed, as shown in Figure5 and Table S2.

Missing information

Two studies were missing critical statistical analysis information in their protocols: One trial did not include an ɑ value, and another did not specify whether the ɑ value was 1 or 2 sided.

Discussion

Across NCTN clinical trials, we found a substantial difference in the design of RT trials compared with systemic trials, with overall higher benchmarks for achieving positivity in studies of RT. The proportion of RT trials with a primary endpoint of overall survival was more than double that of systemic therapy trials, and a larger proportion of RT trials were noninferiority studies assessing omission or deescalation. There were also wide variations in the statistical design of trials both within and between treatment modalities. To the best of our knowledge, our study is the first to assess statistical characteristics in NCI-funded cooperative group trials. Our results have immediate implications for study design and benchmarks.

Primary endpoint

Overall, the proportion of RT trials with a primary endpoint of overall survival was twice that of systemic therapy trials; among the subgroup of studies assessing treatment intensification, the difference was even greater (81% vs 27%). Overall survival is considered a gold-standard metric in oncology; thus, the considerable differences in the use of this endpoint across studies merits further discussion. On one hand, some may argue that for RT to be incorporated into a multimodality approach, often as an additional local therapy (along with surgery) with potential for both acute and late toxicity, the magnitude of benefit must be substantial; thus, the bar should be high, necessitating overall survival as the preferred endpoint. On the other hand, given that RT targets only areas of known or suspected cancer, one could also contend that expecting the local control improvement afforded by RT to translate to a survival benefit may be more difficult to demonstrate in many disease settings and that mitigating tumor growth alone could be sufficient to approve the use of RT, particularly in cases where QOL is improved. We recognize both perspectives, along with the reality that trial design is highly nuanced. In addition, we acknowledge that a comprehensive discussion of the “right” endpoint for oncology studies, including the strength of evidence for surrogate endpoints, is outside the scope of this work, which primarily aimed to illustrate differences in trial design by therapeutic modality across the cooperative groups.

With regard to specific cooperative groups, most RT trials were conducted through NRG Oncology. Because NRG Oncology was formed by combining the National Surgical Adjuvant Breast and Bowel Project (NSABP), the Gynecologic Oncology Group (GOG), and the Radiation Therapy Oncology Group (RTOG), it may not be surprising that there is greater RT focus and expertise for NRG Oncology trials vs those from other cooperative groups. It is possible that groups may have differing trial design preferences, with NRG Oncology historically favoring overall survival as the primary endpoint. At the same time, compared with systemic therapies, a greater proportion of RT trials are conducted through the NCTN because most RT treatments are not tied to industry. In contrast, drugs are often initially evaluated in the phase 1 and phase 2 settings through pharmaceutical company–sponsored trials, with the definitive phase 3 study through NCTN; thus, one might also expect these trials to be more likely to use overall survival as the primary endpoint.

There are several reasons why investigators may select a primary endpoint other than overall survival, including longer projected study timelines and often a larger sample size if overall survival is used. There may also be concern that salvage, second-line therapies, and crossover could mask a survival benefit. Importantly, these considerations apply to all oncologic modalities and are not unique to systemic therapy. One could even argue that crossover is a higher risk for RT-focused studies because radiation delivery for an oncologic indication is not typically restricted by US Food and Drug Administration (FDA) approval requirements. Yet even among trials for stage IV disease, where survival is often limited, 38% of systemic therapy studies had overall survival as an endpoint vs 83% of RT trials.

Another notable finding was that a minority of studies had QOL as a primary or co-primary endpoint, although the proportion was higher in RT trials than in systemic trials (34% vs 2%). We hypothesize several possible explanations. Although QOL has been accepted by the oncology community as an essential patient-centered outcome, measuring it accurately presents a challenge because a variety of QOL instruments exist but there are few standards for timing of assessment or meaningful effect size.6 Studies also have shown disagreement between physician-reported vs patient-reported QOL outcomes, with symptom minimization by physicians.7-10 We argue that patient-reported outcomes should be given greater priority in the NCTN trial portfolio.

Finally, the difference in overall survival as a primary endpoint between RT trials vs systemic trials within phase 2 vs phase 3 trials warrants further consideration. Phase 2 trials are considered “signal finding” studies, conducted with a smaller sample size to determine whether a regimen warrants further confirmation testing in a definitive phase 3 trial.11 If the phase 2 trial is negative, the treatment is considered ineffective, and no additional resources are granted for further assessment. Among RT trials with a phase 2 design, 71% had overall survival as the primary endpoint vs 14% of systemic therapy trials. We worry the high bar for success among RT studies in the phase 2 setting may lead to early termination of a regimen that could have meaningful benefit. Such a finding could inappropriately exclude RT as unhelpful (higher risk of a type II error), when in fact a larger study could demonstrate a different result.

Noninferiority vs superiority trial design

Noninferiority trials are typically conducted with the goal of treatment deescalation. If the oncologic outcome of the experimental regimen is not worse than standard of care by a prespecified margin, it is deemed preferential, given presumed trade-offs for side effects and convenience. Such trials assessing treatment deintensification and omission are critical in oncology because safe therapeutic deescalation represents an advancement for patients and payors. Given limited trial resources, however, individuals may disagree about the appropriate balance between studies attempting to lessen therapy (risking potentially worse outcomes) vs those seeking to improve oncologic endpoints.

Despite overall fewer RT studies, 33% were noninferiority trials assessing deescalation of therapy (vs 7% of systemic trials). Among breast cancer RT trials, the proportion was 75%. Potential reasons for these differences may include negative public perceptions regarding RT12 and concerns regarding toxicity based on historical regimens, which could motivate disproportionate efforts to reduce RT usage.

Alpha, statistical power, effect size, and standardization of statistical design reporting

We did not observe substantial differences in power and effect size in trials of RT vs systemic therapy, but there were outliers for both parameters across modalities. We were surprised that in 2 phase 3 studies, information about trial ɑ was not specified in any part of the entire protocol. Furthermore, in some protocols, study design features were difficult to ascertain for nonstatistician readers. Also, there was variation in reporting of certain parameters. For example, some protocols used hazard ratios, while others used a hazard rate; some included a hypothesized absolute effect difference, but many did not. Because design features such as ɑ, power, and target effect size are critical in trial understanding, it is imperative that these features be easily accessible and interpretable to all individuals involved in the study, including those without extensive statistical training and experience. Also, given our study findings, there should be consideration for consensus on optimal power calculations across NCTN cooperative group trials or, at minimum, careful scrutiny of trials with outlier statistics. The interquartile ranges presented here can be considered good practice recommendations that could enable studies to be held to common standards.

Limitations

Limitations of our study include lack of longitudinal analyses because the list of trials was obtained at 1 point in time from the CTSU website. However since the study activation dates of the trials included span decades, our findings should be representative of the cooperative group trial landscape over a comprehensive period. We also limited our analyses to the 5 sites with the greatest number of RT trials, so our conclusions regarding ɑ, statistical power, effect size, and missing data may not be extrapolatable to other disease sites. Our results are descriptive, yet the marked differences in study design by modality are apparent. Finally, although we advocate for a degree of consensus on standards for study design by therapeutic modality, study phase, and disease setting, we also recognize that each trial design must be tailored to its research question and that a cookie-cutter approach would be detrimental to scientific innovation.

Trials conducted through the NCTN represent one of the most respected and important scientific endeavors funded by the NCI. Our comprehensive analysis of cooperative group trials demonstrates a striking difference in design between studies of RT vs systemic therapy. As compared with trials of systemic therapy, a larger number of RT studies seek to eliminate or deintensify radiation regimens through a noninferiority design. With regard to superiority trials, most in radiation oncology are benchmarked to improve overall survival, a threshold arguably harder to meet. Furthermore, there was variation in reporting of statistical design across modalities, which may create additional barriers to trial implementation and interpretation. Our findings provide impetus to reevaluate the design of future NCI-sponsored pivotal trials and, in doing so, mitigate any potential therapeutic bias in options presented to patients and further accelerate the impact of cooperative group randomized controlled trials.

Author contributions

Nina N. Sanford, MD (Conceptualization; Data curation; Formal analysis; Methodology; Writing—original draft; Writing—review & editing), Qian Shi, PhD (Conceptualization; Formal analysis; Methodology; Writing—original draft; Writing—review & editing), David Hein, MS (Data curation; Formal analysis; Writing—original draft; Writing—review & editing), William Hall, MD (Conceptualization; Methodology; Supervision; Writing—original draft; Writing—review & editing).

Supplementary material

Supplementary material is available at JNCI: Journal of the National Cancer Institute online.

Funding

This work was funded by Dedman Family Scholar in Clinical Care.

Conflicts of interest

W.A.H. reports consulting fees from Aktis Oncology and Sonoptima, departmental research support from Elekta, travel support from Elekta, and an equity interest in Sonoptima. N.N.S. reports consulting fees from Signatera and AstraZeneca. Q.S. and D.M.H. had nothing to disclose.

Data availability

Data are available on a publicly available website and upon request from the authors.

References

© The Author(s) 2024. Published by Oxford University Press. All rights reserved. For commercial re-use, please contact [emailprotected] for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact [emailprotected].

Issue Section:

Article

Email alerts

Citing articles via

More from Oxford Academic

Having trouble contacting the network. Please try again in a moment or two.