Cancer Immunotherapy Machine Learning Challenge

Challenge 1: Predicting T Cell State Proportions

Challenge 1 focused on predicting how specific CRISPR perturbations reshaped the distribution of T cell states within the tumor microenvironment. The underlying dataset consisted of 71,388 tumor-infiltrating T cells, including 4,978 unperturbed cells and 26,031 perturbed cells across 73 single-gene knockouts. After stringent preprocessing to remove doublets and cells with ambiguous guide assignments, the training data included all unperturbed cells and 23,719 perturbed cells corresponding to 66 of the 73 knockouts.

‍

The remaining seven knockouts were withheld for evaluation: Aqr, Bach2, and Bhlhe40 for validation, and Ets1, Fosb, Mafk, and Stat3 for testing. For each cell, participants received a normalized 15,077 gene expression vector, its perturbation condition, and an expert-annotated T cell state label chosen from progenitor, effector, terminal exhausted, cycling, and other. The task was to build a model that, given only the name of a held-out gene knockout and not its expression profile, predicted a five-dimensional probability vector representing the proportions of cells in these five states, with entries summing to one.

Challenge 2: Discovering Novel Perturbations for Immunotherapy

Challenge 2 extended beyond the original set of 73 knockouts and asked participants to propose novel CRISPR targets among all 15077 measured genes that were predicted to induce T cell state distributions favorable for cancer immunotherapy. Building on their Challenge 1 models, participants developed algorithms that, given the name of any gene among the 15077 measured genes passing quality control, output a predicted five-state cell proportion vector representing the effect of knocking out that gene.

These predictions underpinned three related sub-tasks corresponding to different therapeutic strategies, and top-ranked perturbations were experimentally validated in vivo in a mouse tumor model. In Challenge 2A, focused on immune checkpoint blockade, the goal was to identify knockout targets that maximized the proportion of progenitor T cells while ensuring that at least five percent of cells were in the cycling state to maintain sufficient cell expansion. Conceptually, predictions were ranked by their proximity, in L1 distance, to the idealized cell state vector (1, 0, 0, 0, 0), corresponding to 100% progenitor cells.

Challenge 3: Designing New Metrics for Perturbation Scoring

Challenge 3 moved beyond predicting cell state outcomes and instead focused on how perturbations should be scored and prioritized for downstream applications in cancer immunotherapy. Participants were asked to design new metrics that more effectively evaluated the ability of a gene knockout to move cells from undesired to desired states. Formally, let P0 denote the empirical gene expression distribution of unperturbed cells and Pi the distribution resulting from knocking out gene i. Participants proposed a statistic s(·) that summarized Pi (for example, but not limited to, cell state proportions) and a scoring function that combined P0, an optional desired distribution Q, and the predicted statistic Ŝ(Pi) to assign a score to each perturbation, with higher scores indicating more desirable knockouts.

The design had to balance biological richness with statistical feasibility: while higher-dimensional statistics can be more informative, they require larger sample sizes to estimate reliably, whereas lower-dimensional summaries may be easier to predict but less expressive.