The frozen atlas contains 29 element tracks across about twenty loci, 15 promoter and 14 enhancer tracks, and 39,170 measured single-base substitutions. Some loci were assayed in several cell types or constructs, which is why the number of tracks exceeds the number of loci.
An independent reproduction of AlphaGenome on a saturation-mutagenesis atlas: zero-shot recovery of disease regulatory element maps and blind recovery of the known TERT promoter cancer hotspots
Abstract
Sequence-to-function models predict regulatory activity from DNA, but whether they capture the effect of individual base substitutions in disease regulatory elements is best judged against exhaustive experimental maps. This is an independent reproduction of that test. We asked a pretrained DNA sequence model, AlphaGenome, to predict the effect of every single-base substitution across a saturation-mutagenesis reporter atlas of about twenty disease-associated human regulatory elements, without showing it any measured value. AlphaGenome has itself already been evaluated on this benchmark, so the result is a reproduction rather than a new model finding. Across the atlas the model recovers the measured maps above chance. On the TERT promoter the track-averaged predicted effect correlates with the measured effect at Spearman 0.53. The correlation is specific and collapses under a label-permutation null. Following a pre-registered procedure, the model’s most impacted predicted bases are enriched for clinically significant variants. The variant it ranks first is the pathogenic TERT C250T promoter mutation, a recurrent cancer driver that is already known, and the neighbouring C228T hotspot is recovered in the same blind ranking. The analysis uses open data and a public non-commercial model API, costs a few dollars, and runs in an afternoon. We report the method, the per-element recovery, and the conditions under which it succeeds.
1 Introduction
Non-coding variation drives a large fraction of disease risk, yet the functional consequence of a single base change in a regulatory element is hard to read from sequence alone. Massively parallel reporter assays measure that consequence directly. Saturation mutagenesis takes this to its limit and measures the effect of every possible single-base substitution in an element1.
In parallel, DNA sequence models have learned to predict regulatory activity from sequence2. The natural test of such a model is whether it can redraw a measured saturation-mutagenesis map, base by base, for elements it was not trained to fit. If it can, the model becomes a cheap in-silico assay for prioritising causal non-coding variants.
We independently reproduce the reported behaviour of a recent DNA sequence model, AlphaGenome3, on an open saturation-mutagenesis atlas of about twenty disease-associated regulatory elements1. These elements formed the CAGI5 Regulation Saturation community benchmark, in which supervised models were scored against the same measured maps, with sequence-based methods such as deltaSVM reaching only modest per-element correlations4. AlphaGenome has itself already been evaluated on this benchmark, so the result here is a reproduction rather than a new model finding, and that benchmark is the reference point for the correlations reported. The model is used zero-shot. It never sees a measured value. We ask three questions. Does the model recover the measured maps above chance. Is the agreement specific rather than an artifact of the score distribution. And do the bases the model predicts to be most impactful coincide with variants that are independently known to be clinically significant. The aim is a validated, low-cost, reproducible recipe for reading a disease regulatory element base by base, and one concrete variant nomination. AlphaGenome is used under its non-commercial API terms. The code and the data are open. The model weights are not.
2 Methods
2.1 Data
The ground truth is the saturation-mutagenesis reporter atlas of1, retrieved from the authors’ repository and frozen at a fixed commit. The atlas provides, for each element and genome build, one measured log2 activity effect per single-base substitution. We use the GRCh38 coordinates.
2.2 Coordinate validation
Before any prediction, each element’s reference sequence was reconstructed from the frozen table and checked against the GRCh38 reference from the Ensembl-associated UCSC service. This guards against the single most common failure mode, a coordinate or strand error between the measured table and the model’s input.
All 29 of 29 element tracks matched the GRCh38 reference on the forward strand at or above 99 percent. The TERT-HEK promoter, used below as the primary test, matched at 100%.
2.3 Oracle and in-silico saturation mutagenesis
For each element a window of 16,384 bp was centred on the element. AlphaGenome performed in-silico saturation mutagenesis over the element, scoring every single-base substitution. Variant effects were summarised with AlphaGenome’s recommended centre-mask scorer, which reports the log2 change in the summed signal of a regulatory output track over a window centred on the variant. Two output modalities were scored, chromatin accessibility (DNASE) and cap analysis of gene expression (CAGE). The model was run through the public AlphaGenome API and saw only DNA.
2.4 Scoring and controls
For each element and output modality, the predicted effect of a variant was averaged across all cell type tracks of that modality, which involves no within-modality track selection. We report chromatin accessibility (DNASE) as the primary readout because it gives the higher rank correlation across the atlas, and we report the expression readout (CAGE) alongside it with equal prominence; for promoters the two are close and CAGE is marginally higher. Agreement with the measured effect was quantified by the Spearman rank correlation across the element’s substitutions. Two controls were applied. A label-permutation null shuffled the measured effects within each element to confirm the correlation is not a distributional artifact. A pre-registered recovery test asked whether the top decile of predicted absolute effects coincides with clinically significant variants more than a permutation null of the same size.
2.5 Variant annotation and nomination
Known variants overlapping each element were retrieved from the Ensembl variation REST API on GRCh385 for their rsID and consequence. Clinical significance was taken per allele from ClinVar, so a classification applies only to the exact substitution it belongs to and not to the other alternate alleles at the same position. This annotation was queried after the predictions existed and never entered the model. The nomination criterion was fixed in advance (see the pre-registration): the nominated variant maximises the smaller of its predicted and measured absolute rank, subject to a consistent sign between predicted and measured effect.
3 Results
3.1 The model recovers the TERT promoter map zero-shot
The TERT promoter is the primary positive control. Its 259 bp are fully saturated, so all 777 single-base substitutions are measured. Predicting from sequence alone, the model reproduces the measured map.
The track-averaged DNASE predicted effect correlates with the measured effect at Spearman +0.534 across all 777 substitutions, with the best single accessibility track reaching +0.557. The expression readout (CAGE) gives a comparable Spearman +0.495. This is a zero-shot result with no fitted parameters.
3.2 The same holds for a distal enhancer
The SORT1 enhancer at the 1p13 cardiovascular locus is one of the best characterised human enhancers. Predicting its map from sequence alone recovers the measured structure, though the correlation is lower than for the TERT promoter, consistent with the harder grammar of distal elements.
3.3 The agreement holds across the atlas and is specific
Across the 29 element tracks, the median track-averaged DNASE Spearman is +0.368, which reproduces above-chance recovery across the atlas. Under the label-permutation null the median is -0.000 (permutation p = 0.0005), so the agreement is specific to the pairing of predicted and measured effects. The tracks are not independent: several are the same locus assayed in different cell types, so the 29 tracks collapse to 21 distinct loci, which are the proper inferential unit. Averaging contexts within each locus, the median DNASE Spearman across loci is +0.317 (95 percent bootstrap 0.25 to 0.39, 10,000 resamples, seed 0), and 19 of 21 loci are positive. The locus-level median is lower than the track-level median, and it is the value to read as the headline. The TERT enrichment below is a pre-selected positive control and is reported as descriptive, not as a confirmatory test. Promoters are recovered better than enhancers (median +0.403 versus +0.303), consistent with the model’s stronger resolution of promoter-proximal regulatory grammar.
The recovery is not confined to the best element. Across a spread of elements from the strongest to the weakest, the predicted and measured effects track each other, and the few elements with no signal are visible as the diffuse panels.
The gap between promoters and enhancers, and between the two readout modalities, is summarised below.
3.4 Blind recovery of the known TERT promoter cancer hotspots
The model’s predictions are blind to any annotation. We then asked whether the positions it ranks as most impactful are also flagged in an independent clinical database, using allele-specific ClinVar. The three flagged alleles here are not equivalent. TERT C250T is a ClinVar Pathogenic record, while C228T and the c.-57A>C allele carry conflicting classifications of pathogenicity in ClinVar. We treat all three as ClinVar-flagged alleles and do not read a conflicting record as an unqualified Pathogenic one. We test at the position level because the flagged variants sit at only a few positions; scoring each position by its strongest predicted effect avoids treating those few as exchangeable across every substitution. TERT is a pre-selected positive control, so this is a descriptive check rather than a confirmatory test.
The element spans 259 positions, of which 3 carry a ClinVar-flagged variant. Ranking positions by their strongest predicted effect, all 3 of 3 fall in the top decile (26 positions), against a permutation null of 0.29 (permutation p = 0.0010). The ClinVar flag and the measured effect size are not fully mechanistically independent, so this is corroboration rather than a wholly orthogonal validation.
Nomination. The pre-registered criterion nominates chr5:1295135 G>A, which is a known variant, rs1561215364, reported as Pathogenic. Its predicted effect is +1.633 and its measured effect is +1.423, both strongly activating. On the minus-strand TERT gene this genomic substitution is the C250T promoter mutation, one of the two canonical TERT promoter cancer drivers6–8. The model placed it at the top of its blind ranking.
3.5 A supervised baseline, trained on AWS, and where the pretrained model still wins
The natural question is how a purpose-built supervised model, trained on these very effects, compares to the pretrained model that never sees them. We trained one and compared the two head to head. The model is a gradient-boosted decision-tree regressor on sequence-derived features only: the k-mer spectrum of the reference window around each position, the change in that spectrum caused by the substitution (the deltaSVM idea), and a few local context features, with no coordinate or absolute-position feature. This is the feature-and-model family the CAGI5 Regulation Saturation community challenge was built on4,9. It was trained on Amazon Web Services, as a managed SageMaker job launched from the command line; the trainer and the launcher are in the repository.
We score it two ways, both as the field does. The first is the CAGI5 protocol: per element, hold out whole positions, train on the rest, and correlate predicted with measured effect. The second is a deliberately harder cross-element stress test, leave-one-locus-out, in which the model must predict a locus it never saw, with elements that share genomic coordinates grouped so that no locus straddles train and test.
Under the within-element protocol the supervised model reaches a median Pearson +0.485 (Spearman +0.377) across 29 elements, with promoters at +0.509 and enhancers at +0.352. This sits within the band of the strongest CAGI5 supervised submissions (about 0.45 continuous Pearson) and above the classical single-track deltaSVM level, so it is a fair, competitive comparator rather than a strawman.
Scored on the same central substitutions the supervised model uses (34,867 of the 39,170, the rest dropped for lacking a full context window), the zero-shot AlphaGenome model, which sees no measured value, is competitive with the trained one: median Pearson +0.419 (Spearman +0.382) against the supervised model’s +0.485. The supervised model is modestly ahead by the median, but AlphaGenome, with no training, wins the head-to-head on 15 of 29 elements. It wins where it matters most. On the TERT promoter it reaches Pearson +0.708 against the supervised model’s +0.346. Under leave-one-locus-out the supervised model collapses to a median Spearman of +0.057, while the pretrained model needs no training and keeps its per-element Spearman of about +0.382. The right reading is cross-task transfer without fitting the MPRA labels, not a matched generalisation test. AlphaGenome was trained genome-wide and was never put through the supervised model’s leave-one-locus-out split. On this atlas of about twenty loci the supervised model does not transfer across loci. We report this as an observation, not a general law. The cross-locus collapse of trained models has been described before9,10, and the pattern in which a pretrained sequence model matches or beats supervised CAGI5 entries reproduces an earlier result2.
The comparison is the point. A pretrained model, given no measured values and no task-specific training, is competitive with a supervised model built on the community-standard recipe on this atlas, and is far ahead on the flagship cancer promoter. It also transfers to an unseen locus without fitting the measured labels, where the trained model collapses. This is cross-task transfer rather than a matched generalisation test, and it is the practical case for reading a disease element with a pretrained sequence model rather than training a new one for every assay.
4 Discussion
A pretrained DNA sequence model, used with no fitted parameters and no sight of any measured value, reproduces the saturation-mutagenesis map of a disease promoter at a rank correlation of about 0.5, recovers the measured maps across an atlas above a permutation null, and concentrates its highest predicted effects on bases that are independently annotated as clinically significant. The nominated variant is the pathogenic TERT C250T promoter mutation, and the neighbouring C228T mutation is recovered in the same ranking. These two substitutions create de-novo ETS transcription factor binding sites and are among the most common non-coding driver mutations in human cancer8. The zero-shot correlations reported here fall within the range of the supervised methods evaluated on these same CAGI5 elements and exceed the classical deltaSVM baseline, despite the model using no parameters fitted to the assay4.
The contribution is not that a sequence model can read these maps. The AlphaGenome authors already evaluated their model on this exact CAGI5 saturation-mutagenesis benchmark, reaching Pearson 0.57 with cell-type-matched DNase, 0.63 with an all-cell DNase regression, and 0.65 with a multimodal regression3. What is offered here is an independent reproduction with a deliberately simple, tissue-agnostic recipe: a single all-track average with no cell-type selection, run through a public API for a few dollars in an afternoon. On top of it we add locus-level uncertainty across the 21 distinct loci, an allele-specific ClinVar audit of the highest-impact predictions, and a reusable interactive explorer of the measured and predicted maps. The code and the data are open. The model runs through a public non-commercial API and its weights are not open.
4.1 Limitations
The zero-shot correlation of about 0.5 is meaningful but far from the experimental ceiling; a fitted readout head over cell type tracks would likely improve it and is left for future work. The track-averaged readout deliberately avoids cell type matching, which trades peak accuracy for an honest, selection-free headline. The clinical-recovery test rests on the variants that happen to be annotated in a single element and is strongest where annotation is dense, as at the heavily studied TERT promoter; it should be read as a proof of concept rather than a calibrated screen. Enhancers are recovered less well than promoters. This is consistent with the harder regulatory grammar of distal elements, but it is also confounded by two choices: the window is AlphaGenome’s smallest supported context, and the centre-mask scorer is local, so an enhancer whose activity depends on a distal target promoter may have that target clipped from the window. A larger-window sensitivity analysis was not run and is left for future work. The measured atlas is itself a reporter assay and does not observe the native chromatin context.
4.2 Conclusion
Reading a disease regulatory element base by base with a public sequence model is now cheap, open, and reproducible. Used zero-shot, the model recovers the measured maps above chance and places a canonical cancer driver at the top of its blind ranking. That recovery is a strong positive control rather than a novel discovery, and it is exactly the property that makes the approach trustworthy as a triage layer. The method, the per-element recovery, and the conditions under which it succeeds are reported here for reuse.
5 References
1.
Kircher, M. et al. Saturation mutagenesis of twenty disease-associated regulatory elements at single base-pair resolution. Nature Communications 10, 3583 (2019).
2.
Avsec, Ž. et al. Effective gene expression prediction from sequence by integrating long-range interactions. Nature Methods 18, 1196–1203 (2021).
3.
Avsec, Ž., Latysheva, N., Cheng, J., et al. Advancing regulatory variant effect prediction with AlphaGenome. Nature 649, 1206–1218 (2026).
4.
Shigaki, D. et al. Integration of multiple epigenomic marks improves prediction of variant impact in saturation mutagenesis reporter assay. Human Mutation 40, 1280–1291 (2019).
5.
Cunningham, F. et al. Ensembl 2022. Nucleic Acids Research 50, D988–D995 (2022).
6.
Huang, F. W. et al. Highly recurrent TERT promoter mutations in human melanoma. Science 339, 957–959 (2013).
7.
Horn, S. et al. TERT promoter mutations in familial and sporadic melanoma. Science 339, 959–961 (2013).
8.
Vinagre, J. et al. Frequency of TERT promoter mutations in human cancers. Nature Communications 4, 2185 (2013).
9.
Kreimer, A., Yan, Z., Ahituv, N. & Yosef, N. Meta-analysis of massively parallel reporter assays enables prediction of regulatory function across cell types. Human Mutation 40, 1299–1313 (2019).
10.
Sasse, A. et al. Current sequence-based models capture gene expression determinants in promoters but mostly ignore distal enhancers. Genome Biology 24, 181 (2023).
Rendered from committed tables in results/tables/ at source commit 7ffd096. Atlas element tracks: 29. Measured substitutions: 39,170.