OmiCairn

Campaign run · Parkinson's disease

parkinson-prs-portability-r3

PDCompleted the pipelineAttempt 3 of 3 at this topic

When a polygenic risk score (PRS) is built from the Kim 2024 multi-ancestry PD GWAS (GCST90275127) via LDpred-funct on a 1000-Genomes LD reference, does its expected liability-scale R² in 1000 Genomes individuals stratified by Saffie-Awad-2025- style super-population ancestry-clusters (EUR, EAS, AMR, AFR, AJ) match the Saffie-Awad-2025-reported AUC values within ±0.05 per ancestry — purely using public sumstats + 1000 Genomes individual genotypes (no UK Biobank, no GP2 individual access)?

  • Supported 1
  • Not supported 2
  • Inconclusive 1
  • Not testable (data) 1
Open the final report (PDF)This run in the hypothesis explorer

Status

How far it got
Completed the pipeline

Tested at least one hypothesis and reached the multi-agent review stage. This shows the run went through the whole pipeline, not that its work passed a quality bar.

Hypotheses

5 nominated; 5 evaluated.

Recorded status

Halted. The run recorded a final status of halted: it was closed without being marked done. A halted run can still have tested hypotheses and written a final report.

Census class

Success. Census class is a label from an earlier, process-level census of the campaign. It does not map one-to-one onto the stages above. How the classes compare.

Research question

The sentence under the run id is quoted from this brief.

Full research brief given to the run

When a polygenic risk score (PRS) is built from the Kim 2024 multi-ancestry PD GWAS (GCST90275127) via LDpred-funct on a 1000-Genomes LD reference, does its expected liability-scale R² in 1000 Genomes individuals stratified by Saffie-Awad-2025- style super-population ancestry-clusters (EUR, EAS, AMR, AFR, AJ) match the Saffie-Awad-2025-reported AUC values within ±0.05 per ancestry — purely using public sumstats + 1000 Genomes individual genotypes (no UK Biobank, no GP2 individual access)?

Saffie-Awad 2025 trained 105 PRSs across 7 ancestries on GP2 R9 individual data and reported AUC by ancestry (EUR 0.621, EAS 0.578, AAC 0.591, AJ 0.67, AFR 0.583, AMR 0.639, CAS 0.612). Reproducing those AUCs requires GP2 access. This proposal asks the methodological question: how close can a summary-statistics-only + 1000-Genomes-only re-implementation get? The answer is itself an informative methodological contribution about what portability benchmarks are possible without IRB-gated cohort access.

Hypotheses (pre-registered):

H1 (primary — EUR portability matches Saffie-Awad): PRS liability-scale R² in 1000 Genomes EUR individuals (CEU+GBR+FIN+IBS+TSI, n ≈ 503) is within ±0.05 of the liability-R² corresponding to Saffie-Awad 2025's Model 2 EUR AUC of 0.621 (translated via Lee 2012 formula given EUR PD prevalence 0.3%).

H2 (secondary — ancestry-decay ratio): Ratio of AUC(EAS) / AUC(EUR) is within ±0.1 of the Saffie-Awad-reported 0.578 / 0.621 = 0.93. (Ancestry-decay must reproduce roughly, not exactly.)

H3 (secondary — protein-MR PRS overlay): Adding the Storm 2025 protein-MR pQTL-instrumented gene-set as a fixed-effect overlay (PROteinMR-PRS) increases EUR liability-scale R² by ΔR² ≥ 0.01 vs. SNP-only PRS. Methodological test of whether protein-MR adds incremental information.

H4 (secondary — multi-ancestry GWAS does not hurt EUR): The Nalls 2019 EUR-only PRS (GCST009325) does NOT significantly outperform the Kim 2024 multi-ancestry PRS in the EUR 1000 Genomes stratum (DeLong p > 0.1). Tests the field concern that multi-ancestry GWAS hurts EUR-stratum performance.

H5 (secondary — bootstrap CI honesty): Per-ancestry bootstrap 95% CI is reported alongside point estimates. For ancestries with n < 200 in 1000 Genomes (e.g., AMR sub-populations), CI width > 0.15 is the expected outcome; this must be reported in analysis summary and negative_results.md, not buried.

Null payoff. If LD-mismatch and small 1000 Genomes per-ancestry sample sizes inflate error bars beyond ±0.05 (H1), the Wang & Visscher 2024 concern about summary-only PRS portability benchmarking is confirmed, and the run would put a quantitative number on it. Either outcome is informative.

The research question comes from the topic's current research brief. All attempts at a topic share it; earlier attempts may have started from an earlier wording.

Hypotheses and verdicts

Each hypothesis was written with its decision rule before any analysis (or, for on-the-fly hypotheses, recorded during execution). AI adjudication agents applied that rule to the run's latest outputs and quoted the evidence verbatim. Where the data did not meet the rule, the verdict follows the data, even if the run's own label or report claimed support.

Each hypothesis as pre-registered (a condensed statement with its decision rule), the adjudicated verdict, the verbatim evidence and the adjudicator's note.
IDHypothesis and decision ruleVerdict
H1Pre-registeredEUR 1000G analytical PRS liability-R2 (Kim 2024, LDpred) within ±0.05 of Saffie-Awad 2025 EUR benchmark (AUC 0.621)Not supportedIn the explorer

Evidence (verbatim from the run outputs)

[EUR] R²_obs=0.062550 | R²_L=0.026740 | AUC=0.5467 | 95% CI=[0.5440, 0.5501]

Adjudication note

Latest stage EXECUTE stage 9 (liftover + all-chrom AFs fixed) gives AUC 0.547 vs benchmark 0.621 (deviation_auc -0.074349); EXECUTE stage 4 and reviewed report also REJECTED (AUC 0.548, -0.073). Stage 9 lacks analysis summary/completion flag (stage incomplete) but results record is complete.

H2Pre-registeredEAS/EUR analytical AUC ratio within ±0.10 of Saffie-Awad ratio 0.931SupportedIn the explorer

Evidence (verbatim from the run outputs)

"ratio_eas_over_eur": 0.992253,

Adjudication note

Rule met formally (deviation +0.061 within ±0.10), but ratio CI [0.984, 1.000] excludes 0.931 and EAS<EUR ordering is not resolved; in stage 4 (basis of the reviewed report) support was explicitly 'artefactual' (build mismatch). stage 9 fixed liftover but was never analysed/reviewed. Low-confidence support of a weak tolerance test.

H3Pre-registeredStorm 2025 pQTL protein-MR overlay increases EUR liability-R2 by >= 0.01Not testableIn the explorer

Evidence (verbatim from the run outputs)

The Storm 2025 (Brain 148:2412) supplementary pQTL-instrumented gene list is not available in storm_2025_pd_targets.json (empty array). H3 cannot be evaluated.

Adjudication note

DEFERRED in stages 4 and 9 (recorded H3 verdict: DEFERRED).

H4Pre-registeredMulti-ancestry Kim 2024 PRS not significantly outperformed by EUR-only Nalls 2019 PRS in EUR stratumNot supportedIn the explorer

Evidence (verbatim from the run outputs)

[EUR] R²_obs=0.092491 | R²_L=0.039539 | AUC=0.5570 | 95% CI=[0.5539, 0.5608]

Adjudication note

DISCREPANCY: stage 9 code labels H4 SUPPORTED, but with corrected Nalls N_eff (145177) Nalls AUC 0.557 > Kim 0.547; results record delta_kim2024_vs_nalls2019 = -0.01032, CI [-0.014894, -0.005571] (Nalls significantly better) -> pre-registered REJECTED condition; reported delong_p=1.0 is the Kim>Nalls direction. Reviewed report (stage 4 data) said REVERSED, driven by an N_eff artefact. Analytical (label-free) AUCs; interpret cautiously.

H5Pre-registeredBootstrap AUC CI width >0.15 for at least one ancestry with n<200 (CI honesty)InconclusiveIn the explorer

Evidence (verbatim from the run outputs)

H5 is rejected — not because uncertainty is genuinely small, but because the chi-squared parametric bootstrap models uncertainty in a chi-squared sampling model

Adjudication note

Formal rule outcome REJECTED in stages 4 and 9 (AAC CI width ~0.011-0.014), but the run itself states the bootstrap measures analytical sensitivity, not sampling uncertainty (no individual-level labelled genotypes), so the test cannot answer the question. Could also be classed NOT_SUPPORTED by the letter of the rule.