ClinVarbitration arbitration algorithm

Scope and authority

RClinVarbitration implements a policy adapted from Centre for Population Genomics (CPG) ClinVarbitration 2.2.11, pinned at commit 658b9f241eb2d43aa11214b153b19c1e18a16337. The package identifies it as cpg-clinvarbitration-2.2.11.

The result is an alternative aggregation of retained ClinVar submissions. It is not NCBI’s official ClinVar aggregate, a new submission to ClinVar, or a clinical classification by this package. The original VCV, RCV, and SCV values remain available in the source relations so every policy result can be audited.

There are two decision scopes:

  • clinvar_policy_decisions groups one decision by release, profile, VCV, allele, and disease key. This is the preferred XML-derived surface.
  • clinvar_policy_allele_decisions groups by release, profile, VCV, and allele. It is the closest XML-derived counterpart of upstream’s allele-level output.

The separate rclinvarbitration_reproduce_clinvarbitration_parquet() function reads the archived NCBI flat files only for differential validation against an upstream artifact. It is redundant for normal XML-based use.

Inputs retained for arbitration

Each SCV contributes its submitted classification, review status, last assessment date, submitter, source order, assertion identity, and SCV version. Disease-scoped arbitration also uses the condition links attached to the SCV. Named policy profiles and per-call submitter exclusions can remove evidence without deleting it from the imported source tables.

Step 1: classification bins

Matching is case-insensitive for XML-derived policy views. Submitted labels are collapsed as follows.

Policy bin Submitted ClinVar labels
Pathogenic/Likely Pathogenic Pathogenic; Likely pathogenic; Pathogenic, low penetrance; Likely pathogenic, low penetrance; Pathogenic/Likely pathogenic
Benign Benign; Likely benign; Benign/Likely benign; protective
VUS Uncertain significance; Uncertain risk allele
Unknown every other or missing label

Unknown rows are ineligible and do not enter the denominator used below. This follows the actual upstream collection path, which discards unknown classifications before arbitration.

Step 2: exclusions

The intended upstream qualified exclusion removes a benign submission whose normalized submitter is exactly illumina laboratory services; illumina. RClinVarbitration applies that rule. At the pinned upstream commit, an inner-loop continue does not actually remove that row; this package follows the documented intent rather than that Python control-flow accident.

The package then applies exclusions stored in clinvar_policy_submitter_exclusions for the selected profile. A row may apply to every classification from a submitter or to one classification bin only. The Parquet export can add submitter_exclusions for a blinded run. Names are trimmed and matched case-insensitively; the imported evidence is unchanged.

Step 3: SCV version deduplication

XML can expose versioned assertion structure that is absent from the upstream flat-file row model. The XML policy views retain one row per assertion identity within each decision group, preferring the highest SCV version, then a stable assertion/source order. Disease-scoped deduplication includes the disease key. The flat-file differential reproducer intentionally does not add this XML-only step.

Step 4: 2016 evidence window

An eligible submission is called modern when either:

  • date_last_evaluated is on or after 2016-01-01; or
  • its normalized review status is practice guideline or reviewed by expert panel.

If a decision group contains at least one modern submission, only modern submissions are retained. Otherwise, all eligible submissions are retained. A missing date is treated as 1970-01-01. Strong reviews are therefore never removed solely because their date is old or missing.

The views expose both eligible and retained counts and whether this filter was applied.

Step 5: decisive strong review

If any retained submission has review status practice guideline or reviewed by expert panel, the first such submission in source order supplies the decision. This is an order rule, not a vote and not a rule that always puts a practice guideline ahead of an earlier expert-panel row. The source ordinal is retained explicitly so this behavior is reproducible.

Step 6: 60/20 arbitration

When no strong review is decisive, let:

  • \(P\) be retained Pathogenic/Likely Pathogenic submissions;
  • \(B\) be retained Benign submissions;
  • \(U\) be retained VUS submissions; and
  • \(N = P + B + U\).

The first matching rule wins:

  1. If both \(P>0\) and \(B>0\), accept the larger class only when \(\max(P,B) \ge 0.60N\) and \(\min(P,B) \le 0.20N\).
  2. If both pathogenic and benign remain but the 60/20 test fails, return Conflicting.
  3. If \(U > 0.60N\), return VUS. This threshold is strictly greater than 60%.
  4. If \(P>0\), return Pathogenic/Likely Pathogenic.
  5. If \(B>0\), return Benign.
  6. Otherwise return VUS.

VUS submissions are part of the denominator. This matters: six pathogenic, two benign, and two VUS submissions pass the rule (60% and 20%), whereas six pathogenic and three benign submissions fail the minority threshold.

Step 7: star assignment

Stars are calculated from the same retained evidence, but only pathogenic and benign bins can contribute:

  1. any practice guideline: 4 stars;
  2. otherwise any expert-panel review: 3 stars;
  3. otherwise any review status other than no assertion criteria provided: 1 star;
  4. otherwise: 0 stars.

This is a maximum evidence level. A strong VUS submission can determine the classification through the strong-review rule, but it does not itself add stars, matching the pinned upstream implementation.

Disease identity

Disease-scoped decisions prefer a source condition identifier. The normalized key is lower(database):identifier. Where no such pair is available, the fallback order is ClinVar trait-set ID, normalized disease name, then the package’s condition entity ID. Canonical cross-references are selected in this order: MedGen, MONDO, OMIM, Orphanet, MeSH, UMLS, then OMIM phenotypic series. These are pragmatic grouping rules and are audited as deviations from NCBI’s own aggregate semantics in the deviation and differential audit.

Output and audit columns

The decision views retain policy/release/profile identity, source VCV and allele IDs, classification, stars, eligible and retained counts, distinct SCV and submitter counts, per-bin counts, modern-filter status, and the latest retained assessment date. Disease decisions additionally retain disease identity and name.

Complete-release differential validation

The March 2026 archived flat files were executed through both the pinned upstream 2.2.11 TSV code and this package’s flat reproducer. All 4,125,389 keys, classifications, and star values matched exactly. The manifest records input, code, configuration, and output digests.

The monthly XML and flat products are not row-equivalent: 16 shared keys differ and 361 keys occur on only one path. Every key is classified in the audit with source-row receipts. Of 377 total, 369 arise because the flat classification is - while XML has a current germline classification, one is a divergent three-row flat representation of one XML SCV assertion, five use different source vocabularies, and two are child locations of compound CYP2C19 alleles excluded by the top-level allele compatibility export. These are source-model and documented scope differences, not unexplained decision arithmetic.

Use source tables to inspect a result rather than treating the projected label as sufficient evidence:

SELECT d.*, s.scv_accession, s.submitter_name, s.classification,
       s.review_status, s.date_last_evaluated
FROM clinvar_policy_decisions d
JOIN clinvar_disease_submissions s
  ON s.release_id = d.release_id
 AND s.vcv_accession = d.vcv_accession
 AND s.allele_id = d.allele_id
 AND s.disease_key = d.disease_key
WHERE d.release_id = 'my-release'
  AND d.vcv_accession = 'VCV000091629';