Rpgen

Rpgen is the genomic record source for the Rfmalloc stack. It uses PLINK 2’s own pgenlib and import code to read PGEN, PLINK 1 BED, PED/MAP, TPED/TFAM, VCF/BCF, BGEN, Oxford GEN, HAPS/legend, EIGENSTRAT, and legacy PLINK 1 dosage data into file-backed compute layouts.

The central operation is rpgen_ingest():

library(Rfmalloc)

pgen <- system.file("extdata", "chr21_phase3_start.pgen", package = "Rpgen")
backing <- tempfile(fileext = ".bin")
rt <- open_fmalloc(backing, mode = "scratch", size_gb = 1)

dose <- rpgen_ingest(
  pgen,
  representation = "dosage",
  runtime = rt
)

cat(sprintf(
  "%d samples x %d variants, %d payload bytes\n",
  nrow(dose), ncol(dose), length(unclass(dose))
))
#> 2504 samples x 485 variants, 1214464 payload bytes

The abstraction

A format reader emits bounded panels of semantic records. Rfmalloc owns the destination allocation and layout:

Representation	Meaning	Storage
"hardcall"	0, 1, 2, or missing non-reference count	2 bits per sample and variant
"dosage"	fractional non-reference count	1-byte fixed point
"haplotype"	two phased ref/non-ref bits per sample	locus-major, 1 bit per call
"f64"	full-precision dosage	file-backed doubles

This is the useful split. A source declares whether it produced integers, doubles, or packed phase bits. The destination decides whether those records remain full precision or become a compute codec. Readers do not know codec headers, alignment, or allocation details.

PGEN and BED keep one pgenlib reader open and transfer bounded variant panels directly into that destination. They never construct a complete genotype matrix. Haplotype reads use PgrGetP() and preserve phase. A missing genotype or unphased heterozygote is an error because the one-bit HMM layout cannot represent either without a separate mask.

Format closure at the record sink

Source	Companion arguments	Current route
PGEN	optional pvar	persistent pgenlib reader
PLINK 1 BED	optional bim, fam	persistent pgenlib reader
PLINK 1 PED/MAP	map	importer, bounded transpose scratch, record sink
PLINK 1 TPED/TFAM	tfam	importer to record sink
VCF/BCF	none	importer to record sink
BGEN	optional sample	importer to record sink
Oxford GEN	sample	importer to record sink
HAPS/legend	legend, sample, chr	bounded count, phased importer to record sink
EIGENSTRAT	ind, snp	importer to record sink
PLINK 1 dosage	fam, map	importer to record sink

PLINK 2 already exposes each decoded variant immediately before STPgenWriter encodes it. During rpgen_ingest(), Rpgen redirects every writer append shape to Rfmalloc’s record context, including multiallelic, phase, dosage, and sparse records. No genotype PGEN is written or read back. The importers may write temporary .pvar/.psam metadata companions, which are removed on exit. PED/MAP also retains PLINK 2’s sample-major transpose scratch because changing that physical order is an actual algorithmic transpose, not a package-boundary copy. HAPS needs a bounded preliminary line count because its upstream writer receives an upper bound instead of the exact variant count.

The standalone rpgen_import_*() functions do not activate this hook. They continue to produce ordinary PGEN files for interoperability.

The dense rpgen_read_hardcalls() and rpgen_read_dosages() functions remain available for small data, inspection, and reference comparisons. They are not used by the bounded ingestion path.

Phased HAPS

rpgen_ingest(..., representation = "haplotype") stores variants as aligned, contiguous locus rows over donor haplotypes. This is the access direction used by Li and Stephens HMM kernels such as kalis Forward/Backward:

hap <- rpgen_ingest(
  "cohort.haps",
  format = "haps",
  representation = "haplotype",
  legend = "cohort.legend",
  sample = "cohort.sample",
  chr = "1",
  runtime = rt
)

Provenance and license

The vendored pgenlib tree is generated by tools/vendor-pgenlib/vendorpgen.R. The PLINK 2 importer closure is generated by tools/vendor-plink2-import/vendorplink2import.R. Both recipes are pinned and checked for drift in CI. See inst/COPYRIGHTS and the two PROVENANCE.md files for source pins, local patches, and licenses.

Rpgen is GPL-3 because it combines with pgenlib’s LGPL-3 code. The vendored zstd, libdeflate, SIMDe, and SFMT components retain their own licenses.

Installation

From the r-universe:

install.packages("Rpgen",
  repos = c("https://rgenomicsetl.r-universe.dev", getOption("repos")))

or from the monorepo subdirectory:

pak::pak("sounkou-bioinfo/Rfmalloc/packages/Rpgen")

Rpgen depends on Rfmalloc and builds on Linux, macOS, and Windows. CI also checks Apple Silicon and Linux aarch64.