Rgguf reads GGUF model files - the format used by llama.cpp to store large language model tensors and metadata - and exposes their tensors as decoded Rfmalloc arrays, copied native codecs, or zero-copy read-only views over the original model mapping.
GGUF parsing and writing use the official implementation from GGML, carried once by the sibling Rggml package. Rgguf maps tensor data read-only and owns no second GGUF parser.
There are three storage paths:
as = "numeric": R allocates the destination with
Rfmalloc::create_fmalloc_matrix()/create_fmalloc_array(), which
returns a properly classed, file-backed ALTREP object - Rfmalloc’s
Ops/matrix-product dispatch already works on it. Rggml’s official
parser locates the tensor in the read-only GGUF mapping. Native code
writes dequantized values directly into the destination’s REAL()
payload (the ALTREP Dataptr exposes the file-backed fmalloc
memory directly - no extra copy back into an ordinary R vector).as = "native": encoded bytes are copied into owned fmalloc
storage. This is useful when the GGUF file may disappear or the
payload needs an independent lifetime.as = "view": the tensor borrows its exact span in the original
read-only mapping. The view keeps the mapping alive and gives codecs
or GGML the same pointer, so model loading performs no weight copy.GGML/GGUF tensors store dim[0] as the fastest-varying (contiguous)
dimension. R arrays are column-major (the first dimension varies
fastest), so dim[0] maps directly onto R’s first dimension:
dim(gguf_tensor(x, name)) == c(dim[0], dim[1], ...), with no
transposition needed. This is verified by a dedicated round-trip test
(inst/tinytest/test_gguf_roundtrip.R).
# install.packages("remotes")
remotes::install_github("sounkou-bioinfo/Rfmalloc")
remotes::install_github("sounkou-bioinfo/Rgguf")
The example uses local() so its on.exit() cleanup stays scoped to
the snippet. In your own code, keep the runtime handle for as long as
its arrays are live and call Rfmalloc::cleanup_fmalloc() when
finished.
library(Rgguf)
local({
gguf_file <- tempfile(fileext = ".gguf")
alloc_file <- tempfile(fileext = ".bin")
rt <- Rfmalloc::open_fmalloc(alloc_file)
on.exit({
Rfmalloc::cleanup_fmalloc(rt)
unlink(c(gguf_file, alloc_file))
}, add = TRUE)
# Build a small example GGUF file (gguf_write_tensors() only emits F32
# tensors; it exists mainly to make round-trip tests possible without
# shipping binary fixtures).
set.seed(42)
gguf_write_tensors(gguf_file,
tensors = list(
w1 = matrix(rnorm(100 * 50), nrow = 100, ncol = 50),
w2 = matrix(rnorm(50 * 30), nrow = 50, ncol = 30)
),
metadata = list(name = "example-model", version = 1)
)
# Inspect metadata and the tensor directory without reading any payload.
str(gguf_metadata(gguf_file))
print(gguf_tensors(gguf_file))
# Read tensors into Rfmalloc-backed matrices sharing one backing file.
mats <- gguf_import(gguf_file, runtime = rt)
print(Rfmalloc::is_fmalloc_vector(mats$w1))
# Matrix products dispatch through Rfmalloc and stay file-backed.
product <- mats$w1 %*% mats$w2
print(dim(product))
print(Rfmalloc::is_fmalloc_vector(product))
print(max(abs(product - as.matrix(mats$w1[, ]) %*% as.matrix(mats$w2[, ]))))
})
#> List of 2
#> $ name : chr "example-model"
#> $ version: num 1
#> name type n_dims dims n_elements nbytes offset
#> 1 w1 f32 2 100, 50 5000 20000 192
#> 2 w2 f32 2 50, 30 1500 6000 20192
#> [1] TRUE
#> [1] 100 30
#> [1] TRUE
#> [1] 0
gguf_tensor(as = "native") skips dequantization at import: the
tensor’s raw GGUF payload is copied into fmalloc storage at its original
density (4.5 bits/weight for q4_k) and returned as an
Rfmalloc::fmalloc_tensor. Matrix products against dense operands
decode the payload in bounded, block-aligned panels streamed through
BLAS dgemm, so the full double representation is never materialized.
Rgguf registers GGML-backed codecs for q4_0, q4_1, q5_0, q5_1,
q8_0, q2_0, q2_k, q3_k, q4_k, q5_k, and q6_k; f32,
f16, bf16, and f64 codecs are built into Rfmalloc.
q2_0 means GGML’s official group-64 ternary block: one f16 scale and
16 packed bytes for 64 values. Files which reuse type id 42 for an older
group-128 layout are a different format and are deliberately not
accepted as q2_0. A provisional type outside GGML’s enum is likewise
not guessed from its filename. Storage type ids acquire meaning from the
pinned upstream GGML implementation, not from model-repository naming
conventions.
gguf_tensor(as = "view") has the same typed-tensor interface but does
not copy the payload. It is the inference path: Rllm points GGML
directly at the borrowed bytes. Rfmalloc::fmalloc_storage_advise() can
express sequential, prefetch, and release intentions over the view for
out-of-core schedulers.
local({
gguf_file <- tempfile(fileext = ".gguf")
alloc_file <- tempfile(fileext = ".bin")
rt <- Rfmalloc::open_fmalloc(alloc_file)
on.exit({
Rfmalloc::cleanup_fmalloc(rt)
unlink(c(gguf_file, alloc_file))
}, add = TRUE)
set.seed(7)
gguf_write_tensors(gguf_file, list(
w = matrix(rnorm(256 * 64), nrow = 256, ncol = 64)
))
ten <- gguf_tensor(gguf_file, "w", runtime = rt, as = "native")
print(ten)
x <- matrix(rnorm(8 * 256), nrow = 8)
y <- x %*% ten # decoded panel-by-panel inside dgemm
print(dim(y))
print(Rfmalloc::is_fmalloc_vector(y))
w_f64 <- gguf_tensor(gguf_file, "w", runtime = rt)
print(max(abs(y[] - (x %*% w_f64)[])))
})
#> <fmalloc_tensor f32 [256 x 64], 65536 payload bytes>
#> [1] 8 64
#> [1] TRUE
#> [1] 0
gguf_tensor() and gguf_import() use GGML’s authoritative decoder for
the file’s declared type. Plain numeric types are widened directly.
Quantized types decode through a fixed 64 KiB float scratch into the
caller-owned double destination, independent of tensor size.
gguf_open()
gguf_open() parses metadata once and maps tensor bytes read-only. A
readable model file is sufficient; Rgguf never modifies it.
GPL (>= 2). Rgguf contains no vendored third-party source. Rggml documents the pinned official GGML sources it provides.