---
title: "Serialized R objects in BLOBs"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Serialized R objects in BLOBs}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

For objects outside the [native mappings](native-values.html), serialize them
yourself. A raw vector is stored as a `BLOB`. CanardAbsurd never picks a codec,
decodes bytes, or falls back to serialization on your behalf. The same contract
holds over Quack.

## Base R

A matrix and a language object are not native values, so the handler
serializes them in a format it controls:

```{r rds-workflow}
library(CanardAbsurd)
db <- ca_open()
object <- list(matrix = matrix(1:6, 2L), expression = quote(x + 1L))
id <- ca_spawn(db, "rds-v3", serialize(object, NULL, version = 3L))
outcome <- ca_run(ca_claim(db), function(input, task) {
  ca_step(task, "transform", function() {
    value <- unserialize(input)
    value$matrix <- value$matrix * 2L
    serialize(value, NULL, version = 3L)
  })
})
unserialize(outcome$result)
```

SQL sees only the type and size of the bytes, not what is inside them:

```{r blob-inspection}
DBI::dbGetQuery(db@con, "
  WITH payload AS MATERIALIZED (
    SELECT input FROM canard_absurd.tasks WHERE id = ?
  )
  SELECT variant_typeof(input) AS native_type,
         octet_length(input::BLOB) AS bytes
  FROM payload
", params = list(id))
```

## qs2

`qs2` also produces raw vectors. Choose the decoder in the worker:

```{r qs2-workflow}
packed <- qs2::qs_serialize(object, nthreads = 1L)
ca_spawn(db, "qs2-v1", packed)
outcome <- ca_run(ca_claim(db), function(input, task) {
  ca_step(task, "copy", function() {
    value <- qs2::qs_deserialize(input, nthreads = 1L)
    qs2::qs_serialize(value, nthreads = 1L)
  })
})
qs2::qs_deserialize(outcome$result, nthreads = 1L)
```

## Sakura: rebuilding reference objects

Objects with external pointers, such as Arrow tables, torch tensors and open
connections, cannot be copied into another R session.
[Sakura](https://github.com/RConsortium/sakura) serializes them through
class-specific hooks that record how to rebuild them. Here the hook saves a
reader's bytes and cursor position, and the rebuilt object opens a fresh
connection at that position.

```{r sakura-configuration}
cfg <- sakura::serial_config("buffer_reader",
  function(reader) {
    serialize(list(bytes = reader$bytes, position = seek(reader$con)), NULL)
  },
  function(bytes) {
    saved <- unserialize(bytes)
    reader <- new.env(parent = emptyenv())
    reader$bytes <- saved$bytes
    reader$con <- rawConnection(saved$bytes, "rb")
    seek(reader$con, saved$position)
    class(reader) <- "buffer_reader"
    reader
  })

reader <- new.env(parent = emptyenv())
reader$bytes <- charToRaw("alpha\nbeta\n")
reader$con <- rawConnection(reader$bytes, "rb")
class(reader) <- "buffer_reader"
readLines(reader$con, n = 1L)
packed <- sakura::serialize(reader, cfg)
close(reader$con)
```

The handler rebuilds the reader, uses it, and cleans up. Its checkpoint is a
plain string:

```{r sakura-workflow}
ca_spawn(db, "buffer-reader-v1", packed)
outcome <- ca_run(ca_claim(db), function(input, task) {
  ca_step(task, "line", function() {
    restored <- sakura::unserialize(input, cfg)
    on.exit(close(restored$con))
    readLines(restored$con, n = 1L)
  })
})
outcome$result
ca_close(db)
```

## Your responsibilities

- **Version the format.** Record the serializer, its version and the hook
  configuration with the task or its handler name. Sakura selects hooks by
  position, so reordering a configuration changes which decoder reads old bytes.
- **Match the environment.** Workers need compatible packages and decoders.
  Sakura uses native binary serialization, so test every architecture you mix.
- **Test restoration.** Check a rebuilt object's operations in a fresh process.
  Rebuilding does not resume a transaction, socket or call stack, and files or
  services the object refers to must still exist.
- **Deserialize only trusted bytes.** R objects and hooks can carry executable
  code.
- **Resubmit the original bytes.** Equal R objects do not always serialize to
  identical bytes, and idempotent resubmission compares bytes.
- **Keep bulk data elsewhere.** BLOBs travel as hexadecimal SQL text, twice
  their size. Store large datasets in files or dedicated tables and pass
  references.