kd.s11n API

Serialization and deserialization utilities.

kd.s11n.dump(x: DataSlice | DataBag, path: str, /, *, overwrite: bool = False, riegeli_options: str | None = None, fs: Any | None = None) -> None

Aliases:

• kd.dump

Serializes a DataSlice or a DataBag to a file.

In case of a DataSlice, we try to use `x.extract()` to avoid serializing
unnecessary DataBag data. If this is undesirable, consider serializing the
DataBag directly.

Args:
  x: DataSlice or DataBag to serialize.
  path: Path to the file to write.
  overwrite: If True, overwrites the file if it already exists.
  riegeli_options: A string with riegeli/records writer options. See
    https://github.com/google/riegeli/blob/master/doc/record_writer_options.md
      for details. If not provided, 'snappy' will be used.
  fs: FileSystemInterface to use for file I/O. If not provided, the default
    file system interaction is used.

kd.s11n.dumps(x: DataSlice | DataBag, /, *, riegeli_options: str | None = None) -> bytes

Aliases:

• kd.dumps

Serializes a DataSlice or a DataBag.

In case of a DataSlice, we try to use `x.extract()` to avoid serializing
unnecessary DataBag data. If this is undesirable, consider serializing the
DataBag directly.

Due to current limitations of the underlying implementation, this can
only serialize data slices with up to roughly 10**8 items.

Args:
  x: DataSlice or DataBag to serialize.
  riegeli_options: A string with riegeli/records writer options. See
    https://github.com/google/riegeli/blob/master/doc/record_writer_options.md
      for details. If not provided, 'snappy' will be used.

Returns:
  Serialized data.

kd.s11n.experimental_safer_loads(x: bytes) -> Any

Aliases:

• kd.experimental_safer_loads

(experimental) Deserializes a DataSlice or a DataBag.

IMPORTANT: This experimental function restricts the allowed codecs to exclude
PICKLE_PY_OBJECT_CODEC and similar unsafe codecs. However, this restriction
alone does NOT make it safe for use with untrusted data.

Args:
  x: Serialized data.

Returns:
  Deserialized data.

kd.s11n.load(path: str, /, *, fs: Any | None = None) -> Any

Aliases:

• kd.load

Deserializes a DataSlice or a DataBag from a file.

Args:
  path: Path to the file to read.
  fs: FileSystemInterface to use for file I/O. If not provided, the default
    file system interaction is used.

Returns:
  Deserialized data.

kd.s11n.loads(x: bytes) -> Any

Aliases:

• kd.loads

Deserializes a DataSlice or a DataBag.