stack Driver

The stack driver specifies a TensorStore virtually overlays a sequence of TensorStore layers within a common domain.

  • If the domains of the layers may overlap, the last layer that contains any given position within the domain takes precedence. This last layer that contains the position is said to be the backing layer for that position.

  • By choosing appropriate transforms for the layers, this driver may be used to virtually stack a sequence of TensorStores along a new dimension, or to concatenate a sequence of TensorStores along an existing dimension. For these use cases, the layer domains do not overlap.

Python API

Python APIs are provided for constructing a stack-driver-backed TensorStore based on a sequence of Spec or already-open TensorStore objects.

Supported operations

Opening the stack driver itself does not open any of the layers. Instead, layers are opened on demand as needed to satisfy read and write operations.

Reading or writing at a given position is supported, if any only if, the backing layer supports reading or writing, respectively. Reading or writing at a position that has no backing layer results in an error.

json driver/stack : object
Extends:

  • TensorStore — Specifies a TensorStore to open/create.

Required members:
driver : "stack"
layers : array of TensorStore

Array of layered tensorstore drivers.

The stack driver maps each nested driver in layers to the position described by the layer transform. All layers must have the same dtype as well as compatible domains.

Optional members:
context : Context

Specifies context resources that augment/override the parent context.

dtype : dtype

Specifies the data type.

rank : integer[0, 32]

Specifies the rank of the TensorStore.

If transform is also specified, the input rank must match. Otherwise, the rank constraint applies to the driver directly.

transform : IndexTransform

Specifies a transform.

schema : Schema

Specifies constraints on the schema.

When opening an existing array, specifies constraints on the existing schema; opening will fail if the constraints do not match. Any soft constraints specified in the chunk_layout are ignored. When creating a new array, a suitable schema will be selected automatically based on the specified schema constraints in combination with any driver-specific constraints.

data_copy_concurrency : ContextResource = "data_copy_concurrency"

Specifies or references a previously defined Context.data_copy_concurrency. It is normally more convenient to specify a default data_copy_concurrency in the context.

Example

{
  "driver": "stack",
  "layers": [ {"driver": "array", "array": [1, 2, 3], "dtype": "int32"},
    {
      "driver": "array",
      "array": [4, 5, 6],
      "dtype": "int32",
      "transform": {
        "input_inclusive_min": 3,
        "output": {"input_dimension": 0, "offset": -3}
      }
    }]
}

Example

Example of stack driver

>>> ts.open({
...     'driver':
...         'stack',
...     'layers': [
...         {
...             'driver': 'array',
...             'dtype': 'int32',
...             'array': [1, 2, 3, 4],
...         },
...         {
...             'driver': 'array',
...             'dtype': 'int32',
...             'array': [1, 2, 3, 4],
...             'transform': {
...                 'input_inclusive_min': [4],
...                 'input_exclusive_max': [8],
...                 'output': [{
...                     'input_dimension': 0,
...                     'offset': -4
...                 }]
...             },
...         },
...     ]
... }).result().schema.dtype
dtype("int32")

TensorStore Schema

Data type

The dtype must be specified for at least one layer, or in the top-level schema. All specified data types must match.

Domain

The bounds of a stack driver-backed TensorStore, and of each layer, are fixed when it is opened, based on the domains specified for each of the layers, and the domain specified for the stack driver, if any.

Any implicit/resizeable bounds of layers become fixed (explicit) bounds and will not be affected by subsequent resize operations.

By default, the bounds of the stack-driver-backed TensorStore are determined by computing the hull of the effective domains of the layers, but any finite or explicit bound specified on the top-level stack-driver domain takes precedence and overrides the bound determined from the hull.

Note that it is valid for the domain to include positions that are not backed by any layer, but any attempt to read or write such positions results in an error.

The dimension labels are merged and must be compatible.

Dimension units

For each dimension, the corresponding dimension unit is determined as follows:

  1. If a unit for the dimension is specified directly in the top-level stack-driver Schema.dimension_units, the specified unit is assigned.

  2. Otherwise, if there is agreement among all the layers that specify a unit for the dimension, the common unit is assigned.

  3. Otherwise, the unit is unassigned (null).

Fill value

Fill values are not supported.

Codec

Codecs are not supported.

Chunk layout

Chunk layouts are not supported.