JSON Spec

TensorStore relies on a JSON Spec format for specifying all parameters necessary for opening or creating a TensorStore.

json TensorStore : object

Specifies a TensorStore to open/create.

The following properties are supported by all TensorStore drivers. Refer to the driver documentation for the supported driver identifiers and driver-specific properties.

Subtypes:
Required members:
driver : string

Driver identifier

Specifies the TensorStore driver.

Optional members:
context : Context

Specifies context resources that augment/override the parent context.

dtype : dtype

Specifies the data type.

rank : integer[0, 32]

Specifies the rank of the TensorStore.

If transform is also specified, the input rank must match. Otherwise, the rank constraint applies to the driver directly.

transform : IndexTransform

Specifies a transform.

schema : Schema

Specifies constraints on the schema.

When opening an existing array, specifies constraints on the existing schema; opening will fail if the constraints do not match. Any soft constraints specified in the chunk_layout are ignored. When creating a new array, a suitable schema will be selected automatically based on the specified schema constraints in combination with any driver-specific constraints.

json TensorStoreUrl : string

URL representation of a TensorStore to open.

In addition to the JSON object representation, a TensorStore backed by a key-value store may be specified more concisely by a URL pipeline consisting of one or more |-separated components.

The start of the URL pipeline must be a valid KvStoreUrl. This may be followed by a component specifying a key-value store-backed TensorStore, such as zarr3, and then finally zero or more TensorStore adapters, such as cast.

Warning

The URL representation of a TensorStore specification excludes any index transform and parameters such as concurrency limits, cache options, and metadata and schema constraints.

Subtypes:
json dtype : "bool" | "char" | "byte" | "int2" | "int4" | "int8" | "uint8" | "int16" | "uint16" | "int32" | "uint32" | "int64" | "uint64" | "float8_e3m4" | "float8_e4m3fn" | "float8_e4m3fnuz" | "float8_e4m3b11fnuz" | "float8_e5m2" | "float8_e5m2fnuz" | "float16" | "bfloat16" | "float32" | "float64" | "complex64" | "complex128" | "string" | "ustring" | "json"

Specifies a TensorStore data type.

TensorStore data types correspond to the logical data representation, not the precise encoding. There are not separate data types for little endian and big endian byte order.

One of:
"bool"

Boolean value (0 or 1).

Each value occupies an entire byte in memory, but depending on the format may be encoded more efficiently.

"char"

Single byte, interpreted as an ASCII character.

"byte"

Single byte.

"int2"

2-bit signed two’s-complement integer, internally stored as its 8-bit signed integer equivalent (i.e. sign-extended).

"int4"

4-bit signed two’s-complement integer, internally stored as its 8-bit signed integer equivalent (i.e. sign-extended).

"int8"

8-bit signed two’s-complement integer.

"uint8"

8-bit unsigned integer.

"int16"

16-bit signed two’s-complement integer.

"uint16"

16-bit unsigned integer.

"int32"

32-bit signed two’s-complement integer.

"uint32"

32-bit unsigned integer.

"int64"

64-bit signed two’s-complement integer.

"uint64"

64-bit unsigned integer.

"float8_e3m4"

8-bit floating point - Exponent: 3, Mantissa: 4, bias: 3, IEEE 754, with NaN, with infinities .

"float8_e4m3fn"

8-bit floating point - Exponent: 4, Mantissa: 3, bias: 7, with NaN, without infinities .

"float8_e4m3fnuz"

8-bit floating point - Exponent: 4, Mantissa: 3, bias: 8, with NaN, without infinities .

"float8_e4m3b11fnuz"

8-bit floating point - Exponent: 4, Mantissa: 3, bias: 11, with NaN, without infinities .

"float8_e5m2"

8-bit floating point - Exponent: 5, Mantissa: 2, bias: 15, IEEE 754, with NaN and infinities. .

"float8_e5m2fnuz"

8-bit floating point - Exponent: 5, Mantissa: 2, bias: 16, with NaN, without infinities. .

"float16"

IEEE 754 binary16 half-precision floating-point number.

"bfloat16"

bfloat16 floating-point format number.

"float32"

IEEE 754 binary32 single-precision floating-point number.

"float64"

IEEE 754 binary64 double-precision floating-point number.

"complex64"

Complex number, where the real and imaginary components are each represented by a float32.

"complex128"

Complex number, where the real and imaginary components are each represented by a float64.

"string"

Byte string.

"ustring"

Unicode string (represented in memory as UTF-8).

"json"

JSON value.