json Driver

The json driver provides read/write access to JSON values stored in any supported Key-Value Storage Layer. JSON values are accessed as rank-0 arrays with "json" data type.

Note

Even if the JSON value is itself an array value, TensorStore still exposes it as a rank-0 array of a single JSON value.

Sub-values within a JSON file may be specified using JSON Pointer syntax (RFC 6901), and TensorStore guarantees consistency in the presence of concurrent writes because writes are implemented as atomic read-modify-write operations to individual JSON Pointers. If two processes concurrently write to non-overlapping pointers within the same JSON file, it is guaranteed that neither write will be lost.

json driver/json : object
Extends:

  • TensorStore — Specifies a TensorStore to open/create.

Required members:
driver : "json"
kvstore : KvStore | KvStoreUrl

Specifies the underlying storage mechanism.

Optional members:
context : Context

Specifies context resources that augment/override the parent context.

dtype : "json"

Optional. If specified, must be "json".

rank : 0

Optional. If specified, must be 0.

transform : IndexTransform

Specifies a transform.

schema : Schema

Specifies constraints on the schema.

When opening an existing array, specifies constraints on the existing schema; opening will fail if the constraints do not match. Any soft constraints specified in the chunk_layout are ignored. When creating a new array, a suitable schema will be selected automatically based on the specified schema constraints in combination with any driver-specific constraints.

path : string

Additional path to the JSON file within the KvStore specified by kvstore.

This is joined as an additional "/"-separated path component after any path member directly within kvstore. This is supported for backwards compatibility only; the KvStore.path member should be used instead.

Example

"path/to/data.json"
cache_pool : ContextResource = "cache_pool"

Specifies or references a previously defined Context.cache_pool. It is normally more convenient to specify a default cache_pool in the context.

data_copy_concurrency : ContextResource = "data_copy_concurrency"

Specifies or references a previously defined Context.data_copy_concurrency. It is normally more convenient to specify a default data_copy_concurrency in the context.

recheck_cached_data : CacheRevalidationBound = "open"

Time after which cached data is assumed to be fresh. Cached data older than the specified time is revalidated prior to being returned from a read operation. Writes are always consistent regardless of the value of this option.

Specifying true means that the data will be revalidated prior to every read operation. With the default value of "open", any cached data is revalidated when the TensorStore is opened but is not rechecked for each read operation.

json_pointer : string = ""

JSON Pointer (RFC 6901) specifying a path within the JSON file. With the default value of "", read and write operations apply to the entire JSON file. Specifying a non-empty pointer permits reading/writing a sub-value within the full JSON file. For example, if the full JSON file contains {"a": {"b": 42}}, specifying a json_pointer of "/a/b" refers to the value 42. If the pointer refers to a non-existent member, read operations return an error. Write operations attempt to create the member (and any ancestor objects), and return an error if the pointer is inconsistent with the existing value; for example, it is an error to write to a pointer that references a member of a non-object, non-array value.

Example

{
  "driver": "json",
  "kvstore": {"driver": "gcs", "bucket": "my-bucket"},
  "path": "path/to/attributes.json",
  "json_pointer": "/a/2/b"
}