Drivers¶
Single file drivers¶
Chunked storage drivers¶
- json KeyValueStoreBackedChunkDriver : object¶
Common options supported by all chunked storage drivers.
- Extends:¶
TensorStore
— Specifies a TensorStore to open/create.
- Required members:¶
- driver : string¶
Driver identifier
Specifies the TensorStore driver.
- kvstore : KvStore | KvStoreUrl¶
Specifies the underlying storage mechanism.
- Optional members:¶
-
-
rank : integer[
0
,32
]¶ Specifies the rank of the TensorStore.
If
transform
is also specified, the input rank must match. Otherwise, the rank constraint applies to the driver directly.
- transform : IndexTransform¶
Specifies a transform.
- schema : Schema¶
Specifies constraints on the schema.
When opening an existing array, specifies constraints on the existing schema; opening will fail if the constraints do not match. Any soft constraints specified in the
chunk_layout
are ignored. When creating a new array, a suitable schema will be selected automatically based on the specified schema constraints in combination with any driver-specific constraints.
-
path : string =
""
¶ -
This is joined as an additional
"/"
-separated path component after anypath
member directly withinkvstore
. This is supported for backwards compatibility only; theKvStore.path
member should be used instead.Example
"path/to/data"
- open : boolean¶
Open an existing TensorStore. If neither
open
norcreate
is specified, defaults totrue
.
-
create : boolean =
false
¶ Create a new TensorStore. Specify
true
for bothopen
andcreate
to permit either opening an existing TensorStore or creating a new TensorStore if it does not already exist.
-
delete_existing : boolean =
false
¶ Delete any existing data at the specified path before creating a new TensorStore. Requires that
create
istrue
, and thatopen
isfalse
.
-
assume_metadata : boolean =
false
¶ Skip reading the metadata if possible. Instead, just assume any necessary metadata based on constraints in the spec, using the same defaults for any unspecified metadata as when creating a new TensorStore. Requires that
open
istrue
and thatdelete_existing
isfalse
.Warning
This option can lead to data corruption if the assumed metadata does not match the stored metadata, or multiple concurrent writers use different assumed metadata.
-
cache_pool : ContextResource =
"cache_pool"
¶ Specifies or references a previously defined
Context.cache_pool
. It is normally more convenient to specify a defaultcache_pool
in thecontext
.
-
data_copy_concurrency : ContextResource =
"data_copy_concurrency"
¶ Specifies or references a previously defined
Context.data_copy_concurrency
. It is normally more convenient to specify a defaultdata_copy_concurrency
in thecontext
.
-
recheck_cached_metadata : CacheRevalidationBound =
"open"
¶ Time after which cached metadata is assumed to be fresh. Cached metadata older than the specified time is revalidated prior to use. The metadata is used to check the bounds of every read or write operation.
Specifying
true
means that the metadata will be revalidated prior to every read or write operation. With the default value of"open"
, any cached metadata is revalidated when the TensorStore is opened but is not rechecked for each read or write operation.
-
recheck_cached_data : CacheRevalidationBound =
true
¶ Time after which cached data is assumed to be fresh. Cached data older than the specified time is revalidated prior to being returned from a read operation. Partial chunk writes are always consistent regardless of the value of this option.
The default value of
true
means that cached data is revalidated on every read. To enable in-memory data caching, you must both specify acache_pool
with a non-zerototal_bytes_limit
and also specifyfalse
,"open"
, or an explicit time bound forrecheck_cached_data
.
-
rank : integer[
-
json CacheRevalidationBound :
true
|false
|"open"
| number¶ Determines under what circumstances cached data is revalidated.
- One of:¶
-
true
Revalidate cached data at every operation.
-
false
Assume cached data is always fresh and never revalidate.
-
"open"
Revalidate cached data older than the time at which the TensorStore was opened.
- number
Revalidate cached data older than the specified time in seconds since the unix epoch.
-