file Key-Value Store driver¶

The file driver uses the filesystem as a key-value store directly. A key directly specifies a path under a given root directory; the value is stored as the file contents.

Locking provided by the filesystem is used to safely allow concurrent access from multiple processes. (The locking protocol used does not block readers.) Provided that shared locking is supported, concurrent access from multiple machines to a network filesystem is also safe.

json kvstore/file : object¶

Read/write access to the local filesystem.

JSON specification of the key-value store.

Extends:¶

• KvStore — Key-value store specification.

Required members:¶

driver : "file"¶

path : string¶

Path to root directory on local filesystem.

Optional members:¶

context : Context¶

Specifies context resources that augment/override the parent context.

file_io_concurrency : ContextResource¶

Specifies or references a previously defined Context.file_io_concurrency.

file_io_sync : ContextResource¶

Specifies or references a previously defined Context.file_io_sync.

json KvStoreUrl/file : string¶

file:// KvStore URL scheme

File-based key-value stores may be specified using the widely-supported file://path URL syntax.

Examples

URL representation	JSON representation
"file:///tmp/dataset/"	{"driver": "file", "path": "/tmp/dataset/"}
"file://C:/Users/abc/dataset/"	{"driver": "file", "path": "C:/Users/abc/dataset/"}

Extends:¶

• KvStoreUrl — URL representation of a key-value store.

json Context.file_io_concurrency : object¶

Specifies a limit on the number of concurrently local filesystem I/O operations.

Optional members:¶

limit : integer_{[1, +∞)} | "shared" = "shared"¶

The maximum number of concurrent operations. If the special value of "shared" is specified, a shared global limit equal to the number of CPU cores/threads available (or 4 if there are fewer than 4 cores/threads available) applies.

json Context.file_io_sync : boolean = true¶

Specifies durability of writes.

If true, durability is ensured for local file writes (e.g. by calling fsync). If false, durability is not guaranteed, and data may be lost in the event of a crash.

In cases where durability is not required, setting this to false may make write operations faster.

Durability of writes¶

By default, this driver ensures all writes are durable, meaning that committed data won’t be lost in the event that the process or machine crashes.

In cases where durability is not necessary, faster write performance may be achieved by setting Context.file_io_sync to :json:false.

{"driver": "file",
 "path": "/local/path/",
 "file_io_sync": false}

Limitations¶