http Key-Value Store driver¶

The http driver allows arbitrary HTTP servers to be used as read-only key-value stores. Keys directly correspond to HTTP paths.

json kvstore/http : object¶

Read-only access to arbitrary HTTP servers.

JSON specification of the key-value store.

Extends:¶

• KvStore — Key-value store specification.

Required members:¶

driver : "http"¶

base_url : string¶

Base URL.

Must have a scheme of http or https. May include a query string, which will be included in all requests. Must not include a fragment. Any characters not allowed in a URL must be percent encoded.

Example

"https://example.com"

Example

"https://example.com:8000/path/to/data"

Example

"https://example.com:8000/path/to/data?query=value"

Optional members:¶

path : string¶

Path relative to base_url.

If the path begins with a "/", then it is treated as an absolute path, and base_url must not specify a path component. Otherwise, it is interpreted as a relative path and appended as an additional path component to any path specified in base_url.

Unlike base_url, which must be a valid URL, there are no restrictions on the valid characters in path. Any special characters will be percent encoded.

Note

The treatment of relative paths differs slightly from web browsers in that any path specified by base_url is treated as a directory even if it does not end in "/".

The following key-value store specs are all equivalent:

{"driver": "http",
 "base_url": "https://example.com/my/path/?query=value",
 "path": "to/my data"}
{"driver": "http",
 "base_url": "https://example.com/my/path?query=value",
 "path": "to/my data"}
{"driver": "http",
 "base_url": "https://example.com/my/path/to/my%20data?query-value"}
{"driver": "http",
 "base_url": "https://example.com?query=value",
 "path": "/my/path/to/my data"}
"https://example.com/my/path/to/my%20data?query-value"

context : Context¶

Specifies context resources that augment/override the parent context.

headers : array of array¶

Additional HTTP headers to send with every request.

Each header should be specified using the normal HTTP header "name: value" syntax. The header name is case-insensitive while the header value is case-sensitive. Refer to RFC 7230#section-3.2 for details. The obsolete line folding syntax CRLF 1*( SP / HTAB ) is not supported. Multiple headers with the same name are allowed.

Example

["Authorization: Bearer XXXXX"]

http_request_concurrency : ContextResource¶

Specifies or references a previously defined Context.http_request_concurrency.

http_request_retries : ContextResource¶

Specifies or references a previously defined Context.http_request_retries.

json Context.http_request_concurrency : object¶

Specifies a limit on the number of concurrent HTTP requests.

Optional members:¶

limit : integer_{[1, +∞)} | "shared" = "shared"¶

The maximum number of concurrent requests. If the special value of "shared" is specified, a shared global limit of 32 applies.

json Context.http_request_retries : object¶

Specifies retry parameters for handling transient network errors.

Optional members:¶

max_retries : integer_{[1, +∞)} = 32¶

Maximum number of attempts in the case of transient errors.

initial_delay : string = "1s"¶

Initial backoff delay for transient errors.

max_delay : string = "32s"¶

Maximum backoff delay for transient errors.

json KvStoreUrl/http : object¶

http:// and https:// KvStore URL scheme

HTTP key-value stores may be specified using regular HTTP URL syntax.

Examples

URL representation	JSON representation
"http://example.com"	{"driver": "http", "base_url": "http://example.com", "path": "/"}
"https://example.com:8080/path/to/data%20base?query=value"	{"driver": "http", "base_url": "https://example.com:8080?query=value", "path": "/path/to/data base"}

Extends:¶

• KvStoreUrl — URL representation of a key-value store.

Cache behavior¶

When used with an in-memory cache, the staleness of responses is bounded using the Cache-Control request header the Date response header.

If the server supports the ETag response header, the If-Match and If-None-Match request headers are used to revalidate cached responses.

TLS CA certificates¶

For https:// URLs, TensorStore depends on the system certificate authority (CA) store to secure connections. In many cases it will work by default without any additional configuration, but if you receive an error like:

CURL error[77] Problem with the SSL CA cert (path? access rights?):
error setting certificate verify locations:
  CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none

refer to the HTTP request-related environment variables section for information on how to specify the path to the system certificate store at runtime.