http Key-Value Store driver

The http driver allows arbitrary HTTP servers to be used as read-only key-value stores. Keys directly correspond to HTTP paths.

json kvstore/http : object

Extends

  • KvStore — Key-value store specification.

Required members

driver : "http"
base_url : string

Base URL.

Must have a scheme of http or https. May include a query string, which will be included in all requests. Must not include a fragment. Any characters not allowed in a URL must be percent encoded.

Example

"https://example.com"

Example

"https://example.com:8000/path/to/data"

Example

"https://example.com:8000/path/to/data?query=value"

Optional members

path : string

Path relative to base_url.

If the path begins with a "/", then it is treated as an absolute path, and base_url must not specify a path component. Otherwise, it is interpreted as a relative path and appended as an additional path component to any path specified in base_url.

Unlike base_url, which must be a valid URL, there are no restrictions on the valid characters in path. Any special characters will be percent encoded.

Note

The treatment of relative paths differs slightly from web browsers in that any path specified by base_url is treated as a directory even if it does not end in "/".

The following key-value store specs are all equivalent:

{"driver": "http",
 "base_url": "https://example.com/my/path/?query=value",
 "path": "to/my data"}
{"driver": "http",
 "base_url": "https://example.com/my/path?query=value",
 "path": "to/my data"}
{"driver": "http",
 "base_url": "https://example.com/my/path/to/my%20data?query-value"}
{"driver": "http",
 "base_url": "https://example.com?query=value",
 "path": "/my/path/to/my data"}
"https://example.com/my/path/to/my%20data?query-value"
context : Context

Specifies context resources that augment/override the parent context.

headers : array of array

Additional HTTP headers to send with every request.

Each header should be specified using the normal HTTP header "name: value" syntax. The header name is case-insensitive while the header value is case-sensitive. Refer to RFC 7230#section-3.2 for details. The obsolete line folding syntax CRLF 1*( SP / HTAB ) is not supported. Multiple headers with the same name are allowed.

Example

["Authorization: Bearer XXXXX"]
http_request_concurrency : ContextResource

Specifies or references a previously defined Context.http_request_concurrency.

http_request_retries : ContextResource

Specifies or references a previously defined Context.http_request_retries.

json Context.http_request_concurrency : object

Specifies a limit on the number of concurrent HTTP requests.

Optional members

limit : integer[1, +∞) | "shared" = "shared"

The maximum number of concurrent requests. If the special value of "shared" is specified, a shared global limit of 32 applies.

json Context.http_request_retries : object

Specifies retry parameters for handling transient network errors.

Optional members

max_retries : integer[1, +∞) = 32

Maximum number of attempts in the case of transient errors.

initial_delay : string = "1s"

Initial backoff delay for transient errors.

max_delay : string = "32s"

Maximum backoff delay for transient errors.

json KvStoreUrl/http : object

http:// and https:// KvStore URL scheme

HTTP key-value stores may be specified using regular HTTP URL syntax.

Examples

URL representation

JSON representation

"http://example.com"

{"driver": "http",
 "base_url": "http://example.com",
 "path": "/"}

"https://example.com:8080/path/to/data%20base?query=value"

{"driver": "http",
 "base_url": "https://example.com:8080?query=value",
 "path": "/path/to/data base"}

Extends

  • KvStoreUrl — URL representation of a key-value store.

Cache behavior

When used with an in-memory cache, the staleness of responses is bounded using the Cache-Control request header the Date response header.

If the server supports the ETag response header, the If-Match and If-None-Match request headers are used to revalidate cached responses.

TLS CA certificates

For https:// URLs, TensorStore depends on the system certificate authority (CA) store to secure connections. In many cases it will work by default without any additional configuration, but if you receive an error like:

CURL error[77] Problem with the SSL CA cert (path? access rights?):
error setting certificate verify locations:
  CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none

refer to the HTTP request-related environment variables section for information on how to specify the path to the system certificate store at runtime.