Environment variables

The following environment variables may be specified at runtime to configure the behavior of TensorStore.

HTTP requests

TensorStore uses libcurl to make HTTP requests (e.g. to the Google Cloud Storage API), and uses TLS to secure connections to remote servers.

TLS CA certificates

On MS Windows and Mac OS, libcurl uses TLS facilities provided by the operating system and no additional configuration is necessary.

On Linux and BSD, if TensorStore is built with a bundled version of libcurl, as is the default, it expects to find the system certificate authority (CA) bundle in PEM format at /etc/ssl/certs/ca-certificates.crt, which is the location used by most Linux distributions. If the system CA bundle is available at that path, no additional configuration is necessary.

If the system CA bundle is not available at that path, you can specify an alternative certificate bundle path or certificate directory at runtime with the TENSORSTORE_CA_BUNDLE or TENSORSTORE_CA_PATH environment variables:

TENSORSTORE_CA_BUNDLE

Specifies the path to a local file containing one or more CA certificates concatenated into a single file in PEM format. On many Linux distributions, the system certificate bundle is available at /etc/ssl/certs/ca-certificates.crt. Refer to the libcurl documentation for more details.

TENSORSTORE_CA_PATH

Specifies the path of a local directory containing one or more CA certificates in PEM format. Each file in the directory must contain a single certificate, and the directory must be prepared using the OpenSSL c_rehash command. Refer to the libcurl documentation for more details. Note that this is not the most common format for the system CA certificate store. In most cases, the system CA certificate store should instead be specified using TENSORSTORE_CA_BUNDLE.

Note

On Linux and BSD, TensoprStore may optionally be built to dynamically link to a system-provided version of libcurl by specifying TENSORSTORE_SYSTEM_LIBS=se_curl. In this case, the default CA bundle path of /etc/ssl/certs/ca-certificates.crt does not apply; instead, the default depends on how the system-provided libcurl was built, and most likely no additional configuration will be necessary.

Proxy configuration

all_proxy

Specifies a proxy server to use for making any HTTP or HTTPS request. Refer to the libcurl documentation for more details.

http_proxy

Specifies a proxy server to use for making HTTP (not HTTPS) requests. Takes precedence over all_proxy. Refer to the libcurl documentation for more details.

https_proxy

Specifies a proxy server to use for making HTTPS requests. Takes precedence over all_proxy. Refer to the libcurl documentation for more details.

no_proxy

Specifies a comma-separated list of hostnames or ip addresses for which proxying is disabled. Refer to the libcurl documentation for more details.

Debugging

TENSORSTORE_VERBOSE_LOGGING

Enables debug logging for tensorstore internal subsystems. Set to comma separated list of values, where each value is one of name=int or just name. When all is, present, then verbose logging will be enabled for each subsytem, otherwise logging is set only for those subsystems present in the list.

Verbose flag values include: curl, distributed, file, file_detail, gcs, gcs_grpc, gcs_http, gcs_stubby, http_kvstore, http_transport, ocdbt, rate_limiter, s3, thread_pool, tsgrpc_kvstore, zip, zip_details.

TENSORSTORE_CURL_VERBOSE

If set to any value, verbose debugging information will be printed to stderr for all HTTP requests.

SSLKEYLOGFILE

Specifies the path to a local file where information necessary to decrypt TensorStore’s TLS traffic will be saved in a format compatible with Wireshark. Refer to the libcurl documentation for more details.

Google Cloud Credentials

GOOGLE_APPLICATION_CREDENTIALS

Specifies the local path to a Google Cloud JSON credentials file. Refer to the Google Cloud Storage Authentication section for details.

Google Cloud Storage

TENSORSTORE_GCS_HTTP_URL

Specifies to connect to an alternative server in place of https://storage.googleapis.com. Note that the normal Google oauth2 credentials are included in requests, and therefore only trusted servers should be used.

TENSORSTORE_GCS_REQUEST_CONCURRENCY

Specifies the concurrency level used by the shared Context Context.gcs_request_concurrency resource. Defaults to 32.

TENSORSTORE_HTTP2_MAX_CONCURRENT_STREAMS

Specifies the maximum number of concurrent streams per HTTP/2 connection, without limiting the total number of active connections. When unset, a default of 4 concurrent streams are permitted.