API reference¶

Core¶

class tensorstore::Batch;

Batch provides a mechanism for defining a batch of asynchronous operations that will potentially be deferred until all references to the batch are released.

class tensorstore::Transaction;

Shared handle to a transaction.

class tensorstore::Context;

Immutable collection of lazily-constructed “resources”.

class tensorstore::Schema;

Collection of constraints for a TensorStore schema.

class tensorstore::Spec;

Specifies the parameters necessary to open or create a TensorStore.

class tensorstore::TensorStore<ElementType, Rank, Mode>;

Open handle to an asynchronous multi-dimensional array.

Indexing¶

using tensorstore::Index = int64_t;

Type for representing a coordinate in, or a extent of, a multi-dimensional array.

using tensorstore::DimensionIndex = ptrdiff_t;

Type for representing a dimension index or rank.

class tensorstore::IndexInterval;

Represents an interval of index values, with support for +/-inf bounds.

class tensorstore::OptionallyImplicitIndexInterval     : public IndexInterval;

Represents an IndexInterval where the lower/upper bounds may be “implicit”.

class tensorstore::IndexDomainDimension<LabelCKind>
    : public OptionallyImplicitIndexInterval;

Represents an index interval with optionally-implicit bounds and an optionally dimension label.

using tensorstore::InlineRank = DimensionIndex;

Specifies a fixed compile-time rank, or a run-time rank with an optional inline storage limit.

struct tensorstore::RankConstraint;

Represents an optional rank value and provides related operations.

class tensorstore::Box<Rank>;

Value type representing an n-dimensional hyperrectangle.

using tensorstore::DimensionSet = SmallBitSet<kMaxRank>;

Represents a set of dimension indices.

class tensorstore::IndexDomain<Rank, CKind>;

Represents an index domain.

class tensorstore::IndexTransform<InputRank, OutputRank, CKind>;

Transform from an input index space to an output index space.

constexpr bool tensorstore::IsIndexArray<T> =
    IsArray<T> &&
    std::is_convertible_v<T, SharedArrayView<const Index>>;

Bool-valued metafunction that evaluates to true if T is an array type convertible to SharedArrayView<const Index>.

class tensorstore::DimExpression<Op...>;

A DimExpression represents an ordered “selection” of dimensions of an index space and a sequence of “operations” to apply to those dimensions.

Alignment¶

enum class tensorstore::DomainAlignmentOptions;

Specifies options for aligning index domains.

Result<IndexTransform<>> tensorstore::AlignDomainTo(
    IndexDomainView<> source,
    IndexDomainView<> target,
    DomainAlignmentOptions options = DomainAlignmentOptions::all);

Attempts to align the source domain to match the target domain.

absl::Status tensorstore::AlignDimensionsTo(
    IndexDomainView<> source,
    IndexDomainView<> target,
    span<DimensionIndex> source_matches,
    DomainAlignmentOptions options = DomainAlignmentOptions::all);

Same as AlignDomainTo, but instead of computing the alignment transform, simply sets source_matches[i] = j, if source dimension i matches to target dimension j, and sets source_matches[i] = -1 if source dimension i is unmatched.

Result<IndexTransform<>> tensorstore::AlignTransformTo(
    IndexTransform<> source_transform,
    IndexDomainView<> target,
    DomainAlignmentOptions options = DomainAlignmentOptions::all);

Returns ComposeTransforms(source_transform, alignment), where alignment is the result of AlignDomainTo(source_transform.domain(), target, options).

Data types¶

class tensorstore::BFloat16;

Storage-only bfloat16 type.

type tensorstore::Float8e4m3fn;

type tensorstore::Float8e4m3fnuz;

type tensorstore::Float8e4m3b11fnuz;

type tensorstore::Float8e5m2;

type tensorstore::Float8e5m2fnuz;

Storage-only 8-bit floating-point data types.

class tensorstore::Int4Padded;

Int4 type padded to int8.

using tensorstore::dtypes::bool_t = bool;

Boolean value (always represented as 0 or 1).

using tensorstore::dtypes::char_t = char;

Single ASCII/UTF-8 code unit. Primarily intended to represent fixed-width ASCII fields.

using tensorstore::dtypes::byte_t = ::std::byte;

Opaque byte value. Intended to represent opaque binary data.

using tensorstore::dtypes::int4_t = ::tensorstore::Int4Padded;

Signed and unsigned integer types.

using tensorstore::dtypes::int8_t = ::int8_t;

using tensorstore::dtypes::uint8_t = ::uint8_t;

using tensorstore::dtypes::int16_t = ::int16_t;

using tensorstore::dtypes::uint16_t = ::uint16_t;

using tensorstore::dtypes::int32_t = ::int32_t;

using tensorstore::dtypes::uint32_t = ::uint32_t;

using tensorstore::dtypes::int64_t = ::int64_t;

using tensorstore::dtypes::uint64_t = ::uint64_t;

using tensorstore::dtypes::float8_e4m3fn_t =
    ::tensorstore::Float8e4m3fn;

using tensorstore::dtypes::float8_e4m3fnuz_t =
    ::tensorstore::Float8e4m3fnuz;

using tensorstore::dtypes::float8_e4m3b11fnuz_t =
    ::tensorstore::Float8e4m3b11fnuz;

using tensorstore::dtypes::float8_e5m2_t = ::tensorstore::Float8e5m2;

using tensorstore::dtypes::float8_e5m2fnuz_t =
    ::tensorstore::Float8e5m2fnuz;

using tensorstore::dtypes::bfloat16_t = ::tensorstore::BFloat16;

using tensorstore::dtypes::float16_t = ::half_float::half;

IEEE 754 binary16 half-precision floating-point data type.

using tensorstore::dtypes::float32_t = float;

IEEE 754 binary32 single-precision floating-point data type.

using tensorstore::dtypes::float64_t = double;

IEEE 754 binary64 double-precision floating-point data type.

using tensorstore::dtypes::complex64_t = ::std::complex<float32_t>;

Complex number based on float32_t.

using tensorstore::dtypes::complex128_t = std::complex<float64_t>;

Complex number based on float64_t.

using tensorstore::dtypes::string_t = std::string;

Byte string.

using tensorstore::dtypes::ustring_t = Utf8String;

Unicode string, represented in memory as UTF-8.

using tensorstore::dtypes::json_t = ::nlohmann::json;

JSON value.

class tensorstore::DataType;

Run-time representation of a C++ type used as the element type for a multi-dimensional array.

Array¶

enum class tensorstore::ContainerKind;

Specifies whether an object (e.g. strided layout) is stored by value or view/reference.

class tensorstore::ByteStridedPointer<T>;

Wrapper type for a raw pointer for which pointer arithmetic operates with a stride of 1 byte, rather than a stride of sizeof(T) bytes.

class tensorstore::StridedLayout<Rank, OriginKind, CKind>;

A StridedLayout specifies the layout of a multi-dimensional array in terms of an origin vector, a shape vector, and a byte_strides vector, all of length equal to the rank of the multi-dimensional array. The origin and shape vectors specify the domain of the array, and the byte_strides vector specifies the offset in bytes between consecutive elements in each dimension.

class tensorstore::ElementPointer<ElementTagType>;

Pairs an array base pointer (either an Element* or an std::shared_ptr<Element>) with a DataType in order to support a dynamic element type determined at run time.

class tensorstore::
    Array<ElementTagType, Rank, OriginKind, LayoutContainerKind>;

Represents a pointer to an in-memory multi-dimensional array with an arbitrary strided layout.

class tensorstore::
    TransformedArray<ElementTagType, Rank, LayoutCKind>;

View through an index transform of an in-memory array.

Transformation¶

class tensorstore::TransformArrayConstraints;

Specifies constraints on the array returned from TransformArray.

Result<
    SharedArray<const typename Array::Element, InputRank, OriginKind>>
tensorstore::TransformArray(
    const Array& array,
    const IndexTransform<InputRank, OutputRank, CKind>& transform,
    TransformArrayConstraints constraints = skip_repeated_elements);

Returns a strided array that represents the result of applying transform to array.

Result<SharedArray<std::remove_const_t<typename Array::Element>,
                   InputRank,
                   OriginKind>>
tensorstore::MakeCopy(
    const Array& array,
    const IndexTransform<InputRank, OutputRank, CKind>& transform,
    IterationConstraints constraints = skip_repeated_elements);

Equivalent to TransformArray, but always returns a newly allocated array with a non-const element type.

Iteration¶

enum class tensorstore::RepeatedElementsConstraint;

Specifies whether repeated elements may be skipped.

enum class tensorstore::UnspecifiedLayoutOrder;

Specifies that the iteration order is unconstrained.

class tensorstore::LayoutOrderConstraint;

Specifies an order constraint on a strided layout/iteration order, equivalent to an algebraic sum of ContiguousLayoutOrder and unspecified_order.

class tensorstore::IterationConstraints;

Specifies constraints on how multi-dimensional iteration can be performed.

Key-value storage¶

class tensorstore::kvstore::Spec;

Combines a driver-specific kvstore spec with a string path that serves as a key prefix.

class tensorstore::kvstore::KvStore;

Combines a DriverPtr with a string path that serves as a key prefix, and an optional transaction.

struct tensorstore::ByteRange;

Specifies a range of bytes within a stored value.

struct tensorstore::OptionalByteRangeRequest;

Specifies an optional byte range request.

struct tensorstore::StorageGeneration;

Represents a generation identifier associated with a stored object.

struct tensorstore::TimestampedStorageGeneration;

Combines a local timestamp with a StorageGeneration indicating the local time for which the generation is known to be current.

class tensorstore::KeyRange;

Specifies a range of keys according to their unsigned lexicographical order.

Downsampling¶

enum class tensorstore::DownsampleMethod;

Specifies the method by which downsampling is performed.

Result<TensorStore<Element, Rank, ReadWriteMode::read>>
tensorstore::Downsample(
    TensorStore<Element, Rank, Mode> store,
    std::type_identity_t<span<const Index, Rank>> downsample_factors,
    DownsampleMethod downsample_method);

Returns a downsampled view of a TensorStore.

Result<Spec>
tensorstore::Downsample(const Spec& base_spec,
                        span<const Index> downsample_factors,
                        DownsampleMethod downsample_method);

Returns a downsampled view of a Spec.

Utilities¶

class tensorstore::span<T, Extent>;

Unowned view of a contiguous 1-d array of elements, similar to std::span.

struct tensorstore::Utf8String;

Wrapper around std::string to indicate a UTF-8 encoded string.

class tensorstore::SmallBitSet<N>;

Bit set that fits in a single unsigned integer.

class tensorstore::MaybeHardConstraintSpan<T> : public span<const T>;

Combines a span (with length assumed to be a valid rank) with a bit vector indicating which values are hard constraints as opposed to soft constraints.

class tensorstore::MaybeHardConstraintIndex;

Combines an Index (which defaults to kImplicit) with a bool indicating whether the value is a hard constraint as opposed to a default value.

constexpr bool
tensorstore::IsCompatibleOptionSequence<Options, Option...> =
        ((!std::is_same_v<Options, std::remove_cvref_t<Option>> &&
          Options::template IsOption<std::remove_cvref_t<Option>>) &&
         ...);

Checks if every Option is compatible with the specified Options type.

Error handling¶

TENSORSTORE_RETURN_IF_ERROR(...);

Causes the containing function to return the specified absl::Status value if it is an error status.

TENSORSTORE_CHECK_OK(...);

Logs an error and terminates the program if the specified absl::Status is an error status.

absl::Status tensorstore::MaybeAnnotateStatus(
    absl::Status source,
    std::string_view message,
    std::source_location loc = std::source_location::current());

absl::Status tensorstore::MaybeAnnotateStatus(
    absl::Status source,
    std::string_view message,
    absl::StatusCode new_code,
    std::source_location loc = std::source_location::current());

If status is not absl::StatusCode::kOk, then annotate the status message.

class [[nodiscard]] tensorstore::Result<T>;

Result<T> implements a value-or-error concept providing a discriminated union of a usable value, T, or an absl::Status error describing why the value is not present. Result<void> is permitted, which becomes a wrapper for absl::Status, and which enables use of Result<T> in a wide variety of template contexts.

Asynchronous support¶

type tensorstore::Executor;

Function object that is callable with nullary noexcept functions.

class [[nodiscard]] tensorstore::Promise<T>;

“Producer” interface to a one-time channel.

class [[nodiscard]] tensorstore::AnyFuture;

Generic consumer interface to a one-time channel.

class [[nodiscard]] tensorstore::Future<T> : public AnyFuture;

“Consumer” interface to a one-time channel.

class [[nodiscard]] tensorstore::PromiseFuturePair<T>;

Pairs a Promise with a Future.

struct [[nodiscard]] tensorstore::WriteFutures;

Handle for consuming the result of an asynchronous write operation.

JSON serialization¶

class tensorstore::IncludeDefaults;

Specifies whether members equal to their default values are included when converting to JSON.

class tensorstore::JsonSerializationOptions;

Defines options for conversion to/from JSON.

Compile-time data type/rank/mode constraints¶

enum class tensorstore::CastChecking;

Enum type used as a template parameter to casts to choose between an unchecked cast (that returns the bare value T) and a checked cast that returns Result<T>.

struct tensorstore::unchecked_t;

Tag type used as the first parameter of some constructors to request an unchecked conversion (e.g. from dynamic rank to a compile-time rank).

constexpr bool
tensorstore::IsStaticCastConstructible<Target, SourceRef>;

bool-valued metafunction that evaluates to true if a value of type SourceRef&& can be converted to a type of Target using StaticCast.

using tensorstore::StaticCastResultType<Target, SourceRef, Checking>;

Evaluates to the result of casting a value of type SourceRef&& to Target with a checking mode of Checking.

StaticCastResultType<Target, SourceRef, Checking>
tensorstore::StaticCast(SourceRef&& source);

Type-erasure-aware conversion of source to the specified Target type.

using tensorstore::RebindRank<SourceRef, TargetRank>;

Evaluates to a type similar to SourceRef but with a static rank of TargetRank.

StaticCastResultType<RebindRank<SourceRef, TargetRank>,
                     SourceRef,
                     Checking>
tensorstore::StaticRankCast(SourceRef&& source);

Casts source to have a static rank of TargetRank.

using tensorstore::RebindDataType<SourceRef, TargetElement>;

Evaluates to a type similar to SourceRef but with a static data type of TargetElement.

StaticCastResultType<RebindDataType<SourceRef, TargetElement>,
                     SourceRef,
                     Checking>
tensorstore::StaticDataTypeCast(SourceRef&& source);

Casts source to have a static data type of TargetElement.

StaticCastResultType<RebindDataType<SourceRef, TargetElement>,
                     SourceRef>
tensorstore::ConstDataTypeCast(SourceRef&& source);

Casts source to a specified target element type which must differ from the existing element type only in const qualification.

using tensorstore::RebindMode<SourceRef, TargetMode>;

Evaluates to a type similar to SourceRef but with a ReadWriteMode of TargetMode.

StaticCastResultType<RebindMode<SourceRef, TargetMode>,
                     SourceRef,
                     Checking>
tensorstore::ModeCast(SourceRef&& source);

Casts source to have a static ReadWriteMode of TargetMode.

String Utilities¶

std::string tensorstore::StrCat(const Arg&... arg);

Concatenates the string representation of arg... and returns the result.

void tensorstore::StrAppend(std::string* result, const Arg&... arg);

Appends a string representation of arg… to *result.

Index vectors¶

constexpr bool
tensorstore::IsCompatibleFullIndexVector<Rank, Indices>;

Bool-valued metafunction that evaluates to true if Indices is span-compatible with a span::value_type convertible without narrowing to Index and a static span::extent compatible with Rank.

constexpr bool
tensorstore::IsImplicitlyCompatibleFullIndexVector<Rank, Indices>;

Bool-valued metafunction that evaluates to true if Indices is span-compatible with a span::value_type convertible without narrowing to Index and a static span::extent implicitly compatible with Rank.

constexpr bool
tensorstore::IsCompatiblePartialIndexVector<Rank, Indices>;

Bool-valued metafunction that evaluates to true if Indices is span-compatible with a span::value_type convertible without narrowing to Index and a static span::extent <= Rank.

constexpr bool
tensorstore::IsCompatibleFullIndexPack<Rank, IndexType...>;

Bool-valued metafunction that evaluates to true if every IndexType is convertible without narrowing to Index, and sizeof...(IndexType) is compatible with Rank.

constexpr bool tensorstore::IsIndexConvertibleVector<Indices>;

Bool-valued metafunction that evaluates to true if Indices is span-compatible with a span::value_type convertible without narrowing to Index.

constexpr bool tensorstore::IsIndexVector<Indices>;

Bool-valued metafunction that evaluates to true if Indices is span-compatible with a span::value_type of Index.

constexpr bool tensorstore::IsMutableIndexVector<Indices>;

Bool-valued metafunction that evaluates to true if Indices is span-compatible with a span::element_type of Index.

using tensorstore::SpanStaticExtent<X0, Xs...>;

ptrdiff_t-valued metafunction with a static constexpr ptrdiff_t value member that is equal to the common static extent of X0, Xs... if X0, Xs... are all span-compatible types with the same static extent.