zip Key-Value Store driver

The zip driver implements support for reading and writing ZIP format files on top of a base key-value store. (Not all ZIP features are supported.)

json kvstore/zip : object

Key-value store adapter for the ZIP archive format.

JSON specification of the ZIP key-value store.

Extends:

  • KvStoreAdapter — Key-value store adapter that adapts a single base key-value store.

Required members:
driver : "zip"
base : KvStore | KvStoreUrl

Base key-value store to adapt.

Optional members:
path : string

Key prefix within the key-value store.

If the prefix is intended to correspond to a Unix-style directory path, it should end with "/".

context : Context

Specifies context resources that augment/override the parent context.

cache_pool : ContextResource = "cache_pool"

Specifies or references a previously defined Context.cache_pool. It is typically more convenient to specify a default cache_pool in the context.

data_copy_concurrency : ContextResource = "data_copy_concurrency"

Specifies or references a previously defined Context.data_copy_concurrency. It is typically more convenient to specify a default data_copy_concurrency in the context.

json KvStoreUrl/zip : string

zip: KvStore URL scheme

Zip key-value store adapters may be specified using the zip:path URL syntax.

Examples

URL representation

JSON representation

"file:///tmp/archive.zip|zip:path/within/archive"

 
{"driver": "zip",
 "path": "path/within/archive",
 "base": {"driver": "file",
          "path": "/tmp/archive.zip"}
}

"file:///tmp/archive.zip|zip:nested.zip|zip:path/within/inner_zip"

 
{"driver": "zip",
 "path": "path/within/inner_zip",
 "base": {"driver": "zip",
          "path": "nested.zip",
          "base": {"driver": "file",
                   "path": "/tmp/archive.zip"}}
}
Extends:

  • KvStoreUrl — URL representation of a key-value store.

Example JSON specifications

{ "driver": "zip",
  "kvstore": "gs://my-bucket/path/to/file.zip" }

Auto detection

This driver supports auto-detection based on the signature at the end of the file.

Warning

To limit the amount of data that is read, auto-detection is only supported if the end of file comment does not exceed 4096 bytes. ZIP archives with comments up to the maximum length of 65535 bytes are still supported without auto-detection, however.

Transactions and Write Amplification

This driver supports writing and deleting keys inside the ZIP archive. Since ZIP archives do not support random modification, each write or delete operation may rewrite the entire ZIP archive. Therefore, using transactions is strongly recommended to batch multiple write operations and minimize write amplification.

Limitations

Not all ZIP features and compression formats are supported:

Compression formats

  • Reading: Support is limited to deflate, bzip2, zstd, xz, and store (no compression).

  • Writing: Support is limited to deflate, zstd, xz, and store. Other compression methods (such as lzma) are not supported.

Standard ZIP encryption, strong encryption, and AES encryption are not supported.