Sedpack Installation

Installation of the Sedpack Python package.

Python Package Index

All one should need to install the package from PyPI is:

pip install sedpack

If you plan to use TensorFlow features (such as the "tfrec" shard format or as_tfdataset), TensorFlow needs to be installed as well:

pip install "sedpack[tf]"

Note that this is the latest stable version of the package. If you want the bleeding edge features install from source instead.

Installing from Source

One can always opt for installation from the source.

Development dependencies:

• python-dev and gcc for xxhash
• Install Rust

Clone and create a Python virtual environment:

git clone github.com/google/sedpack/  # Clone the repository
python3 -m venv my_env  # Create Python virtual environment
source my_env/bin/activate  # Activate your virtual environment
cd sedpack/  # Change directory to the cloned git repository

[optional] Install pinned Python dependencies:

Linux

python3 -m pip install --require-hashes -r requirements.txt

Windows

Currently the file requirements.txt is compiled for Linux and some versions of PyPI packages might be different or even not existing elsewhere. One can always recreate those:

python3 -m pip install pip-tools
python3 -m pip-compile --generate-hashes pyproject.toml > requirements_win.txt
python3 -m pip install --require-hashes -r requirements_win.txt

macOS

Currently the file requirements.txt is compiled for Linux and some versions of PyPI packages might be different or even not existing elsewhere. One can always recreate those:

python3 -m pip install pip-tools
python3 -m pip-compile --generate-hashes pyproject.toml > requirements_mac.txt
python3 -m pip install --require-hashes -r requirements_mac.txt

Install sedpack with development dependencies and TensorFlow:

python3 -m pip install --editable '.[dev,tf]'

Or for a minimal installation without development or TensorFlow dependencies:

python3 -m pip install --editable .

[optional] Run unit-tests (none should be skipped when TensorFlow is installed):

python -m pytest

[optional] Rebuild the Rust library after a change:

maturin develop --release