Scripts

smart_control.reinforcement_learning.scripts.populate_starter_buffer

Script to populate an initial replay buffer for RL training.

This creates a starter buffer with exploration data that can be used to bootstrap the training process.

populate_replay_buffer

populate_replay_buffer(
    buffer_name,
    buffer_capacity,
    steps_per_run,
    num_runs,
    sequence_length,
    env_gin_config_file_path,
)

Populates a replay buffer with initial exploration data.

Parameters:

Name	Type	Description	Default
buffer_name		Name with which to save replay buffer. Buffer will be at smart_control/reinforcement_learning/data/starter_buffers/{buffer_name}	required
buffer_capacity		Maximum size of the replay buffer	required
steps_per_run		Number of steps per actor run	required
num_runs		Number of actor runs to perform	required
sequence_length		Length of sequences to store in the replay buffer	required
env_gin_config_file_path		Path to the environment configuration file	required

Returns:

Type	Description
	The replay buffer.

smart_control.reinforcement_learning.scripts.train

Trains a reinforcement learning agent using a pre-populated replay buffer.

This script sets up the training process with separate collection and evaluation components.

train_agent

train_agent(
    starter_buffer_path,
    experiment_name,
    agent_type="sac",
    train_iterations=100000,
    collect_steps_per_iteration=1,
    batch_size=256,
    log_interval=100,
    eval_interval=1000,
    num_eval_episodes=5,
    checkpoint_interval=1000,
    learner_iterations=200,
)

Trains a reinforcement learning agent using a pre-populated replay buffer.

Parameters:

Name	Type	Description	Default
starter_buffer_path		Path to the pre-populated replay buffer	required
experiment_name		Name of the experiment - used to name the experiment results directory	required
agent_type		Type of agent to train ('sac' or 'td3')	'sac'
train_iterations		Number of training iterations	100000
collect_steps_per_iteration		Number of collection steps per training iteration	1
batch_size		Batch size for training	256
log_interval		Interval for logging training metrics	100
eval_interval		Interval for evaluating the agent	1000
num_eval_episodes		Number of episodes for evaluation	5
checkpoint_interval		Interval for checkpointing the replay buffer	1000
learner_iterations		Number of iterations to run the agent learner per training loop	200

Returns:

Type	Description
	The trained agent.