beaker-py

A lightweight pure-Python RPC-based client for Beaker.

See also

There are also higher-level Python packages available for interacting with Beaker, such as Gantry.

Installing

Installing with pip

beaker-py is available on PyPI. Just run

pip install beaker-py

Installing from source

To install beaker-py from source, first clone the repository:

git clone https://github.com/allenai/beaker.git

Then create or activate a Python virtual environment, and run:

cd beaker/bindings/python
make dev-install

Quick start

If you’ve already configured the Beaker command-line client, beaker-py will find and use the existing configuration file (usually located at $HOME/.beaker/config.yml) or BEAKER_TOKEN environment variable.

Then you can instantiate the Beaker Python client with the Beaker.from_env() class method:

from beaker import Beaker

with Beaker.from_env() as beaker:
    ...

With the Python client, you can:

• Query Clusters with Beaker.cluster methods. For example:

  beaker.cluster.get("ai2/jupiter-cirrascale-2")
  

• Manage Datasets with Beaker.dataset methods. For example:

  beaker.dataset.create(dataset_name, source_dir)
  

• Manage Experiments with Beaker.experiment and Beaker.workload methods. For example:

  beaker.experiment.create(spec=spec, name=name)
  

• Manage Groups with Beaker.group methods. For example:

  beaker.group.create(name)
  

• Manage Images with Beaker.image methods. For example:

  beaker.image.update(image, name=name)
  

• Manage Secrets with Beaker.secret methods. For example:

  beaker.secret.write(name, value)
  

• Manage Workspaces with Beaker.workspace methods. For example:

  beaker.workspace.create("ai2/new_workspace")
  

• Track Jobs with Beaker.job methods. For example:

  beaker.job.logs(job, follow=True)
  

• Create and process Queues with Beaker.queue methods. For example:

  beaker.queue.create("my-work-queue", batch_size=4)
  

If you’re coming from v1 of beaker-py, consider reading the migration guide.

Example workflow

Launch and follow an experiment like Gantry does:

import time
from beaker import Beaker, BeakerExperimentSpec, BeakerJobPriority


with Beaker.from_env() as beaker:
    # Build experiment spec...
    spec = BeakerExperimentSpec.new(
        description="beaker-py test run",
        beaker_image="petew/hello-world",
        priority=BeakerJobPriority.low,
        preemptible=True,
    )

    # Create experiment workload...
    workload = beaker.experiment.create(spec=spec)

    # Wait for job to be created...
    while (job := beaker.workload.get_latest_job(workload)) is None:
        print("waiting for job to start...")
        time.sleep(1.0)

    # Follow logs...
    print("Job logs:")
    for job_log in beaker.job.logs(job, follow=True):
        print(job_log.message.decode())

See the integration tests for more examples.

Indices and tables

• Index

• Module Index