Attaching existing data to your experiment

Ax provides a straightforward way to attach custom trials to your experiment -- we can use this to attach data from trials we conducted before beginning our experiment with Ax.

Introduction

It is not uncommon to have conducted some manual trials before formally beginning our experiment with Ax. These could have been from some non-adaptive experiment, a previous Ax experiment (though in this case it is often more convenient to save and load the Ax either to a JSON file or to a database), or some baseline observation for our system's status quo.

When this occurs it is useful to give Ax access to these observations, both to provide extra information to the surrogate model used in Bayesian optimization and for later analyses of these trials along with the trials generated by Ax.

In any of these cases, we can use the Client's attach_trial method to manually attach a trial with a custom parameterization to our experiment.

Prerequisites

We will assume you are familiar with basic Ax usage already.

Setup

Before we begin you must instantiate the Client and configure it your your experiment.

client = Client()

client.configure_experiment(...)
client.configure_optimization(...)

Steps

1. Call attach_trial with your custom parameterization
2. Attach data and complete the trial as you typically would

1. Call attach_trial with your custom parameterization

attach_trial takes two parameters: the parameters of the trial and optionally an arm name (here, arm is synonymous with trial). The parameters dictionary must contain a value for each parameter in the experiment's search space. The arm name is not necessary, but it can be useful for keeping track of which arms were not generated by Ax when looking at analyses. Once called attach_trial will return the newly created trial's index -- we'll need this when we want to attach data and complete the trial.

2. Attach data and complete the trial as you typically would

Once the trial index has been generated you may evaluate the trial and tell the results to Ax using complete_trial.

Steps 1 and 2 are often done together using a loop which iterates over each existing trial.

# For simplicity we'll assume the existing data is in the form of a list of tuples
# of dictionaries, with the parameterization first and the raw data second.
my_data = [
    ({"x1": 0, "x2": 0, ...}, {"my_objective": 0}),
    ...
]

for paramters, raw_data in my_data.items():
    # First attach the trial and note the trial index
    trial_index = client.attach_trial(
        parameters=my_parameters,
    )

    # Then complete the trial with the existing data
    client.complete_trial(
        trial_index=trial_index, raw_data=raw_data,
    )

Learn more

Take a look at these other recipes to continue your learning:

• Attaching a custom parameterization to your experiment