Attaching custom trials to your experiment
While Ax excels at generating high-quality trials using Bayesian optimization, we may wish to attach trials with specific parameterizations to their experiment.
Introduction
This could be because we have an intuition that certain parameterizations will be successful and want to encourage the optimizer to explore this region of the search space. It could be that we have some “baseline” configuration which represents the status quo of our system (see attaching a baseline trial to your experiment for further reading). Or, we may already have some previously collected data from manual "trials" conducted before the Ax experiment began.
In any of these cases, we can use the Client
's attach_trial
method to
manually attach a trial with a custom parameterization to our experiment.
Prerequisites
We will assume you are already familiar with using Ax for ask-tell optimization, though this can be used for closed-loop experiments as well.
Setup
Before we begin you must instantiate the Client
and configure it your your
experiment.
client = Client()
client.configure_experiment(...)
client.configure_optimization(...)
Steps
- Call
attach_trial
with your custom parameterization - Attach data and complete the trial as you typically would
1. Call attach_trial
with your custom parameterization
attach_trial
takes two parameters: the parameters of the trial and optionally
an arm name (here, arm is synonymous with trial; see the glossary for more
info). The parameters dictionary must contain a value for each parameter in the
experiment's search space. The arm name is not necessary, but it can be useful
for keeping track of which arms were not generated by Ax when looking at
analyses. Once called attach_trial
will return the newly created trial's index
-- we'll need this when we want to attach data and complete the trial.
my_parameters = {"x1": 0, "x2": 0, "x3": 0, "x4": 0, "x5": 0, "x6": 0}
trial_index = client.attach_trial(
parameters=my_parameters,
arm_name="origin",
)
2. Attach data and complete the trial as you typically would
Once the trial index has been generated you may evaluate the trial and tell the
results to Ax using complete_trial
.
client.complete_trial(
trial_index=trial_index,
raw_data={"hartmann6": hartmann6(**my_parameters)}
)
Learn more
Take a look at these other recipes to continue your learning: