ax.core¶
Core Classes¶
Arm¶
-
class
ax.core.arm.
Arm
(parameters, name=None)[source]¶ Bases:
ax.core.base.Base
Base class for defining arms.
Randomization in experiments assigns units to a given arm. Thus, the arm encapsulates the parametrization needed by the unit.
-
property
name_or_short_signature
¶ Returns arm name if exists; else last 4 characters of the hash.
Used for presentation of candidates (e.g. plotting and tables), where the candidates do not yet have names (since names are automatically set upon addition to a trial).
- Return type
-
property
parameters
¶ Get mapping from parameter names to values.
-
property
BaseTrial¶
-
class
ax.core.base_trial.
BaseTrial
(experiment, trial_type=None)[source]¶ Bases:
abc.ABC
,ax.core.base.Base
Base class for representing trials.
Trials are containers for arms that are deployed together. There are two types of trials: regular Trial, which only contains a single arm, and BatchTrial, which contains an arbitrary number of arms.
-
abstract property
abandoned_arms
¶ All abandoned arms, associated with this trial.
-
assign_runner
()[source]¶ Assigns default experiment runner if trial doesn’t already have one.
- Return type
-
complete
()[source]¶ - Stops the trial if functionality is defined on runner
and marks trial completed.
- Return type
- Returns
The trial instance.
-
property
deployed_name
¶ Name of the experiment created in external framework.
This property is derived from the name field in run_metadata.
-
property
experiment
¶ The experiment this trial belongs to.
- Return type
-
fetch_data
(metrics=None, **kwargs)[source]¶ Fetch data for this trial for all metrics on experiment.
- Parameters
- Return type
- Returns
Data for this trial.
-
mark_abandoned
(reason=None)[source]¶ Mark trial as abandoned.
- Parameters
abandoned_reason – The reason the trial was abandoned.
- Return type
- Returns
The trial instance.
-
mark_dispatched
()[source]¶ Mark trial as dispatched through the service API to await completion.
- Return type
- Returns
The trial instance.
-
mark_staged
()[source]¶ Mark the trial as being staged for running.
- Return type
- Returns
The trial instance.
-
run
()[source]¶ Deploys the trial according to the behavior on the runner.
The runner returns a run_metadata dict containining metadata of the deployment process. It also returns a deployed_name of the trial within the system to which it was deployed. Both these fields are set on the trial.
- Return type
- Returns
The trial instance.
-
property
run_metadata
¶ Dict containing metadata from the deployment process.
This is set implicitly during trial.run().
-
property
status
¶ The status of the trial in the experimentation lifecycle.
- Return type
-
property
time_run_started
¶ Time the trial was started running (i.e. collecting data).
-
abstract property
-
class
ax.core.base_trial.
TrialStatus
[source]¶ Bases:
enum.Enum
Enum of trial status.
General lifecycle of a trial is::
CANDIDATE --> STAGED --> RUNNING --> COMPLETED -------------> --> FAILED (machine failure) -------------------------> ABANDONED (human-initiated action) --> DISPATCHED ---------->
Trials may be abandoned at any time prior to completion or failure via human intervention. The difference between abandonment and failure is that the former is human-directed, while the latter is an internal failure state.
Additionally, when trials are deployed, they may be in an intermediate staged state (e.g. scheduled but waiting for resources) or immediately transition to running.
When used though the service API, Ax proposes trials and expects the client application to compelete them with evaluation data when available. In this case, a trial is set to ‘dispatched’ right after the trial is created, and when user completes the trial with data, its status is set to ‘completed’.
-
ABANDONED
= 5¶
-
CANDIDATE
= 0¶
-
COMPLETED
= 3¶
-
DISPATCHED
= 6¶
-
FAILED
= 2¶
-
RUNNING
= 4¶
-
STAGED
= 1¶
-
BatchTrial¶
-
class
ax.core.batch_trial.
AbandonedArm
[source]¶ Bases:
tuple
Tuple storing metadata of arm that has been abandoned within a BatchTrial.
-
property
name
¶ Alias for field number 0
-
property
reason
¶ Alias for field number 2
-
property
time
¶ Alias for field number 1
-
property
-
class
ax.core.batch_trial.
BatchTrial
(experiment, generator_run=None, trial_type=None)[source]¶ Bases:
ax.core.base_trial.BaseTrial
-
property
abandoned_arms
¶ List of arms that have been abandoned within this trial
-
property
abandoned_arms_metadata
¶ - Return type
-
add_arm
(*args, **kwargs)¶
-
add_arms_and_weights
(*args, **kwargs)¶
-
add_generator_run
(*args, **kwargs)¶
-
property
arm_weights
¶ The set of arms and associated weights for the trial.
These are constructed by merging the arms and weights from each generator run that is attached to the trial.
-
property
arms_by_name
¶ Map from arm name to object for all arms in trial.
-
property
experiment
¶ The experiment this batch belongs to.
- Return type
-
property
generator_run_structs
¶ List of generator run structs attached to this trial.
Struct holds generator_run object and the weight with which it was added.
- Return type
-
property
is_factorial
¶ Return true if the trial’s arms are a factorial design with no linked factors.
- Return type
-
mark_arm_abandoned
(arm_name, reason=None)[source]¶ Mark a arm abandoned.
Usually done after deployment when one arm causes issues but user wants to continue running other arms in the batch.
- Parameters
- Return type
- Returns
The batch instance.
-
normalized_arm_weights
(total=1, trunc_digits=None)[source]¶ Returns arms with a new set of weights normalized to the given total.
This method is useful for many runners where we need to normalize weights to a certain total without mutating the weights attached to a trial.
- Parameters
total (
float
) – The total weight to which to normalize. Default is 1, in which case arm weights can be interpreted as probabilities.trunc_digits (
Optional
[int
]) – The number of digits to keep. If the resulting total weight is not equal to total, re-allocate weight in such a way to maintain relative weights as best as possible.
- Return type
- Returns
Mapping from arms to the new set of weights.
-
reweight_status_quo
(*args, **kwargs)¶
-
run
()[source]¶ Deploys the trial according to the behavior on the runner.
The runner returns a run_metadata dict containining metadata of the deployment process. It also returns a deployed_name of the trial within the system to which it was deployed. Both these fields are set on the trial.
- Return type
- Returns
The trial instance.
-
set_status_quo_and_optimize_power
(*args, **kwargs)¶
-
set_status_quo_with_weight
(*args, **kwargs)¶
-
property
Data¶
-
ax.core.data.
COLUMN_DATA_TYPES
= {'arm_name': <class 'str'>, 'end_time': <class 'pandas._libs.tslibs.timestamps.Timestamp'>, 'fidelities': <class 'str'>, 'frac_nonnull': <class 'numpy.float64'>, 'mean': <class 'numpy.float64'>, 'metric_name': <class 'str'>, 'n': <class 'numpy.int64'>, 'random_split': <class 'numpy.int64'>, 'sem': <class 'numpy.float64'>, 'start_time': <class 'pandas._libs.tslibs.timestamps.Timestamp'>, 'trial_index': <class 'numpy.int64'>}¶
-
class
ax.core.data.
Data
(df=None, description=None)[source]¶ Bases:
ax.core.base.Base
Class storing data for an experiment.
The dataframe is retrieved via the df property. The data can be stored to gluster for future use by attaching it to an experiment using experiment.add_data() (this requires a description to be set.)
-
df
¶ DataFrame with underlying data, and required columns.
-
description
¶ Human-readable description of data.
-
property
df
- Return type
DataFrame
-
property
df_hash
¶ Compute hash of pandas DataFrame.
This first serializes the DataFrame and computes the md5 hash on the resulting string. Note that this may cause performance issue for very large DataFrames.
- Parameters
df – The DataFrame for which to compute the hash.
- Returns
str: The hash of the DataFrame.
- Return type
-
static
from_evaluations
(evaluations, trial_index, sample_sizes=None)[source]¶ Convert dict of evaluations to Ax data object.
- Parameters
evaluations (
Dict
[str
,Dict
[str
,Tuple
[float
,Optional
[float
]]]]) – Map from arm name to metric outcomes (itself a mapping of metric names to tuples of mean and optionally a SEM).trial_index (
int
) – Trial index to which this data belongs.sample_sizes (
Optional
[Dict
[str
,int
]]) – Number of samples collected for each arm.
- Return type
- Returns
Ax Data object.
-
static
from_fidelity_evaluations
(evaluations, trial_index, sample_sizes=None)[source]¶ Convert dict of fidelity evaluations to Ax data object.
- Parameters
evaluations (
Dict
[str
,List
[Tuple
[Dict
[str
,Union
[str
,bool
,float
,int
,None
]],Dict
[str
,Tuple
[float
,Optional
[float
]]]]]]) – Map from arm name to list of (fidelity, metric outcomes) (where metric outcomes is itself a mapping of metric names to tuples of mean and SEM).trial_index (
int
) – Trial index to which this data belongs.sample_sizes (
Optional
[Dict
[str
,int
]]) – Number of samples collected for each arm.
- Return type
- Returns
Ax Data object.
-
-
ax.core.data.
REQUIRED_COLUMNS
= {'arm_name', 'mean', 'metric_name'}¶
-
ax.core.data.
TPdTimestamp
¶ alias of
pandas._libs.tslibs.timestamps.Timestamp
-
ax.core.data.
custom_data_class
(column_data_types=None, required_columns=None, time_columns=None)[source]¶ Creates a custom data class with additional columns.
All columns and their designations on the base data class are preserved, the inputs here are appended to the definitions on the base class.
- Parameters
- Return type
- Returns
New data subclass with amended column definitions.
Experiment¶
-
class
ax.core.experiment.
Experiment
(search_space, name=None, optimization_config=None, tracking_metrics=None, runner=None, status_quo=None, description=None, is_test=False, experiment_type=None)[source]¶ Bases:
ax.core.base.Base
Base class for defining an experiment.
-
add_tracking_metric
(metric)[source]¶ Add a new metric to the experiment.
- Parameters
metric (
Metric
) – Metric to be added.- Return type
-
property
arms_by_name
¶ The arms belonging to this experiment, by their name.
-
property
arms_by_signature
¶ The arms belonging to this experiment, by their signature.
-
property
data_by_trial
¶ Data stored on the experiment, indexed by trial index and storage time.
First key is trial index and second key is storage time in milliseconds. For a given trial, data is ordered by storage time, so first added data will appear first in the list.
- Return type
-
property
default_trial_type
¶ Default trial type assigned to trials in this experiment.
In the base experiment class this is always None. For experiments with multiple trial types, use the MultiTypeExperiment class.
-
fetch_data
(metrics=None, **kwargs)[source]¶ Fetches data for all metrics and trials on this experiment.
-
property
is_simple_experiment
¶ Whether this experiment is a regular Experiment or the subclassing SimpleExperiment.
-
lookup_data_for_trial
(trial_index)[source]¶ Lookup stored data for a specific trial.
Returns latest data object present for this trial. Returns empty data if no data present.
-
lookup_data_for_ts
(timestamp)[source]¶ Collect data for all trials stored at this timestamp.
Useful when many trials’ data was fetched and stored simultaneously and user wants to retrieve same collection of data later.
Can also be used to lookup specific data for a single trial when storage time is known.
-
new_batch_trial
(generator_run=None, trial_type=None)[source]¶ Create a new batch trial associated with this experiment.
- Return type
-
new_trial
(generator_run=None, trial_type=None)[source]¶ Create a new trial associated with this experiment.
- Return type
-
property
num_abandoned_arms
¶ How many arms attached to this experiment are abandoned.
- Return type
-
property
optimization_config
¶ The experiment’s optimization config.
- Return type
-
property
parameters
¶ The parameters in the experiment’s search space.
-
remove_tracking_metric
(metric_name)[source]¶ Remove a metric that already exists on the experiment.
- Parameters
metric_name (
str
) – Unique name of metric to remove.- Return type
-
runner_for_trial
(trial)[source]¶ The default runner to use for a given trial.
In the base experiment class, this is always the default experiment runner. For experiments with multiple trial types, use the MultiTypeExperiment class.
-
property
search_space
¶ The search space for this experiment.
When setting a new search space, all parameter names and types must be preserved. However, if no trials have been created, all modifications are allowed.
- Return type
-
property
status_quo
¶ The existing arm that new arms will be compared against.
-
property
sum_trial_sizes
¶ Sum of numbers of arms attached to each trial in this experiment.
- Return type
-
supports_trial_type
(trial_type)[source]¶ Whether this experiment allows trials of the given type.
The base experiment class only supports None. For experiments with multiple trial types, use the MultiTypeExperiment class.
- Return type
-
-
ax.core.experiment.
logger
= <Logger ax.core.experiment (INFO)>¶
GeneratorRun¶
-
class
ax.core.generator_run.
ArmWeight
[source]¶ Bases:
tuple
NamedTuple for tying together arms and weights.
-
property
arm
¶ Alias for field number 0
-
property
weight
¶ Alias for field number 1
-
property
-
class
ax.core.generator_run.
GeneratorRun
(arms, weights=None, optimization_config=None, search_space=None, model_predictions=None, best_arm_predictions=None, type=None, fit_time=None, gen_time=None, model_key=None, model_kwargs=None, bridge_kwargs=None)[source]¶ Bases:
ax.core.base.Base
An object that represents a single run of a generator.
This object is created each time the
gen
method of a generator is called. It stores the arms and (optionally) weights that were generated by the run. When we add a generator run to a trial, its arms and weights will be merged with those from previous generator runs that were already attached to the trial.-
property
arm_weights
¶ Mapping from arms to weights (order matches order in arms property).
-
property
best_arm_predictions
¶
-
property
index
¶ The index of this generator run within a trial’s list of generator run structs. This field is set when the generator run is added to a trial.
-
property
model_predictions
¶
-
property
model_predictions_by_arm
¶
-
property
optimization_config
¶ The optimization config used during generation of this run.
- Return type
-
property
param_df
¶ Constructs a Pandas dataframe with the parameter values for each arm.
Useful for inspecting the contents of a generator run.
- Returns
a dataframe with the generator run’s arms.
- Return type
pd.DataFrame
-
property
search_space
¶ The search used during generation of this run.
- Return type
-
property
-
class
ax.core.generator_run.
GeneratorRunType
[source]¶ Bases:
enum.Enum
Class for enumerating generator run types.
-
MANUAL
= 1¶
-
STATUS_QUO
= 0¶
-
Metric¶
-
class
ax.core.metric.
Metric
(name, lower_is_better=None)[source]¶ Bases:
ax.core.base.Base
Base class for representing metrics.
-
lower_is_better
¶ Flag for metrics which should be minimized.
-
fetch_experiment_data
(experiment, **kwargs)[source]¶ Fetch this metric’s data for an experiment.
Default behavior is to fetch data from all trials expecting data and concatenate the results.
- Return type
-
classmethod
fetch_experiment_data_multi
(experiment, metrics, **kwargs)[source]¶ Fetch multiple metrics data for an experiment.
Default behavior calls fetch_trial_data_multi for each trial. Subclasses should override to batch data computation across trials + metrics.
- Return type
-
MultiTypeExperiment¶
-
class
ax.core.multi_type_experiment.
MultiTypeExperiment
(name, search_space, default_trial_type, default_runner, optimization_config=None, status_quo=None, description=None, is_test=False, experiment_type=None)[source]¶ Bases:
ax.core.experiment.Experiment
Class for experiment with multiple trial types.
A canonical use case for this is tuning a large production system with limited evaluation budget and a simulator which approximates evaluations on the main system. Trial deployment and data fetching is separate for the two systems, but the final data is combined and fed into multi-task models.
See the Multi-Task Modeling tutorial for more details.
-
name
¶ Name of the experiment.
-
description
¶ Description of the experiment.
-
add_tracking_metric
(metric, trial_type, canonical_name=None)[source]¶ Add a new metric to the experiment.
- Parameters
- Return type
-
add_trial_type
(trial_type, runner)[source]¶ Add a new trial_type to be supported by this experiment.
- Parameters
- Return type
-
property
default_trial_type
¶ Default trial type assigned to trials in this experiment.
-
fetch_data
(metrics=None, **kwargs)[source]¶ Fetches data for all metrics and trials on this experiment.
-
property
metric_to_trial_type
¶ Map metrics to trial types.
Adds in default trial type for OC metrics to custom defined trial types..
-
remove_tracking_metric
(metric_name)[source]¶ Remove a metric that already exists on the experiment.
- Parameters
metric_name (
str
) – Unique name of metric to remove.- Return type
-
runner_for_trial
(trial)[source]¶ The default runner to use for a given trial.
Looks up the appropriate runner for this trial type in the trial_type_to_runner.
-
supports_trial_type
(trial_type)[source]¶ Whether this experiment allows trials of the given type.
Only trial types defined in the trial_type_to_runner are allowed.
- Return type
-
update_runner
(trial_type, runner)[source]¶ Update the default runner for an existing trial_type.
- Parameters
- Return type
-
-
ax.core.multi_type_experiment.
logger
= <Logger ax.core.multi_type_experiment (INFO)>¶
Objective¶
-
class
ax.core.objective.
Objective
(metric, minimize=False)[source]¶ Bases:
ax.core.base.Base
Base class for representing an objective.
-
minimize
¶ If True, minimize metric.
-
-
class
ax.core.objective.
ScalarizedObjective
(metrics, weights=None, minimize=False)[source]¶ Bases:
ax.core.objective.Objective
Class for an objective composed of a linear scalarization of metrics.
-
metrics
¶ List of metrics.
-
weights
¶ Weights for scalarization; default to 1.
-
property
metric_weights
¶ Get the objective metrics and weights.
-
Observation¶
-
class
ax.core.observation.
Observation
(features, data, arm_name=None)[source]¶ Bases:
ax.core.base.Base
Represents an observation.
A set of features (ObservationFeatures) and corresponding measurements (ObservationData). Optionally, a arm name associated with the features.
-
features
¶ - Type
-
data
¶ - Type
-
-
class
ax.core.observation.
ObservationData
(metric_names, means, covariance)[source]¶ Bases:
ax.core.base.Base
Outcomes observed at a point.
The “point” corresponding to this ObservationData would be an ObservationFeatures object.
-
metric_names
¶ A list of k metric names that were observed
-
means
¶ a k-array of observed means
-
covariance
¶ a (k x k) array of observed covariances
-
-
class
ax.core.observation.
ObservationFeatures
(parameters, trial_index=None, start_time=None, end_time=None, random_split=None)[source]¶ Bases:
ax.core.base.Base
The features of an observation.
These include both the arm parameters and the features of the observation found in the Data object: trial index, times, and random split. This object is meant to contain everything needed to represent this observation in a model feature space. It is essentially a row of Data joined with the arm parameters.
An ObservationFeatures object would typically have a corresponding ObservationData object that provides the observed outcomes.
-
parameters
¶ arm parameters
-
trial_index
¶ trial index
-
start_time
¶ batch start time
-
end_time
¶ batch end time
-
random_split
¶ random split
-
static
from_arm
(arm, trial_index=None, start_time=None, end_time=None, random_split=None)[source]¶ Convert a Arm to an ObservationFeatures, including additional data as specified.
- Return type
-
-
ax.core.observation.
observations_from_data
(experiment, data)[source]¶ Convert Data to observations.
Converts a Data object to a list of Observation objects. Pulls arm parameters from experiment. Overrides fidelity parameters in the arm with those found in the Data object.
Uses a diagonal covariance matrix across metric_names.
- Parameters
experiment (
Experiment
) – Experiment with arm parameters.data (
Data
) – Data of observations
Returns: List of Observation objects.
- Return type
-
ax.core.observation.
separate_observations
(observations, copy=False)[source]¶ Split out observations into features+data.
- Parameters
observations (
List
[Observation
]) – input observations- Returns
ObservationFeatures observation_data: ObservationData
- Return type
observation_features
OptimizationConfig¶
-
ax.core.optimization_config.
MAX_OBJECTIVES
= 4¶
-
ax.core.optimization_config.
OC_TEMPLATE
= 'OptimizationConfig(objective={objective}, outcome_constraints=[{constraints}])'¶
-
class
ax.core.optimization_config.
OptimizationConfig
(objective, outcome_constraints=None)[source]¶ Bases:
ax.core.base.Base
An optimization configuration, which comprises an objective and outcome constraints.
There is no minimum or maximum number of outcome constraints, but an individual metric can have at most two constraints–which is how we represent metrics with both upper and lower bounds.
-
property
outcome_constraints
¶ Get outcome constraints.
- Return type
-
property
OutcomeConstraint¶
-
ax.core.outcome_constraint.
CONSTRAINT_WARNING_MESSAGE
= 'Specified constraint appears invalid: {bound} bound on metric for which {is_better} values are better.'¶
-
ax.core.outcome_constraint.
LOWER_BOUND_MISMATCH
= {'bound': 'Lower', 'is_better': 'lower'}¶
-
class
ax.core.outcome_constraint.
OutcomeConstraint
(metric, op, bound, relative=True)[source]¶ Bases:
ax.core.base.Base
Base class for representing outcome constraints.
Outcome constraints may of the form metric >= bound or metric <= bound, where the bound can be expressed as an absolute measurement or relative to the status quo (if applicable).
-
metric
¶ Metric to constrain.
-
op
¶ Specifies whether metric should be greater or equal to, or less than or equal to, some bound.
-
bound
¶ The bound in the constraint.
-
relative
¶ Whether you want to bound on an absolute or relative scale. If relative, bound is the acceptable percent change.
-
property
metric
- Return type
-
property
op
- Return type
-
-
ax.core.outcome_constraint.
UPPER_BOUND_MISMATCH
= {'bound': 'Upper', 'is_better': 'higher'}¶
-
ax.core.outcome_constraint.
logger
= <Logger ax.core.outcome_constraint (INFO)>¶
Parameter¶
-
class
ax.core.parameter.
ChoiceParameter
(name, parameter_type, values, is_ordered=False, is_task=False, is_fidelity=False, target_value=None)[source]¶ Bases:
ax.core.parameter.Parameter
Parameter object that specifies a discrete set of values.
-
add_values
(values)[source]¶ Add input list to the set of allowed values for parameter.
Cast all input values to the parameter type.
-
property
parameter_type
¶ - Return type
-
set_values
(values)[source]¶ Set the list of allowed values for parameter.
Cast all input values to the parameter type.
-
-
ax.core.parameter.
FIXED_CHOICE_PARAM_ERROR
= 'ChoiceParameters require multiple feasible values. Please use FixedParameter instead when setting a single possible value.'¶
-
class
ax.core.parameter.
FixedParameter
(name, parameter_type, value, is_fidelity=False, target_value=None)[source]¶ Bases:
ax.core.parameter.Parameter
Parameter object that specifies a single fixed value.
-
property
parameter_type
¶ - Return type
-
property
-
ax.core.parameter.
PARAMETER_PYTHON_TYPE_MAP
= {<ParameterType.INT: 1>: <class 'int'>, <ParameterType.FLOAT: 2>: <class 'float'>, <ParameterType.STRING: 3>: <class 'str'>, <ParameterType.BOOL: 0>: <class 'bool'>}¶
-
class
ax.core.parameter.
Parameter
[source]¶ Bases:
ax.core.base.Base
-
is_valid_type
(value)[source]¶ Whether a given value’s type is allowed by this parameter.
- Return type
-
abstract property
parameter_type
¶ - Return type
-
property
python_type
¶ The python type for the corresponding ParameterType enum.
Used primarily for casting values of unknown type to conform to that of the parameter.
-
-
class
ax.core.parameter.
ParameterType
[source]¶ Bases:
enum.Enum
An enumeration.
-
BOOL
= 0¶
-
FLOAT
= 2¶
-
INT
= 1¶
-
STRING
= 3¶
-
-
class
ax.core.parameter.
RangeParameter
(name, parameter_type, lower, upper, log_scale=False, digits=None, is_fidelity=False, target_value=None)[source]¶ Bases:
ax.core.parameter.Parameter
Parameter object that specifies a continuous numerical range of values.
-
property
digits
¶ Number of digits to round values to for float type.
Upper and lower bound are re-cast after this property is changed.
-
is_valid_type
(value)[source]¶ Same as default except allows floats whose value is an int for Int parameters.
- Return type
-
property
log_scale
¶ Whether to sample in log space when drawing random values of the parameter.
- Return type
-
property
lower
¶ Lower bound of the parameter range.
Value is cast to parameter type upon set and also validated to ensure the bound is strictly less than upper bound.
- Return type
-
property
parameter_type
¶ - Return type
-
update_range
(lower=None, upper=None)[source]¶ Set the range to the given values.
If lower or upper is not provided, it will be left at its current value.
- Parameters
- Return type
-
property
upper
¶ Upper bound of the parameter range.
Value is cast to parameter type upon set and also validated to ensure the bound is strictly greater than lower bound.
- Return type
-
property
-
ax.core.parameter.
TParameterType
= typing.Union[typing.Type[int], typing.Type[float], typing.Type[str], typing.Type[bool]]¶
ParameterConstraint¶
-
class
ax.core.parameter_constraint.
OrderConstraint
(lower_parameter, upper_parameter)[source]¶ Bases:
ax.core.parameter_constraint.ParameterConstraint
Constraint object for specifying one parameter to be smaller than another.
-
clone_with_transformed_parameters
(transformed_parameters)[source]¶ Clone, but replace parameters with transformed versions.
- Return type
-
property
constraint_dict
¶ Weights on parameters for linear constraint representation.
-
-
class
ax.core.parameter_constraint.
ParameterConstraint
(constraint_dict, bound)[source]¶ Bases:
ax.core.base.Base
Base class for linear parameter constraints.
Constraints are expressed using a map from parameter name to weight followed by a bound.
- The constraint is satisfied if w * v <= b where:
w is the vector of parameter weights. v is a vector of parameter values. b is the specified bound. * is the dot product operator.
-
check
(parameter_dict)[source]¶ Whether or not the set of parameter values satisfies the constraint.
Does a weighted sum of the parameter values based on the constraint_dict and checks that the sum is less than the bound.
-
class
ax.core.parameter_constraint.
SumConstraint
(parameters, is_upper_bound, bound)[source]¶ Bases:
ax.core.parameter_constraint.ParameterConstraint
Constraint on the sum of parameters being greater or less than a bound.
-
clone_with_transformed_parameters
(transformed_parameters)[source]¶ Clone, but replace parameters with transformed versions.
- Return type
-
property
constraint_dict
¶ Weights on parameters for linear constraint representation.
-
property
op
¶ Whether the sum is constrained by a <= or >= inequality.
- Return type
-
Runner¶
-
class
ax.core.runner.
Runner
[source]¶ Bases:
ax.core.base.Base
,abc.ABC
Abstract base class for custom runner classes
-
property
staging_required
¶ Whether the trial goes to staged or running state once deployed.
- Return type
-
property
SearchSpace¶
-
class
ax.core.search_space.
SearchSpace
(parameters, parameter_constraints=None)[source]¶ Bases:
ax.core.base.Base
Base object for SearchSpace object.
Contains a set of Parameter objects, each of which have a name, type, and set of valid values. The search space also contains a set of ParameterConstraint objects, which can be used to define restrictions across parameters (e.g. p_a < p_b).
-
cast_arm
(arm)[source]¶ Cast parameterization of given arm to the types in this SearchSpace.
For each parameter in given arm, cast it to the proper type specified in this search space. Throws if there is a mismatch in parameter names. This is mostly useful for int/float, which user can be sloppy with when hand written.
-
check_membership
(parameterization, raise_error=False)[source]¶ Whether the given parameterization belongs in the search space.
Checks that the given parameter values have the same name/type as search space parameters, are contained in the search space domain, and satisfy the parameter constraints.
- Parameters
- Return type
- Returns
Whether the parameterization is contained in the search space.
-
check_types
(parameterization, allow_none=True, raise_error=False)[source]¶ Checks that the given parameterization’s types match the search space.
Checks that the names of the parameterization match those specified in the search space, and the given values are of the correct type.
- Parameters
- Return type
- Returns
Whether the parameterization has valid types.
-
out_of_design_arm
()[source]¶ Create a default out-of-design arm.
An out of design arm contains values for some parameters which are outside of the search space. In the modeling conversion, these parameters are all stripped down to an empty dictionary, since the point is already outside of the modeled space.
- Return type
- Returns
New arm w/ null parameter values.
-
property
parameter_constraints
¶ - Return type
-
SimpleExperiment¶
-
ax.core.simple_experiment.
DEFAULT_OBJECTIVE_NAME
= 'objective'¶
-
class
ax.core.simple_experiment.
SimpleExperiment
(search_space, name=None, objective_name=None, evaluation_function=<function unimplemented_evaluation_function>, minimize=False, outcome_constraints=None, status_quo=None)[source]¶ Bases:
ax.core.experiment.Experiment
Simplified experiment class with defaults.
- Parameters
search_space (
SearchSpace
) – parameter spaceobjective_name (
Optional
[str
]) – which of the metrics computed by the evaluation function is the objectiveevaluation_function (
Callable
[[Dict
[str
,Union
[str
,bool
,float
,int
,None
]],Optional
[float
]],Union
[Dict
[str
,Tuple
[float
,Optional
[float
]]],Tuple
[float
,Optional
[float
]],float
,List
[Tuple
[Dict
[str
,Union
[str
,bool
,float
,int
,None
]],Dict
[str
,Tuple
[float
,Optional
[float
]]]]]]]) – function that evaluates mean and standard error for a parameter configuration. This function should accept a dictionary of parameter names to parameter values (TParametrization) and optionally a weight, and return a dictionary of metric names to a tuple of means and standard errors (TEvaluationOutcome). The function can also return a single tuple, in which case we assume the metric is the objective.minimize (
bool
) – whether the objective should be minimized, defaults to Falseoutcome_constraints (
Optional
[List
[OutcomeConstraint
]]) – constraints on the outcome, if anystatus_quo (
Optional
[Arm
]) – Arm representing existing “control” arm
-
add_tracking_metric
(metric)[source]¶ Add a new metric to the experiment.
- Parameters
metric (
Metric
) – Metric to be added.- Return type
-
eval
()[source]¶ Evaluate all arms in the experiment with the evaluation function passed as argument to this SimpleExperiment.
- Return type
-
property
evaluation_function
¶ Get the evaluation function.
-
fetch_data
(metrics=None, **kwargs)[source]¶ Fetches data for all metrics and trials on this experiment.
-
property
has_evaluation_function
¶ Whether this SimpleExperiment has a valid evaluation function attached.
- Return type
-
property
is_simple_experiment
¶ Whether this experiment is a regular Experiment or the subclassing SimpleExperiment.
-
ax.core.simple_experiment.
TEvaluationFunction
= typing.Callable[[typing.Dict[str, typing.Union[str, bool, float, int, NoneType]], typing.Union[float, NoneType]], typing.Union[typing.Dict[str, typing.Tuple[float, typing.Union[float, NoneType]]], typing.Tuple[float, typing.Union[float, NoneType]], float, typing.List[typing.Tuple[typing.Dict[str, typing.Union[str, bool, float, int, NoneType]], typing.Dict[str, typing.Tuple[float, typing.Union[float, NoneType]]]]]]]¶
Trial¶
-
class
ax.core.trial.
Trial
(experiment, generator_run=None, trial_type=None)[source]¶ Bases:
ax.core.base_trial.BaseTrial
Trial that only has one attached arm and no arm weights.
- Parameters
experiment (
Experiment
) – experiment, to which this trial is attachedgenerator_run (
Optional
[GeneratorRun
]) – generator_run associated with this trial. Trial has only one generator run (and thus arm) attached to it. This can also be set later through add_arm or add_generator_run, but a trial’s associated genetor run is immutable once set.
-
add_arm
(*args, **kwargs)¶
-
add_generator_run
(*args, **kwargs)¶
-
property
arms
¶ All arms attached to this trial.
- Returns
- list of a single arm
attached to this trial if there is one, else None.
- Return type
arms
-
property
arms_by_name
¶ Dictionary of all arms attached to this trial with their names as keys.
- Returns
- dictionary of a single
arm name to arm if one is attached to this trial, else None.
- Return type
arms
-
property
generator_run
¶ Generator run attached to this trial.
- Return type
Core Types¶
-
class
ax.core.types.
ComparisonOp
[source]¶ Bases:
enum.Enum
Class for enumerating comparison operations.
-
GEQ
= 0¶
-
LEQ
= 1¶
-
-
ax.core.types.
TBounds
= typing.Union[typing.Tuple[numpy.ndarray, numpy.ndarray], NoneType]¶
-
ax.core.types.
TBucket
= typing.List[typing.Dict[str, typing.List[str]]]¶
-
ax.core.types.
TConfig
= typing.Dict[str, typing.Union[int, float, str, botorch.acquisition.acquisition.AcquisitionFunction]]¶
-
ax.core.types.
TContextStratum
= typing.Union[typing.Dict[str, typing.Union[str, float, int]], NoneType]¶
-
ax.core.types.
TEvaluationOutcome
= typing.Union[typing.Dict[str, typing.Tuple[float, typing.Union[float, NoneType]]], typing.Tuple[float, typing.Union[float, NoneType]], float, typing.List[typing.Tuple[typing.Dict[str, typing.Union[str, bool, float, int, NoneType]], typing.Dict[str, typing.Tuple[float, typing.Union[float, NoneType]]]]]]¶
-
ax.core.types.
TFidelityTrialEvaluation
= typing.List[typing.Tuple[typing.Dict[str, typing.Union[str, bool, float, int, NoneType]], typing.Dict[str, typing.Tuple[float, typing.Union[float, NoneType]]]]]¶
-
ax.core.types.
TModelCov
= typing.Dict[str, typing.Dict[str, typing.List[float]]]¶
-
ax.core.types.
TModelMean
= typing.Dict[str, typing.List[float]]¶
-
ax.core.types.
TModelPredict
= typing.Tuple[typing.Dict[str, typing.List[float]], typing.Dict[str, typing.Dict[str, typing.List[float]]]]¶
-
ax.core.types.
TModelPredictArm
= typing.Tuple[typing.Dict[str, float], typing.Union[typing.Dict[str, typing.Dict[str, float]], NoneType]]¶
-
ax.core.types.
TParamCounter
= typing.DefaultDict[int, int]¶
-
ax.core.types.
TParamValue
= typing.Union[str, bool, float, int, NoneType]¶
-
ax.core.types.
TParamValueList
= typing.List[typing.Union[str, bool, float, int, NoneType]]¶
-
ax.core.types.
TParameterization
= typing.Dict[str, typing.Union[str, bool, float, int, NoneType]]¶
-
ax.core.types.
TTrialEvaluation
= typing.Dict[str, typing.Tuple[float, typing.Union[float, NoneType]]]¶