ax.plot¶

Base¶

class ax.plot.base.AxPlotConfig[source]¶

Bases: ax.plot.base._AxPlotConfigBase

Config for plots

class ax.plot.base.AxPlotTypes[source]¶

Bases: enum.Enum

Enum of Ax plot types.

BANDIT_ROLLOUT = 4¶

CONTOUR = 0¶

GENERIC = 1¶

INTERACT_CONTOUR = 3¶

SLICE = 2¶

class ax.plot.base.PlotData[source]¶

Bases: tuple

Struct for plot data, including both in-sample and out-of-sample arms

in_sample¶

Alias for field number 1

metrics¶

Alias for field number 0

out_of_sample¶

Alias for field number 2

status_quo_name¶

Alias for field number 3

class ax.plot.base.PlotInSampleArm[source]¶

Bases: tuple

Struct for in-sample arms (both observed and predicted data)

context_stratum¶

Alias for field number 6

name¶

Alias for field number 0

parameters¶

Alias for field number 1

se¶

Alias for field number 4

se_hat¶

Alias for field number 5

y¶

Alias for field number 2

y_hat¶

Alias for field number 3

class ax.plot.base.PlotMetric[source]¶

Bases: tuple

Struct for metric

metric¶

Alias for field number 0

pred¶

Alias for field number 1

class ax.plot.base.PlotOutOfSampleArm[source]¶

Bases: tuple

Struct for out-of-sample arms (only predicted data)

context_stratum¶

Alias for field number 4

name¶

Alias for field number 0

parameters¶

Alias for field number 1

se_hat¶

Alias for field number 3

y_hat¶

Alias for field number 2

Color¶

class ax.plot.color.COLORS[source]¶

Bases: enum.Enum

An enumeration.

CORAL = (251, 128, 114)¶

LIGHT_PURPLE = (190, 186, 218)¶

ORANGE = (253, 180, 98)¶

PINK = (188, 128, 189)¶

STEELBLUE = (128, 177, 211)¶

TEAL = (141, 211, 199)¶

ax.plot.color.plotly_color_scale(list_of_rgb_tuples, reverse=False, alpha=1)[source]¶

Convert list of RGB tuples to list of tuples, where each tuple is break in [0, 1] and stringified RGBA color.

Return type:	List[Tuple[float, str]]

ax.plot.color.rgba(rgb_tuple, alpha=1)[source]¶

Convert RGB tuple to an RGBA string.

Return type:	str

Contour Plot¶

ax.plot.contour.interact_contour(model, metric_name, generator_runs_dict=None, relative=False, density=50, slice_values=None, lower_is_better=False)[source]¶

Create interactive plot with predictions for a 2-d slice of the parameter space.

Parameters:

model (ModelBridge) – ModelBridge that contains model for predictions metric_name (str) – Name of metric to plot generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – A dictionary {name: generator run} of generator runs whose arms will be plotted, if they lie in the slice. relative (bool) – Predictions relative to status quo density (int) – Number of points along slice to evaluate predictions. slice_values (Optional[Dict[str, Any]]) – A dictionary {name: val} for the fixed values of the other parameters. If not provided, then the status quo values will be used if there is a status quo, otherwise the mean of numeric parameters or the mode of choice parameters. lower_is_better (bool) – Lower values for metric are better.

Return type:

AxPlotConfig

ax.plot.contour.plot_contour(model, param_x, param_y, metric_name, generator_runs_dict=None, relative=False, density=50, slice_values=None, lower_is_better=False)[source]¶

Plot predictions for a 2-d slice of the parameter space.

Parameters:

model (ModelBridge) – ModelBridge that contains model for predictions param_x (str) – Name of parameter that will be sliced on x-axis param_y (str) – Name of parameter that will be sliced on y-axis metric_name (str) – Name of metric to plot generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – A dictionary {name: generator run} of generator runs whose arms will be plotted, if they lie in the slice. relative (bool) – Predictions relative to status quo density (int) – Number of points along slice to evaluate predictions. slice_values (Optional[Dict[str, Any]]) – A dictionary {name: val} for the fixed values of the other parameters. If not provided, then the status quo values will be used if there is a status quo, otherwise the mean of numeric parameters or the mode of choice parameters. lower_is_better (bool) – Lower values for metric are better.

Return type:

AxPlotConfig

Model Diagnostic Plot¶

ax.plot.diagnostic.interact_batch_comparison(observations, experiment, batch_x, batch_y, rel=False, status_quo_name=None)[source]¶

Compare repeated arms from two trials; select metric via dropdown.

Parameters:	observations (List[Observation]) – List of observations to compute comparison. batch_x (int) – Index of batch for x-axis. batch_y (int) – Index of bach for y-axis. rel (bool) – Whether to relativize data against status_quo arm. status_quo_name (Optional[str]) – Name of the status_quo arm.
Return type:	AxPlotConfig

ax.plot.diagnostic.interact_cross_validation(cv_results, show_context=True)[source]¶

Interactive cross-validation (CV) plotting; select metric via dropdown.

Note: uses the Plotly version of dropdown (which means that all data is stored within the notebook).

Parameters:	cv_results (List[CVResult]) – cross-validation results. show_context (bool) – if True, show context on hover.
Return type:	AxPlotConfig

ax.plot.diagnostic.interact_empirical_model_validation(batch, data)[source]¶

Compare the model predictions for the batch arms against observed data.

Relies on the model predictions stored on the generator_runs of batch.

Parameters:	batch (BatchTrial) – Batch on which to perform analysis. data (Data) – Observed data for the batch.
Return type:	AxPlotConfig
Returns:	AxPlotConfig for the plot.

ax.plot.diagnostic.tile_cross_validation(cv_results, show_arm_details_on_hover=True, show_context=True)[source]¶

Tile version of CV plots; sorted by ‘best fitting’ outcomes.

Plots are sorted in decreasing order using the p-value of a Fisher exact test statistic.

Parameters:	cv_results (List[CVResult]) – cross-validation results. include_measurement_error – if True, include measurement_error metrics in plot. show_arm_details_on_hover (bool) – if True, display parameterizations of arms on hover. Default is True. show_context (bool) – if True (default), display context on hover.
Return type:	AxPlotConfig

Helpers¶

ax.plot.helper.arm_name_to_tuple(arm_name)[source]¶

Return type:	Union[Tuple[int, int], Tuple[int]]

ax.plot.helper.get_fixed_values(model, slice_values=None)[source]¶

Get fixed values for parameters in a slice plot.

If there is an in-design status quo, those values will be used. Otherwise, the mean of RangeParameters or the mode of ChoiceParameters is used.

Any value in slice_values will override the above.

Parameters:	model (ModelBridge) – ModelBridge being used for plotting slice_values (Optional[Dict[str, Any]]) – Map from parameter name to value at which is should be fixed.

Returns: Map from parameter name to fixed value.

Return type:	Dict[str, Union[str, bool, float, int, None]]

ax.plot.helper.get_grid_for_parameter(parameter, density)[source]¶

Get a grid of points along the range of the parameter.

Will be a log-scale grid if parameter is log scale.

Parameters:	parameter (RangeParameter) – Parameter for which to generate grid. density (int) – Number of points in the grid.
Return type:	ndarray

ax.plot.helper.get_plot_data(model, generator_runs_dict, metric_names=None)[source]¶

Format data object with metrics for in-sample and out-of-sample arms.

Calculate both observed and predicted metrics for in-sample arms. Calculate predicted metrics for out-of-sample arms passed via the generator_runs_dict argument.

In PlotData, in-sample observations are merged with IVW. In RawData, they are left un-merged and given as a list of dictionaries, one for each observation and having keys ‘arm_name’, ‘mean’, and ‘sem’.

Parameters:	model (ModelBridge) – The model. generator_runs_dict (Dict[str, GeneratorRun]) – a mapping from generator run name to generator run. metric_names (Optional[Set[str]]) – Restrict predictions to this set. If None, all metrics in the model will be returned.
Return type:	Tuple[PlotData, List[Dict[str, Union[float, str]]], Dict[str, Dict[str, Union[str, bool, float, int, None]]]]
Returns:	A tuple containing PlotData object with in-sample and out-of-sample predictions. List of observations like: {'metric_name': 'likes', 'arm_name': '0_1', 'mean': 1., 'sem': 0.1}. Mapping from arm name to parameters.

ax.plot.helper.get_range_parameter(model, param_name)[source]¶

Get the range parameter with the given name from the model.

Throws if parameter doesn’t exist or is not a range parameter.

Parameters:	model (ModelBridge) – The model. param_name (str) – The name of the RangeParameter to be found.

Returns: The RangeParameter named param_name.

Return type:	RangeParameter

ax.plot.helper.get_range_parameters(model)[source]¶

Get a list of range parameters from a model.

Parameters:	model (ModelBridge) – The model.

Returns: List of RangeParameters.

Return type:	List[RangeParameter]

ax.plot.helper.resize_subtitles(figure, size)[source]¶

Rendering¶

ax.plot.render.plot_config_to_html(plot_config, plot_module_name='ax.plot', plot_resources={<AxPlotTypes.CONTOUR: 0>: 'contour.js', <AxPlotTypes.GENERIC: 1>: 'generic_plotly.js', <AxPlotTypes.INTERACT_CONTOUR: 3>: 'interact_contour.js', <AxPlotTypes.SLICE: 2>: 'slice.js', <AxPlotTypes.BANDIT_ROLLOUT: 4>: 'bandit_rollout.js'}, inject_helpers=False)[source]¶

Generate HTML + JS corresponding from a plot config.

Return type:	str

Scatter Plots¶

ax.plot.scatter.interact_fitted(model, generator_runs_dict=None, rel=True, show_arm_details_on_hover=True, show_CI=True, arm_noun='arm')[source]¶

Interactive fitted outcome plots for each arm used in fitting the model.

Choose the outcome to plot using a dropdown.

Parameters:

model (ModelBridge) – model to use for predictions. generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – a mapping from generator run name to generator run. rel (bool) – if True, use relative effects. Default is True. show_arm_details_on_hover (bool) – if True, display parameterizations of arms on hover. Default is True. show_CI (bool) – if True, render confidence intervals. arm_noun (str) – noun to use instead of “arm” (e.g. group)

Return type:

AxPlotConfig

ax.plot.scatter.lattice_multiple_metrics(model, generator_runs_dict=None, rel=True, show_arm_details_on_hover=False)[source]¶

Plot raw values or predictions of combinations of two metrics for arms.

Parameters:	model (ModelBridge) – model to draw predictions from. generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – a mapping from generator run name to generator run. rel (bool) – if True, use relative effects. Default is True. show_arm_details_on_hover (bool) – if True, display parameterizations of arms on hover. Default is False.
Return type:	AxPlotConfig

ax.plot.scatter.plot_fitted(model, metric, generator_runs_dict=None, rel=True, custom_arm_order=None, custom_arm_order_name='Custom', show_CI=True)[source]¶

Plot fitted metrics.

Parameters:

model (ModelBridge) – model to use for predictions. metric (str) – metric to plot predictions for. generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – a mapping from generator run name to generator run. rel (bool) – if True, use relative effects. Default is True. custom_arm_order (Optional[List[str]]) – a list of arm names in the order corresponding to how they should be plotted on the x-axis. If not None, this is the default ordering. custom_arm_order_name (str) – name for custom ordering to show in the ordering dropdown. Default is ‘Custom’. show_CI (bool) – if True, render confidence intervals.

Return type:

AxPlotConfig

ax.plot.scatter.plot_multiple_metrics(model, metric_x, metric_y, generator_runs_dict=None, rel=True)[source]¶

Plot raw values or predictions of two metrics for arms.

All arms used in the model are included in the plot. Additional arms can be passed through the generator_runs_dict argument.

Parameters:	model (ModelBridge) – model to draw predictions from. metric_x (str) – metric to plot on the x-axis. metric_y (str) – metric to plot on the y-axis. generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – a mapping from generator run name to generator run. rel (bool) – if True, use relative effects. Default is True.
Return type:	AxPlotConfig

ax.plot.scatter.plot_objective_vs_constraints(model, objective, subset_metrics=None, generator_runs_dict=None, rel=True)[source]¶

Plot the tradeoff between an objetive and all other metrics in a model.

All arms used in the model are included in the plot. Additional arms can be passed through via the generator_runs_dict argument.

Parameters:

model (ModelBridge) – model to draw predictions from. objective (str) – metric to optimize. Plotted on the x-axis. subset_metrics (Optional[List[str]]) – list of metrics to plot on the y-axes if need a subset of all metrics in the model. generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – a mapping from generator run name to generator run. rel (bool) – if True, use relative effects. Default is True.

Return type:

AxPlotConfig

ax.plot.scatter.tile_fitted(model, generator_runs_dict=None, rel=True, show_arm_details_on_hover=False, show_CI=True, arm_noun='arm')[source]¶

Tile version of fitted outcome plots.

Parameters:

model (ModelBridge) – model to use for predictions. generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – a mapping from generator run name to generator run. rel (bool) – if True, use relative effects. Default is True. show_arm_details_on_hover (bool) – if True, display parameterizations of arms on hover. Default is False. show_CI (bool) – if True, render confidence intervals. arm_noun (str) – noun to use instead of “arm” (e.g. group)

Return type:

AxPlotConfig

Slice Plot¶

ax.plot.slice.plot_slice(model, param_name, metric_name, generator_runs_dict=None, relative=False, density=50, slice_values=None)[source]¶

Plot predictions for a 1-d slice of the parameter space.

Parameters:

model (ModelBridge) – ModelBridge that contains model for predictions param_name (str) – Name of parameter that will be sliced metric_name (str) – Name of metric to plot generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – A dictionary {name: generator run} of generator runs whose arms will be plotted, if they lie in the slice. relative (bool) – Predictions relative to status quo density (int) – Number of points along slice to evaluate predictions. slice_values (Optional[Dict[str, Any]]) – A dictionary {name: val} for the fixed values of the other parameters. If not provided, then the status quo values will be used if there is a status quo, otherwise the mean of numeric parameters or the mode of choice parameters.

Return type:

AxPlotConfig