ax.plot¶
Plots¶
Base¶
-
class
ax.plot.base.
AxPlotTypes
[source]¶ Bases:
enum.Enum
Enum of Ax plot types.
-
BANDIT_ROLLOUT
= 4¶
-
CONTOUR
= 0¶
-
GENERIC
= 1¶
-
INTERACT_CONTOUR
= 3¶
-
INTERACT_SLICE
= 5¶
-
SLICE
= 2¶
-
-
ax.plot.base.
CI_OPACITY
= 0.4¶
-
ax.plot.base.
DECIMALS
= 3¶
-
class
ax.plot.base.
PlotData
[source]¶ Bases:
tuple
Struct for plot data, including both in-sample and out-of-sample arms
-
property
in_sample
¶ Alias for field number 1
-
property
metrics
¶ Alias for field number 0
-
property
out_of_sample
¶ Alias for field number 2
-
property
status_quo_name
¶ Alias for field number 3
-
property
-
class
ax.plot.base.
PlotInSampleArm
[source]¶ Bases:
tuple
Struct for in-sample arms (both observed and predicted data)
-
property
context_stratum
¶ Alias for field number 6
-
property
name
¶ Alias for field number 0
-
property
parameters
¶ Alias for field number 1
-
property
se
¶ Alias for field number 4
-
property
se_hat
¶ Alias for field number 5
-
property
y
¶ Alias for field number 2
-
property
y_hat
¶ Alias for field number 3
-
property
-
class
ax.plot.base.
PlotMetric
[source]¶ Bases:
tuple
Struct for metric
-
property
metric
¶ Alias for field number 0
-
property
pred
¶ Alias for field number 1
-
property
-
class
ax.plot.base.
PlotOutOfSampleArm
[source]¶ Bases:
tuple
Struct for out-of-sample arms (only predicted data)
-
property
context_stratum
¶ Alias for field number 4
-
property
name
¶ Alias for field number 0
-
property
parameters
¶ Alias for field number 1
-
property
se_hat
¶ Alias for field number 3
-
property
y_hat
¶ Alias for field number 2
-
property
-
ax.plot.base.
Z
= 1.96¶
Color¶
-
ax.plot.color.
BLUE_SCALE
= [(255, 247, 251), (236, 231, 242), (208, 209, 230), (166, 189, 219), (116, 169, 207), (54, 144, 192), (5, 112, 176), (3, 78, 123)]¶
-
class
ax.plot.color.
COLORS
[source]¶ Bases:
enum.Enum
An enumeration.
-
CORAL
= (251, 128, 114)¶
-
LIGHT_PURPLE
= (190, 186, 218)¶
-
ORANGE
= (253, 180, 98)¶
-
PINK
= (188, 128, 189)¶
-
STEELBLUE
= (128, 177, 211)¶
-
TEAL
= (141, 211, 199)¶
-
-
ax.plot.color.
DISCRETE_COLOR_SCALE
= [(128, 177, 211), (251, 128, 114), (141, 211, 199), (188, 128, 189), (190, 186, 218), (253, 180, 98)]¶
-
ax.plot.color.
GREEN_PINK_SCALE
= [(142, 1, 82), (197, 27, 125), (222, 119, 174), (241, 182, 218), (253, 224, 239), (247, 247, 247), (230, 245, 208), (184, 225, 134), (127, 188, 65), (77, 146, 33), (39, 100, 25)]¶
-
ax.plot.color.
GREEN_SCALE
= [(247, 252, 253), (229, 245, 249), (204, 236, 230), (153, 216, 201), (102, 194, 164), (65, 174, 118), (35, 139, 69), (0, 109, 44), (0, 68, 27)]¶
-
ax.plot.color.
MIXED_SCALE
= [(2, 63, 165), (125, 135, 185), (190, 193, 212), (214, 188, 192), (187, 119, 132), (142, 6, 59), (74, 111, 227), (133, 149, 225), (181, 187, 227), (230, 175, 185), (224, 123, 145), (211, 63, 106), (17, 198, 56), (141, 213, 147), (198, 222, 199), (234, 211, 198), (240, 185, 141), (239, 151, 8), (15, 207, 192), (156, 222, 214), (213, 234, 231), (243, 225, 235), (246, 196, 225), (247, 156, 212)]¶
Contour Plot¶
-
ax.plot.contour.
ContourPredictions
= typing.Tuple[ax.plot.base.PlotData, numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray, typing.Dict[str, bool]]¶
-
ax.plot.contour.
MAX_PARAM_LENGTH
= 40¶
-
ax.plot.contour.
interact_contour
(model, metric_name, generator_runs_dict=None, relative=False, density=50, slice_values=None, lower_is_better=False, fixed_features=None)[source]¶ Create interactive plot with predictions for a 2-d slice of the parameter space.
- Parameters
model (
ModelBridge
) – ModelBridge that contains model for predictionsmetric_name (
str
) – Name of metric to plotgenerator_runs_dict (
Optional
[Dict
[str
,GeneratorRun
]]) – A dictionary {name: generator run} of generator runs whose arms will be plotted, if they lie in the slice.relative (
bool
) – Predictions relative to status quodensity (
int
) – Number of points along slice to evaluate predictions.slice_values (
Optional
[Dict
[str
,Any
]]) – A dictionary {name: val} for the fixed values of the other parameters. If not provided, then the status quo values will be used if there is a status quo, otherwise the mean of numeric parameters or the mode of choice parameters.lower_is_better (
bool
) – Lower values for metric are better.fixed_features (
Optional
[ObservationFeatures
]) – An ObservationFeatures object containing the values of features (including non-parameter features like context) to be set in the slice.
- Return type
-
ax.plot.contour.
plot_contour
(model, param_x, param_y, metric_name, generator_runs_dict=None, relative=False, density=50, slice_values=None, lower_is_better=False, fixed_features=None)[source]¶ Plot predictions for a 2-d slice of the parameter space.
- Parameters
model (
ModelBridge
) – ModelBridge that contains model for predictionsparam_x (
str
) – Name of parameter that will be sliced on x-axisparam_y (
str
) – Name of parameter that will be sliced on y-axismetric_name (
str
) – Name of metric to plotgenerator_runs_dict (
Optional
[Dict
[str
,GeneratorRun
]]) – A dictionary {name: generator run} of generator runs whose arms will be plotted, if they lie in the slice.relative (
bool
) – Predictions relative to status quodensity (
int
) – Number of points along slice to evaluate predictions.slice_values (
Optional
[Dict
[str
,Any
]]) – A dictionary {name: val} for the fixed values of the other parameters. If not provided, then the status quo values will be used if there is a status quo, otherwise the mean of numeric parameters or the mode of choice parameters.lower_is_better (
bool
) – Lower values for metric are better.fixed_features (
Optional
[ObservationFeatures
]) – An ObservationFeatures object containing the values of features (including non-parameter features like context) to be set in the slice.
- Return type
Model Diagnostic Plot¶
-
ax.plot.diagnostic.
FloatList
= typing.List[float]¶
-
ax.plot.diagnostic.
interact_batch_comparison
(observations, experiment, batch_x, batch_y, rel=False, status_quo_name=None)[source]¶ Compare repeated arms from two trials; select metric via dropdown.
- Parameters
- Return type
-
ax.plot.diagnostic.
interact_cross_validation
(cv_results, show_context=True)[source]¶ Interactive cross-validation (CV) plotting; select metric via dropdown.
Note: uses the Plotly version of dropdown (which means that all data is stored within the notebook).
- Parameters
- Return type
-
ax.plot.diagnostic.
interact_empirical_model_validation
(batch, data)[source]¶ Compare the model predictions for the batch arms against observed data.
Relies on the model predictions stored on the generator_runs of batch.
- Parameters
batch (
BatchTrial
) – Batch on which to perform analysis.data (
Data
) – Observed data for the batch.
- Return type
- Returns
AxPlotConfig for the plot.
-
ax.plot.diagnostic.
tile_cross_validation
(cv_results, show_arm_details_on_hover=True, show_context=True)[source]¶ Tile version of CV plots; sorted by ‘best fitting’ outcomes.
Plots are sorted in decreasing order using the p-value of a Fisher exact test statistic.
Helpers¶
-
ax.plot.helper.
RawData
= typing.List[typing.Dict[str, typing.Union[float, str]]]¶
-
ax.plot.helper.
TNullableGeneratorRunsDict
= typing.Union[typing.Dict[str, ax.core.generator_run.GeneratorRun], NoneType]¶
-
ax.plot.helper.
get_fixed_values
(model, slice_values=None)[source]¶ Get fixed values for parameters in a slice plot.
If there is an in-design status quo, those values will be used. Otherwise, the mean of RangeParameters or the mode of ChoiceParameters is used.
Any value in slice_values will override the above.
- Parameters
model (
ModelBridge
) – ModelBridge being used for plottingslice_values (
Optional
[Dict
[str
,Any
]]) – Map from parameter name to value at which is should be fixed.
Returns: Map from parameter name to fixed value.
-
ax.plot.helper.
get_grid_for_parameter
(parameter, density)[source]¶ Get a grid of points along the range of the parameter.
Will be a log-scale grid if parameter is log scale.
- Parameters
parameter (
RangeParameter
) – Parameter for which to generate grid.density (
int
) – Number of points in the grid.
- Return type
ndarray
-
ax.plot.helper.
get_plot_data
(model, generator_runs_dict, metric_names=None, fixed_features=None)[source]¶ Format data object with metrics for in-sample and out-of-sample arms.
Calculate both observed and predicted metrics for in-sample arms. Calculate predicted metrics for out-of-sample arms passed via the generator_runs_dict argument.
In PlotData, in-sample observations are merged with IVW. In RawData, they are left un-merged and given as a list of dictionaries, one for each observation and having keys ‘arm_name’, ‘mean’, and ‘sem’.
- Parameters
model (
ModelBridge
) – The model.generator_runs_dict (
Dict
[str
,GeneratorRun
]) – a mapping from generator run name to generator run.metric_names (
Optional
[Set
[str
]]) – Restrict predictions to this set. If None, all metrics in the model will be returned.fixed_features (
Optional
[ObservationFeatures
]) – Fixed features to use when making model predictions.
- Return type
Tuple
[PlotData
,List
[Dict
[str
,Union
[float
,str
]]],Dict
[str
,Dict
[str
,Union
[str
,bool
,float
,int
,None
]]]]- Returns
A tuple containing
PlotData object with in-sample and out-of-sample predictions.
List of observations like:
{'metric_name': 'likes', 'arm_name': '0_1', 'mean': 1., 'sem': 0.1}.
Mapping from arm name to parameters.
-
ax.plot.helper.
get_range_parameter
(model, param_name)[source]¶ Get the range parameter with the given name from the model.
Throws if parameter doesn’t exist or is not a range parameter.
- Parameters
model (
ModelBridge
) – The model.param_name (
str
) – The name of the RangeParameter to be found.
Returns: The RangeParameter named param_name.
- Return type
-
ax.plot.helper.
get_range_parameters
(model)[source]¶ Get a list of range parameters from a model.
- Parameters
model (
ModelBridge
) – The model.
Returns: List of RangeParameters.
- Return type
-
ax.plot.helper.
logger
= <Logger PlotHelper (INFO)>¶
Rendering¶
-
ax.plot.render.
Ax_PLOT_REGISTRY
= {<AxPlotTypes.GENERIC: 1>: 'generic_plotly.js'}¶
-
ax.plot.render.
CSS_FILE
= 'ax/ax/plot/css/base.css'¶
-
ax.plot.render.
DEFAULT_HEIGHT
= 550¶
-
ax.plot.render.
DEFAULT_WIDTH
= '100%'¶
Scatter Plots¶
-
ax.plot.scatter.
Traces
= typing.List[typing.Dict[str, typing.Any]]¶
-
ax.plot.scatter.
interact_fitted
(model, generator_runs_dict=None, rel=True, show_arm_details_on_hover=True, show_CI=True, arm_noun='arm', metrics=None, fixed_features=None)[source]¶ Interactive fitted outcome plots for each arm used in fitting the model.
Choose the outcome to plot using a dropdown.
- Parameters
model (
ModelBridge
) – model to use for predictions.generator_runs_dict (
Optional
[Dict
[str
,GeneratorRun
]]) – a mapping from generator run name to generator run.rel (
bool
) – if True, use relative effects. Default is True.show_arm_details_on_hover (
bool
) – if True, display parameterizations of arms on hover. Default is True.show_CI (
bool
) – if True, render confidence intervals.arm_noun (
str
) – noun to use instead of “arm” (e.g. group)metrics (
Optional
[List
[str
]]) – List of metric names to restrict to when plotting.fixed_features (
Optional
[ObservationFeatures
]) – Fixed features to use when making model predictions.
- Return type
-
ax.plot.scatter.
lattice_multiple_metrics
(model, generator_runs_dict=None, rel=True, show_arm_details_on_hover=False)[source]¶ Plot raw values or predictions of combinations of two metrics for arms.
- Parameters
model (
ModelBridge
) – model to draw predictions from.generator_runs_dict (
Optional
[Dict
[str
,GeneratorRun
]]) – a mapping from generator run name to generator run.rel (
bool
) – if True, use relative effects. Default is True.show_arm_details_on_hover (
bool
) – if True, display parameterizations of arms on hover. Default is False.
- Return type
-
ax.plot.scatter.
plot_fitted
(model, metric, generator_runs_dict=None, rel=True, custom_arm_order=None, custom_arm_order_name='Custom', show_CI=True)[source]¶ Plot fitted metrics.
- Parameters
model (
ModelBridge
) – model to use for predictions.metric (
str
) – metric to plot predictions for.generator_runs_dict (
Optional
[Dict
[str
,GeneratorRun
]]) – a mapping from generator run name to generator run.rel (
bool
) – if True, use relative effects. Default is True.custom_arm_order (
Optional
[List
[str
]]) – a list of arm names in the order corresponding to how they should be plotted on the x-axis. If not None, this is the default ordering.custom_arm_order_name (
str
) – name for custom ordering to show in the ordering dropdown. Default is ‘Custom’.show_CI (
bool
) – if True, render confidence intervals.
- Return type
-
ax.plot.scatter.
plot_multiple_metrics
(model, metric_x, metric_y, generator_runs_dict=None, rel=True)[source]¶ Plot raw values or predictions of two metrics for arms.
All arms used in the model are included in the plot. Additional arms can be passed through the generator_runs_dict argument.
- Parameters
model (
ModelBridge
) – model to draw predictions from.metric_x (
str
) – metric to plot on the x-axis.metric_y (
str
) – metric to plot on the y-axis.generator_runs_dict (
Optional
[Dict
[str
,GeneratorRun
]]) – a mapping from generator run name to generator run.rel (
bool
) – if True, use relative effects. Default is True.
- Return type
-
ax.plot.scatter.
plot_objective_vs_constraints
(model, objective, subset_metrics=None, generator_runs_dict=None, rel=True, fixed_features=None)[source]¶ Plot the tradeoff between an objetive and all other metrics in a model.
All arms used in the model are included in the plot. Additional arms can be passed through via the generator_runs_dict argument.
Fixed features input can be used to override fields of the insample arms when making model predictions.
- Parameters
model (
ModelBridge
) – model to draw predictions from.objective (
str
) – metric to optimize. Plotted on the x-axis.subset_metrics (
Optional
[List
[str
]]) – list of metrics to plot on the y-axes if need a subset of all metrics in the model.generator_runs_dict (
Optional
[Dict
[str
,GeneratorRun
]]) – a mapping from generator run name to generator run.rel (
bool
) – if True, use relative effects. Default is True.fixed_features (
Optional
[ObservationFeatures
]) – Fixed features to use when making model predictions.
- Return type
-
ax.plot.scatter.
tile_fitted
(model, generator_runs_dict=None, rel=True, show_arm_details_on_hover=False, show_CI=True, arm_noun='arm', metrics=None, fixed_features=None)[source]¶ Tile version of fitted outcome plots.
- Parameters
model (
ModelBridge
) – model to use for predictions.generator_runs_dict (
Optional
[Dict
[str
,GeneratorRun
]]) – a mapping from generator run name to generator run.rel (
bool
) – if True, use relative effects. Default is True.show_arm_details_on_hover (
bool
) – if True, display parameterizations of arms on hover. Default is False.show_CI (
bool
) – if True, render confidence intervals.arm_noun (
str
) – noun to use instead of “arm” (e.g. group)metrics (
Optional
[List
[str
]]) – List of metric names to restrict to when plotting.fixed_features (
Optional
[ObservationFeatures
]) – Fixed features to use when making model predictions.
- Return type
Slice Plot¶
-
ax.plot.slice.
SlicePredictions
= typing.Tuple[ax.plot.base.PlotData, typing.List[typing.Dict[str, typing.Union[float, str]]], typing.List[float], numpy.ndarray, numpy.ndarray, str, str, bool, typing.Dict[str, typing.Union[str, bool, float, int, NoneType]], numpy.ndarray, bool]¶
-
ax.plot.slice.
interact_slice
(model, param_name, metric_name='', generator_runs_dict=None, relative=False, density=50, slice_values=None, fixed_features=None)[source]¶ Create interactive plot with predictions for a 1-d slice of the parameter space.
- Parameters
model (
ModelBridge
) – ModelBridge that contains model for predictionsparam_name (
str
) – Name of parameter that will be slicedmetric_name (
str
) – Name of metric to plotgenerator_runs_dict (
Optional
[Dict
[str
,GeneratorRun
]]) – A dictionary {name: generator run} of generator runs whose arms will be plotted, if they lie in the slice.relative (
bool
) – Predictions relative to status quodensity (
int
) – Number of points along slice to evaluate predictions.slice_values (
Optional
[Dict
[str
,Any
]]) – A dictionary {name: val} for the fixed values of the other parameters. If not provided, then the status quo values will be used if there is a status quo, otherwise the mean of numeric parameters or the mode of choice parameters. Ignored if fixed_features is specified.fixed_features (
Optional
[ObservationFeatures
]) – An ObservationFeatures object containing the values of features (including non-parameter features like context) to be set in the slice.
- Return type
-
ax.plot.slice.
plot_slice
(model, param_name, metric_name, generator_runs_dict=None, relative=False, density=50, slice_values=None, fixed_features=None)[source]¶ Plot predictions for a 1-d slice of the parameter space.
- Parameters
model (
ModelBridge
) – ModelBridge that contains model for predictionsparam_name (
str
) – Name of parameter that will be slicedmetric_name (
str
) – Name of metric to plotgenerator_runs_dict (
Optional
[Dict
[str
,GeneratorRun
]]) – A dictionary {name: generator run} of generator runs whose arms will be plotted, if they lie in the slice.relative (
bool
) – Predictions relative to status quodensity (
int
) – Number of points along slice to evaluate predictions.slice_values (
Optional
[Dict
[str
,Any
]]) – A dictionary {name: val} for the fixed values of the other parameters. If not provided, then the status quo values will be used if there is a status quo, otherwise the mean of numeric parameters or the mode of choice parameters. Ignored if fixed_features is specified.fixed_features (
Optional
[ObservationFeatures
]) – An ObservationFeatures object containing the values of features (including non-parameter features like context) to be set in the slice.
- Return type
Trace Plots¶
-
ax.plot.trace.
Traces
= typing.List[typing.Dict[str, typing.Any]]¶
-
ax.plot.trace.
generator_changes_scatter
(generator_changes, y_range, generator_change_color=(141, 211, 199))[source]¶ Creates a graph object for the line(s) representing generator changes.
- Parameters
- Returns
- plotly graph objects for the lines representing generator
changes
- Return type
go.Scatter
-
ax.plot.trace.
mean_trace_scatter
(y, trace_color=(128, 177, 211), legend_label='mean')[source]¶ Creates a graph object for trace of the mean of the given series across runs.
-
ax.plot.trace.
optimization_times
(fit_times, gen_times, title='')[source]¶ Plots wall times for each method as a bar chart.
- Parameters
Returns: AxPlotConfig with the plot
- Return type
-
ax.plot.trace.
optimization_trace_all_methods
(y_dict, optimum=None, title='', ylabel='', trace_colors=[(128, 177, 211), (251, 128, 114), (141, 211, 199), (188, 128, 189), (190, 186, 218), (253, 180, 98)], optimum_color=(253, 180, 98))[source]¶ Plots a comparison of optimization traces with 2-SEM bands for multiple methods on the same problem.
- Parameters
y – a mapping of method names to (r x t) arrays, where r is the number of runs in the test, and t is the number of trials.
title (
str
) – Title for this plot.ylabel (
str
) – Label for y axistrace_colors (
List
[Tuple
[int
]]) – tuples of 3 int values representing RGB colors to use for different methods shown in the combination plot. Defaults to Ax discrete color scale.optimum_color (
Tuple
[int
]) – tuple of 3 int values representing an RGB color. Defaults to orange.
- Returns
plot of the comparison of optimization traces with IQR
- Return type
-
ax.plot.trace.
optimization_trace_single_method
(y, optimum=None, generator_changes=None, title='', ylabel='', trace_color=(128, 177, 211), optimum_color=(253, 180, 98), generator_change_color=(141, 211, 199))[source]¶ Plots an optimization trace with mean and 2 SEMs
- Parameters
y (
ndarray
) – (r x t) array; result to plot, with r runs and t trialsgenerator_changes (
Optional
[List
[int
]]) – iterations, before which generators changedtitle (
str
) – title of this plotylabel (
str
) – Label for y axistrace_color (
Tuple
[int
]) – tuple of 3 int values representing an RGB color. Defaults to orange.optimum_color (
Tuple
[int
]) – tuple of 3 int values representing an RGB color. Defaults to orange.generator_change_color (
Tuple
[int
]) – tuple of 3 int values representing an RGB color. Defaults to orange.
- Returns
plot of the optimization trace with IQR
- Return type
-
ax.plot.trace.
optimum_objective_scatter
(optimum, num_iterations, optimum_color=(253, 180, 98))[source]¶ Creates a graph object for the line representing optimal objective.
- Parameters
- Returns
plotly graph objects for the optimal objective line
- Return type
go.Scatter