{
"cells": [
{
"cell_type": "markdown",
"id": "f23d463f",
"metadata": {
"code_folding": [],
"customInput": null,
"hidden_ranges": [],
"originalKey": "95e7a97a-bf78-48d4-a0c1-c0e8dfc4fed9",
"papermill": {
"duration": 0.009263,
"end_time": "2024-07-23T19:54:29.983535",
"exception": false,
"start_time": "2024-07-23T19:54:29.974272",
"status": "completed"
},
"showInput": true,
"tags": []
},
"source": [
"# Multi-Objective Optimization Ax API\n",
"### Using the Service API\n",
"For Multi-objective optimization (MOO) in the `AxClient`, objectives are specified through the `ObjectiveProperties` dataclass. An `ObjectiveProperties` requires a boolean `minimize`, and also accepts an optional floating point `threshold`. If a `threshold` is not specified, Ax will infer it through the use of heuristics. If the user knows the region of interest (because they have specs or prior knowledge), then specifying the thresholds is preferable to inferring it. But if the user would need to guess, inferring is preferable.\n",
"\n",
"\n",
"To learn more about how to choose a threshold, see [Set Objective Thresholds to focus candidate generation in a region of interest](#Set-Objective-Thresholds-to-focus-candidate-generation-in-a-region-of-interest). See the [Service API Tutorial](/tutorials/gpei_hartmann_service.html) for more infomation on running experiments with the Service API."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "d6048efd",
"metadata": {
"code_folding": [],
"customInput": null,
"execution": {
"iopub.execute_input": "2024-07-23T19:54:30.002412Z",
"iopub.status.busy": "2024-07-23T19:54:30.001960Z",
"iopub.status.idle": "2024-07-23T19:54:33.610729Z",
"shell.execute_reply": "2024-07-23T19:54:33.609973Z"
},
"hidden_ranges": [],
"originalKey": "06bf2029-0ea4-40b4-aced-956f1411cb6e",
"papermill": {
"duration": 3.632948,
"end_time": "2024-07-23T19:54:33.625260",
"exception": false,
"start_time": "2024-07-23T19:54:29.992312",
"status": "completed"
},
"showInput": true,
"tags": []
},
"outputs": [
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.utils.notebook.plotting: Injecting Plotly library into cell. Do not overwrite or delete cell.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.utils.notebook.plotting: Please see\n",
" (https://ax.dev/tutorials/visualizations.html#Fix-for-plots-that-are-not-rendering)\n",
" if visualizations are not rendering.\n"
]
},
{
"data": {
"text/html": [
" \n",
" "
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import torch\n",
"from ax.plot.pareto_frontier import plot_pareto_frontier\n",
"from ax.plot.pareto_utils import compute_posterior_pareto_frontier\n",
"from ax.service.ax_client import AxClient\n",
"from ax.service.utils.instantiation import ObjectiveProperties\n",
"\n",
"# Plotting imports and initialization\n",
"from ax.utils.notebook.plotting import init_notebook_plotting, render\n",
"from botorch.test_functions.multi_objective import BraninCurrin\n",
"\n",
"init_notebook_plotting()"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "5caad1f0",
"metadata": {
"execution": {
"iopub.execute_input": "2024-07-23T19:54:33.702958Z",
"iopub.status.busy": "2024-07-23T19:54:33.702470Z",
"iopub.status.idle": "2024-07-23T19:54:33.706715Z",
"shell.execute_reply": "2024-07-23T19:54:33.706165Z"
},
"papermill": {
"duration": 0.044777,
"end_time": "2024-07-23T19:54:33.708009",
"exception": false,
"start_time": "2024-07-23T19:54:33.663232",
"status": "completed"
},
"tags": []
},
"outputs": [],
"source": [
"# Load our sample 2-objective problem\n",
"branin_currin = BraninCurrin(negate=True).to(\n",
" dtype=torch.double,\n",
" device=torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\"),\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "247c5843",
"metadata": {
"code_folding": [],
"customInput": null,
"execution": {
"iopub.execute_input": "2024-07-23T19:54:33.788697Z",
"iopub.status.busy": "2024-07-23T19:54:33.788185Z",
"iopub.status.idle": "2024-07-23T19:54:33.801902Z",
"shell.execute_reply": "2024-07-23T19:54:33.801233Z"
},
"executionStartTime": 1628191188673,
"executionStopTime": 1628191188746,
"hidden_ranges": [],
"originalKey": "c687973d-1b09-4a8f-9108-1f74adf64d4d",
"papermill": {
"duration": 0.057077,
"end_time": "2024-07-23T19:54:33.803491",
"exception": false,
"start_time": "2024-07-23T19:54:33.746414",
"status": "completed"
},
"requestMsgId": "ea523260-8896-48e4-a62f-3530d268b209",
"showInput": true,
"tags": []
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.service.ax_client: Starting optimization with verbose logging. To disable logging, set the `verbose_logging` argument to `False`. Note that float values in the logs are rounded to 6 decimal points.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x1. If that is not the expected value type, you can explicitly specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x2. If that is not the expected value type, you can explicitly specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.service.utils.instantiation: Created search space: SearchSpace(parameters=[RangeParameter(name='x1', parameter_type=FLOAT, range=[0.0, 1.0]), RangeParameter(name='x2', parameter_type=FLOAT, range=[0.0, 1.0])], parameter_constraints=[]).\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.core.experiment: The is_test flag has been set to True. This flag is meant purely for development and integration testing purposes. If you are running a live experiment, please set this flag to False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.modelbridge.dispatch_utils: Using Models.BOTORCH_MODULAR since there is at least one ordered parameter and there are no unordered categorical parameters.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.modelbridge.dispatch_utils: Calculating the number of remaining initialization trials based on num_initialization_trials=None max_initialization_trials=None num_tunable_parameters=2 num_trials=None use_batch_trials=False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.modelbridge.dispatch_utils: calculated num_initialization_trials=5\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.modelbridge.dispatch_utils: num_completed_initialization_trials=0 num_remaining_initialization_trials=5\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.modelbridge.dispatch_utils: `verbose`, `disable_progbar`, and `jit_compile` are not yet supported when using `choose_generation_strategy` with ModularBoTorchModel, dropping these arguments.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:33] ax.modelbridge.dispatch_utils: Using Bayesian Optimization generation strategy: GenerationStrategy(name='Sobol+BoTorch', steps=[Sobol for 5 trials, BoTorch for subsequent trials]). Iterations after 5 will take longer to generate due to model-fitting.\n"
]
}
],
"source": [
"ax_client = AxClient()\n",
"ax_client.create_experiment(\n",
" name=\"moo_experiment\",\n",
" parameters=[\n",
" {\n",
" \"name\": f\"x{i+1}\",\n",
" \"type\": \"range\",\n",
" \"bounds\": [0.0, 1.0],\n",
" }\n",
" for i in range(2)\n",
" ],\n",
" objectives={\n",
" # `threshold` arguments are optional\n",
" \"a\": ObjectiveProperties(minimize=False, threshold=branin_currin.ref_point[0]),\n",
" \"b\": ObjectiveProperties(minimize=False, threshold=branin_currin.ref_point[1]),\n",
" },\n",
" overwrite_existing_experiment=True,\n",
" is_test=True,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "4426adbb",
"metadata": {
"code_folding": [],
"customInput": null,
"hidden_ranges": [],
"originalKey": "70fd45e1-a2ce-4034-bb44-086507833472",
"papermill": {
"duration": 0.039056,
"end_time": "2024-07-23T19:54:33.881615",
"exception": false,
"start_time": "2024-07-23T19:54:33.842559",
"status": "completed"
},
"showInput": true,
"tags": []
},
"source": [
"### Create an Evaluation Function\n",
"In the case of MOO experiments, evaluation functions can be any arbitrary function that takes in a `dict` of parameter names mapped to values and returns a `dict` of objective names mapped to a `tuple` of mean and SEM values."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "b174e1ab",
"metadata": {
"code_folding": [],
"customInput": null,
"execution": {
"iopub.execute_input": "2024-07-23T19:54:33.961671Z",
"iopub.status.busy": "2024-07-23T19:54:33.961094Z",
"iopub.status.idle": "2024-07-23T19:54:33.965177Z",
"shell.execute_reply": "2024-07-23T19:54:33.964609Z"
},
"executionStartTime": 1628191201840,
"executionStopTime": 1628191201871,
"hidden_ranges": [],
"originalKey": "a0e4fa8d-ebc7-4dc6-b370-ed4a83e3208f",
"papermill": {
"duration": 0.045651,
"end_time": "2024-07-23T19:54:33.966417",
"exception": false,
"start_time": "2024-07-23T19:54:33.920766",
"status": "completed"
},
"requestMsgId": "9cfd336d-c317-4d1c-a028-42d45903bac6",
"showInput": true,
"tags": []
},
"outputs": [],
"source": [
"def evaluate(parameters):\n",
" evaluation = branin_currin(\n",
" torch.tensor([parameters.get(\"x1\"), parameters.get(\"x2\")])\n",
" )\n",
" # In our case, standard error is 0, since we are computing a synthetic function.\n",
" # Set standard error to None if the noise level is unknown.\n",
" return {\"a\": (evaluation[0].item(), 0.0), \"b\": (evaluation[1].item(), 0.0)}"
]
},
{
"cell_type": "markdown",
"id": "09ac0857",
"metadata": {
"code_folding": [],
"customInput": null,
"hidden_ranges": [],
"originalKey": "4200cd7c-8e13-4cbf-b0c1-72b52d900aaf",
"papermill": {
"duration": 0.038783,
"end_time": "2024-07-23T19:54:34.044323",
"exception": false,
"start_time": "2024-07-23T19:54:34.005540",
"status": "completed"
},
"showInput": true,
"tags": []
},
"source": [
"### Run Optimization"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "70a54cb9",
"metadata": {
"customInput": null,
"execution": {
"iopub.execute_input": "2024-07-23T19:54:34.123376Z",
"iopub.status.busy": "2024-07-23T19:54:34.122874Z",
"iopub.status.idle": "2024-07-23T19:55:20.068995Z",
"shell.execute_reply": "2024-07-23T19:55:20.068410Z"
},
"executionStartTime": 1628191208271,
"executionStopTime": 1628191238749,
"originalKey": "f91b1a1e-c78a-4262-a211-a13115c007c1",
"papermill": {
"duration": 45.987263,
"end_time": "2024-07-23T19:55:20.070430",
"exception": false,
"start_time": "2024-07-23T19:54:34.083167",
"status": "completed"
},
"requestMsgId": "842a1cf8-97a3-43d6-83a3-f258ea96ae20",
"showInput": true,
"tags": []
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/tmp.DL1QmpHQMI/Ax-main/ax/modelbridge/cross_validation.py:462: UserWarning:\n",
"\n",
"Encountered exception in computing model fit quality: RandomModelBridge does not support prediction.\n",
"\n",
"[INFO 07-23 19:54:34] ax.service.ax_client: Generated new trial 0 with parameters {'x1': 0.915697, 'x2': 0.299481} using model Sobol.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:34] ax.service.ax_client: Completed trial 0 with data: {'a': (-9.028192, 0.0), 'b': (-8.33193, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/tmp.DL1QmpHQMI/Ax-main/ax/modelbridge/cross_validation.py:462: UserWarning:\n",
"\n",
"Encountered exception in computing model fit quality: RandomModelBridge does not support prediction.\n",
"\n",
"[INFO 07-23 19:54:34] ax.service.ax_client: Generated new trial 1 with parameters {'x1': 0.580851, 'x2': 0.149961} using model Sobol.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:34] ax.service.ax_client: Completed trial 1 with data: {'a': (-2.064706, 0.0), 'b': (-10.83757, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/tmp.DL1QmpHQMI/Ax-main/ax/modelbridge/cross_validation.py:462: UserWarning:\n",
"\n",
"Encountered exception in computing model fit quality: RandomModelBridge does not support prediction.\n",
"\n",
"[INFO 07-23 19:54:34] ax.service.ax_client: Generated new trial 2 with parameters {'x1': 0.838598, 'x2': 0.164636} using model Sobol.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:34] ax.service.ax_client: Completed trial 2 with data: {'a': (-13.842742, 0.0), 'b': (-9.887474, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/tmp.DL1QmpHQMI/Ax-main/ax/modelbridge/cross_validation.py:462: UserWarning:\n",
"\n",
"Encountered exception in computing model fit quality: RandomModelBridge does not support prediction.\n",
"\n",
"[INFO 07-23 19:54:34] ax.service.ax_client: Generated new trial 3 with parameters {'x1': 0.716842, 'x2': 0.932837} using model Sobol.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:34] ax.service.ax_client: Completed trial 3 with data: {'a': (-183.998077, 0.0), 'b': (-4.43332, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/tmp.DL1QmpHQMI/Ax-main/ax/modelbridge/cross_validation.py:462: UserWarning:\n",
"\n",
"Encountered exception in computing model fit quality: RandomModelBridge does not support prediction.\n",
"\n",
"[INFO 07-23 19:54:34] ax.service.ax_client: Generated new trial 4 with parameters {'x1': 0.883702, 'x2': 0.440028} using model Sobol.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:34] ax.service.ax_client: Completed trial 4 with data: {'a': (-30.602318, 0.0), 'b': (-7.000249, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:34] ax.service.ax_client: Generated new trial 5 with parameters {'x1': 0.0, 'x2': 1.0} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:34] ax.service.ax_client: Completed trial 5 with data: {'a': (-17.508297, 0.0), 'b': (-1.180408, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/linear_operator/utils/cholesky.py:40: NumericalWarning:\n",
"\n",
"A not p.d., added jitter of 1.0e-08 to the diagonal\n",
"\n",
"[INFO 07-23 19:54:35] ax.service.ax_client: Generated new trial 6 with parameters {'x1': 0.0, 'x2': 0.808202} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:35] ax.service.ax_client: Completed trial 6 with data: {'a': (-38.371155, 0.0), 'b': (-1.383998, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:36] ax.service.ax_client: Generated new trial 7 with parameters {'x1': 0.151649, 'x2': 1.0} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:36] ax.service.ax_client: Completed trial 7 with data: {'a': (-14.931427, 0.0), 'b': (-5.200738, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:37] ax.service.ax_client: Generated new trial 8 with parameters {'x1': 0.0, 'x2': 0.178435} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:37] ax.service.ax_client: Completed trial 8 with data: {'a': (-223.288132, 0.0), 'b': (-2.81796, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:38] ax.service.ax_client: Generated new trial 9 with parameters {'x1': 0.062442, 'x2': 1.0} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:38] ax.service.ax_client: Completed trial 9 with data: {'a': (-4.356442, 0.0), 'b': (-3.516149, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:39] ax.service.ax_client: Generated new trial 10 with parameters {'x1': 0.028274, 'x2': 1.0} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:40] ax.service.ax_client: Completed trial 10 with data: {'a': (-9.668906, 0.0), 'b': (-2.315367, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:41] ax.service.ax_client: Generated new trial 11 with parameters {'x1': 1.0, 'x2': 1.0} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:41] ax.service.ax_client: Completed trial 11 with data: {'a': (-145.872208, 0.0), 'b': (-4.005316, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:42] ax.service.ax_client: Generated new trial 12 with parameters {'x1': 0.0673, 'x2': 0.937939} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:42] ax.service.ax_client: Completed trial 12 with data: {'a': (-3.769835, 0.0), 'b': (-3.847423, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:46] ax.service.ax_client: Generated new trial 13 with parameters {'x1': 0.076933, 'x2': 0.963646} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:46] ax.service.ax_client: Completed trial 13 with data: {'a': (-2.861984, 0.0), 'b': (-4.049937, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:48] ax.service.ax_client: Generated new trial 14 with parameters {'x1': 0.013615, 'x2': 1.0} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:48] ax.service.ax_client: Completed trial 14 with data: {'a': (-13.372099, 0.0), 'b': (-1.735031, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:51] ax.service.ax_client: Generated new trial 15 with parameters {'x1': 0.042489, 'x2': 0.98635} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:51] ax.service.ax_client: Completed trial 15 with data: {'a': (-7.078302, 0.0), 'b': (-2.876585, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:53] ax.service.ax_client: Generated new trial 16 with parameters {'x1': 0.095148, 'x2': 0.912659} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:53] ax.service.ax_client: Completed trial 16 with data: {'a': (-1.402514, 0.0), 'b': (-4.695683, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:56] ax.service.ax_client: Generated new trial 17 with parameters {'x1': 0.42263, 'x2': 0.799052} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:56] ax.service.ax_client: Completed trial 17 with data: {'a': (-74.387772, 0.0), 'b': (-5.717248, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:58] ax.service.ax_client: Generated new trial 18 with parameters {'x1': 0.050799, 'x2': 1.0} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:54:58] ax.service.ax_client: Completed trial 18 with data: {'a': (-5.61823, 0.0), 'b': (-3.137197, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:55:02] ax.service.ax_client: Generated new trial 19 with parameters {'x1': 0.006678, 'x2': 1.0} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:55:02] ax.service.ax_client: Completed trial 19 with data: {'a': (-15.398723, 0.0), 'b': (-1.45336, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:55:05] ax.service.ax_client: Generated new trial 20 with parameters {'x1': 0.111035, 'x2': 0.859094} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:55:05] ax.service.ax_client: Completed trial 20 with data: {'a': (-0.596418, 0.0), 'b': (-5.260275, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:55:08] ax.service.ax_client: Generated new trial 21 with parameters {'x1': 0.020711, 'x2': 1.0} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:55:08] ax.service.ax_client: Completed trial 21 with data: {'a': (-11.479138, 0.0), 'b': (-2.019151, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:55:11] ax.service.ax_client: Generated new trial 22 with parameters {'x1': 0.035465, 'x2': 1.0} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:55:11] ax.service.ax_client: Completed trial 22 with data: {'a': (-8.153015, 0.0), 'b': (-2.588728, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:55:15] ax.service.ax_client: Generated new trial 23 with parameters {'x1': 0.084317, 'x2': 0.931818} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:55:15] ax.service.ax_client: Completed trial 23 with data: {'a': (-2.093678, 0.0), 'b': (-4.35646, 0.0)}.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:55:20] ax.service.ax_client: Generated new trial 24 with parameters {'x1': 0.00331, 'x2': 1.0} using model BoTorch.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 07-23 19:55:20] ax.service.ax_client: Completed trial 24 with data: {'a': (-16.443575, 0.0), 'b': (-1.315804, 0.0)}.\n"
]
}
],
"source": [
"for i in range(25):\n",
" parameters, trial_index = ax_client.get_next_trial()\n",
" # Local evaluation here can be replaced with deployment to external system.\n",
" ax_client.complete_trial(trial_index=trial_index, raw_data=evaluate(parameters))"
]
},
{
"cell_type": "markdown",
"id": "5b74edfe",
"metadata": {
"code_folding": [],
"customInput": null,
"hidden_ranges": [],
"originalKey": "e0a6feb4-8c38-42e4-9d7c-62b79307e043",
"papermill": {
"duration": 0.04139,
"end_time": "2024-07-23T19:55:20.153396",
"exception": false,
"start_time": "2024-07-23T19:55:20.112006",
"status": "completed"
},
"showInput": false,
"tags": []
},
"source": [
"### Plot Pareto Frontier"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "431c6c9a",
"metadata": {
"code_folding": [],
"customInput": null,
"execution": {
"iopub.execute_input": "2024-07-23T19:55:20.236983Z",
"iopub.status.busy": "2024-07-23T19:55:20.236329Z",
"iopub.status.idle": "2024-07-23T19:55:37.419741Z",
"shell.execute_reply": "2024-07-23T19:55:37.419051Z"
},
"executionStartTime": 1628191262231,
"executionStopTime": 1628191270720,
"hidden_ranges": [],
"originalKey": "c2c2b222-6b68-4f1a-839f-16b50019ada4",
"papermill": {
"duration": 17.226933,
"end_time": "2024-07-23T19:55:37.421266",
"exception": false,
"start_time": "2024-07-23T19:55:20.194333",
"status": "completed"
},
"requestMsgId": "563d345b-573c-4d93-a480-5db88a283250",
"showInput": true,
"tags": []
},
"outputs": [
{
"data": {
"application/vnd.plotly.v1+json": {
"config": {
"linkText": "Export to plot.ly",
"plotlyServerURL": "https://plot.ly",
"showLink": false
},
"data": [
{
"error_x": {
"array": [
0.07706968227546693,
0.07706968227546693,
0.07706968227546693,
0.05812323671531533,
0.056540859676414895,
0.052639776611758234,
0.07706968233102279,
0.07706968227546693,
0.05715826391437495,
0.06308179180261336,
0.05795097401869168,
0.057676895492742616,
0.05462591624744181,
0.057162412378072784,
0.07706968227546693,
0.07147278292751416,
0.05247230343144794,
0.05501974916038185,
0.07218118786815939,
0.18078653761981398
],
"color": "rgba(128,177,211,0.4)",
"thickness": 2,
"type": "data"
},
"error_y": {
"array": [
0.003473670900448312,
0.003473670900448312,
0.0034736708981227553,
0.002665489473700985,
0.002582748295762392,
0.0023285544773174413,
0.003473670900448312,
0.0034736708981227553,
0.0026027456291768296,
0.0028600072836633805,
0.002640488700704067,
0.0026251998801553145,
0.002557955084736955,
0.0026162711330671136,
0.003473670895797199,
0.0032275062947090404,
0.0024494093409982813,
0.0025750744230017223,
0.0032351620438386334,
0.006107349690095123
],
"color": "rgba(128,177,211,0.4)",
"thickness": 2,
"type": "data"
},
"hoverinfo": "text",
"legendgroup": "mean",
"marker": {
"color": "rgba(128,177,211,1)"
},
"mode": "markers",
"name": "mean",
"text": [
"Parameterization 0
a: -17.503 [-17.581, -17.426]
b: -1.18 [-1.184, -1.177]
Parameterization:
x1: 0.0
x2: 0.9999999999999926",
"Parameterization 1
a: -17.503 [-17.581, -17.426]
b: -1.18 [-1.184, -1.177]
Parameterization:
x1: 0.0
x2: 0.9999999999999103",
"Parameterization 2
a: -17.503 [-17.581, -17.426]
b: -1.18 [-1.184, -1.177]
Parameterization:
x1: 4.4713622221032485e-17
x2: 1.0",
"Parameterization 3
a: -12.971 [-13.029, -12.913]
b: -1.794 [-1.797, -1.791]
Parameterization:
x1: 0.01507426881234419
x2: 1.0",
"Parameterization 4
a: -10.954 [-11.010, -10.897]
b: -2.102 [-2.105, -2.100]
Parameterization:
x1: 0.02282172054158936
x2: 1.0",
"Parameterization 5
a: -16.269 [-16.322, -16.216]
b: -1.339 [-1.341, -1.336]
Parameterization:
x1: 0.0038653612077254695
x2: 0.9999999999999999",
"Parameterization 6
a: -17.503 [-17.581, -17.426]
b: -1.18 [-1.184, -1.177]
Parameterization:
x1: 4.597463221352049e-16
x2: 1.0",
"Parameterization 7
a: -17.503 [-17.581, -17.426]
b: -1.18 [-1.184, -1.177]
Parameterization:
x1: 3.6692429135730665e-17
x2: 1.0",
"Parameterization 8
a: -11.557 [-11.614, -11.500]
b: -2.007 [-2.010, -2.005]
Parameterization:
x1: 0.02041321785767573
x2: 1.0",
"Parameterization 9
a: -8.079 [-8.142, -8.016]
b: -2.603 [-2.606, -2.600]
Parameterization:
x1: 0.03584351472329541
x2: 1.0",
"Parameterization 10
a: -12.306 [-12.364, -12.248]
b: -1.893 [-1.895, -1.890]
Parameterization:
x1: 0.017536634547526655
x2: 1.0",
"Parameterization 11
a: -12.011 [-12.068, -11.953]
b: -1.937 [-1.940, -1.935]
Parameterization:
x1: 0.018656665473700734
x2: 1.0",
"Parameterization 12
a: -14.509 [-14.564, -14.455]
b: -1.575 [-1.578, -1.573]
Parameterization:
x1: 0.009662523945446798
x2: 1.0",
"Parameterization 13
a: -9.557 [-9.614, -9.499]
b: -2.335 [-2.337, -2.332]
Parameterization:
x1: 0.02878088817341213
x2: 1.0",
"Parameterization 14
a: -17.503 [-17.581, -17.426]
b: -1.18 [-1.184, -1.177]
Parameterization:
x1: 1.2974519589954713e-16
x2: 1.0",
"Parameterization 15
a: -5.714 [-5.786, -5.643]
b: -3.113 [-3.116, -3.109]
Parameterization:
x1: 0.05006478580232103
x2: 1.0",
"Parameterization 16
a: -15.036 [-15.089, -14.984]
b: -1.503 [-1.505, -1.500]
Parameterization:
x1: 0.007886909903450558
x2: 1.0",
"Parameterization 17
a: -14.409 [-14.464, -14.354]
b: -1.589 [-1.592, -1.586]
Parameterization:
x1: 0.010005122257584785
x2: 1.0",
"Parameterization 18
a: -4.791 [-4.863, -4.719]
b: -3.369 [-3.372, -3.366]
Parameterization:
x1: 0.05777075769427637
x2: 0.9998837052549892",
"Parameterization 19
a: -0.948 [-1.129, -0.768]
b: -4.951 [-4.957, -4.945]
Parameterization:
x1: 0.10251321902301119
x2: 0.8903898016822577"
],
"type": "scatter",
"x": [
-17.503480841410237,
-17.503480841423812,
-17.50348084141363,
-12.970931471864134,
-10.953862519550611,
-16.269054428896723,
-17.50348084140345,
-17.50348084140345,
-11.557002547527834,
-8.079194728298276,
-12.3057784346259,
-12.010585153874722,
-14.509234799558666,
-9.55651006068262,
-17.503480841417023,
-5.714484247428416,
-15.036235568434709,
-14.40879893999912,
-4.790748903356739,
-0.9483785482559455
],
"y": [
-1.1801659125690835,
-1.1801659125697466,
-1.180165912569894,
-1.7939170015841084,
-2.1024792017735074,
-1.3386328685529452,
-1.1801659125687887,
-1.180165912569894,
-2.0072605624118998,
-2.6028157286413736,
-1.8926706169183256,
-1.9373927664062571,
-1.5750944717527187,
-2.334764031362969,
-1.1801659125697466,
-3.1125169803394845,
-1.5028580621195573,
-1.5890099165200344,
-3.368748593750326,
-4.95112564690293
]
}
],
"layout": {
"height": 500,
"hovermode": "closest",
"legend": {
"orientation": "h"
},
"margin": {
"b": 75,
"l": 225,
"pad": 4,
"t": 75
},
"template": {
"data": {
"bar": [
{
"error_x": {
"color": "#2a3f5f"
},
"error_y": {
"color": "#2a3f5f"
},
"marker": {
"line": {
"color": "#E5ECF6",
"width": 0.5
},
"pattern": {
"fillmode": "overlay",
"size": 10,
"solidity": 0.2
}
},
"type": "bar"
}
],
"barpolar": [
{
"marker": {
"line": {
"color": "#E5ECF6",
"width": 0.5
},
"pattern": {
"fillmode": "overlay",
"size": 10,
"solidity": 0.2
}
},
"type": "barpolar"
}
],
"carpet": [
{
"aaxis": {
"endlinecolor": "#2a3f5f",
"gridcolor": "white",
"linecolor": "white",
"minorgridcolor": "white",
"startlinecolor": "#2a3f5f"
},
"baxis": {
"endlinecolor": "#2a3f5f",
"gridcolor": "white",
"linecolor": "white",
"minorgridcolor": "white",
"startlinecolor": "#2a3f5f"
},
"type": "carpet"
}
],
"choropleth": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"type": "choropleth"
}
],
"contour": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0.0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1.0,
"#f0f921"
]
],
"type": "contour"
}
],
"contourcarpet": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"type": "contourcarpet"
}
],
"heatmap": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0.0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1.0,
"#f0f921"
]
],
"type": "heatmap"
}
],
"heatmapgl": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0.0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1.0,
"#f0f921"
]
],
"type": "heatmapgl"
}
],
"histogram": [
{
"marker": {
"pattern": {
"fillmode": "overlay",
"size": 10,
"solidity": 0.2
}
},
"type": "histogram"
}
],
"histogram2d": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0.0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1.0,
"#f0f921"
]
],
"type": "histogram2d"
}
],
"histogram2dcontour": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0.0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1.0,
"#f0f921"
]
],
"type": "histogram2dcontour"
}
],
"mesh3d": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"type": "mesh3d"
}
],
"parcoords": [
{
"line": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "parcoords"
}
],
"pie": [
{
"automargin": true,
"type": "pie"
}
],
"scatter": [
{
"fillpattern": {
"fillmode": "overlay",
"size": 10,
"solidity": 0.2
},
"type": "scatter"
}
],
"scatter3d": [
{
"line": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatter3d"
}
],
"scattercarpet": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scattercarpet"
}
],
"scattergeo": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scattergeo"
}
],
"scattergl": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scattergl"
}
],
"scattermapbox": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scattermapbox"
}
],
"scatterpolar": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatterpolar"
}
],
"scatterpolargl": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatterpolargl"
}
],
"scatterternary": [
{
"marker": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"type": "scatterternary"
}
],
"surface": [
{
"colorbar": {
"outlinewidth": 0,
"ticks": ""
},
"colorscale": [
[
0.0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1.0,
"#f0f921"
]
],
"type": "surface"
}
],
"table": [
{
"cells": {
"fill": {
"color": "#EBF0F8"
},
"line": {
"color": "white"
}
},
"header": {
"fill": {
"color": "#C8D4E3"
},
"line": {
"color": "white"
}
},
"type": "table"
}
]
},
"layout": {
"annotationdefaults": {
"arrowcolor": "#2a3f5f",
"arrowhead": 0,
"arrowwidth": 1
},
"autotypenumbers": "strict",
"coloraxis": {
"colorbar": {
"outlinewidth": 0,
"ticks": ""
}
},
"colorscale": {
"diverging": [
[
0,
"#8e0152"
],
[
0.1,
"#c51b7d"
],
[
0.2,
"#de77ae"
],
[
0.3,
"#f1b6da"
],
[
0.4,
"#fde0ef"
],
[
0.5,
"#f7f7f7"
],
[
0.6,
"#e6f5d0"
],
[
0.7,
"#b8e186"
],
[
0.8,
"#7fbc41"
],
[
0.9,
"#4d9221"
],
[
1,
"#276419"
]
],
"sequential": [
[
0.0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1.0,
"#f0f921"
]
],
"sequentialminus": [
[
0.0,
"#0d0887"
],
[
0.1111111111111111,
"#46039f"
],
[
0.2222222222222222,
"#7201a8"
],
[
0.3333333333333333,
"#9c179e"
],
[
0.4444444444444444,
"#bd3786"
],
[
0.5555555555555556,
"#d8576b"
],
[
0.6666666666666666,
"#ed7953"
],
[
0.7777777777777778,
"#fb9f3a"
],
[
0.8888888888888888,
"#fdca26"
],
[
1.0,
"#f0f921"
]
]
},
"colorway": [
"#636efa",
"#EF553B",
"#00cc96",
"#ab63fa",
"#FFA15A",
"#19d3f3",
"#FF6692",
"#B6E880",
"#FF97FF",
"#FECB52"
],
"font": {
"color": "#2a3f5f"
},
"geo": {
"bgcolor": "white",
"lakecolor": "white",
"landcolor": "#E5ECF6",
"showlakes": true,
"showland": true,
"subunitcolor": "white"
},
"hoverlabel": {
"align": "left"
},
"hovermode": "closest",
"mapbox": {
"style": "light"
},
"paper_bgcolor": "white",
"plot_bgcolor": "#E5ECF6",
"polar": {
"angularaxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
},
"bgcolor": "#E5ECF6",
"radialaxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
}
},
"scene": {
"xaxis": {
"backgroundcolor": "#E5ECF6",
"gridcolor": "white",
"gridwidth": 2,
"linecolor": "white",
"showbackground": true,
"ticks": "",
"zerolinecolor": "white"
},
"yaxis": {
"backgroundcolor": "#E5ECF6",
"gridcolor": "white",
"gridwidth": 2,
"linecolor": "white",
"showbackground": true,
"ticks": "",
"zerolinecolor": "white"
},
"zaxis": {
"backgroundcolor": "#E5ECF6",
"gridcolor": "white",
"gridwidth": 2,
"linecolor": "white",
"showbackground": true,
"ticks": "",
"zerolinecolor": "white"
}
},
"shapedefaults": {
"line": {
"color": "#2a3f5f"
}
},
"ternary": {
"aaxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
},
"baxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
},
"bgcolor": "#E5ECF6",
"caxis": {
"gridcolor": "white",
"linecolor": "white",
"ticks": ""
}
},
"title": {
"x": 0.05
},
"xaxis": {
"automargin": true,
"gridcolor": "white",
"linecolor": "white",
"ticks": "",
"title": {
"standoff": 15
},
"zerolinecolor": "white",
"zerolinewidth": 2
},
"yaxis": {
"automargin": true,
"gridcolor": "white",
"linecolor": "white",
"ticks": "",
"title": {
"standoff": 15
},
"zerolinecolor": "white",
"zerolinewidth": 2
}
}
},
"title": {
"text": "Pareto Frontier"
},
"width": 750,
"xaxis": {
"ticksuffix": "",
"title": {
"text": "a"
},
"zeroline": true
},
"yaxis": {
"ticksuffix": "",
"title": {
"text": "b"
},
"zeroline": true
}
}
},
"text/html": [
"