{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Loop API Example on Hartmann6\n",
"\n",
"The loop API is the most lightweight way to do optimization in Ax. The user makes one call to `optimize`, which performs all of the optimization under the hood and returns the optimized parameters.\n",
"\n",
"For more customizability of the optimization procedure, consider the Service or Developer API."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 04-30 16:00:39] ipy_plotting: Injecting Plotly library into cell. Do not overwrite or delete cell.\n"
]
}
],
"source": [
"import numpy as np\n",
"\n",
"from ax.plot.contour import plot_contour\n",
"from ax.plot.trace import optimization_trace_single_method\n",
"from ax.service.managed_loop import optimize\n",
"from ax.metrics.branin import branin\n",
"from ax.utils.measurement.synthetic_functions import hartmann6\n",
"from ax.utils.notebook.plotting import render, init_notebook_plotting\n",
"\n",
"init_notebook_plotting()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. Define evaluation function\n",
"\n",
"First, we set up an evaluation function that returns evaluations for two metrics: \"hartmann6\" and \"l2norm\"."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def hartmann_evaluation_function(parameterization):\n",
" x = np.array([parameterization.get(f\"x{i+1}\") for i in range(6)])\n",
" # In our case, standard error is 0, since we are computing a synthetic function.\n",
" return {\"hartmann6\": (hartmann6(x), 0.0), \"l2norm\": (np.sqrt((x ** 2).sum()), 0.0)}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Run optimization\n",
"The setup for the loop is fully compatible with JSON. The optimization algorithm is selected based on the properties of the problem search space."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[INFO 04-30 16:00:39] ax.service.utils.dispatch: Using Bayesian Optimization generation strategy. Iterations after 6 will take longer to generate due to model-fitting.\n",
"[INFO 04-30 16:00:39] ax.service.managed_loop: Started full optimization with 30 steps.\n",
"[INFO 04-30 16:00:39] ax.service.managed_loop: Running optimization trial 1...\n",
"[INFO 04-30 16:00:39] ax.service.managed_loop: Running optimization trial 2...\n",
"[INFO 04-30 16:00:39] ax.service.managed_loop: Running optimization trial 3...\n",
"[INFO 04-30 16:00:39] ax.service.managed_loop: Running optimization trial 4...\n",
"[INFO 04-30 16:00:39] ax.service.managed_loop: Running optimization trial 5...\n",
"[INFO 04-30 16:00:39] ax.service.managed_loop: Running optimization trial 6...\n",
"[INFO 04-30 16:00:39] ax.service.managed_loop: Running optimization trial 7...\n",
"[INFO 04-30 16:03:22] ax.service.managed_loop: Running optimization trial 8...\n",
"[INFO 04-30 16:06:32] ax.service.managed_loop: Running optimization trial 9...\n",
"[INFO 04-30 16:09:53] ax.service.managed_loop: Running optimization trial 10...\n",
"[INFO 04-30 16:14:00] ax.service.managed_loop: Running optimization trial 11...\n",
"[INFO 04-30 16:18:21] ax.service.managed_loop: Running optimization trial 12...\n",
"[INFO 04-30 16:22:16] ax.service.managed_loop: Running optimization trial 13...\n",
"[INFO 04-30 16:24:24] ax.service.managed_loop: Running optimization trial 14...\n",
"[INFO 04-30 16:28:02] ax.service.managed_loop: Running optimization trial 15...\n",
"[INFO 04-30 16:31:10] ax.service.managed_loop: Running optimization trial 16...\n",
"[INFO 04-30 16:35:04] ax.service.managed_loop: Running optimization trial 17...\n",
"[INFO 04-30 16:39:13] ax.service.managed_loop: Running optimization trial 18...\n",
"[INFO 04-30 16:43:41] ax.service.managed_loop: Running optimization trial 19...\n",
"[INFO 04-30 16:47:26] ax.service.managed_loop: Running optimization trial 20...\n",
"[INFO 04-30 16:51:29] ax.service.managed_loop: Running optimization trial 21...\n",
"[INFO 04-30 16:52:26] ax.service.managed_loop: Running optimization trial 22...\n",
"[INFO 04-30 16:54:26] ax.service.managed_loop: Running optimization trial 23...\n",
"[INFO 04-30 16:59:06] ax.service.managed_loop: Running optimization trial 24...\n",
"[INFO 04-30 17:00:50] ax.service.managed_loop: Running optimization trial 25...\n",
"[INFO 04-30 17:03:20] ax.service.managed_loop: Running optimization trial 26...\n",
"[INFO 04-30 17:05:30] ax.service.managed_loop: Running optimization trial 27...\n",
"[INFO 04-30 17:07:03] ax.service.managed_loop: Running optimization trial 28...\n",
"[INFO 04-30 17:11:53] ax.service.managed_loop: Running optimization trial 29...\n",
"[INFO 04-30 17:13:45] ax.service.managed_loop: Running optimization trial 30...\n"
]
}
],
"source": [
"best_parameters, values, experiment, model = optimize(\n",
" parameters=[\n",
" {\n",
" \"name\": \"x1\",\n",
" \"type\": \"range\",\n",
" \"bounds\": [0.0, 1.0],\n",
" \"value_type\": \"float\", # Optional, defaults to inference from type of \"bounds\".\n",
" \"log_scale\": False, # Optional, defaults to False.\n",
" },\n",
" {\n",
" \"name\": \"x2\",\n",
" \"type\": \"range\",\n",
" \"bounds\": [0.0, 1.0],\n",
" },\n",
" {\n",
" \"name\": \"x3\",\n",
" \"type\": \"range\",\n",
" \"bounds\": [0.0, 1.0],\n",
" },\n",
" {\n",
" \"name\": \"x4\",\n",
" \"type\": \"range\",\n",
" \"bounds\": [0.0, 1.0],\n",
" },\n",
" {\n",
" \"name\": \"x5\",\n",
" \"type\": \"range\",\n",
" \"bounds\": [0.0, 1.0],\n",
" },\n",
" {\n",
" \"name\": \"x6\",\n",
" \"type\": \"range\",\n",
" \"bounds\": [0.0, 1.0],\n",
" },\n",
" ],\n",
" experiment_name=\"test\",\n",
" objective_name=\"hartmann6\",\n",
" evaluation_function=hartmann_evaluation_function,\n",
" minimize=True, # Optional, defaults to False.\n",
" parameter_constraints=[\"x1 + x2 <= 20\"], # Optional.\n",
" outcome_constraints=[\"l2norm <= 1.25\"], # Optional.\n",
" total_trials=30, # Optional.\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And we can introspect optimization results:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'x1': 0.2518118433201088,\n",
" 'x2': 0.12875223997444202,\n",
" 'x3': 0.432035773040578,\n",
" 'x4': 0.2853897302458177,\n",
" 'x5': 0.31579857892882346,\n",
" 'x6': 0.6537879408811222}"
]
},
"execution_count": 4,
"metadata": {
"bento_obj_id": "140074817154360"
},
"output_type": "execute_result"
}
],
"source": [
"best_parameters"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'l2norm': 0.9355511573766859, 'hartmann6': -3.272066359418134}"
]
},
"execution_count": 5,
"metadata": {
"bento_obj_id": "140074197040944"
},
"output_type": "execute_result"
}
],
"source": [
"means, covariances = values\n",
"means"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For comparison, minimum of Hartmann6 is:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"-3.32237"
]
},
"execution_count": 6,
"metadata": {
"bento_obj_id": "140075678065648"
},
"output_type": "execute_result"
}
],
"source": [
"hartmann6.fmin"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Plot results\n",
"Here we arbitrarily select \"x1\" and \"x2\" as the two parameters to plot for both metrics, \"hartmann6\" and \"l2norm\"."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"render(plot_contour(model=model, param_x='x1', param_y='x2', metric_name='hartmann6'))"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"render(plot_contour(model=model, param_x='x1', param_y='x2', metric_name='l2norm'))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We also plot optimization trace, which shows best hartmann6 objective value seen by each iteration of the optimization:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# `plot_single_method` expects a 2-d array of means, because it expects to average means from multiple \n",
"# optimization runs, so we wrap out best objectives array in another array.\n",
"best_objectives = np.array([[trial.objective_mean for trial in experiment.trials.values()]])\n",
"best_objective_plot = optimization_trace_single_method(\n",
" y=np.minimum.accumulate(best_objectives, axis=1),\n",
" optimum=hartmann6.fmin,\n",
" title=\"Model performance vs. # of iterations\",\n",
" ylabel=\"Hartmann6\",\n",
")\n",
"render(best_objective_plot)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "python3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}