{ "cells": [ { "cell_type": "markdown", "metadata": { "originalKey": "6dba2bea-d97e-4545-9803-4242850e1807" }, "source": [ "# Ax Service API with RayTune on PyTorch CNN\n", "\n", "Ax integrates easily with different scheduling frameworks and distributed training frameworks. In this example, Ax-driven optimization is executed in a distributed fashion using [RayTune](https://ray.readthedocs.io/en/latest/tune.html). \n", "\n", "RayTune is a scalable framework for hyperparameter tuning that provides many state-of-the-art hyperparameter tuning algorithms and seamlessly scales from laptop to distributed cluster with fault tolerance. RayTune leverages [Ray](https://ray.readthedocs.io/)'s Actor API to provide asynchronous parallel and distributed execution.\n", "\n", "Ray 'Actors' are a simple and clean abstraction for replicating your Python classes across multiple workers and nodes. Each hyperparameter evaluation is asynchronously executed on a separate Ray actor and reports intermediate training progress back to RayTune. Upon reporting, RayTune then uses this information to performs actions such as early termination, re-prioritization, or checkpointing." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2022-12-29T21:34:47.572156Z", "iopub.status.busy": "2022-12-29T21:34:47.571940Z", "iopub.status.idle": "2022-12-29T21:34:50.218431Z", "shell.execute_reply": "2022-12-29T21:34:50.217642Z" }, "originalKey": "fe7a9417-4bde-46d2-9de3-af1bc73bde45" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/tmp/ipykernel_3063/1865771151.py:5: DeprecationWarning: The module `ray.tune.suggest` has been moved to `ray.tune.search` and the old location will be deprecated soon. Please adjust your imports to point to the new location. Example: Do a global search and replace `ray.tune.suggest` with `ray.tune.search`.\n", " from ray.tune.suggest.ax import AxSearch\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/tmp/ipykernel_3063/1865771151.py:5: DeprecationWarning: The module `ray.tune.suggest.ax` has been moved to `ray.tune.search.ax` and the old location will be deprecated soon. Please adjust your imports to point to the new location. Example: Do a global search and replace `ray.tune.suggest.ax` with `ray.tune.search.ax`.\n", " from ray.tune.suggest.ax import AxSearch\n" ] } ], "source": [ "import logging\n", "\n", "from ray import tune\n", "from ray.tune import report\n", "from ray.tune.suggest.ax import AxSearch\n", "\n", "logger = logging.getLogger(tune.__name__)\n", "logger.setLevel(\n", " level=logging.CRITICAL\n", ") # Reduce the number of Ray warnings that are not relevant here." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2022-12-29T21:34:50.221592Z", "iopub.status.busy": "2022-12-29T21:34:50.220965Z", "iopub.status.idle": "2022-12-29T21:34:50.329072Z", "shell.execute_reply": "2022-12-29T21:34:50.328456Z" }, "originalKey": "19956234-25ae-4e72-9d72-dbcd1b90e530" }, "outputs": [ { "data": { "text/html": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:34:50] ax.utils.notebook.plotting: Injecting Plotly library into cell. Do not overwrite or delete cell.\n" ] }, { "data": { "text/html": [ " \n", " " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import torch\n", "from ax.plot.contour import plot_contour\n", "from ax.plot.trace import optimization_trace_single_method\n", "from ax.service.ax_client import AxClient\n", "from ax.utils.notebook.plotting import init_notebook_plotting, render\n", "from ax.utils.tutorials.cnn_utils import CNN, evaluate, load_mnist, train\n", "\n", "init_notebook_plotting()" ] }, { "cell_type": "markdown", "metadata": { "originalKey": "a26e18f8-caa7-411d-809a-61a9229cd6c6" }, "source": [ "## 1. Initialize client\n", "We specify `enforce_sequential_optimization` as False, because Ray runs many trials in parallel. With the sequential optimization enforcement, `AxClient` would expect the first few trials to be completed with data before generating more trials.\n", "\n", "When high parallelism is not required, it is best to enforce sequential optimization, as it allows for achieving optimal results in fewer (but sequential) trials. In cases where parallelism is important, such as with distributed training using Ray, we choose to forego minimizing resource utilization and run more trials in parallel." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2022-12-29T21:34:50.388327Z", "iopub.status.busy": "2022-12-29T21:34:50.387467Z", "iopub.status.idle": "2022-12-29T21:34:50.392078Z", "shell.execute_reply": "2022-12-29T21:34:50.391423Z" }, "originalKey": "a91e1cb2-999a-4b88-a2d2-85d0acaa8854" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:34:50] ax.service.ax_client: Starting optimization with verbose logging. To disable logging, set the `verbose_logging` argument to `False`. Note that float values in the logs are rounded to 6 decimal points.\n" ] } ], "source": [ "ax = AxClient(enforce_sequential_optimization=False)" ] }, { "cell_type": "markdown", "metadata": { "originalKey": "1766919c-fb6f-4271-a8e1-6f972eee78f3" }, "source": [ "## 2. Set up experiment\n", "Here we set up the search space and specify the objective; refer to the Ax API tutorials for more detail." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2022-12-29T21:34:50.394659Z", "iopub.status.busy": "2022-12-29T21:34:50.394125Z", "iopub.status.idle": "2022-12-29T21:34:50.397255Z", "shell.execute_reply": "2022-12-29T21:34:50.396654Z" }, "originalKey": "37e367d4-d09d-425b-98f7-c8849d9be4b7" }, "outputs": [], "source": [ "MINIMIZE = False # Whether we should be minimizing or maximizing the objective" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2022-12-29T21:34:50.399692Z", "iopub.status.busy": "2022-12-29T21:34:50.399167Z", "iopub.status.idle": "2022-12-29T21:34:50.409094Z", "shell.execute_reply": "2022-12-29T21:34:50.408499Z" }, "originalKey": "777c8d33-2cd1-4425-b45f-2a44922dce7d" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:34:50] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter lr. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:34:50] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter momentum. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:34:50] ax.service.utils.instantiation: Created search space: SearchSpace(parameters=[RangeParameter(name='lr', parameter_type=FLOAT, range=[1e-06, 0.4], log_scale=True), RangeParameter(name='momentum', parameter_type=FLOAT, range=[0.0, 1.0])], parameter_constraints=[]).\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:34:50] ax.modelbridge.dispatch_utils: Using Bayesian optimization since there are more ordered parameters than there are categories for the unordered categorical parameters.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:34:50] ax.modelbridge.dispatch_utils: Calculating the number of remaining initialization trials based on num_initialization_trials=None max_initialization_trials=None num_tunable_parameters=2 num_trials=None use_batch_trials=False\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:34:50] ax.modelbridge.dispatch_utils: calculated num_initialization_trials=5\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:34:50] ax.modelbridge.dispatch_utils: num_completed_initialization_trials=0 num_remaining_initialization_trials=5\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:34:50] ax.modelbridge.dispatch_utils: Using Bayesian Optimization generation strategy: GenerationStrategy(name='Sobol+GPEI', steps=[Sobol for 5 trials, GPEI for subsequent trials]). Iterations after 5 will take longer to generate due to model-fitting.\n" ] } ], "source": [ "ax.create_experiment(\n", " name=\"mnist_experiment\",\n", " parameters=[\n", " {\"name\": \"lr\", \"type\": \"range\", \"bounds\": [1e-6, 0.4], \"log_scale\": True},\n", " {\"name\": \"momentum\", \"type\": \"range\", \"bounds\": [0.0, 1.0]},\n", " ],\n", " objective_name=\"mean_accuracy\",\n", " minimize=MINIMIZE,\n", ")" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2022-12-29T21:34:50.411541Z", "iopub.status.busy": "2022-12-29T21:34:50.411024Z", "iopub.status.idle": "2022-12-29T21:34:50.417524Z", "shell.execute_reply": "2022-12-29T21:34:50.416937Z" }, "originalKey": "589e4d80-02ae-461d-babc-0f96718f623e" }, "outputs": [ { "data": { "text/plain": [ "False" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ax.experiment.optimization_config.objective.minimize" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2022-12-29T21:34:50.419933Z", "iopub.status.busy": "2022-12-29T21:34:50.419384Z", "iopub.status.idle": "2022-12-29T21:34:51.189751Z", "shell.execute_reply": "2022-12-29T21:34:51.189080Z" }, "originalKey": "773a2c32-4ff3-4e92-8996-325504ce953e" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n", "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to /home/runner/.data/MNIST/raw/train-images-idx3-ubyte.gz\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "0a34097995724b4a9bb3652959f6b2e7", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/9912422 [00:00,\n", " ,\n", " )" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "load_mnist(data_path=\"~/.data\") # Pre-load the dataset before the initial evaluations are executed." ] }, { "cell_type": "markdown", "metadata": { "originalKey": "5fec848a-3538-489c-bcdd-a74051f48140" }, "source": [ "## 3. Define how to evaluate trials\n", "Since we use the Ax Service API here, we evaluate the parameterizations that Ax suggests, using RayTune. The evaluation function follows its usual pattern, taking in a parameterization and outputting an objective value. For detail on evaluation functions, see [Trial Evaluation](https://ax.dev/docs/runner.html). " ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2022-12-29T21:34:51.192690Z", "iopub.status.busy": "2022-12-29T21:34:51.192235Z", "iopub.status.idle": "2022-12-29T21:34:51.197431Z", "shell.execute_reply": "2022-12-29T21:34:51.196871Z" }, "originalKey": "75fce84d-35bd-45b5-b55e-f52baf26db03" }, "outputs": [], "source": [ "def train_evaluate(parameterization):\n", " device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", " train_loader, valid_loader, test_loader = load_mnist(data_path=\"~/.data\")\n", " net = train(\n", " net=CNN(),\n", " train_loader=train_loader,\n", " parameters=parameterization,\n", " dtype=torch.float,\n", " device=device,\n", " )\n", " report(\n", " mean_accuracy=evaluate(\n", " net=net,\n", " data_loader=valid_loader,\n", " dtype=torch.float,\n", " device=device,\n", " )\n", " )" ] }, { "cell_type": "markdown", "metadata": { "originalKey": "dda3574c-5967-43ea-8d23-7a151dc59ec9" }, "source": [ "## 4. Run optimization\n", "Execute the Ax optimization and trial evaluation in RayTune using [AxSearch algorithm](https://ray.readthedocs.io/en/latest/tune-searchalg.html#ax-search):" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "code_folding": [], "execution": { "iopub.execute_input": "2022-12-29T21:34:51.200013Z", "iopub.status.busy": "2022-12-29T21:34:51.199713Z", "iopub.status.idle": "2022-12-29T21:37:39.432447Z", "shell.execute_reply": "2022-12-29T21:37:39.431787Z" }, "hidden_ranges": [], "originalKey": "1d768bb2-d46b-4c4c-879e-3242af7555f4" }, "outputs": [ { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "/home/runner/work/Ax/Ax/ax/core/observation.py:274: FutureWarning:\n", "\n", "In a future version of pandas, a length 1 tuple will be returned when iterating over a groupby with a grouper equal to a list of length 1. Don't supply a list with a single grouper to avoid this warning.\n", "\n", "[INFO 12-29 21:34:54] ax.service.ax_client: Generated new trial 0 with parameters {'lr': 0.000448, 'momentum': 0.15958}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/home/runner/work/Ax/Ax/ax/core/observation.py:274: FutureWarning:\n", "\n", "In a future version of pandas, a length 1 tuple will be returned when iterating over a groupby with a grouper equal to a list of length 1. Don't supply a list with a single grouper to avoid this warning.\n", "\n", "[INFO 12-29 21:34:54] ax.service.ax_client: Generated new trial 1 with parameters {'lr': 3e-06, 'momentum': 0.796108}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/home/runner/work/Ax/Ax/ax/core/observation.py:274: FutureWarning:\n", "\n", "In a future version of pandas, a length 1 tuple will be returned when iterating over a groupby with a grouper equal to a list of length 1. Don't supply a list with a single grouper to avoid this warning.\n", "\n", "[INFO 12-29 21:34:54] ax.service.ax_client: Generated new trial 2 with parameters {'lr': 1.2e-05, 'momentum': 0.538357}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:07] ax.service.ax_client: Completed trial 1 with data: {'mean_accuracy': (0.5675, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:07] ax.service.ax_client: Generated new trial 3 with parameters {'lr': 0.015297, 'momentum': 0.677089}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:07] ax.service.ax_client: Completed trial 0 with data: {'mean_accuracy': (0.929667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:07] ax.service.ax_client: Generated new trial 4 with parameters {'lr': 0.121624, 'momentum': 0.407587}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:16] ax.service.ax_client: Completed trial 2 with data: {'mean_accuracy': (0.746, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:16] ax.service.ax_client: Generated new trial 5 with parameters {'lr': 1.2e-05, 'momentum': 0.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:17] ax.service.ax_client: Completed trial 3 with data: {'mean_accuracy': (0.112667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:17] ax.service.ax_client: Generated new trial 6 with parameters {'lr': 0.006591, 'momentum': 0.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:24] ax.service.ax_client: Completed trial 4 with data: {'mean_accuracy': (0.105333, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:25] ax.service.ax_client: Generated new trial 7 with parameters {'lr': 7e-05, 'momentum': 0.227281}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:26] ax.service.ax_client: Completed trial 5 with data: {'mean_accuracy': (0.487333, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:27] ax.service.ax_client: Generated new trial 8 with parameters {'lr': 0.000214, 'momentum': 0.375103}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:34] ax.service.ax_client: Completed trial 6 with data: {'mean_accuracy': (0.115667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:35] ax.service.ax_client: Generated new trial 9 with parameters {'lr': 0.00021, 'momentum': 0.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:36] ax.service.ax_client: Completed trial 7 with data: {'mean_accuracy': (0.869, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:37] ax.service.ax_client: Generated new trial 10 with parameters {'lr': 0.000118, 'momentum': 0.697606}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:43] ax.service.ax_client: Completed trial 8 with data: {'mean_accuracy': (0.918167, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:45] ax.service.ax_client: Generated new trial 11 with parameters {'lr': 0.000223, 'momentum': 0.216812}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:46] ax.service.ax_client: Completed trial 9 with data: {'mean_accuracy': (0.906667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:47] ax.service.ax_client: Generated new trial 12 with parameters {'lr': 5e-05, 'momentum': 1.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:53] ax.service.ax_client: Completed trial 10 with data: {'mean_accuracy': (0.931167, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:55] ax.service.ax_client: Generated new trial 13 with parameters {'lr': 0.000275, 'momentum': 1.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:58] ax.service.ax_client: Completed trial 11 with data: {'mean_accuracy': (0.909167, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:35:59] ax.service.ax_client: Generated new trial 14 with parameters {'lr': 6.3e-05, 'momentum': 0.611518}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:04] ax.service.ax_client: Completed trial 12 with data: {'mean_accuracy': (0.849667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:06] ax.service.ax_client: Generated new trial 15 with parameters {'lr': 0.000407, 'momentum': 0.66411}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:08] ax.service.ax_client: Completed trial 13 with data: {'mean_accuracy': (0.268333, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:10] ax.service.ax_client: Generated new trial 16 with parameters {'lr': 4.1e-05, 'momentum': 0.735952}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:15] ax.service.ax_client: Completed trial 14 with data: {'mean_accuracy': (0.9135, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:17] ax.service.ax_client: Generated new trial 17 with parameters {'lr': 1.6e-05, 'momentum': 1.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:19] ax.service.ax_client: Completed trial 15 with data: {'mean_accuracy': (0.93, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:21] ax.service.ax_client: Generated new trial 18 with parameters {'lr': 0.00029, 'momentum': 0.556927}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:26] ax.service.ax_client: Completed trial 16 with data: {'mean_accuracy': (0.899, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:28] ax.service.ax_client: Generated new trial 19 with parameters {'lr': 0.000624, 'momentum': 0.476343}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:32] ax.service.ax_client: Completed trial 17 with data: {'mean_accuracy': (0.875667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:33] ax.service.ax_client: Generated new trial 20 with parameters {'lr': 0.000202, 'momentum': 0.584793}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:38] ax.service.ax_client: Completed trial 18 with data: {'mean_accuracy': (0.945167, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:39] ax.service.ax_client: Generated new trial 21 with parameters {'lr': 0.000462, 'momentum': 0.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:42] ax.service.ax_client: Completed trial 19 with data: {'mean_accuracy': (0.934667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:43] ax.service.ax_client: Generated new trial 22 with parameters {'lr': 0.000451, 'momentum': 0.564712}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:50] ax.service.ax_client: Completed trial 20 with data: {'mean_accuracy': (0.932333, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:52] ax.service.ax_client: Generated new trial 23 with parameters {'lr': 0.000496, 'momentum': 0.327785}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:54] ax.service.ax_client: Completed trial 21 with data: {'mean_accuracy': (0.935333, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:55] ax.modelbridge.base: Untransformed parameter 0.40000000000000013 greater than upper bound 0.4, clamping\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:36:55] ax.service.ax_client: Generated new trial 24 with parameters {'lr': 0.4, 'momentum': 1.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:02] ax.service.ax_client: Completed trial 22 with data: {'mean_accuracy': (0.957, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:03] ax.service.ax_client: Generated new trial 25 with parameters {'lr': 1e-06, 'momentum': 0.249746}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:06] ax.service.ax_client: Completed trial 23 with data: {'mean_accuracy': (0.946, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:08] ax.service.ax_client: Generated new trial 26 with parameters {'lr': 0.000413, 'momentum': 0.432858}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:14] ax.service.ax_client: Completed trial 24 with data: {'mean_accuracy': (0.101167, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:15] ax.service.ax_client: Generated new trial 27 with parameters {'lr': 3e-05, 'momentum': 0.905257}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:19] ax.service.ax_client: Completed trial 25 with data: {'mean_accuracy': (0.1205, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:21] ax.service.ax_client: Generated new trial 28 with parameters {'lr': 4.8e-05, 'momentum': 0.411281}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:25] ax.service.ax_client: Completed trial 26 with data: {'mean_accuracy': (0.948833, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:26] ax.service.ax_client: Generated new trial 29 with parameters {'lr': 0.000384, 'momentum': 0.586623}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:30] ax.service.ax_client: Completed trial 27 with data: {'mean_accuracy': (0.918333, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:35] ax.service.ax_client: Completed trial 28 with data: {'mean_accuracy': (0.876833, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 12-29 21:37:39] ax.service.ax_client: Completed trial 29 with data: {'mean_accuracy': (0.942, None)}.\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Set up AxSearcher in RayTune\n", "algo = AxSearch(ax_client=ax)\n", "# Wrap AxSearcher in a concurrently limiter, to ensure that Bayesian optimization receives the\n", "# data for completed trials before creating more trials\n", "algo = tune.suggest.ConcurrencyLimiter(algo, max_concurrent=3)\n", "tune.run(\n", " train_evaluate,\n", " num_samples=30,\n", " search_alg=algo,\n", " verbose=0, # Set this level to 1 to see status updates and to 2 to also see trial results.\n", " # To use GPU, specify: resources_per_trial={\"gpu\": 1}.\n", ")" ] }, { "cell_type": "markdown", "metadata": { "originalKey": "cb00f812-e9e5-4208-a680-adf6619d74c4" }, "source": [ "## 5. Retrieve the optimization results" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2022-12-29T21:37:39.435184Z", "iopub.status.busy": "2022-12-29T21:37:39.434949Z", "iopub.status.idle": "2022-12-29T21:37:40.090019Z", "shell.execute_reply": "2022-12-29T21:37:40.089327Z" }, "originalKey": "2ec54675-d0ad-4eac-aaf3-66b593037cce" }, "outputs": [ { "data": { "text/plain": [ "{'lr': 0.00045105615350493, 'momentum': 0.5647121361805426}" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "best_parameters, values = ax.get_best_parameters()\n", "best_parameters" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "execution": { "iopub.execute_input": "2022-12-29T21:37:40.095532Z", "iopub.status.busy": "2022-12-29T21:37:40.094881Z", "iopub.status.idle": "2022-12-29T21:37:40.098965Z", "shell.execute_reply": "2022-12-29T21:37:40.098483Z" }, "originalKey": "50c764a6-a630-4935-9c07-ea84045e0ecc" }, "outputs": [ { "data": { "text/plain": [ "{'mean_accuracy': 0.9519302757733763}" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "means, covariances = values\n", "means" ] }, { "cell_type": "markdown", "metadata": { "originalKey": "12a87817-4409-4f07-a912-8d60eff71d68" }, "source": [ "## 6. Plot the response surface and optimization trace" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "execution": { "iopub.execute_input": "2022-12-29T21:37:40.101308Z", "iopub.status.busy": "2022-12-29T21:37:40.100911Z", "iopub.status.idle": "2022-12-29T21:37:40.713040Z", "shell.execute_reply": "2022-12-29T21:37:40.712479Z" }, "originalKey": "3742f35b-6b28-49ae-a606-a138459f4964", "scrolled": false }, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "linkText": "Export to plot.ly", "plotlyServerURL": "https://plot.ly", "showLink": false }, "data": [ { "autocolorscale": false, "autocontour": true, "colorbar": { "tickfont": { "size": 8 }, "ticksuffix": "", "x": 0.45, "y": 0.5 }, "colorscale": [ [ 0.0, "rgb(247,252,253)" ], [ 0.125, "rgb(229,245,249)" ], [ 0.25, "rgb(204,236,230)" ], [ 0.375, "rgb(153,216,201)" ], [ 0.5, "rgb(102,194,164)" ], [ 0.625, "rgb(65,174,118)" ], [ 0.75, "rgb(35,139,69)" ], [ 0.875, "rgb(0,109,44)" ], [ 1.0, "rgb(0,68,27)" ] ], "contours": { "coloring": "heatmap" }, "hoverinfo": "x+y+z", "ncontours": 25, "type": "contour", "x": [ 1e-06, 1.3011511650442548e-06, 1.6929943542960217e-06, 2.2028415765056147e-06, 2.866229883678204e-06, 3.729398352432554e-06, 4.852511011181743e-06, 6.3138503555892e-06, 8.215273746089953e-06, 1.0689313005882422e-05, 1.390841207112662e-05, 1.809694657026198e-05, 2.354686311364001e-05, 3.063802837345029e-05, 3.986470631277378e-05, 5.1870009063012666e-05, 6.749072272319499e-05, 8.781563250096393e-05, 0.00011426141253772723, 0.00014867137004306603, 0.00019344392634026088, 0.0002516997901283655, 0.0003274994751669172, 0.00042612632366481585, 0.0005544547624925005, 0.0007214294601814526, 0.000938688782612345, 0.0012213760031100258, 0.0015891948094037057, 0.002067782677737912, 0.002690497840197013, 0.0035007443993213955, 0.004554997653699184, 0.005926740503884541, 0.007711585311544345, 0.010033938212454076, 0.01305567039511669, 0.01698740074503987, 0.02210317627048227, 0.028759573555516532, 0.03742055263793628, 0.04868979566145066, 0.06335278435066323, 0.0824315491666629, 0.10725590623460621, 0.13955614735503497, 0.18158364372009145, 0.23626776957937787, 0.3074200836506151, 0.39999999999999997 ], "xaxis": "x", "y": [ 0.0, 0.02040816326530612, 0.04081632653061224, 0.061224489795918366, 0.08163265306122448, 0.1020408163265306, 0.12244897959183673, 0.14285714285714285, 0.16326530612244897, 0.18367346938775508, 0.2040816326530612, 0.22448979591836732, 0.24489795918367346, 0.26530612244897955, 0.2857142857142857, 0.3061224489795918, 0.32653061224489793, 0.3469387755102041, 0.36734693877551017, 0.3877551020408163, 0.4081632653061224, 0.42857142857142855, 0.44897959183673464, 0.4693877551020408, 0.4897959183673469, 0.5102040816326531, 0.5306122448979591, 0.5510204081632653, 0.5714285714285714, 0.5918367346938775, 0.6122448979591836, 0.6326530612244897, 0.6530612244897959, 0.673469387755102, 0.6938775510204082, 0.7142857142857142, 0.7346938775510203, 0.7551020408163265, 0.7755102040816326, 0.7959183673469387, 0.8163265306122448, 0.836734693877551, 0.8571428571428571, 0.8775510204081632, 0.8979591836734693, 0.9183673469387754, 0.9387755102040816, 0.9591836734693877, 0.9795918367346939, 1.0 ], "yaxis": "y", "z": [ [ 0.07465039832593201, 0.09940414778151052, 0.1299336335858371, 0.1658839925492097, 0.2067227610663338, 0.2518072598634238, 0.30044129863419433, 0.3519077395045591, 0.40546927803097643, 0.460331748192255, 0.5155637410527966, 0.5699959985598498, 0.6222826463684938, 0.6711301499236721, 0.7155297473033708, 0.7549698986538776, 0.7895957936064891, 0.8202283806090463, 0.8481240044294887, 0.8744094992551178, 0.8992632630031148, 0.9210445473510811, 0.9357845629620422, 0.9378786851718692, 0.9219520879612955, 0.8850525907297127, 0.8271084028630653, 0.7502128046003584, 0.658047702849702, 0.5553869702817152, 0.44763533949245804, 0.3404016664739263, 0.23908662897227112, 0.14844020414682657, 0.0720275604353553, 0.011676291974249242, -0.0324533616263597, -0.06131666479976583, -0.076537187394085, -0.08009385770026745, -0.07410339013970901, -0.06066936527658495, -0.04177758125886766, -0.019224133409209143, 0.005431792205533026, 0.03089436642190868, 0.05613856649903748, 0.08040023222826109, 0.10315167057594243, 0.12406736563122 ], [ 0.07473965396098259, 0.1004200632671024, 0.1320513752299205, 0.16923763332414565, 0.21139367746247079, 0.2578199568015208, 0.3077640104242062, 0.3604552168226784, 0.41510597487723877, 0.4708741717217819, 0.5267812494963073, 0.5816134092825196, 0.6339804099824168, 0.6825401364768636, 0.7262363727252436, 0.7645277631754734, 0.7975777710971881, 0.8263091028526827, 0.8521774027601203, 0.876575854175247, 0.8999470936735298, 0.9208358148688598, 0.9353456221499117, 0.9378285156885995, 0.9227643891787474, 0.8869445874240396, 0.8300331855353782, 0.7539574057545152, 0.6623383217701077, 0.5599603775429275, 0.4522736710852374, 0.3449424911174326, 0.2434197431622872, 0.15249814438692022, 0.07577473744130958, 0.015113705113505849, -0.02928761350157849, -0.05836036212118545, -0.07371665504236369, -0.07733541643204789, -0.07134318207262469, -0.05786101001926236, -0.03889706035513485, -0.01627138695769914, 0.008434752285355174, 0.03390844196127207, 0.059114586732828656, 0.08328646103411241, 0.1059003158431957, 0.12663913097676083 ], [ 0.07538607434912958, 0.10200725013260337, 0.1347521088443283, 0.17318150715583802, 0.21665560652450216, 0.2644165239754397, 0.31565435393002217, 0.3695436938878421, 0.4252448897751295, 0.4818654336036169, 0.5383774424834243, 0.593527005680384, 0.6458896770422239, 0.6940827735872137, 0.7370055664731361, 0.7740832443471057, 0.8054885253382538, 0.8322370499627553, 0.8559805023367807, 0.8783796373757358, 0.9001418789836366, 0.9199976182975127, 0.9341445608338319, 0.9369414121512868, 0.9227783731638243, 0.8881746834781, 0.8324797599098527, 0.7574314344244437, 0.6665837949073511, 0.5647279420341602, 0.45735130701047977, 0.35016024971119836, 0.24864139969210852, 0.15760662484671661, 0.0806613448604736, 0.019707821910686096, -0.02499810152320958, -0.0543445936827176, -0.06991844517635937, -0.07368881311056474, -0.06778556423665272, -0.054343313198518706, -0.035390606203029806, -0.012770823919079355, 0.011912591108537596, 0.03732932886314011, 0.062434104424762715, 0.08645783338065027, 0.10888113722857828, 0.1293959257426499 ], [ 0.07662202355495396, 0.10419365697057292, 0.13805750947841056, 0.17772922252043333, 0.22251260028720243, 0.27159037683841786, 0.3240943775216771, 0.37914336716542196, 0.43584412308675524, 0.49325194941929107, 0.5502897509650085, 0.6056686435023735, 0.6579395477631775, 0.7056886713704175, 0.7477759427649273, 0.7835914173278226, 0.8133086440704452, 0.8380266355245979, 0.8595885856671658, 0.8799219958790447, 0.8999982358006557, 0.9187358537542538, 0.9324384742133832, 0.9355032521091241, 0.9222627308493835, 0.8889612918781168, 0.834608779250648, 0.7607422652198728, 0.6708474462805762, 0.5697182566308494, 0.4628694265668759, 0.3560336875148589, 0.2547111860865398, 0.16370979283972653, 0.08662604165935894, 0.025399715264282197, -0.019640040315555773, -0.04932097649668232, -0.06519091278970446, -0.06919946465229754, -0.06347327324726171, -0.05015652185655728, -0.0312961249943976, -0.008758150718921831, 0.01583164825599226, 0.04112526043518905, 0.0660671392468698, 0.08988613165010584, 0.11206774189077473, 0.13231329371012213 ], [ 0.07847885789750442, 0.10700572959036336, 0.14198698417098388, 0.18289115718923854, 0.22896443574567343, 0.2793296041317331, 0.33305975831445805, 0.3892170224510163, 0.44685341237683196, 0.5049712479103698, 0.562446354615206, 0.617960594492306, 0.6700492178414599, 0.717277767059544, 0.7584743455206596, 0.7929947870140546, 0.8210063624804878, 0.8436816117965638, 0.8630497596192387, 0.8813022697731446, 0.8996728401597467, 0.9172714730069261, 0.9305085490756168, 0.9338264846712501, 0.9215112277449989, 0.8895404097979189, 0.8365885731342588, 0.7639973030493095, 0.675187241836422, 0.5749502601932556, 0.4688154505404798, 0.36252356438992145, 0.2615668307062141, 0.1707280803762149, 0.09358306139231443, 0.03210703621715838, -0.013289321831394196, -0.0433588270973424, -0.059597406693008614, -0.06392546791727938, -0.058459797826220905, -0.045350043981221, -0.026659367817751023, -0.00427582017350514, 0.02015244424331286, 0.04525944435445728, 0.06997937861462744, 0.09353942913394553, 0.11543060022563911, 0.1353641690177203 ], [ 0.08098645582326569, 0.11046798105343503, 0.1465572838180187, 0.18867415735569604, 0.23600648369344684, 0.2876171007958912, 0.3425203070437288, 0.39972105771605854, 0.4582158623082147, 0.5169543686205056, 0.5747688667179888, 0.6303181060486911, 0.6821302747388938, 0.728761297605659, 0.7690172325667693, 0.8022237606839044, 0.82853695872178, 0.8491933573900724, 0.8664021935947268, 0.8826139668175645, 0.899321228685083, 0.9158271601856918, 0.9286382707374654, 0.9322230009439723, 0.9208153603427924, 0.8901438850940083, 0.8385802414636058, 0.7672946273171575, 0.679650504840733, 0.5804308689601376, 0.47516290920165577, 0.36957458808879057, 0.2691284167091621, 0.17856406987213735, 0.10142820188456891, 0.039728563893495994, -0.006039491541097841, -0.03654311305068891, -0.05321482322187021, -0.05793654138499349, -0.05280858527536114, -0.03998184559017359, -0.021533469305301423, 0.0006273246732032556, 0.02482996025000983, 0.04969028898780925, 0.07413236956630975, 0.09738226304934794, 0.1189372109009127, 0.13851903817421019 ], [ 0.08417263053192547, 0.11460245303138694, 0.15178202245189665, 0.19508118253444506, 0.24362957893894438, 0.2964307610274104, 0.3524405844810536, 0.4106066380629124, 0.46986971452243087, 0.5291280917969111, 0.5871749196990065, 0.6426522205571603, 0.6940895388820987, 0.7400445405423991, 0.7793129984974718, 0.8111980893347319, 0.8358430180629579, 0.8545398050688906, 0.8696715071581074, 0.8839398532394135, 0.8990884107281621, 0.9146113273115187, 0.927090449507205, 0.9309784266565613, 0.9204382603268563, 0.8909771065036565, 0.840722263087806, 0.7707136789147134, 0.6842690043367217, 0.586153130888739, 0.4818718873961254, 0.37711784583562363, 0.2773025649151183, 0.18710744986268668, 0.11004398737393073, 0.04814885581831285, 0.002001747018333111, -0.028971932535910017, -0.04613176878289349, -0.05131265673624752, -0.04659200407427111, -0.03411765585818183, -0.01597833478660249, 0.00589652713794675, 0.029814015095268043, 0.05437171013871145, 0.07848377923054384, 0.10137586792482012, 0.1225523199189874, 0.14174615003754942 ], [ 0.08806239857462406, 0.11942803987873485, 0.15767107810372782, 0.2021108790992706, 0.2518198757831611, 0.3057437007081627, 0.3627805646386411, 0.4218208638674517, 0.48175001416959046, 0.5414169867102698, 0.5995806208694126, 0.654872672637229, 0.7058322933657482, 0.7510302689654322, 0.7892653278618128, 0.8198294924411, 0.8428558387016766, 0.8596852999757986, 0.8728685209316454, 0.8853465187645979, 0.8990985988791296, 0.9138012893667653, 0.9260840054175307, 0.9303270758134998, 0.9205899489937576, 0.8921983699643555, 0.8431165707233068, 0.7743071845652824, 0.689055042800384, 0.5920951408578631, 0.4888899875308528, 0.3850734062223992, 0.2859861942774762, 0.19623917004942415, 0.11930402452531397, 0.057242524932261785, 0.010716109888802894, -0.020753749921227138, -0.03844645862848073, -0.044142432032646894, -0.03989010316707853, -0.027830003043564422, -0.010059888440245968, 0.011471707401274234, 0.035049736637105755, 0.05925351784784072, 0.08298772523260822, 0.10547847183896741, 0.1262381965827084, 0.14501177626550954 ], [ 0.09267707411855719, 0.12495964356999889, 0.16422985228144238, 0.2097570746665251, 0.2605586820864459, 0.3155244870422826, 0.3734962949943455, 0.4333078757278298, 0.4937901169139387, 0.5537452686267185, 0.6119028458045724, 0.6668907412460704, 0.7172657322149684, 0.7616228004918956, 0.7987776144568359, 0.8280256941725415, 0.8494983137041612, 0.8645818206006339, 0.875987812951574, 0.8868789968759933, 0.8994452419241022, 0.9135274836620342, 0.9257716880378231, 0.9304270530930236, 0.9214044300613533, 0.893902079271748, 0.8458179487402244, 0.7780954545331731, 0.6939991188973753, 0.598219920045924, 0.4961537653378739, 0.3933529076906995, 0.2950697827023058, 0.20583491165135614, 0.12907664897902393, 0.06687798185113614, 0.01997813495759959, -0.012004581978733109, -0.03026447604661997, -0.03652136210852641, -0.03278921298502391, -0.021197107396049075, -0.0038491972166471156, 0.017288158971720646, 0.040478123920789755, 0.06428188246786481, 0.0875951773089585, 0.10964565807245819, 0.12995496922121197, 0.14828052446813844 ], [ 0.09803315492753062, 0.13120712537946666, 0.1714583755650364, 0.2180082029085843, 0.2698222799403245, 0.3257373665971844, 0.3845405240074987, 0.44500985695038714, 0.5059230173710427, 0.5660384492700866, 0.6240613241861191, 0.6786219441173093, 0.7283024320025874, 0.7717324256613262, 0.8077583613349371, 0.8356960804347506, 0.8556896750843321, 0.8691721262125797, 0.8790076828738268, 0.8885560268444808, 0.900182209062323, 0.9138605006129722, 0.9262218498942845, 0.9313365304857777, 0.922921877887106, 0.8961083578204176, 0.8488283637045012, 0.7820639735924887, 0.6990696006242607, 0.6044763896626825, 0.5035905970210774, 0.4018620503459689, 0.30444015826730414, 0.21576797075921894, 0.13922795439139513, 0.07692062433550528, 0.02965826945781569, -0.0028452790618729296, -0.021696499811726166, -0.028549958262142128, -0.025380433737449026, -0.014301658442093967, 0.002578513739076582, 0.02327734254696434, 0.04603669608703065, 0.06939987919034818, 0.09225443157886537, 0.11383079495162662, 0.13366102349001607, 0.15151570585463114 ], [ 0.10414096229910885, 0.13817402120893008, 0.17935027202329523, 0.2268466950485511, 0.2795817590852269, 0.33634249800406163, 0.39586328570023177, 0.4568679203380077, 0.5180824945760092, 0.578224763083427, 0.6359804653407325, 0.6899884561997311, 0.7388636198638806, 0.7812798824511484, 0.8161272488353672, 0.8427590657409899, 0.86135250730321, 0.8733954403046614, 0.881892153592422, 0.8903666344110496, 0.9013164631612566, 0.9148021720983339, 0.9274093178342921, 0.9330036944722878, 0.9250835419081731, 0.8987615301081509, 0.852097350155187, 0.7861648462263666, 0.7042146230687779, 0.6108014575329055, 0.5111209154561687, 0.41050295079116145, 0.3139828689216183, 0.2259116325202023, 0.14962428311382447, 0.08723550928190338, 0.03962558523022419, 0.006601004894489226, -0.012856084868508866, -0.02033186066303183, -0.01775805381993434, -0.007229503348415656, 0.009144938727621565, 0.02936776894313531, 0.051660224301205315, 0.07454811055698729, 0.09691165938453239, 0.11798553700057679, 0.13731346590741156, 0.15467975846745874 ], [ 0.11100299315756379, 0.1458560017678504, 0.18789164625450716, 0.23624840870261937, 0.2898029073716836, 0.3472962091799166, 0.40741244689954714, 0.4688228826338995, 0.5302040773683828, 0.5902363534989223, 0.647590871354372, 0.700921132686911, 0.7488819905392523, 0.7902004300245552, 0.8238212131483146, 0.8491508273216876, 0.866422410329383, 0.8771960196852624, 0.8845954135159358, 0.8922692044786513, 0.9028022894148255, 0.916281017202057, 0.9292186460198444, 0.9352809300722469, 0.9277432809790249, 0.9017384266260918, 0.8555288182876496, 0.7903221864443065, 0.7093661444372773, 0.6171230944472095, 0.5186607075932606, 0.41917632044716, 0.32358416559216735, 0.23614109512660741, 0.1601342448270806, 0.09768955816248137, 0.04975011297042975, 0.016211101316841448, -0.003857557380169019, -0.011971975638736976, -0.010017936490023471, -6.827100734618785e-05, 0.015769848084508986, 0.03548596379505908, 0.057281543782091826, 0.07966540725260363, 0.10151153336461372, 0.12206040091288328, 0.1408686548841115, 0.1577347259984343 ], [ 0.1186119381425802, 0.1542391284438941, 0.19706004235982055, 0.246182199326621, 0.30044621583403447, 0.3585513083291385, 0.4191342336573581, 0.48081594025306246, 0.5422258384234544, 0.6020102100554426, 0.6588304958768223, 0.7113610318096693, 0.7583038366082602, 0.7984470230511245, 0.8307995236899375, 0.8548339184760076, 0.8708604427474496, 0.8805336725858789, 0.8870685225253452, 0.8941949855961974, 0.9045398920496643, 0.9181541656179686, 0.9314593978650165, 0.937948950362054, 0.9306900561565057, 0.9048653684141374, 0.8589937767101977, 0.794441013112777, 0.7144457843815998, 0.6233641225174538, 0.5261241056234607, 0.4277834071566073, 0.33313261279691553, 0.2463349863891211, 0.17063031831620956, 0.10815334835854151, 0.059904812489998016, 0.02586385290854909, 0.005185935915175688, -0.003574678410778964, -0.002255907501560128, 0.007094046203049209, 0.022372311599017758, 0.041557508454117986, 0.06283244436682778, 0.0846896086543164, 0.10599793445781314, 0.12600542016746485, 0.144282800617489, 0.1606427905111496 ], [ 0.12694832971209769, 0.16329820630425018, 0.20682372486853862, 0.25660975885420556, 0.31146705893692184, 0.37005748123189414, 0.4309737593262525, 0.492789266344835, 0.5540890319301623, 0.6134888581354645, 0.6696454262858142, 0.7212603663942532, 0.7670903127124303, 0.8059921576724547, 0.8370467706787115, 0.8598027411935788, 0.8746638121912864, 0.883392482626927, 0.8892674834717736, 0.896057684817859, 0.906387798125234, 0.9202256285434849, 0.933892198021717, 0.9407453865408999, 0.933675771841409, 0.9079415905348737, 0.8623476838923723, 0.7984186845273107, 0.7193717675191355, 0.6294462927813592, 0.533425846515048, 0.4362276152120261, 0.342520322646088, 0.2563765036075136, 0.18099008292748886, 0.11850253978295455, 0.06996720813960788, 0.035441874519257954, 0.01416420090394177, 0.0047578884490413476, 0.0054338286652096945, 0.014170533499291316, 0.028871978804250298, 0.047508153769387995, 0.06824463990492724, 0.08955842618935184, 0.1103147447949615, 0.12977088208037535, 0.1475126335126059, 0.16336685495226388 ], [ 0.13597864715056818, 0.17299600799048365, 0.2171415700052446, 0.26748583186698544, 0.32281609959237073, 0.38176180374255375, 0.4428755783269651, 0.5046865532486152, 0.5657385968656835, 0.6246208172263146, 0.679990293726272, 0.7305828635719908, 0.775217753613439, 0.8128281666198396, 0.8425731352357952, 0.8640835209591922, 0.8778644429972435, 0.8857839107785641, 0.8911614765134117, 0.8977684074713561, 0.9081897249778574, 0.9222812245282981, 0.9362634209097432, 0.943398101108792, 0.9364467893648301, 0.9107669253380605, 0.8654505147944268, 0.802157434117003, 0.7240660440424348, 0.6352941196459068, 0.5404833347719177, 0.44441570313860307, 0.3516437978016319, 0.2661542006551674, 0.19109712054618255, 0.12861898232280622, 0.07982072318738498, 0.04483305892441736, 0.02297158080422823, 0.012927398617655461, 0.012960220036919212, 0.02107679942275853, 0.03519040776818827, 0.0532650060012112, 0.0734508190757619, 0.09421039451538038, 0.11440673280450009, 0.13330815020832898, 0.15051613777810335, 0.16587116792722167 ], [ 0.14565657796905784, 0.18328410582688803, 0.22796373802013536, 0.2787588645336935, 0.33443994418034345, 0.3936093888273906, 0.45478428771555435, 0.5164535268399074, 0.5771235550436286, 0.6353608593127948, 0.6898283461701584, 0.7393035711975903, 0.7826770872492004, 0.8189660222670178, 0.8474121361757252, 0.8677299067506326, 0.880522066670849, 0.8877457845796304, 0.8927403414750859, 0.8992526096578458, 0.9098033207564582, 0.9241252928159986, 0.9383440793152057, 0.9456628077595566, 0.9387787672189725, 0.9131716318918365, 0.8681870349110856, 0.8055762021757773, 0.7284605021929179, 0.6408378903670628, 0.5472180360161133, 0.45225846009239695, 0.36040436937623405, 0.27556244528489593, 0.20084162631383595, 0.13839154638856466, 0.0893557478081588, 0.053931843849613914, 0.03150834917737311, 0.020840790903299822, 0.020236823601659015, 0.027732407149627103, 0.04125243392205302, 0.05875778659429087, 0.07838578291630782, 0.09858591786089677, 0.11822053801897159, 0.1365705724016879, 0.15325334191669904, 0.1681219787256618 ], [ 0.15592605140933347, 0.19410499630387235, 0.23923305420745944, 0.290372061654066, 0.34628204070194457, 0.4055441740317449, 0.46664519433009743, 0.5280384605431474, 0.5881973399688492, 0.6456701143801252, 0.699131249789919, 0.7474082072269906, 0.7894724980157695, 0.8244329529356554, 0.8516166669166811, 0.8708170011021776, 0.8827185159608625, 0.8893405093031207, 0.8940207272593208, 0.9004654406587023, 0.9111237886934627, 0.9256113574604138, 0.9399670139611257, 0.9473637174778429, 0.9405144544323627, 0.9150451673483764, 0.8704841419525302, 0.8086197681216262, 0.7325011668643165, 0.6460152987543449, 0.5535559638842684, 0.45967078498748626, 0.36870822778419593, 0.28450157455546254, 0.21012076655017387, 0.1477167166455129, 0.09847047444014989, 0.06264025929288408, 0.03968191250156872, 0.028411569659272606, 0.027183164212686628, 0.0340622683237658, 0.04698756695593642, 0.06392016910976472, 0.0829876773791084, 0.10262842152490803, 0.1217057632348646, 0.13951446890294394, 0.15568715036152792, 0.17008820507874023 ], [ 0.16672401023895012, 0.20539461193863706, 0.2508868327275636, 0.3022647554602725, 0.35828378204120626, 0.41750984079190506, 0.47840505960531654, 0.5393927171116956, 0.5989180983442142, 0.6555160832048547, 0.7078787095232436, 0.7548921950808809, 0.7956195771027528, 0.8292693074187739, 0.8552542289630574, 0.8734354322017223, 0.8845529881119664, 0.8906533226394988, 0.8950503178331887, 0.9014031721999252, 0.912100603117345, 0.9266638545214533, 0.9410555307767229, 0.9484325934145148, 0.9416016480078403, 0.9163580282240119, 0.8723217233533946, 0.8112632782448552, 0.7361494286783614, 0.650771274454194, 0.5594271028549272, 0.466571137688059, 0.3764660672506284, 0.2928777860871753, 0.21883882486565093, 0.1564989877313745, 0.10707153261250568, 0.07086877466460795, 0.04740782560447099, 0.0355609441110204, 0.03372597070588679, 0.039997968705808296, 0.05233139431573419, 0.0686911952395387, 0.08719933256365386, 0.10628562149628873, 0.124816178738683, 0.14210018333018237, 0.1577841901667686, 0.17174209002746 ], [ 0.1779827951313867, 0.21708468976620238, 0.2628588476483818, 0.3143739528349715, 0.3703857537156746, 0.42945084395429634, 0.4900129277970571, 0.5504713456402635, 0.6092490102749173, 0.6648726271520832, 0.7160580176656267, 0.76175955794933, 0.8011432351093712, 0.8335251079388081, 0.8584020089588802, 0.8756858019617338, 0.8861375320438948, 0.8917899253069065, 0.8959093876843341, 0.9021095781430079, 0.9127467838667831, 0.9272890867528916, 0.9416363263882669, 0.9489286884296726, 0.942113142124573, 0.9171686762753252, 0.8737338705051991, 0.8135107277193385, 0.7393796804222943, 0.6550557896627395, 0.5647637303057886, 0.4728803949303028, 0.3835923952880588, 0.3006028159219669, 0.22690718130916043, 0.1646511000403429, 0.11507445335384403, 0.07853696442485492, 0.05461062378001047, 0.04221879404737694, 0.03980026052946817, 0.0454789840416403, 0.05722695121127741, 0.07301676206448504, 0.09096972377032131, 0.10951092826783015, 0.12751103170027533, 0.144293159336205, 0.15951563399604596, 0.1730598184772012 ], [ 0.1896321790355272, 0.22910484680111365, 0.27508123785276173, 0.3266359286855032, 0.38252905469384346, 0.44131351952182407, 0.5014210364720396, 0.5612337593831922, 0.6191586764022412, 0.6737200096563792, 0.7236636480536571, 0.7680218565580746, 0.8060756518131541, 0.8372566613166386, 0.8611421461019926, 0.8776733680603124, 0.8875921391126032, 0.8928726330750757, 0.8967089074259812, 0.902676540235529, 0.9131408023410187, 0.9275747019439399, 0.9418327346348204, 0.9490213013666235, 0.9422257165172424, 0.9176107667290362, 0.874798853793571, 0.8153867895428722, 0.7421732296743448, 0.6588197047166598, 0.5694977495754081, 0.47852021746130724, 0.39000459266051063, 0.30759346779806956, 0.23424417339642478, 0.17209415291906982, 0.1224039910309368, 0.0855740093400904, 0.061224475199421424, 0.04832444927528845, 0.04535024128004039, 0.05045373266883468, 0.061625987637118906, 0.07685114101362933, 0.0942555724790165, 0.11226500309218412, 0.12975642022007117, 0.14606497596146084, 0.16085794789703667, 0.1740220599241722 ], [ 0.20160108453229064, 0.24138434644122198, 0.2874862284431401, 0.33898775769596934, 0.3946566193369141, 0.45304723087613474, 0.5125858000561396, 0.571644516023507, 0.6286216201090774, 0.6820450641567793, 0.7306970127197228, 0.773697350026828, 0.8104545113621868, 0.8405235085372321, 0.8635573490104573, 0.8795027664266969, 0.8890390068679033, 0.8940344272514587, 0.8975846070352655, 0.9032381279357333, 0.9134203449870633, 0.9276773486953143, 0.9418404658394953, 0.9489486968065195, 0.9421761873990901, 0.9178635698840371, 0.8756184954079789, 0.8169224474688567, 0.7445089468635676, 0.6620090272973359, 0.5735573022365756, 0.4834111087213677, 0.3956218418311888, 0.3137710709981113, 0.24077489069014685, 0.1787576312428596, 0.12899432782360465, 0.09191904852526944, 0.06719365683643841, 0.053827270950809525, 0.05032999714088071, 0.05488040310223263, 0.0654900278000754, 0.08015841068286189, 0.09702309771539563, 0.11451747313429717, 0.13152661341397287, 0.14739424054789996, 0.16179350387694247, 0.1746144031704947 ], [ 0.21381901333947906, 0.2538535707836312, 0.3000076163655835, 0.351368707182464, 0.4067144741370115, 0.46460550735187145, 0.5234688466333304, 0.5816742144684655, 0.6376189502723313, 0.6898415588051247, 0.737166488689344, 0.7788105413613937, 0.8143217300588288, 0.8433859175513163, 0.8657269521387596, 0.8812726775479883, 0.8905957787856307, 0.8954106308667626, 0.8986868134675868, 0.9039578271314053, 0.9137649749237405, 0.9277981943732414, 0.9418919460599493, 0.9489713165957848, 0.9422139769031577, 0.9181124879148662, 0.8762894234342243, 0.8181359789515177, 0.7463517084054047, 0.66455825903617, 0.576863067750744, 0.4874704073753568, 0.40036406698635496, 0.3190609519368049, 0.24643095498582446, 0.1845793787136476, 0.1347891828199247, 0.09752139520900416, 0.07247285689041116, 0.058687024980876856, 0.05470393204769908, 0.05872749626842655, 0.06879109464091249, 0.08291355478624318, 0.09924977312977756, 0.1162486429932813, 0.13280507505260297, 0.14826721693873568, 0.16231099873281385, 0.17482765303419445 ], [ 0.22621721393178956, 0.26644521899011236, 0.312582002444376, 0.36372143978956495, 0.4186528705436699, 0.47594712266957595, 0.5340380732574123, 0.591300508107764, 0.6461392239062177, 0.6971108201054002, 0.7430878063198975, 0.7833922401559276, 0.8177228333802652, 0.8459030699863628, 0.8677235084099414, 0.8830704857200898, 0.892367819024542, 0.897128318450052, 0.9001664203311708, 0.9050098967258764, 0.9143702807197603, 0.9281501299216007, 0.9422153068801897, 0.9493240241016025, 0.942552476073679, 0.9185048628458664, 0.8768696973782429, 0.8190117938425002, 0.7476402034916624, 0.6663837343673311, 0.5793247569756101, 0.4906104947653843, 0.4041510413886321, 0.32339200607198554, 0.25115033506609513, 0.1895055468485386, 0.13974184370826237, 0.10234062664751742, 0.07702730668262558, 0.06287404126933394, 0.05844695015329171, 0.061974037738339316, 0.07151198900222866, 0.08510290032165879, 0.10092520541111127, 0.11745034108419916, 0.13358488245427513, 0.14867808359869494, 0.1624056359487207, 0.1746579685230294 ], [ 0.23872961072098747, 0.27909525146618963, 0.32514976800068907, 0.3759929932867966, 0.4304272423385464, 0.487037053667575, 0.544268664290251, 0.6005092089727724, 0.6541795355047254, 0.703862667006355, 0.7484848689330469, 0.7874802342814294, 0.8207070753911371, 0.8481320500145298, 0.8696100552853703, 0.8849671375437869, 0.894439840123182, 0.8992939314749563, 0.9021578333443735, 0.9065568529772454, 0.9154192149550977, 0.9289226517774638, 0.9429922474722955, 0.9501682206450346, 0.9433181448987449, 0.9191036227135511, 0.8773451079464156, 0.8194803761531004, 0.7482759852704377, 0.6673779690514826, 0.5808383414738655, 0.4927375051938595, 0.4069018144684711, 0.32669645009996295, 0.25487723779885274, 0.19349054220289963, 0.14381513408936042, 0.10634655572340246, 0.08083274506475036, 0.0663691572394356, 0.06154436688428411, 0.06460944424668535, 0.07364608120118965, 0.08672373990309279, 0.10205028471508892, 0.11812511364841305, 0.13386839447998378, 0.14862878125195533, 0.16207905546900314, 0.1741068355160953 ], [ 0.2512935164077433, 0.2917435985115381, 0.33765580135995604, 0.38813551704166127, 0.4419989423137347, 0.49784725274197583, 0.554143990859007, 0.6092954139594268, 0.6617468320931799, 0.7101166953088697, 0.7533910482370737, 0.79112060162788, 0.8233283063007776, 0.8501276978586799, 0.8714382364259179, 0.8870125541427106, 0.8968674471200089, 0.9019798954745951, 0.90476003677685, 0.9087249951625631, 0.9170528920322399, 0.9302478977615463, 0.9443179725927922, 0.9515446744086231, 0.9444978430250885, 0.9198416183099078, 0.877600946446351, 0.8194029332656766, 0.7481156625357125, 0.6674059940585909, 0.5812845209980921, 0.49375079766497204, 0.40853459036317197, 0.3289098193900491, 0.2575621059933269, 0.19649698583597686, 0.1469813234621783, 0.10951908837305768, 0.08387521819921251, 0.06916344896157534, 0.06399155813044977, 0.06663306314672113, 0.07519666470288078, 0.08778331433839581, 0.1026355319119262, 0.11828462440210319, 0.1336663324709948, 0.14812849641602954, 0.1613390290984651, 0.17318088205820414 ], [ 0.2638501479636044, 0.30433465149683686, 0.3500499832949971, 0.40010675351500274, 0.45333571955014274, 0.5083571628918537, 0.5636562811357566, 0.6176645183446587, 0.6688593865856144, 0.7159039401623667, 0.7578509727634863, 0.7943695882326433, 0.8256464694281244, 0.8519433279957411, 0.8732474898167442, 0.8892320743675132, 0.8996693806889147, 0.9052113231348873, 0.9080170918694529, 0.9115793931912266, 0.9193417570361331, 0.93216947942888, 0.9461670760050822, 0.9533361867035695, 0.9458967154496929, 0.920489053662589, 0.8774064274198787, 0.8185639897016199, 0.746967781952967, 0.6663044457559791, 0.5805288175116372, 0.4935433838514036, 0.4089671528252658, 0.3299712531483082, 0.25916173844520357, 0.19849568859918487, 0.149221979993021, 0.11184796698671051, 0.08615071678696473, 0.07125775738816575, 0.06579336789072565, 0.06805343237766892, 0.07617598775301626, 0.08829748481597499, 0.10269947952053682, 0.11794807839898702, 0.1329965855638432, 0.1471928897241328, 0.1601989666458048, 0.17189155771539155 ], [ 0.2763449661839533, 0.3168175548525214, 0.36228744351231973, 0.4118702607946955, 0.46441190718902475, 0.5185539061523923, 0.5728069297120058, 0.6256328993355058, 0.675548209769826, 0.7212689202986584, 0.7619227933716709, 0.797295776045678, 0.8277294445714334, 0.8536322257564125, 0.8750655091174103, 0.8916244868512532, 0.9028214195014085, 0.9089541604341753, 0.9118995687743916, 0.9150995557243367, 0.9222577321502151, 0.9346144850122233, 0.9483684836956932, 0.9552494085085763, 0.9471335566932868, 0.9206559396261816, 0.876417047411842, 0.8166738088614145, 0.744595267604773, 0.6638838431027979, 0.5784235104398021, 0.4920034159764223, 0.40811788236704494, 0.329824082055734, 0.2596395302073411, 0.19946563401510164, 0.15052775842594035, 0.1133323952741555, 0.08766465114280153, 0.07266201997061139, 0.06696330383451954, 0.06888732459870028, 0.07660409384501932, 0.0882893397935045, 0.10226721363539903, 0.11714081385965025, 0.13188297753593892, 0.14584319198724305, 0.15867729233012784, 0.1702547075049542 ], [ 0.2887278587815095, 0.32914631823740975, 0.37432860384982874, 0.4233953804032888, 0.47520830284671123, 0.528432088529557, 0.5816063133319085, 0.6332279849885003, 0.6818579050448191, 0.7262718724556573, 0.7656807538162356, 0.7999819017443328, 0.8296547793711031, 0.8552497254710976, 0.8769101296076622, 0.8941622277102552, 0.9062530140054866, 0.9131063901950114, 0.9162887727904323, 0.9191572243983761, 0.9256482913357258, 0.9373675773502195, 0.9505897806575034, 0.9568179501872457, 0.9476704765965748, 0.9198250632384048, 0.874194081332232, 0.8133804813608477, 0.7407234122316771, 0.6599340690446426, 0.5748114212444857, 0.48901673202606544, 0.4059073560099621, 0.328416703320233, 0.25896581318348766, 0.1993939481123591, 0.15089810721243657, 0.11398053483255344, 0.08843116186471223, 0.07339441928215662, 0.06752255528511619, 0.0691586419053668, 0.07650757921564622, 0.08778785310877879, 0.10136906839509308, 0.11589305591281662, 0.13035410941696401, 0.14410526816257707, 0.15679675150029582, 0.16829007463391044 ], [ 0.3009531874278417, 0.34127977004723614, 0.38613902825289315, 0.43465696409942195, 0.48571174038585513, 0.5379931896152503, 0.5900730123066288, 0.6404874307681873, 0.687846206037541, 0.7309897716214878, 0.7692166371948249, 0.8025252953162957, 0.8315107334443653, 0.8568555426698531, 0.8787926553115432, 0.8967942541400848, 0.9098477323045131, 0.9174940842931805, 0.920966195920638, 0.9234984680618101, 0.9292147661832169, 0.9400463760293946, 0.952329627012558, 0.9574264468218647, 0.9468569111523373, 0.9173949746727637, 0.8702352590424323, 0.8082896461965031, 0.7350525837861761, 0.6542326675624326, 0.5695313486808539, 0.48447034864036265, 0.402260464927982, 0.32570370008470406, 0.2571182611817152, 0.19827582371763675, 0.15034087049884293, 0.11380885792978934, 0.08847226043959866, 0.07348036110466172, 0.06749887047851266, 0.0688972218577476, 0.07591834219777727, 0.08682663550721581, 0.10003945859853747, 0.11423881621397147, 0.1284423167090224, 0.14200871482059296, 0.1545836985730552, 0.16602076514852193 ], [ 0.31297971981433587, 0.35318137566936925, 0.3976891046677967, 0.4456348839981147, 0.4959143718814608, 0.5472445415590075, 0.5982324047319305, 0.6474572550213195, 0.6935816021086649, 0.7355133862545522, 0.7726368436952741, 0.8050359962071265, 0.8333960726375831, 0.8585158875881487, 0.880722384791119, 0.8994519164450463, 0.9134484564250063, 0.9218740233748459, 0.9256112164522203, 0.9277340298666791, 0.9324985281173046, 0.942088004616816, 0.9529251842478678, 0.9563554026687862, 0.9439800888305091, 0.9127244380704227, 0.8640101449254229, 0.8009888460922628, 0.727274226836448, 0.6465552443544973, 0.5624247804147601, 0.47825669818443917, 0.39710893731474184, 0.32164713989959254, 0.25408231278949284, 0.1961143570600007, 0.1488717524319152, 0.11284133504610883, 0.087816790728656, 0.07295129681436874, 0.06692533210609264, 0.06813760746744502, 0.07487237227024257, 0.08544279238189967, 0.09831583893426765, 0.11221492428182545, 0.12618274774990357, 0.13958602699905254, 0.15206740389815798, 0.16347270309014672 ], [ 0.3247704684726831, 0.364818946253571, 0.40895358927239644, 0.45631336125488314, 0.5058127002037602, 0.5561979445585642, 0.6061146905021583, 0.6541890075439553, 0.6991391845927216, 0.7399409176402167, 0.7760556655239159, 0.8076324029928139, 0.8354181854720976, 0.8603047393387748, 0.8827116648331848, 0.9020577293571527, 0.9168679282295786, 0.9259440077158614, 0.9298100746251412, 0.9313452255083212, 0.9348807463382637, 0.9427713138416824, 0.9515929605829557, 0.952829592682696, 0.9383124186576617, 0.9051752188156224, 0.854995754831235, 0.7910735140401915, 0.7170884873296323, 0.6366870638364682, 0.5533433847505571, 0.47027833690221676, 0.3903941183148162, 0.3162179737905876, 0.2498515637020488, 0.1929202500676317, 0.14651360377957945, 0.11110842976579882, 0.08649920077815287, 0.07184341221442525, 0.06583907466254046, 0.06691782743649399, 0.07340861014384692, 0.08367589154670263, 0.09623778125753046, 0.10986017882965693, 0.12361255825354267, 0.13687185029819804, 0.1492794042664386, 0.16067409857445714 ], [ 0.33629245851810163, 0.37616426586371965, 0.41991104833888554, 0.4666801586102265, 0.5154064227533044, 0.5648680063928062, 0.613752488193838, 0.6607362553122996, 0.7045954815392184, 0.744370952400192, 0.7795877258496845, 0.810435032167379, 0.837689256570538, 0.8623035417924199, 0.8847802580394466, 0.904535808464815, 0.919904404025633, 0.9293606624613926, 0.9330777726125087, 0.9337134303971876, 0.9356218820383101, 0.9412796664624177, 0.9474955326377078, 0.9460610075546754, 0.9291531168544113, 0.8941518223346352, 0.8427097927781616, 0.7781721144714397, 0.7042218312986857, 0.62443487741421, 0.5421567285187896, 0.46045281206983923, 0.382069838908944, 0.3093974550242981, 0.24442809144901662, 0.18871133855922972, 0.14329548469333087, 0.10864586797212195, 0.08455811976252336, 0.07019622178191809, 0.06427999536072138, 0.06527822955468043, 0.07156790178694972, 0.08156704309437623, 0.09384616367040466, 0.10721460936356364, 0.12077021360101492, 0.13390232228339494, 0.1462529111139642, 0.15765494475735242 ], [ 0.347516446392039, 0.38719266629549814, 0.43054323685867835, 0.47672569004007703, 0.5246971625968873, 0.5732703238992161, 0.6211782058749666, 0.6671507717830167, 0.7100230888124941, 0.7488957406970647, 0.7833404461758805, 0.8135590222097212, 0.8403203506600645, 0.8645985453920433, 0.8869574531417563, 0.9068205658156324, 0.9223584040507298, 0.9317629970692402, 0.9348918658106093, 0.9341721827849138, 0.9339435327132679, 0.9367857571585041, 0.9398174014822587, 0.935304639815219, 0.9158724473233149, 0.8791379417988983, 0.8267398782818715, 0.7619685950111663, 0.6884432591042501, 0.609638077898409, 0.5287596721376644, 0.4487173689190659, 0.3721052015873309, 0.3011785040877558, 0.23782270195630928, 0.1835119376731592, 0.1392514542983957, 0.10549314341419758, 0.08203476028602996, 0.06805113762127102, 0.062289521263843906, 0.06326040856765858, 0.06939206510181462, 0.07915809343011926, 0.09118246665335661, 0.10431883972560962, 0.11769488976225728, 0.13071450088729974, 0.14302228263321093, 0.15444655379853034 ], [ 0.3584166108216296, 0.3978825791829066, 0.4408344553020103, 0.4864421056185074, 0.5336871724397898, 0.5814196398364242, 0.6284214062882745, 0.6734788031910156, 0.715485631807475, 0.7535950130953911, 0.7874067073962204, 0.8171058307754511, 0.8434133756849869, 0.8672751004081594, 0.8892805385640197, 0.9088603725816259, 0.9240449344691123, 0.9327995273999596, 0.9347347166538613, 0.9320703449786387, 0.9291105810640751, 0.9285357687228781, 0.9278441840471767, 0.9199280430978908, 0.8979599655058245, 0.8597284636862285, 0.8067674701275426, 0.742220849720474, 0.6695780361336184, 0.5921784179951749, 0.5130789499946349, 0.4350331995558174, 0.36048712033619756, 0.29156695955464335, 0.23005512470937195, 0.17735207251267815, 0.13441905115109076, 0.10169172107047741, 0.07897124115091003, 0.06545012402885175, 0.05990950436223119, 0.060906267453199714, 0.06692308485441634, 0.07649093532332674, 0.08828817209938467, 0.1012135460617889, 0.11442596383471981, 0.12734587423753885, 0.1396225606316378, 0.15108113708171123 ], [ 0.3689702353410628, 0.408215093856147, 0.45077092611047115, 0.4958224109316395, 0.5423780986664601, 0.5893281084903771, 0.6355063723683628, 0.6797577106137394, 0.721033366750593, 0.7585305076664882, 0.791857946023327, 0.8211545651725701, 0.847051125883501, 0.8704084042581528, 0.8917888277837972, 0.9106162009220524, 0.924802832705599, 0.9321569964366577, 0.9321402579804963, 0.9268346748990524, 0.9204985670917432, 0.9159167711674816, 0.9110315847012802, 0.8994831224581682, 0.8750661257856734, 0.8356539687785078, 0.7825853094306967, 0.7187743470228165, 0.6475182033103053, 0.5719877211853942, 0.4950785383914116, 0.4193889799001111, 0.34722247213367846, 0.2805826668862751, 0.2211542151996062, 0.17026678891770286, 0.12883757632614368, 0.09728298222267484, 0.0754090786992142, 0.06243458384341216, 0.05718131629798762, 0.05825724512920927, 0.0642024474096774, 0.07360693451975686, 0.08520426083691346, 0.09793900193241323, 0.11100258538262764, 0.12383394470773779, 0.13608906946573807, 0.1475914311664357 ], [ 0.3791573986622595, 0.41817354630940995, 0.4603402293816077, 0.5048596779577756, 0.5507698875020508, 0.5970037901438178, 0.6424500429345605, 0.6860132013634475, 0.7266996189161632, 0.763741421963457, 0.7967380596448533, 0.8257535808410724, 0.8512860778789035, 0.8740505729952983, 0.8945127740298722, 0.9120555206495754, 0.9245009607576681, 0.9295867982202196, 0.9267422309763264, 0.9180233682303282, 0.907641948851043, 0.8984998692973906, 0.8890425484188741, 0.873737065402264, 0.847023422054597, 0.806794452304401, 0.7541075474148001, 0.6915704967934884, 0.6222294633458473, 0.5490532209471275, 0.4747635282035415, 0.4018034992178097, 0.3323397427223722, 0.26826036267322306, 0.21115822421730907, 0.16229583130616088, 0.12254692562720937, 0.09230669229509358, 0.07138824078410633, 0.05904461757383672, 0.05414519855184308, 0.055353732512670284, 0.061270620955556354, 0.07054647097415345, 0.0819708032153661, 0.09453470304382372, 0.10746331981897506, 0.12021587916503929, 0.13245707214258717, 0.14401036811715384 ], [ 0.3889606849097322, 0.42774315910327393, 0.4695308321996615, 0.513546398660254, 0.5588599045429511, 0.6044494719389434, 0.6492604420258133, 0.6922572892148338, 0.7324982010271573, 0.7692410255746166, 0.8020585859173619, 0.8309132318894806, 0.8561293516792675, 0.8782148561139739, 0.8974576982048597, 0.9131409424105633, 0.9230389466172746, 0.9249213764170795, 0.9183117998030499, 0.9053599915022731, 0.890260719152002, 0.876058451886051, 0.8617494558343908, 0.8426567216562624, 0.813844351990482, 0.773181298067481, 0.7213724446976614, 0.6606497542910067, 0.5937543437063337, 0.523420366559844, 0.45218234054713363, 0.3823272526195724, 0.3158900777520406, 0.2546503085750673, 0.20011514817331877, 0.15348386272368197, 0.11558795300811475, 0.08680135513333742, 0.06694703225582643, 0.05531872611006983, 0.05083989131276012, 0.0522346830035092, 0.058166679377453345, 0.0673485893621425, 0.07862663578152995, 0.0910390634929703, 0.10384585495169107, 0.11652821692739179, 0.1287614772695843, 0.1403707867988655 ], [ 0.3983649204910655, 0.4369107436686506, 0.4783317370164224, 0.5218740242628019, 0.5666423242966256, 0.6116618837683905, 0.6559356753397083, 0.6984870609557999, 0.7384219127198008, 0.7750146491486576, 0.8077956248366536, 0.8366007855371043, 0.8615419999581475, 0.8828603006119862, 0.9005820119908252, 0.9138156602595094, 0.9203423919671557, 0.9180745979171113, 0.906756759129261, 0.8887394298836846, 0.8682659229611703, 0.848566338103466, 0.8292187914728341, 0.8063862753641731, 0.7757056928659394, 0.7349894913250113, 0.684538390634326, 0.6261498549672476, 0.5622117994441125, 0.4951931145085898, 0.4274272394144948, 0.36104293357662015, 0.2979476798446164, 0.23981862187628533, 0.18808310142236273, 0.14388110325632608, 0.10800378489696816, 0.08080563591834822, 0.06212266525698429, 0.05129391623753443, 0.0473025231671258, 0.04893740556405257, 0.05492805973713033, 0.06405074972464064, 0.07520911528327034, 0.0874891746422869, 0.10018676155727368, 0.11280662663588392, 0.1250365896334562, 0.13670518025726652 ], [ 0.4073569388184868, 0.44566446548556243, 0.4867322587826902, 0.5298327224251957, 0.5741078298184487, 0.6186313425770406, 0.6624635162215947, 0.7046842716879741, 0.744442171763351, 0.7810192010923431, 0.8138888718886742, 0.8427383710270392, 0.8674309016066876, 0.8878834478205553, 0.9037804709903559, 0.9139932342650219, 0.9163552974523718, 0.9090298130460428, 0.8920997812075986, 0.868212633525659, 0.8417484460233591, 0.8161826918278577, 0.7916904609275797, 0.7652258956437097, 0.7329282604462103, 0.6925234669435614, 0.6438746155637167, 0.5882998801817146, 0.5277936230559428, 0.46453187301813254, 0.40063319704509165, 0.3380648261741338, 0.2786095222982813, 0.22384724668293993, 0.1751305947042472, 0.1335440659625664, 0.09984105092745466, 0.07435963032040505, 0.056952111507489245, 0.04700607876690621, 0.0435687079405076, 0.045497512496238235, 0.051590435782605915, 0.06068866551479202, 0.07175393952126607, 0.08392061695939812, 0.09652129852805602, 0.10908570300132236, 0.1213158965986545, 0.13304547325372496 ], [ 0.41592536934980506, 0.45399366072926484, 0.49472191476921606, 0.537411366210377, 0.5812436451765142, 0.625341810999083, 0.6688215536800867, 0.7108157402381817, 0.7505097664163337, 0.7871842611185982, 0.8202429313232995, 0.8492043945441761, 0.8736504559088802, 0.8931216960376831, 0.9068928340429668, 0.913558336580853, 0.9110332098679723, 0.8978242751077797, 0.874456848230762, 0.8439633303570785, 0.8109575507143505, 0.7792296656869576, 0.7495553493263849, 0.7196107537306713, 0.6859560104419846, 0.6461996085002938, 0.5997481601067598, 0.5474110212764737, 0.49075816401363015, 0.4316493777359554, 0.3719752506120522, 0.3135371521381731, 0.2579943808524333, 0.20683351814663042, 0.1613365941077286, 0.12253611090324767, 0.0911507377822054, 0.06750578153248754, 0.05147291788438768, 0.042490484357703084, 0.039672775499704604, 0.04194898320357299, 0.048187684349681126, 0.05729621354701875, 0.0682950232386823, 0.08036731454597457, 0.09288325285749399, 0.10539879417529874, 0.11763188210864761, 0.12942282327348997 ], [ 0.42406044481627575, 0.461888684325258, 0.5022903781845555, 0.5445977072552277, 0.5880338837357377, 0.6317712902354458, 0.674977831695166, 0.7168344699073916, 0.7565566506806556, 0.7934146777957489, 0.8267308318385137, 0.8558382623486742, 0.8800094754886725, 0.8983651046582614, 0.9097213019011543, 0.9123747407431154, 0.9043388413081238, 0.8845351442299367, 0.8540178307822582, 0.8162836804253029, 0.7762749852925308, 0.7381668891441461, 0.703332128029208, 0.670090042242674, 0.6353353640998659, 0.5965273116807036, 0.552608494235217, 0.5038649358489992, 0.4514219273905892, 0.39680484820404605, 0.3416645504425369, 0.2876314735729791, 0.23624121542460758, 0.18888929268945753, 0.14679026739357115, 0.11092767551204075, 0.0819886609112439, 0.0602894677400061, 0.0457238416005884, 0.037782271167480896, 0.03564806245283636, 0.03832430126265962, 0.044751918715741246, 0.053905398819982686, 0.06486441654609831, 0.07686142181150124, 0.08930480458475232, 0.1017778503330582, 0.11401585977742845, 0.12586743777410603 ], [ 0.4317538209880606, 0.4693407691461823, 0.5094274296194745, 0.551378534534104, 0.5944599635230323, 0.6378924111385906, 0.6808918827581758, 0.7226813818565372, 0.7624986383620048, 0.799594476992838, 0.8331994362405643, 0.8624477581265724, 0.8862814362001743, 0.9033704389308451, 0.9120443147807953, 0.9102937427449311, 0.8962398773248851, 0.8692683591589205, 0.8310293022474583, 0.7855511275020086, 0.7381884282187784, 0.6935652852393711, 0.6536438299618864, 0.6173058434905746, 0.5816947378963553, 0.5440895716414531, 0.5029708428874153, 0.45810051378869227, 0.4101496298396143, 0.3602968110377902, 0.30994334069443363, 0.2605432877977327, 0.21350696242508355, 0.1701396455387173, 0.13159037402711904, 0.09879614465508602, 0.07241558973907969, 0.05275929484177633, 0.039745267313016974, 0.03291684583615495, 0.03152720123397945, 0.03465462451149792, 0.04131356308418266, 0.05054635709313127, 0.061492253490069704, 0.07343323196983986, 0.08581640692707637, 0.09825328396849253, 0.11049781633004052, 0.12240840002051012 ], [ 0.43899840670592366, 0.4763418862025989, 0.5161228692678423, 0.5577396420914658, 0.6005007602999393, 0.6436731172730564, 0.6865160558028326, 0.7282875216839187, 0.7682387957987384, 0.8055917887441773, 0.8394762738666289, 0.8688181847626247, 0.8922161823601976, 0.9078742604785972, 0.9136271594601604, 0.9071606487225568, 0.8867080262461683, 0.8521500657359873, 0.8057794845668163, 0.7522070809005212, 0.6972660894078765, 0.6460814999666551, 0.6011945139756235, 0.5619717720395432, 0.5257240873539291, 0.4895234916772079, 0.4513989454498682, 0.4105997458183231, 0.36734325983480276, 0.3224549832693302, 0.2770791351794818, 0.23248797543842997, 0.18996382314662197, 0.15072116647943135, 0.11584430100227383, 0.08622537856337864, 0.06249706872790661, 0.04496713052631429, 0.03357941226309791, 0.027930160278010097, 0.027342364301703226, 0.030969954665035937, 0.03790144498759829, 0.04724737939333279, 0.05820671935347643, 0.07011109787462788, 0.08244667232005964, 0.09485383244210055, 0.10710625646560257, 0.11907349551369106 ], [ 0.44578820666295416, 0.48288461097464497, 0.5223664118738621, 0.5636657467067521, 0.6061327594639062, 0.6490774344758848, 0.6917970523346082, 0.7335765851481639, 0.7736712929485945, 0.8112644326362995, 0.8453772474142829, 0.8747224016900744, 0.897551833013347, 0.9116045257132572, 0.9142305202125035, 0.9028196776046813, 0.8757187352702707, 0.833320040619283, 0.7785851004835046, 0.7167374435526462, 0.6541329985790102, 0.5964333916035629, 0.5467461272316569, 0.5048513064472278, 0.4681542898879204, 0.43350086404166466, 0.39848773898270323, 0.3618732601312832, 0.32343063647649345, 0.2836315960108599, 0.24335835687516538, 0.2036962773946731, 0.16579615301434036, 0.13077991029259517, 0.09966677996548823, 0.07330494172066881, 0.05230297915609228, 0.03696791633316532, 0.027270341183265878, 0.022858854196624923, 0.02312543810836365, 0.02729928187992614, 0.03454288672422612, 0.04403494506967931, 0.05503402708834948, 0.06692135726305137, 0.0792222561375644, 0.09160641358604338, 0.1038680400603812, 0.11588903072378642 ], [ 0.4521181834080626, 0.4889620123755053, 0.528147618180809, 0.569140529275199, 0.6113304074059683, 0.654066363872399, 0.6966775984390661, 0.7384676059814896, 0.7786854519728617, 0.8164657682193626, 0.8507146587439115, 0.8799310450725477, 0.902026304887098, 0.9142911150494739, 0.9136180265821067, 0.8971182282227874, 0.863251570928548, 0.8129267644659649, 0.7497798763041688, 0.679654685339623, 0.6094487694673449, 0.545376467928071, 0.491095452274071, 0.4467356712550259, 0.40973625571948874, 0.37670892320152716, 0.3448463262076972, 0.31244599053761773, 0.278853902450792, 0.24419251190638253, 0.2090797029463447, 0.17440948131159173, 0.14119706904185814, 0.11046907701504138, 0.08317834205366459, 0.06012908652888782, 0.04190688762187489, 0.02881929443835063, 0.020863817935010465, 0.017740269219966187, 0.01890811608896248, 0.02367068878560752, 0.031263782119613004, 0.04093375323595261, 0.05199839579929699, 0.06388825681115506, 0.07616773172277425, 0.08853596584874668, 0.10080820250136546, 0.1128796355573517 ], [ 0.45798414666954007, 0.49456758284086044, 0.5334559039931475, 0.5741468290138692, 0.6160666389721371, 0.6585989008274681, 0.7010981861787215, 0.7428776570396852, 0.7831697340374106, 0.8210504130008719, 0.8553050295160278, 0.8842223790771812, 0.905388255436531, 0.9156759127644752, 0.9115637662788135, 0.8899114827156299, 0.8492916676930367, 0.79112408768458, 0.7197044768558238, 0.6414810125830588, 0.5638862350785963, 0.49368080080250215, 0.43505082306854503, 0.38842107975002166, 0.35121976400959426, 0.31983143838410677, 0.2910815507893577, 0.26284337073882463, 0.23405833100276813, 0.2045084555722344, 0.17454748092715588, 0.14487449490290782, 0.11636489837939079, 0.08994650823319983, 0.06650357792553829, 0.04679554958045751, 0.03138522739485439, 0.0205810855286922, 0.014407022261839542, 0.012612349018088609, 0.014721910455687048, 0.02011140655071575, 0.02808865017040363, 0.03796674602773964, 0.049122027203775454, 0.06103387251578829, 0.07330545336191929, 0.08566526596696145, 0.09794974887450769, 0.1100680406982919 ], [ 0.4633826770434743, 0.49969522226580143, 0.5382806403215843, 0.5786669661946489, 0.6203135336449377, 0.6626331450100402, 0.7049988203590688, 0.7467244363589456, 0.7870154316754724, 0.8248794520385582, 0.8589762337488007, 0.887391317985325, 0.9074073450476886, 0.9155229852883899, 0.9078606560073536, 0.8810682618367375, 0.8338328662985182, 0.7680696199412865, 0.6886977276030325, 0.6027322009025519, 0.5181101788903963, 0.4421077272587152, 0.379408131160893, 0.33068516215212196, 0.2933322088681991, 0.2635304536551539, 0.23778252403686534, 0.21357840870606265, 0.18948177696829327, 0.1649466380356549, 0.14006513867364945, 0.1153389732098119, 0.09149958976937478, 0.06937208971648623, 0.049769274479094405, 0.03340421870408372, 0.020816357843248312, 0.012314653208548743, 0.0079481598788409, 0.00751344379698371, 0.010598089521435261, 0.016647821991087497, 0.02504066191504095, 0.03515512065059723, 0.04642507919843586, 0.058378027676560196, 0.07065540833987805, 0.0830147203428051, 0.09531341287763984, 0.1074748206412981 ], [ 0.4683110881384966, 0.5043392798748811, 0.5426113381370982, 0.5826831616292096, 0.6240430558788013, 0.6661274539513168, 0.7083207131023276, 0.7499286421772684, 0.7901198896899417, 0.8278248134218479, 0.8615734967827352, 0.8892571689936816, 0.9078836021275711, 0.9136291555963318, 0.9023304824043743, 0.8704790009040432, 0.8168832168629327, 0.7439249568873748, 0.6570890559573503, 0.5639018817095527, 0.47275543346074367, 0.39138551795522597, 0.32492553060904017, 0.2742625908954937, 0.2367577489721347, 0.2084291686242955, 0.1855064953840293, 0.16513996970976796, 0.14554505634764314, 0.12586301575564318, 0.10592918273253593, 0.08604665099893016, 0.06679920292811281, 0.048905148985630476, 0.03310249882948757, 0.02005572692095825, 0.010279545301241644, 0.004082188892359229, 0.0015359931441230046, 0.002482039061417196, 0.00656755142561305, 0.013305439434447064, 0.022141640834949183, 0.032518330351392066, 0.04392563868808119, 0.05593821379820385, 0.06823506541363233, 0.08060213279370498, 0.09291737232997588, 0.10511809286586737 ], [ 0.47276742759305546, 0.508494650443891, 0.5464379016085894, 0.5861780218394123, 0.6272278395906504, 0.6690415947494657, 0.7110078808738352, 0.7524160823769155, 0.7923891641341684, 0.8297725846374999, 0.8629638519185234, 0.8896696089263061, 0.9066554555543173, 0.9098347701571454, 0.8948362431839041, 0.8580666786873907, 0.7984733085358962, 0.7188575409464436, 0.6251921392593778, 0.5254465555478588, 0.428404062819806, 0.342183129006863, 0.27229623077906334, 0.21982047832661983, 0.18211779436607867, 0.15509663796433304, 0.13476649955079045, 0.1179825707568175, 0.10264349348289759, 0.08759538248485477, 0.07242364828613701, 0.05723301131370817, 0.042456579468708955, 0.02870193020303813, 0.016628695760624645, 0.006850025705620544, -0.00014609401423748558, -0.004054050907418505, -0.004780681951291776, -0.002443571426007707, 0.0026606472735563047, 0.010108804177812458, 0.019412040051904667, 0.03007407692610431, 0.041639698268540304, 0.05372952340437376, 0.06605923443154116, 0.07844246345479922, 0.0907769178166048, 0.10301316315871434 ], [ 0.4767505139526356, 0.5121569160536672, 0.54975093029401, 0.5891350611019381, 0.6298419817659294, 0.6713378557961424, 0.7130086130257849, 0.7541195055396925, 0.7937401364852502, 0.8306252425249478, 0.8630387094937277, 0.8885124066231458, 0.9036058327461765, 0.9040336176779996, 0.8852966755542309, 0.8438014930316069, 0.7786669430095323, 0.6930433151663209, 0.5932997661234558, 0.4877723098206421, 0.38556268504459895, 0.29508117737012185, 0.22211920055992418, 0.1679353250822576, 0.1299542431439572, 0.10403508687435159, 0.086021213581432, 0.07251794893278829, 0.061139825242561896, 0.05045744777851768, 0.03981524989596574, 0.029121397940384708, 0.018656284158227177, 0.008913219292188801, 0.00046985811854227677, -0.006115015083723363, -0.010382651955790134, -0.012032525534578342, -0.010953588650923107, -0.00722564940958792, -0.0010930325632113602, 0.007081396155028674, 0.016870901524196347, 0.027838298999361633, 0.03958114279767333, 0.05176460610430522, 0.06413995858590571, 0.07654761236110119, 0.08890409072850236, 0.10117210656360931 ] ], "zauto": true, "zmax": 0.9574264468218647, "zmin": -0.9574264468218647 }, { "autocolorscale": false, "autocontour": true, "colorbar": { "tickfont": { "size": 8 }, "ticksuffix": "", "x": 1, "y": 0.5 }, "colorscale": [ [ 0.0, "rgb(255,247,251)" ], [ 0.14285714285714285, "rgb(236,231,242)" ], [ 0.2857142857142857, "rgb(208,209,230)" ], [ 0.42857142857142855, "rgb(166,189,219)" ], [ 0.5714285714285714, "rgb(116,169,207)" ], [ 0.7142857142857143, "rgb(54,144,192)" ], [ 0.8571428571428571, "rgb(5,112,176)" ], [ 1.0, "rgb(3,78,123)" ] ], "contours": { "coloring": "heatmap" }, "hoverinfo": "x+y+z", "ncontours": 25, "type": "contour", "x": [ 1e-06, 1.3011511650442548e-06, 1.6929943542960217e-06, 2.2028415765056147e-06, 2.866229883678204e-06, 3.729398352432554e-06, 4.852511011181743e-06, 6.3138503555892e-06, 8.215273746089953e-06, 1.0689313005882422e-05, 1.390841207112662e-05, 1.809694657026198e-05, 2.354686311364001e-05, 3.063802837345029e-05, 3.986470631277378e-05, 5.1870009063012666e-05, 6.749072272319499e-05, 8.781563250096393e-05, 0.00011426141253772723, 0.00014867137004306603, 0.00019344392634026088, 0.0002516997901283655, 0.0003274994751669172, 0.00042612632366481585, 0.0005544547624925005, 0.0007214294601814526, 0.000938688782612345, 0.0012213760031100258, 0.0015891948094037057, 0.002067782677737912, 0.002690497840197013, 0.0035007443993213955, 0.004554997653699184, 0.005926740503884541, 0.007711585311544345, 0.010033938212454076, 0.01305567039511669, 0.01698740074503987, 0.02210317627048227, 0.028759573555516532, 0.03742055263793628, 0.04868979566145066, 0.06335278435066323, 0.0824315491666629, 0.10725590623460621, 0.13955614735503497, 0.18158364372009145, 0.23626776957937787, 0.3074200836506151, 0.39999999999999997 ], "xaxis": "x2", "y": [ 0.0, 0.02040816326530612, 0.04081632653061224, 0.061224489795918366, 0.08163265306122448, 0.1020408163265306, 0.12244897959183673, 0.14285714285714285, 0.16326530612244897, 0.18367346938775508, 0.2040816326530612, 0.22448979591836732, 0.24489795918367346, 0.26530612244897955, 0.2857142857142857, 0.3061224489795918, 0.32653061224489793, 0.3469387755102041, 0.36734693877551017, 0.3877551020408163, 0.4081632653061224, 0.42857142857142855, 0.44897959183673464, 0.4693877551020408, 0.4897959183673469, 0.5102040816326531, 0.5306122448979591, 0.5510204081632653, 0.5714285714285714, 0.5918367346938775, 0.6122448979591836, 0.6326530612244897, 0.6530612244897959, 0.673469387755102, 0.6938775510204082, 0.7142857142857142, 0.7346938775510203, 0.7551020408163265, 0.7755102040816326, 0.7959183673469387, 0.8163265306122448, 0.836734693877551, 0.8571428571428571, 0.8775510204081632, 0.8979591836734693, 0.9183673469387754, 0.9387755102040816, 0.9591836734693877, 0.9795918367346939, 1.0 ], "yaxis": "y2", "z": [ [ 0.179400075993246, 0.16986472833479727, 0.16118011152493308, 0.15140505899949996, 0.13874125946948995, 0.12193379751252974, 0.10046805936834943, 0.0746304563773537, 0.0455157795495788, 0.015140088643408808, 0.015108085161260462, 0.04041061076675189, 0.059850335897321444, 0.072259056032669, 0.07749998041198673, 0.07606866911422108, 0.06877269957103135, 0.05653546945858099, 0.040503036044888974, 0.022489145331575373, 0.005942812523019672, 0.008118485049219817, 0.011280793689953236, 0.005027232468573694, 0.012114687198996265, 0.03158101960902414, 0.051501161789839225, 0.06833370692869532, 0.07945659480562815, 0.0830473188748432, 0.07806913404809356, 0.06427577814222744, 0.04223931820638134, 0.013552498161215452, 0.020732152374302752, 0.056046437739227586, 0.09101845136897282, 0.12382717926878363, 0.15334325787044345, 0.17899001996723732, 0.2006734913161448, 0.21870844618619562, 0.23372847526043514, 0.24657212316058172, 0.2581453540364038, 0.26927543242787827, 0.28058598313661026, 0.29242549465824214, 0.30486531764463376, 0.3177568041271435 ], [ 0.16654306960090728, 0.1579081024542236, 0.15075036681361806, 0.14283916287195836, 0.13212760998540027, 0.11722411294006368, 0.09759073526138654, 0.07360153245125327, 0.04666648982000867, 0.02062688049229167, 0.018746798452297844, 0.03941282356076232, 0.05685641868216288, 0.06792790681856363, 0.0723341481740611, 0.07064875226307055, 0.06374819071190116, 0.05249785842216983, 0.03785821809118114, 0.021535559569162338, 0.0081188488240183, 0.010135584818199723, 0.012425433471151551, 0.007029421092886347, 0.012504402793429063, 0.0310901749446387, 0.05067446057162323, 0.06739793042786263, 0.07859872561349994, 0.08245290527870648, 0.07794868025997563, 0.06493172615571335, 0.04434417914159371, 0.020513920618731477, 0.02535451899315621, 0.05680424753852339, 0.09016120472479315, 0.12174880487845463, 0.1501379211476792, 0.17467190165291727, 0.19524561448709182, 0.2122018129736388, 0.22623174642949886, 0.23825134168420034, 0.24924694607960254, 0.2601070075589949, 0.2714773349089729, 0.28368194015770437, 0.29672938231148327, 0.3103889047045012 ], [ 0.1532614965186217, 0.1456785460682401, 0.14031906215681506, 0.13459104350548695, 0.12616431955814655, 0.11352840583558099, 0.09618623680052159, 0.07472839698052118, 0.051152675719751475, 0.03082598003351316, 0.027837893503787903, 0.04154141475176565, 0.05548663347478012, 0.06441350961452041, 0.067513387764718, 0.06534583611260382, 0.05884920998680938, 0.048820493388681135, 0.03600918378163734, 0.022076518361115665, 0.012119039389734004, 0.013034874074178606, 0.014092470381970585, 0.009543413602906035, 0.013958596871998014, 0.031313169220784275, 0.05045618228253791, 0.06708092621509819, 0.07847323256647958, 0.08283485936338199, 0.07924371095225677, 0.06778800976256345, 0.05016324800207539, 0.032953913198473336, 0.036271965102923215, 0.061399890026406855, 0.09165790738112177, 0.1212012041867265, 0.14793262641901408, 0.17097158215014924, 0.190137628970489, 0.2057713399409803, 0.21861008860599718, 0.22964741910962388, 0.2399561592237284, 0.25049095250369324, 0.2619176715181774, 0.2745248592421559, 0.2882418542726249, 0.30273957769235543 ], [ 0.13956476846388086, 0.13320034530527783, 0.12993829662100556, 0.12671720916384246, 0.12087496392053791, 0.11079630631914558, 0.09606726007043725, 0.0775270253704552, 0.05761115879826252, 0.04161124666387935, 0.037840938578765124, 0.04577056626917976, 0.055488493608164005, 0.06161574580229595, 0.06294093064975963, 0.060018732145483894, 0.053880966977851816, 0.045234920031860776, 0.034496802541531396, 0.023022147468354022, 0.015240698319833232, 0.01515026049823564, 0.015116305227294723, 0.010857543155890147, 0.01506687495678297, 0.031678707953264246, 0.05050804580821053, 0.06713204384623726, 0.07885811580296684, 0.08395578951757338, 0.08163451689962228, 0.07226522660149338, 0.05815230215729472, 0.046059134020982585, 0.04888000180701025, 0.06866600261699839, 0.09513380323413241, 0.12202831282297051, 0.14665362363971768, 0.1678525875617119, 0.18533034481057328, 0.19940378283940988, 0.21084929035399078, 0.22074227436634702, 0.23025241743727676, 0.2404088488304049, 0.25189597522741664, 0.2649536266839852, 0.2794121401573108, 0.29482544554307305 ], [ 0.12546943846512915, 0.12051206822812974, 0.11968095663326861, 0.11928444879808198, 0.1162696190172644, 0.10893371673111049, 0.0969694145736923, 0.08142457468103376, 0.06488950669166521, 0.051889562545227114, 0.047458892822084785, 0.05103930585057002, 0.05651223853951813, 0.05942724223315108, 0.058548292157080004, 0.05455453086479054, 0.04867248546004984, 0.041505163193974765, 0.03292510791476942, 0.02361921800221285, 0.017044405290915568, 0.01609187207528345, 0.015154004362835686, 0.010785696594628393, 0.015351748155311106, 0.03185191385437271, 0.05058739910075461, 0.06734546843738072, 0.07955018109943975, 0.08557726505863716, 0.08478006846353374, 0.07776342518210931, 0.0671090728382734, 0.058831885071072704, 0.0616036214108983, 0.07743536887290907, 0.1000882485207407, 0.1239992432444273, 0.14618699027307852, 0.16525809661579985, 0.18079573953316883, 0.19308350129064197, 0.202935699114597, 0.21151915518638426, 0.22011656447991843, 0.2298443969871519, 0.2414049368383578, 0.25497306456256885, 0.27025609143727114, 0.2866695202065789 ], [ 0.11100061539021785, 0.10767315874331777, 0.10964906092621776, 0.11237222866513961, 0.11234497389833326, 0.10781238021073095, 0.09859259369611117, 0.08588197953291196, 0.07221951261088909, 0.06128948599144077, 0.0562755303954206, 0.05658318843049279, 0.05819558318734608, 0.05775235088837459, 0.054315292789191394, 0.04888822180864415, 0.0430943898628313, 0.03746065534965814, 0.03103206792246733, 0.023503832709933357, 0.017500688343540124, 0.015843394498995477, 0.014247653647326395, 0.009483673031416381, 0.014848941934107865, 0.031719984557992346, 0.05055305595463932, 0.06756943537495697, 0.08037808524832886, 0.08748174483652563, 0.08836391786480621, 0.0837735845911658, 0.07628732682487337, 0.07095257779051474, 0.07388054822892977, 0.08684032414810003, 0.10600583387383297, 0.12684049057121013, 0.14638847972571126, 0.16311434068039635, 0.17649869677458307, 0.18679397377190815, 0.19485769645170534, 0.20196383487961375, 0.20953170593392367, 0.21878420358137157, 0.2304420241804874, 0.24459476756300433, 0.2607972752481206, 0.2783021455640611 ], [ 0.09619347888628135, 0.09477461279645404, 0.09998455827804283, 0.10607388467444026, 0.10908515187720612, 0.10728337480975904, 0.10063950673869601, 0.0904542023304719, 0.07912467067674545, 0.06963931740797705, 0.06411310982236135, 0.06192662487412158, 0.0602249406133651, 0.05652069066988229, 0.050287723473421406, 0.043023846162535496, 0.03707098911796634, 0.03301761162108521, 0.02871278393052745, 0.02259370383489261, 0.016721785734910233, 0.014539604469759259, 0.012658153610727304, 0.007306326695194666, 0.013876405970171667, 0.03132096935800106, 0.0503435970655118, 0.06770117733644236, 0.08120552220252963, 0.08948376430519804, 0.09211770264287161, 0.08990683738883956, 0.08524556110898897, 0.08226848998399548, 0.08545251877337054, 0.09630286023145244, 0.1124270866752746, 0.13026783467654765, 0.14709556038278981, 0.16133516305303655, 0.1723992180719547, 0.18051958411554114, 0.18660744031931015, 0.1920660193875528, 0.19848414404161788, 0.20721865464268813, 0.21901075309200044, 0.23383868732036625, 0.2510684198159606, 0.26976197429357807 ], [ 0.08109493698399943, 0.0819576708695704, 0.09088237121480798, 0.10049528765436253, 0.10646301535917779, 0.10719137851540889, 0.10284223944820321, 0.09480062154999527, 0.08531061056641547, 0.07684444369464427, 0.07088155496550153, 0.0667877181547732, 0.062358953028646164, 0.05569113131178099, 0.04658932033681292, 0.03706253940496537, 0.030587571343958914, 0.028194228004398, 0.026018465910115533, 0.021004995644624748, 0.0149161769491939, 0.012408827004235281, 0.010818625652382063, 0.004922341630902459, 0.012903840618263048, 0.030758271015642262, 0.049944408273128515, 0.06767459637822665, 0.08192812621342291, 0.0914327195018741, 0.09582700061215013, 0.095881538812358, 0.09371803519904078, 0.09268909056083634, 0.09617548854843254, 0.1054463896012835, 0.11897482605517044, 0.13401060639029086, 0.1481393225446653, 0.15982712099695223, 0.16845491437116672, 0.17424760340390985, 0.1781828791315593, 0.18182099743026522, 0.18696440350359883, 0.19514288026407028, 0.20712222515561843, 0.22273504435329516, 0.24111304566481456, 0.261096940920804 ], [ 0.06576560170085204, 0.06944907503507661, 0.08260348178292379, 0.09574936786448325, 0.10444165140672602, 0.10738694297024803, 0.10497555369756216, 0.09867214093577786, 0.09059326942276869, 0.0828491848647841, 0.07653237923638338, 0.07100138445000945, 0.06442484278679067, 0.055245126306313924, 0.04342094055128214, 0.03124880583538755, 0.023696521018635282, 0.023132935888881948, 0.023149982196278656, 0.019009718167871916, 0.012394206908112581, 0.009771718027235508, 0.009303908881291298, 0.0038352724953288325, 0.012353406118694404, 0.03012022785354551, 0.04935628499022996, 0.06744690728488419, 0.08246779863697283, 0.09321080613606487, 0.09932759060412188, 0.10149944156182554, 0.10154032903091481, 0.10215570174516414, 0.10595931511399173, 0.11402241973511427, 0.1253531933721404, 0.13782612895755844, 0.14935450004292744, 0.15849450419366468, 0.16462355956218716, 0.16797027752927407, 0.16959005249702538, 0.1712315940236104, 0.17496835184410117, 0.18255782344913368, 0.19479703386597713, 0.21132665968286557, 0.2309872821916455, 0.2523651693455211 ], [ 0.05028276398096596, 0.057631206727267105, 0.07548087883721183, 0.09194513480852565, 0.10297575407068382, 0.10773523785012665, 0.10686023318307306, 0.10189264001416579, 0.09485740242591215, 0.08762165877491777, 0.08104058838103782, 0.0744713028353096, 0.06630262342195503, 0.055171480097980674, 0.04103374778530954, 0.026050520549073093, 0.016534615543893846, 0.018159570243203752, 0.02044849912643768, 0.01700597930980062, 0.00962083493227017, 0.007116215945680894, 0.008656419537154538, 0.0051157954588295345, 0.012318094127270167, 0.029426441937434926, 0.048574759471769584, 0.06698845733823314, 0.08276711996727946, 0.09472905034922732, 0.1024982214543952, 0.10662411458632508, 0.10860944890025948, 0.11063008704227077, 0.11474462453328438, 0.12186348655620872, 0.1313355381167565, 0.14150576904515208, 0.15058673788256122, 0.15724375795638956, 0.1608654930948037, 0.16168692121510753, 0.16084570929461334, 0.16031053491627492, 0.16249842916578291, 0.1694714402770397, 0.1820677028285873, 0.1996718269966948, 0.22076184441933489, 0.24363571980657833 ], [ 0.03474757027323975, 0.04717984796240272, 0.06990231130018545, 0.08917116294640645, 0.10201273567542381, 0.10812102340799003, 0.10836070154802009, 0.1043422470546229, 0.09803299299213787, 0.09114647772260202, 0.08439595586503347, 0.07714146758986233, 0.06790721327662246, 0.05544736789668903, 0.03966269193450132, 0.02224851595453734, 0.009429156030288276, 0.013944318517872315, 0.01835598854834898, 0.015448300601198844, 0.007345701801419598, 0.0053303112243783665, 0.00897045523525782, 0.007035516968697191, 0.012489535323818586, 0.028621782147534348, 0.04758455875456514, 0.0662774994164379, 0.08278520975137461, 0.09592323050119184, 0.10525318914908238, 0.11116469583137004, 0.11486220485855814, 0.11808914327765481, 0.12249233799709376, 0.1288547882729549, 0.13675024228346425, 0.1448753314603013, 0.15169702850062977, 0.15598698326959753, 0.15714568738849946, 0.15540591093611564, 0.15198029331016266, 0.14908340985304355, 0.14956502413040473, 0.15590008605007208, 0.16898192352964334, 0.18784787636291925, 0.21052409697566665, 0.23498903515214398 ], [ 0.019319988311653123, 0.03927527675984598, 0.06624688702361058, 0.08747612119902855, 0.10149356823846922, 0.1084504745947913, 0.10938019175632818, 0.10594441181014377, 0.10008161382690478, 0.0934207187607007, 0.08659821951308194, 0.07897941974355145, 0.0691733105632904, 0.05602193677835656, 0.039428929874137056, 0.020789808907254112, 0.0041984732956752285, 0.011721864818187683, 0.01727832395756511, 0.01467082541938276, 0.006555229851551522, 0.005456410268033836, 0.009797621924737477, 0.008456438295752405, 0.012423154243918868, 0.027617511078032914, 0.04636731665476959, 0.06529939808019883, 0.08249532137274476, 0.09675052731426395, 0.10753606419469444, 0.11506450023208517, 0.12026302236867092, 0.12452194452460884, 0.1291781746654359, 0.13491705477786542, 0.14146813516481227, 0.14779231823223773, 0.15256373469359732, 0.15464437871962147, 0.1534353399618662, 0.14914645754955028, 0.14304136383806085, 0.13759255968951695, 0.13618807687182802, 0.14187019694350195, 0.15560703696425776, 0.17595559924655949, 0.20038004678011762, 0.22651688938984146 ], [ 0.00491452670183538, 0.0355788734578853, 0.06476876126377468, 0.08685182459803616, 0.1013535062097169, 0.10865081992791488, 0.10985545578715308, 0.10665664555763268, 0.10098842479056627, 0.09445159481724405, 0.08765437056739507, 0.07996655236265703, 0.07004485620593899, 0.05680807880733472, 0.04026347658238301, 0.022032889471303954, 0.007565614366140679, 0.012531081877564342, 0.0173441301461208, 0.014665915821673757, 0.007325537634822415, 0.006891196723014562, 0.01061202587953103, 0.00905880327596746, 0.011807956052922881, 0.026352332576587978, 0.044915039190537985, 0.06404816938317308, 0.08188381335444594, 0.09718713773078845, 0.10931488252395277, 0.11829324932876159, 0.12479674144112081, 0.12992784170979207, 0.1347893674195322, 0.13999590642972684, 0.1453924010886533, 0.15014182676383878, 0.15308281477900218, 0.1531456496609734, 0.14971290164425796, 0.1429400197093287, 0.13409751850989282, 0.12590244830404215, 0.12239908104934447, 0.12742049136772107, 0.14203650616665542, 0.16412467828765032, 0.19045596272540835, 0.21832157732727683 ], [ 0.011953858156970205, 0.03711774325964419, 0.06547710370421062, 0.08722522599575105, 0.10152290713529394, 0.10866871930234535, 0.10975201003919607, 0.10646425562203035, 0.10075755283023945, 0.09425522287213146, 0.0875773499874784, 0.08009302379741018, 0.07046941037798239, 0.057684811610479926, 0.041912050122018145, 0.025247876764108074, 0.013859029505079992, 0.015549560897744868, 0.018296107953598238, 0.015087771959817756, 0.008502280886954011, 0.008323519075115958, 0.011098515347973075, 0.00884328220975012, 0.01056772695613024, 0.024839453896876206, 0.043239784488415554, 0.06252791264820254, 0.08094997293821056, 0.09722680182017698, 0.11057867521452396, 0.12084181249892444, 0.12846416080900916, 0.1343151145088339, 0.13932248692558397, 0.1440549890087521, 0.1484508174129551, 0.1518321441436347, 0.15316686164170928, 0.15143052025290504, 0.14596450516345855, 0.13683118749285975, 0.12524283448379658, 0.11410748676742323, 0.10824386071157249, 0.1126051705981415, 0.12839964150213015, 0.15252011559312556, 0.1808990994419249, 0.21051402588006446 ], [ 0.02637018704752889, 0.042970651286287624, 0.06810420845733269, 0.08846299549584793, 0.10192832124412746, 0.10846808101599337, 0.10906029822429349, 0.1053764166162461, 0.09940967882274239, 0.09285622827736058, 0.08638584780599103, 0.07935591750361504, 0.07039698656226084, 0.05850866675047023, 0.044015899049049526, 0.029334042467646444, 0.019896583776538516, 0.019309427444273736, 0.01969192923453221, 0.015518472759419179, 0.009183336995818104, 0.009133436844595944, 0.0111461716436446, 0.00801582155113663, 0.008881863450302464, 0.023176877889355775, 0.04137260270521738, 0.06075269245022363, 0.07970639066082828, 0.09688010786885422, 0.1113350694731874, 0.12271863701054073, 0.13127916511382962, 0.13769996591854392, 0.14278189917356596, 0.1470713747219459, 0.15058987630662127, 0.15279058673757723, 0.1527434673764587, 0.1494485467086459, 0.14218381071755798, 0.13087782209361, 0.11660167204213527, 0.10234396226860523, 0.09378698740681757, 0.09749921216130893, 0.11487667938392118, 0.14134921620948834, 0.17187669744169135, 0.2032104820363858 ], [ 0.04040060451100474, 0.0512627027023556, 0.07219375441209466, 0.090387030205654, 0.10249389035525934, 0.10802774972127678, 0.10779281471411714, 0.10342407301704248, 0.09698122435175976, 0.09028804864302603, 0.0841050184328189, 0.07775980654441336, 0.06978243971705704, 0.059129827131774136, 0.04620544174357941, 0.033470093759911126, 0.02523286376161021, 0.022980899393390278, 0.021159557997609168, 0.015699615008695805, 0.009013076519346547, 0.009133916016146054, 0.010774930672350792, 0.006946706611901164, 0.007205628769122271, 0.02151074618760701, 0.03935092860085467, 0.0587453103237966, 0.07817997570029095, 0.09617443564110732, 0.1116086682532892, 0.1239472916819619, 0.13326681264535536, 0.14010577329186422, 0.14517864692532906, 0.14903239001498464, 0.15177035400769015, 0.15295982979309183, 0.15175330267126172, 0.14715845770437255, 0.13837133776011798, 0.1251501900684585, 0.10833326076038953, 0.09080878230282982, 0.0791200016742998, 0.08220847060323945, 0.10172249918854621, 0.13086766294039243, 0.16357208824441444, 0.19652748233860948 ], [ 0.0537787019343617, 0.06055970420424241, 0.07723587581120638, 0.09279505328666025, 0.1031429444567784, 0.10733925778637558, 0.10598204877031475, 0.10065931704460471, 0.0935248527373824, 0.08659384693114298, 0.08076794935683594, 0.07531912574553261, 0.06859029890808119, 0.059408534607117466, 0.04816025670910884, 0.037156371888692766, 0.029691931871562636, 0.02619081267393798, 0.02248764652277105, 0.015605444489761134, 0.007991693268889419, 0.0083701151996254, 0.010066561089002593, 0.006066086437223331, 0.0062055930694740625, 0.019956840073857576, 0.03720121862903318, 0.05653725142505337, 0.07641396458836124, 0.09515436674520118, 0.11143991871441337, 0.12456472420476074, 0.13446204119432942, 0.14156256951051616, 0.14652966941766232, 0.14993340307869268, 0.15196398373004483, 0.15229481066778933, 0.15014819229179566, 0.14452725461703328, 0.13453340722476306, 0.11972880198970366, 0.10063461069492802, 0.07978892378701519, 0.06437948430044678, 0.06689210291659174, 0.08930312885407936, 0.1213812180612617, 0.15617650489200194, 0.19057498700913528 ], [ 0.06637643302924556, 0.07002649118543626, 0.08276446230818749, 0.09547996233728007, 0.10379957732648445, 0.1064046521341507, 0.1036790005836031, 0.09715599495485312, 0.08911121895855532, 0.08182797074770828, 0.07641772335336668, 0.07206190538872441, 0.0668009523894251, 0.05922880472050107, 0.0496322671966947, 0.04009883683301907, 0.03317432174697252, 0.028770063642427984, 0.02359712046984172, 0.015415037047713019, 0.006461624743012406, 0.007091786603002075, 0.009114239678695681, 0.005577011808933778, 0.0062034498605518735, 0.018517424648560273, 0.034929423910717364, 0.05417283331041448, 0.07447113981693664, 0.09388226907757247, 0.11088417475465079, 0.12461994857077811, 0.13490880312931988, 0.1421067632189351, 0.1468573402658439, 0.14977631444736145, 0.15115098523252163, 0.15076021395598652, 0.14788938745106275, 0.14152930133472857, 0.130680876329078, 0.11470069786198141, 0.09373877468288928, 0.06970473259383791, 0.049794495537253604, 0.051821211025444236, 0.07814613144458274, 0.11323649354320613, 0.14987540500742763, 0.18544790864592411 ], [ 0.07809741231498708, 0.07918837907385669, 0.08839578435109932, 0.09824405782365538, 0.10438995794035312, 0.10523429481275094, 0.1009519350155641, 0.09301135321849932, 0.08383211504615903, 0.0760579402415437, 0.07110993249384362, 0.06803458006777434, 0.06441728432185542, 0.05850834619129798, 0.05044763112794135, 0.04212649642832516, 0.03562299472613778, 0.03064131153723303, 0.02447741036855513, 0.015409163964382898, 0.0053272056270402485, 0.005827390050115459, 0.008013292662934338, 0.005241354782639114, 0.006643131823434739, 0.017059962035868927, 0.03253169312567012, 0.05171944216645556, 0.0724378104848376, 0.09243854544925485, 0.11001064436417214, 0.12417295751281004, 0.13465953155786783, 0.14178113304779633, 0.14618934480351728, 0.1485686179901616, 0.14931828265990638, 0.14832851715876633, 0.14494618209717505, 0.13814562772409922, 0.12682790517363762, 0.11015407528210547, 0.08790355487366795, 0.061159098241570065, 0.03583900229880027, 0.037563029541949684, 0.0689864922280805, 0.10679143655047078, 0.14482919027204186, 0.18121676529400763 ], [ 0.08886404553978584, 0.08775946615666529, 0.0938294376314068, 0.10090729784285377, 0.1048431894260586, 0.10384448809677467, 0.0978849548840783, 0.08834852463711379, 0.07780544035083943, 0.06936705511137935, 0.06491556882494993, 0.06330782111189787, 0.061471130626818755, 0.057204868841519124, 0.05050057045949964, 0.04314748500219704, 0.037011364142576685, 0.03176964591845566, 0.025126254049434187, 0.015793712779843063, 0.0058507563450952, 0.005393444948922404, 0.006907629953404341, 0.004690688446847361, 0.006805379915792008, 0.015390415671740503, 0.030029874773548768, 0.04928261591562072, 0.07042701548545789, 0.0909207936116317, 0.10890090862091976, 0.12329371336245401, 0.13377489780673, 0.14063515379908748, 0.14455895065506835, 0.14632297796908578, 0.14645830023136977, 0.1449785675773686, 0.1412949886410262, 0.13436366594601346, 0.12299104070992672, 0.10617148557684711, 0.08338490396745939, 0.05494166008669992, 0.023837108097445752, 0.025661937779137615, 0.06273033751705354, 0.10235840435574253, 0.14115038897107876, 0.1779187236520122 ], [ 0.09861436905456918, 0.0955554837331699, 0.09883480554565453, 0.10331063870651898, 0.10509163143404063, 0.1022547912346513, 0.09457588959391133, 0.08331952135300225, 0.07118286951206335, 0.06185797596998106, 0.057924390337759736, 0.05798371543821474, 0.0580291495563488, 0.055319481467593204, 0.04974587828103077, 0.043126602730403016, 0.03733770511265172, 0.032140219182154824, 0.02551169548241359, 0.016543425947430815, 0.007794806921007593, 0.006163907208267324, 0.006052973757972304, 0.003864130536094275, 0.0063825626637904085, 0.013376253928972287, 0.027525028070002924, 0.04702021761571612, 0.06857826912796819, 0.08944099190022446, 0.10764673609792352, 0.12206112101989232, 0.1323238596820144, 0.13872573473248032, 0.14200575773338317, 0.14305732460605178, 0.14256826665681052, 0.14069466190948804, 0.13691897565485994, 0.13017763679244376, 0.1191889236548998, 0.10282232427537187, 0.08039297807626028, 0.05185920138373838, 0.01804904613457904, 0.020503147560486015, 0.060189100726485784, 0.10012666760242855, 0.13888211178626028, 0.1755506468817301 ], [ 0.10730097414013103, 0.10245153301820867, 0.10323557902801155, 0.10531613165152959, 0.10507071788987031, 0.10048497172690814, 0.09113292842232189, 0.07810803629277283, 0.0641617875432766, 0.05365831443440022, 0.05024930253629572, 0.05220521881086838, 0.054197760902948956, 0.05289781930031284, 0.04819259556034975, 0.04207392087975972, 0.03662327043539836, 0.03175123299752295, 0.02556517801627307, 0.01740847154542777, 0.009977507134754973, 0.007502944894451487, 0.005738319287261909, 0.003235418379375614, 0.005446029248254832, 0.011065148576728146, 0.025251076195361724, 0.0451451713182361, 0.0670506574464252, 0.08811997301426372, 0.10634702120740139, 0.12056193285744987, 0.1303840229629706, 0.13611847076047853, 0.13857705240633408, 0.13879552222864272, 0.13764998942380713, 0.13546610373652906, 0.13180837576380397, 0.12558980703347683, 0.11544289956399384, 0.10015588719000341, 0.0790400053578185, 0.0523171286322214, 0.02316461901642394, 0.025921168052867396, 0.06159613539670516, 0.10009460266475709, 0.13798425649252932, 0.1740656777651165 ], [ 0.11489066064653734, 0.1083610539133331, 0.10689682757392059, 0.10680527782234142, 0.10471840175002257, 0.09855166039810484, 0.08766943719685859, 0.07293057253252519, 0.05700395768958619, 0.044931178534375546, 0.042033531443772365, 0.04616974198086788, 0.05012651906966485, 0.05002911518968247, 0.045899381447648976, 0.040039459960676815, 0.03491331761370997, 0.0306174770698028, 0.025202031988464924, 0.018057839548586842, 0.011689562392615522, 0.00866744466059841, 0.005943579824462873, 0.0034932332789896605, 0.004344604927828167, 0.008822935288436222, 0.02359209681704592, 0.04390386552095668, 0.06600713204939616, 0.08707901631561624, 0.10510385599606852, 0.11888957314952854, 0.12804234086938632, 0.13288952297371684, 0.1343299415650408, 0.13356872592129482, 0.13171008573094678, 0.12928721936110546, 0.12596160298910652, 0.12061285381534328, 0.11177873162982364, 0.0981965838397226, 0.0793027189923651, 0.055985911579693376, 0.0341265213682505, 0.03703448375525337, 0.06637495162240589, 0.10205180907731422, 0.1383326654028201, 0.1733742665904912 ], [ 0.12136450538502742, 0.11322501104362072, 0.10971543103870723, 0.10767671895738788, 0.10397442626228422, 0.09646493826495531, 0.08429657991301082, 0.06803303550502932, 0.05006353457640999, 0.035899748915350166, 0.033466387094393917, 0.04014976932602654, 0.04600835840429957, 0.046842734734518415, 0.04297126140392541, 0.037111566575984835, 0.03228028672943189, 0.02878069710756555, 0.024355652801298405, 0.018213031953973422, 0.012587854789369699, 0.009219388330821067, 0.00623024112963192, 0.004301744006535621, 0.003625458206634994, 0.007525886771403448, 0.022995915289046164, 0.04352372467203944, 0.06559115381445063, 0.08642936761263482, 0.10401797925261838, 0.11714288829220564, 0.1253961474045099, 0.12912824719230687, 0.12933451278071767, 0.12741763488163685, 0.12476068621888602, 0.12215782147843691, 0.11938739235937133, 0.11527360598343936, 0.10822946243235909, 0.09694269966606099, 0.08102238478562424, 0.06200148770073703, 0.046483071520769616, 0.04963230320399889, 0.07348563262705912, 0.10562547651238109, 0.13973207953023503, 0.17334955642648808 ], [ 0.1267182770793473, 0.11700631839983366, 0.11161347501516797, 0.10784394032397225, 0.10277965315659054, 0.09422524237892574, 0.08111383946532107, 0.06367811422137537, 0.04382183110668608, 0.02691793079097079, 0.02482930013700914, 0.03452240619050223, 0.04207320980863759, 0.04350073810179533, 0.03955689154030625, 0.033417879633889425, 0.02882819472640302, 0.026323146796461343, 0.023010761766156254, 0.01771165992920491, 0.012523495594735222, 0.008945493476929937, 0.006126275184615817, 0.004867788867824595, 0.0036860383116432867, 0.008164264714682496, 0.023752121405097446, 0.044144740592069215, 0.06590174936044559, 0.0862616047729559, 0.10318407611387856, 0.11542481628336011, 0.12255444563813434, 0.12494066105913162, 0.12367836304870936, 0.12039600649129353, 0.11682067455473766, 0.11408412947957593, 0.11210832644881194, 0.10961848996434818, 0.10483925433620919, 0.09636927960911111, 0.0839445519007668, 0.0694388439850882, 0.058874889881111565, 0.0622966769105394, 0.08191222688506666, 0.11036330885795305, 0.14193870195981223, 0.1738360521857122 ], [ 0.13096320143829487, 0.11968721389327848, 0.1125340104819912, 0.10723337208644418, 0.10107567284548279, 0.09182109280389276, 0.07819834664591603, 0.06011799144521982, 0.03890144697483543, 0.01871811580633969, 0.016672620081687396, 0.029796787406691436, 0.03856866240483438, 0.040183932494700716, 0.035844653612014686, 0.029128661556957362, 0.02469745803683183, 0.023381195531683673, 0.021226258681444597, 0.01652487352328474, 0.011485166410715005, 0.007808316272139217, 0.005415862262945183, 0.00481283627266234, 0.004310430730091302, 0.010470335945325432, 0.025806449013185757, 0.045773867769768065, 0.06697648314959152, 0.08663745023463988, 0.10268654352325846, 0.11384091885882862, 0.11963923500853145, 0.12045374812523627, 0.11747295970313325, 0.11257607458171388, 0.10791761812342099, 0.1050801997735052, 0.10416641520026682, 0.10372107816213645, 0.10166775427354902, 0.09643453250928173, 0.08777668795271731, 0.07757578176179815, 0.07083190109160092, 0.07450512516627442, 0.0908780505336556, 0.11581070591305322, 0.14468557635164392, 0.17465991255519384 ], [ 0.1341271011333146, 0.12126851853873824, 0.11243869293000985, 0.1057830909057059, 0.09880488423054366, 0.08922816711631691, 0.07559492957314552, 0.057550700985746196, 0.035972925452786235, 0.013287542245644617, 0.0106466903151073, 0.026578724476340677, 0.03572022443913537, 0.03706836547588274, 0.03205424623852235, 0.02446345924852491, 0.020070328695936272, 0.02015663488471996, 0.01914196636904075, 0.014758776339882648, 0.009584802338320472, 0.0059625985240399915, 0.00425518661417126, 0.0041542169418751565, 0.005222413769686507, 0.013530443272391604, 0.0288382387231022, 0.048294600937448036, 0.06878995653939626, 0.08758631963754351, 0.10259631717491882, 0.11249764256348282, 0.1167864529117783, 0.11582038079172638, 0.11086241287139016, 0.10405703123388206, 0.09809074705252403, 0.09517002550300611, 0.09563199045719498, 0.09769215678093504, 0.09879417741514554, 0.09708815129923716, 0.09223782657866347, 0.0859192593963678, 0.08213955317054188, 0.0860101253175718, 0.0998439254343079, 0.12155879922724262, 0.14770483549197272, 0.17563917876978152 ], [ 0.13625593540932673, 0.12177030243838935, 0.11130699781257342, 0.10344221642550089, 0.09591116942710673, 0.08641015431809257, 0.07330952768897868, 0.056070010554763905, 0.035457736372906355, 0.014068980457022493, 0.010823729681922302, 0.02534628134118674, 0.03367160995463588, 0.0342912250708538, 0.028417626380751917, 0.019704324453281404, 0.01518066451392487, 0.016922681679956288, 0.016961859920806274, 0.012652534326108805, 0.007079356868558421, 0.003906072281559486, 0.0033971350480920975, 0.003327023435785155, 0.006486158471481833, 0.016852035601190474, 0.03248400149143355, 0.05152205074958629, 0.0712681235033834, 0.08910748786071665, 0.10296914884390723, 0.1115001038394145, 0.11414583885318723, 0.11122420651734806, 0.10403523240591601, 0.0949786907567052, 0.08739581788315759, 0.0843907291518977, 0.0866183506108219, 0.09169249811449645, 0.096319899577167, 0.09827955031078693, 0.09708796293748458, 0.09414950010719456, 0.09268060620382051, 0.09667117131847251, 0.10844470428342091, 0.1272636664617779, 0.15074372628365795, 0.17659267539632748 ], [ 0.13741574701535372, 0.12123378512988715, 0.1091368937543989, 0.10017104372370965, 0.09234121271187283, 0.0833216194055328, 0.07130846350009802, 0.05563258490761723, 0.03720503325602411, 0.01984185277530022, 0.016422262347365133, 0.026082815342726922, 0.032427831220809604, 0.03191213845732832, 0.025140650135050662, 0.01522246515740894, 0.010347300898404785, 0.014014415688952493, 0.01490133879948372, 0.010565049906514967, 0.004545734305190221, 0.0031600527741307153, 0.0039275801015668885, 0.003160355403738538, 0.008189494016632869, 0.020264888532765642, 0.03647048113335558, 0.055264287763521464, 0.07431126919597977, 0.0911768714995525, 0.10384538311030408, 0.1109491810815026, 0.1118787707390652, 0.1068830685008222, 0.09723917429521023, 0.08554419366609693, 0.07591393756777383, 0.07279798601583834, 0.07730687203827452, 0.08594754413281044, 0.09436797110981983, 0.09996432158875196, 0.10213875913792639, 0.10206368332338511, 0.10238621473204372, 0.10640071912136696, 0.1164321425390307, 0.13264735181752674, 0.15357409243808606, 0.1773469201194555 ], [ 0.13769498384174067, 0.1197244719777011, 0.10594704927813355, 0.0959419435590375, 0.08804643805269138, 0.0799128653610899, 0.06952473913700163, 0.05606521958187906, 0.040562000657106266, 0.027027999183995302, 0.023243592972663537, 0.028228392725718976, 0.03184288565103391, 0.02988648208064314, 0.022341493010128752, 0.011519806537429941, 0.006142866529082412, 0.011769981420175523, 0.013092741109797749, 0.008902788499735121, 0.0036329054929680753, 0.004865961903662938, 0.00538189703147136, 0.0037684470136031645, 0.010203640194113616, 0.023713503674785752, 0.040632106399051, 0.05936159258399272, 0.07781635120794374, 0.09375605498823393, 0.10525093160916317, 0.11093779875470819, 0.11015302643582008, 0.10304839166105209, 0.0907975413109672, 0.07605842319799858, 0.06376999354606405, 0.06047699751646315, 0.06799138822077253, 0.0807605058252155, 0.09307784495894723, 0.10210794919159126, 0.10725252691981246, 0.10953665737798457, 0.11121745380286939, 0.11514344313127921, 0.12363607833548249, 0.13749067851437893, 0.1559965987996232, 0.17774092087769236 ], [ 0.13720708460988995, 0.11733662819334269, 0.1017808908084752, 0.0907411069942729, 0.08298548336723371, 0.07613659012622516, 0.06787048430958019, 0.05711141208312362, 0.04476176971341237, 0.034171044857004375, 0.029872868836613147, 0.0310675075709966, 0.0316731318015619, 0.028071375781804696, 0.019984795668721456, 0.009192415981171088, 0.004184055011110905, 0.0103736981225234, 0.011487417942396893, 0.007916299764360476, 0.0054264871729639156, 0.007341815601472382, 0.0067669613581332125, 0.004261174160574595, 0.01230708779944435, 0.02717943143994147, 0.044885277574055193, 0.06370031684757854, 0.08169258410072487, 0.09680094435120584, 0.10719891577256282, 0.1115465278513068, 0.10913373285432316, 0.09999678355216089, 0.08512081552123395, 0.0669915280978828, 0.05117809781011528, 0.04757020095410023, 0.05915567733141243, 0.07651477586801424, 0.09259415876363533, 0.10468663291375216, 0.1123357563706661, 0.11649604734203577, 0.11915709110732589, 0.12286705015158102, 0.12994014341195553, 0.14162389994078398, 0.15784143073194876, 0.17762909700978513 ], [ 0.13609307950617422, 0.11419921548007098, 0.0967131803750735, 0.08457134939506658, 0.07712711290288747, 0.07195610963976604, 0.06625296680302693, 0.05849208813732541, 0.04918854283610019, 0.040770021414459665, 0.035900581859810286, 0.034047698233632365, 0.031667330137700574, 0.026272256205593793, 0.0178614532553124, 0.008475086319188075, 0.005726913530392159, 0.009688424727885241, 0.00984750917465804, 0.007500024626257608, 0.007997280417309285, 0.009749931558776001, 0.007734268150590425, 0.004118982069764888, 0.014405939370607855, 0.030676075311301346, 0.049201896127985784, 0.0682105866528869, 0.08586877558116754, 0.10026818852856474, 0.1096914229585758, 0.1128389559943999, 0.10897070656371753, 0.0980099784371404, 0.08069907279771489, 0.05907017625959044, 0.03857690918440786, 0.03436864647203499, 0.0515911845214885, 0.07364898640723032, 0.09305002374384692, 0.10768570751824516, 0.11733103132345275, 0.12290670319740485, 0.12620539024668684, 0.12955774733065306, 0.13526687570594237, 0.14491798415235926, 0.1589669887502631, 0.1768827270574514 ], [ 0.13452372973420793, 0.11048329537278631, 0.09086032374258333, 0.07745652604760382, 0.07045352193921803, 0.0673551488729364, 0.06459102608661219, 0.059954110021388334, 0.05342473842754436, 0.04661667299543877, 0.04117847609153924, 0.036840137907482085, 0.03164519883472293, 0.024317259433501427, 0.015652549849622478, 0.008721580939431648, 0.008132643875377942, 0.009369010386141517, 0.007878294979297151, 0.0073901099754538685, 0.010471527585597008, 0.01195406701962207, 0.008438956659421909, 0.003724875959244006, 0.01662429626461253, 0.034259174620696525, 0.05359035244476908, 0.07285686228569896, 0.09029395026407407, 0.10411865670480862, 0.11272097916169899, 0.11485757231970795, 0.10978391431875283, 0.09734207022625699, 0.0780521591076844, 0.05334632698835981, 0.027117959568942644, 0.0217362447067001, 0.04648473217893651, 0.07258733088614172, 0.0945478037097543, 0.11109650871667016, 0.12220919333277992, 0.12876079554612832, 0.1323777359147017, 0.13521786369782618, 0.13956861302834, 0.14727748906933122, 0.15925766880633366, 0.17539035091350805 ], [ 0.13270039413896276, 0.10641046946787601, 0.08439642773791305, 0.06944892731921935, 0.06296417654791364, 0.062349781173819424, 0.06282942961514046, 0.061296352486533215, 0.057208812272394154, 0.05161634535840067, 0.04565671790878484, 0.039289308149060176, 0.03153980466997103, 0.0221400978620626, 0.013050092224414295, 0.009079174391030016, 0.010171751276438655, 0.009234851694875267, 0.005479994044330115, 0.007629829705568529, 0.012782289222684927, 0.014066357526188403, 0.009415832640805514, 0.004789356858062384, 0.019236779554057164, 0.038018790815072576, 0.05807992160861205, 0.07762640034353542, 0.09493375735825195, 0.10831823540753423, 0.11627159248721582, 0.1176210077845679, 0.11165029126830826, 0.09817976644571597, 0.07762317022275898, 0.05104457834723504, 0.0203359832613672, 0.01372086176712181, 0.045175724807084636, 0.07363125717851467, 0.09714260456746951, 0.11491264570644737, 0.1269624753280512, 0.13407129873811013, 0.13770320258290858, 0.1398646248524595, 0.1428219681238707, 0.14863515986946726, 0.15862141595053195, 0.17305750110737736 ], [ 0.13085338810548625, 0.102260868677078, 0.07757808138673652, 0.06064311701504769, 0.05468085415365611, 0.05700414309576639, 0.06094982515804972, 0.06237696770298154, 0.06038562859515896, 0.055729688651477746, 0.04933321374343926, 0.041351986924472166, 0.03140266113408067, 0.019863149511205606, 0.009878431203498305, 0.009247162161843767, 0.011802466933692073, 0.009509962604909534, 0.003542962054566171, 0.008698651863310751, 0.0151116438840792, 0.016295025916873308, 0.011245661031441246, 0.00802497522253726, 0.02249581784158794, 0.04204759357669934, 0.06270494631614865, 0.08251787304059174, 0.09976493539368618, 0.11283676193689111, 0.12031945451718788, 0.12112327627310496, 0.11459562579962451, 0.10060909392711388, 0.07964630942230694, 0.05296561585029189, 0.02372319609660179, 0.019179310822844492, 0.04832743760748851, 0.07686451901269066, 0.10083380267799867, 0.11912654329466581, 0.13159878583236498, 0.13886754214427097, 0.14222368073324754, 0.14352965049333533, 0.1450246172721336, 0.14894809372395382, 0.15698744607588236, 0.16980605612753327 ], [ 0.12923621911747205, 0.0983771204136559, 0.07078088690854306, 0.05120557325499174, 0.04565612169435855, 0.05145319570683115, 0.05897792926632477, 0.06310968848094735, 0.0628695130776888, 0.058946760846938034, 0.05222850753450217, 0.04304649744877882, 0.03137439534872985, 0.017873352991851832, 0.00626021987959391, 0.009514840345521405, 0.013322428936440802, 0.010677704064471261, 0.005194548160563101, 0.011009445437928119, 0.01768274968133226, 0.01881623927685889, 0.014096165931780421, 0.012513014335201833, 0.026483502681601576, 0.0464020209339904, 0.06748938241542435, 0.08753138902915779, 0.10476954260146056, 0.11764606350456049, 0.12483352450580543, 0.12533522513415732, 0.11859396110291386, 0.10460320580171327, 0.08407244505429208, 0.05890100814974029, 0.03475572065605883, 0.03217567821108551, 0.055378370894465574, 0.0821378112691188, 0.10556715160506808, 0.12372687607093029, 0.13613710454482858, 0.14319205585094302, 0.1459933669674152, 0.14625897759736112, 0.14619369200684004, 0.14819527526630513, 0.15430434875996835, 0.16557342685191387 ], [ 0.12811405772516815, 0.09515756846039215, 0.06454562428385205, 0.04144782255250495, 0.03599316008477181, 0.04593769593537794, 0.05698693841687753, 0.06345556476170583, 0.06461935021582209, 0.061271818315812784, 0.05436821319812134, 0.04441087044405083, 0.031622627916013515, 0.016812236413107796, 0.003617774101970606, 0.010488204556388029, 0.0150952761930822, 0.012988694165975142, 0.009443675115550654, 0.014448271321605003, 0.020617881786800386, 0.0216788981701651, 0.01770986673728836, 0.017601112299300378, 0.0310918140968244, 0.05107716542153768, 0.07243507621239556, 0.09266098087423659, 0.10993012357389463, 0.12271795765436949, 0.1297762427853394, 0.13020790472895846, 0.12357443206621056, 0.11003654728502878, 0.0906103047666709, 0.06788346544905578, 0.048562814865517605, 0.04689506962977457, 0.06514405135438646, 0.0891407519289664, 0.11124585683686916, 0.12869720363804657, 0.14060387074168548, 0.14709823736120106, 0.1490784947601164, 0.14811350702023318, 0.1463653905739478, 0.14637634361438812, 0.15053869005639628, 0.16031171851653137 ], [ 0.12774561936406278, 0.09302937846714801, 0.05960995985686566, 0.032031294798367534, 0.025911270823359162, 0.04085533313456948, 0.05509683405671075, 0.06341403789561774, 0.06562226296686266, 0.06271300812426314, 0.055769333985858434, 0.045468986362702156, 0.032260076001108924, 0.017265155245112386, 0.00594553147327301, 0.01253476714131323, 0.017339999116751836, 0.016236247642200764, 0.014408651712246145, 0.01860036125604244, 0.02387695966235374, 0.024782834286310767, 0.021686394357549984, 0.02287525247737295, 0.03609560691183495, 0.05600499528272209, 0.07751685459005833, 0.09789040718749228, 0.11522651895888124, 0.12802282062286205, 0.13510455027073687, 0.13567720538121933, 0.1294330934410378, 0.11671725420521514, 0.09884462085879085, 0.07886570394719246, 0.06331775656310949, 0.06208263495797664, 0.0765751709122104, 0.09750331330795156, 0.11774616120586245, 0.134015807194449, 0.14503021829140203, 0.150648525502705, 0.15155719652490116, 0.1491697993883655, 0.14559561806805182, 0.14351154894339233, 0.14567420493039562, 0.15398695857345387 ], [ 0.1283596749098041, 0.09239481121323118, 0.05685086306709157, 0.02454623599539922, 0.016066548126215774, 0.036813028749082585, 0.05346852942865498, 0.06301555514150797, 0.06588357671629842, 0.06327562333019317, 0.056430539675576014, 0.04620702767850798, 0.03327721109757855, 0.019237753449702084, 0.010597529069655389, 0.01545466123711255, 0.020024819448448538, 0.020004068869763254, 0.019510126435104986, 0.023040428970046416, 0.027282821385678863, 0.02791775472282335, 0.025639768123720454, 0.028011885247534545, 0.041232506849995625, 0.06107142273858141, 0.08268516122631238, 0.10319257011765556, 0.12063460004667084, 0.13352904109029393, 0.14077127771208942, 0.14166899215450163, 0.1360463402476398, 0.12442332230084326, 0.10834780386141166, 0.09105394048524608, 0.07838066792379865, 0.07735439576339498, 0.08895820008583052, 0.10687115378442305, 0.12493278524598297, 0.1396564917920858, 0.14944990002168418, 0.15391285021864948, 0.15351937150464437, 0.14952113206423612, 0.14396157381167765, 0.13964297690908514, 0.13971169414862716, 0.14657844391766806 ], [ 0.13013034259672987, 0.09355770120659383, 0.057036521360522625, 0.022257904596238547, 0.009794748104455213, 0.034597162243297806, 0.0522912749969922, 0.062317069512065386, 0.06542208096223419, 0.06295892802460236, 0.056327217945360615, 0.04656307894536584, 0.03452580007213092, 0.02214867100171175, 0.015400607024730234, 0.018750926401890862, 0.022916002258032617, 0.023882709882408152, 0.024420622535224947, 0.027407022948496677, 0.03058455894864186, 0.03081879627183102, 0.029232464919787662, 0.03274158757842723, 0.04625398911650706, 0.06614149260708807, 0.08787455832796304, 0.10853214052590221, 0.12612682568749833, 0.13920340965399966, 0.14672685646303002, 0.1481040686962769, 0.14328308829109102, 0.13293156485739419, 0.11874251817202444, 0.10391491234037441, 0.09346105922832469, 0.09252656921096603, 0.10183952641087947, 0.11694096786084547, 0.13267125509242084, 0.1455899932949718, 0.15389774091073355, 0.15696717059862225, 0.15506640083143655, 0.14927867813848902, 0.1415642516485899, 0.13483726588869707, 0.13266982653240045, 0.13807823636391478 ], [ 0.1331576201669983, 0.09665816374526379, 0.060460341935865136, 0.027215301022676615, 0.014687272324871124, 0.03490468467617928, 0.051762424208709225, 0.06140129852049551, 0.0642704792505902, 0.06175691427254111, 0.05541170953866122, 0.04643017606078096, 0.035756484394341606, 0.025295922813619025, 0.019775239016318804, 0.02192589555015887, 0.02569054492109556, 0.027525308048285062, 0.028885558474952822, 0.031402432800158614, 0.033514049765870545, 0.03321053362912958, 0.032176000606828854, 0.03684381138441894, 0.050952079259141414, 0.07108387660469404, 0.09301523061471928, 0.11387041603433273, 0.13167421786063166, 0.14501228820092493, 0.1529212213179449, 0.15490251820348283, 0.15101419176186906, 0.14203564605028457, 0.12972109957898492, 0.11709750974848526, 0.10839462809620923, 0.1074852928216122, 0.1149277268431671, 0.1274680364719301, 0.14083616023851225, 0.1517856137066443, 0.15840846508364606, 0.15989194257335534, 0.1563105108891492, 0.14857257819246517, 0.13853179681677205, 0.12919022500422717, 0.12458722826467779, 0.12849082011002957 ], [ 0.13745924968958542, 0.10165052859635816, 0.06681000036819547, 0.03688021807046289, 0.025236402745269063, 0.037912314199560315, 0.05206154017501537, 0.06038004766063867, 0.06248148580598217, 0.05966345038776481, 0.053618342999004295, 0.045669698309669764, 0.0366797170057006, 0.028119857756180384, 0.023381280874421366, 0.02456406750310111, 0.028021724707030498, 0.03064999063742003, 0.03269900621813837, 0.0347878014541404, 0.03582283273574323, 0.03483464213342715, 0.03422939298572873, 0.04015087367794578, 0.055173371229179, 0.07579047839860426, 0.09804475366903474, 0.11917119108586081, 0.1372491766934631, 0.15092327688410787, 0.1593057362325905, 0.16198719718284504, 0.15911878775413904, 0.1515546432361665, 0.14104231273351217, 0.13036631687170752, 0.12307455204295405, 0.1221506364302803, 0.12802927099452732, 0.13826086262000958, 0.14931572965573692, 0.15821277272187825, 0.1630157581476096, 0.1627703881473872, 0.1573735510980583, 0.14755254302376142, 0.13502353673601178, 0.12283398142473013, 0.11552660086487818, 0.1178329274743828 ], [ 0.14297632628669446, 0.10833608093613392, 0.07541981226301427, 0.04860063704311695, 0.03698707263328962, 0.04320052999416041, 0.053325300750993135, 0.05940082528471177, 0.060140475391449594, 0.05668257570820942, 0.050872236939925826, 0.0441301474115367, 0.03701934215371642, 0.03022507624896437, 0.025960534992079832, 0.026331868624341177, 0.02962359003393745, 0.033039311847681684, 0.0357051077429808, 0.03738379045775426, 0.037304310834181786, 0.03546592675742235, 0.035200817030630334, 0.04255651182419057, 0.05882786153854196, 0.08019034789888822, 0.10291819720251266, 0.12440648489663057, 0.142828515580211, 0.1569070497430039, 0.16583497223337218, 0.16928633342509025, 0.16748798248928976, 0.16133532778873888, 0.15252040919723517, 0.14355920333093694, 0.13742598165837627, 0.13646360564616541, 0.1410114661523416, 0.14917199888819382, 0.1580136567910813, 0.1648422658537414, 0.16775145403534497, 0.165686480156967, 0.158384939981528, 0.146387457755283, 0.13123420432209112, 0.11594749854343944, 0.10558234244226643, 0.10613353978687452 ], [ 0.14959003448461167, 0.11642806399572919, 0.08559781374643202, 0.06117978118278604, 0.04908506655072557, 0.05012918570647892, 0.055632231872399435, 0.05865406522193106, 0.057385938522675574, 0.05284572796713829, 0.047100706764466475, 0.04166691460344892, 0.036548663575268016, 0.03134793583558008, 0.027316690992626166, 0.026968016268446554, 0.030272756623042636, 0.034545733499190365, 0.03780686090073376, 0.03907655349429759, 0.037809283113926756, 0.034921951503722716, 0.03495314174920763, 0.04402880150536117, 0.06189592596250391, 0.08425848763477738, 0.10761549837085652, 0.12956123095553543, 0.14839616692477783, 0.16293904892355934, 0.17246819301382588, 0.17673528696360646, 0.1760265506967832, 0.17125101919224062, 0.16401358457287088, 0.15656200160057474, 0.15139484955742705, 0.1503802870453972, 0.15378135312458918, 0.16008906066791453, 0.16684912198763208, 0.17164712494031248, 0.17264477142806475, 0.16872261515097758, 0.159478567022009, 0.14526328813278455, 0.12739730775426936, 0.10877132047755034, 0.09489469832726587, 0.09343408416436552 ], [ 0.1571436198532555, 0.12561530474925023, 0.09678459173855684, 0.07410033355852312, 0.06127689891678389, 0.058156529800542985, 0.05900292190015685, 0.05837555910616833, 0.054438651243191895, 0.048240395938149355, 0.04224679560325954, 0.03816092533053871, 0.03511456926942196, 0.031336828704731964, 0.02731780880401998, 0.026277941356465863, 0.029824141232483233, 0.0351043491478336, 0.038979705563900985, 0.03982916867881593, 0.037261811000992795, 0.03307015327118252, 0.0334154269008883, 0.04462875611402554, 0.06443417131566383, 0.08802020211747646, 0.11214565324955336, 0.1346363485092647, 0.15394514576115037, 0.1690007903898078, 0.17917044556150788, 0.18427758329750782, 0.18465329226628036, 0.1811989196709524, 0.17541422498449655, 0.16929334580803476, 0.1649421551570501, 0.16386863034572333, 0.16627330375515378, 0.17092717731730217, 0.17575576074298696, 0.17860306370034273, 0.17772156412706383, 0.17195702061282622, 0.16078854155220726, 0.14437849778077316, 0.12378570476816736, 0.10162663256235854, 0.08367686180652238, 0.07978889414530767 ], [ 0.16546365542411234, 0.1356051252233753, 0.1085724642694568, 0.08710585260612119, 0.07345001617744044, 0.06691518666047913, 0.06341323759657119, 0.05883635322216231, 0.05163718494314501, 0.0430609246970378, 0.03628673823907349, 0.03353755286020435, 0.032661903147249476, 0.030154973207716047, 0.025909588816306072, 0.024132297722879042, 0.02823336178953566, 0.034755377324537276, 0.039291008190786084, 0.03970038155469054, 0.03568301527733418, 0.029836247290587523, 0.03060759125767237, 0.04453597977406425, 0.06657901951170529, 0.09155117723843878, 0.11654761429082287, 0.13964990935357216, 0.15947852063061602, 0.1750806190983958, 0.18591320046080345, 0.19186534226131255, 0.19330054989292647, 0.19109699838205518, 0.18664129834733584, 0.18169530922280833, 0.17804060294772944, 0.17690634441660028, 0.17844136361224425, 0.18162299994927533, 0.1846800937518933, 0.18568855120622135, 0.18300358679198386, 0.17546102275860373, 0.16244386759559395, 0.14393630529436127, 0.12070633696175466, 0.09493577281098482, 0.07226889114676203, 0.06526616076366538 ], [ 0.17437687318604803, 0.1461440985071846, 0.12067518373609618, 0.10005310491317256, 0.08554507202070058, 0.07617694830275863, 0.06881205930133548, 0.060313668905025965, 0.0494658347082753, 0.037704325709579496, 0.02926096371002396, 0.02779441118545388, 0.02927729089355709, 0.027913576746657585, 0.02314898651792352, 0.020470044144523383, 0.025604243704534898, 0.03368177730350671, 0.038927536269448774, 0.038873807527728926, 0.03323484749625128, 0.02522189929548709, 0.026700375725844598, 0.04407954729522575, 0.06854504952711692, 0.0949729932050375, 0.12088796391182476, 0.14463635527083274, 0.165009299709848, 0.1811738411987332, 0.19267453212213592, 0.19945922158626916, 0.2019132389940393, 0.20088094225400124, 0.19763457632356188, 0.1937273638287371, 0.1906723578911013, 0.18947932336188528, 0.19025424907510174, 0.19213002529192297, 0.19357974546315854, 0.19288459060849938, 0.18850780707989495, 0.17929637250936115, 0.1645623818224626, 0.14413359882302787, 0.11848538459030031, 0.0892337221462778, 0.06124698117071371, 0.0499502347271596 ], [ 0.18372140510275942, 0.15702356812911022, 0.13289367156704096, 0.11285647157205508, 0.09753007313989624, 0.08580460122110857, 0.07513568870037773, 0.06304520852538091, 0.04853663875968202, 0.032942430170970545, 0.021364703977970297, 0.02107449556191681, 0.025293046942949766, 0.02496259203511129, 0.019301565887736575, 0.01531534738550773, 0.022304775910319784, 0.0322670554371368, 0.03823037701270334, 0.03769897536925545, 0.030304646766642074, 0.019362420476850307, 0.02217950433352854, 0.043760746941036564, 0.07061347317797767, 0.0984436060591976, 0.1252555953946049, 0.14964391983775507, 0.17055927987635483, 0.18728224300305893, 0.1994388687719276, 0.20702798315851725, 0.21044761811761065, 0.2105013810370104, 0.20835025242715854, 0.20536228604551696, 0.202827382221518, 0.2015803576870294, 0.2016918854750441, 0.2024149415266235, 0.20242164609504643, 0.20017429431662165, 0.19424581365451693, 0.1835128717589003, 0.16724456606085217, 0.14514725268986886, 0.1174411482238953, 0.085146682675804, 0.05163006285617216, 0.033949597415423256 ], [ 0.19335300949973397, 0.16807718199071198, 0.14508967780850024, 0.12546374956109557, 0.10938939684306542, 0.09571617647978627, 0.08231584736428214, 0.06718306349600191, 0.04947088362891451, 0.030108968521002533, 0.013395718643044, 0.013990491799123227, 0.021537617484541405, 0.02209349666579605, 0.015168228345552103, 0.008891339599549913, 0.019241619075731933, 0.031159195254489302, 0.037723725112282475, 0.0367319399636729, 0.02764877573635967, 0.012806014299694423, 0.018291898984546626, 0.04423033947666326, 0.07310601117491243, 0.1021427331177694, 0.12975386468604486, 0.15473057809759733, 0.17615702055573026, 0.19341307428940707, 0.20619637334204077, 0.21454777821620313, 0.2188699422095557, 0.21992144058890994, 0.2187575787174125, 0.21658323858002948, 0.21450210450715795, 0.21320802097372316, 0.21274288059317492, 0.21245474576184994, 0.2111803279207194, 0.20754234613365036, 0.20022337509876378, 0.18814654429337024, 0.17056804372731568, 0.1471197733624453, 0.11784623627166155, 0.0833046559210786, 0.045131540913247295, 0.017448552594984585 ], [ 0.2031475628308242, 0.17917524759873532, 0.1571674750300133, 0.13784383828792449, 0.12111751302892296, 0.10586156986650638, 0.09028229516675591, 0.07276931742095791, 0.052688475630787414, 0.03080390229595104, 0.009570523834541947, 0.009537062009958002, 0.019721255277125625, 0.020799651142904087, 0.01300247704825731, 0.0037623075399895157, 0.01821375249518662, 0.03125512142899056, 0.03809259740611698, 0.036723088027917655, 0.026502959297054676, 0.008389406028018555, 0.017611093809888113, 0.04616761152822893, 0.07634317487448346, 0.1062531485734562, 0.1344909973858284, 0.15995899310362308, 0.1818351894939339, 0.1995776233873314, 0.21294203845557738, 0.22200123733703309, 0.22715508645972946, 0.22911461196546318, 0.22883623588118104, 0.227381589659016, 0.2256982985594294, 0.22436568101361543, 0.2234025954239015, 0.22223444410275428, 0.21983637553852411, 0.21497443082651652, 0.2064401980370662, 0.1932185495072461, 0.17458359313326033, 0.15014712601063981, 0.11988768989916042, 0.0841818323193759, 0.043863267531813216, 0.003314430541148221 ] ] }, { "hoverinfo": "text", "legendgroup": "In-sample", "marker": { "color": "black", "opacity": 0.5, "symbol": 1 }, "mode": "markers", "name": "In-sample", "text": [ "Arm 0_0
mean_accuracy: 0.929667 (SEM: None)
lr: 0.000447872
momentum: 0.15958", "Arm 1_0
mean_accuracy: 0.5675 (SEM: None)
lr: 2.63915e-06
momentum: 0.796108", "Arm 2_0
mean_accuracy: 0.746 (SEM: None)
lr: 1.23545e-05
momentum: 0.538357", "Arm 3_0
mean_accuracy: 0.112667 (SEM: None)
lr: 0.015297
momentum: 0.677089", "Arm 4_0
mean_accuracy: 0.105333 (SEM: None)
lr: 0.121624
momentum: 0.407587", "Arm 5_0
mean_accuracy: 0.487333 (SEM: None)
lr: 1.21574e-05
momentum: 0", "Arm 6_0
mean_accuracy: 0.115667 (SEM: None)
lr: 0.00659114
momentum: 0", "Arm 7_0
mean_accuracy: 0.869 (SEM: None)
lr: 7.00249e-05
momentum: 0.227281", "Arm 8_0
mean_accuracy: 0.918167 (SEM: None)
lr: 0.000213762
momentum: 0.375103", "Arm 9_0
mean_accuracy: 0.906667 (SEM: None)
lr: 0.000210491
momentum: 0", "Arm 10_0
mean_accuracy: 0.931167 (SEM: None)
lr: 0.000118015
momentum: 0.697606", "Arm 11_0
mean_accuracy: 0.909167 (SEM: None)
lr: 0.000223372
momentum: 0.216812", "Arm 12_0
mean_accuracy: 0.849667 (SEM: None)
lr: 5.04164e-05
momentum: 1", "Arm 13_0
mean_accuracy: 0.268333 (SEM: None)
lr: 0.000274805
momentum: 1", "Arm 14_0
mean_accuracy: 0.9135 (SEM: None)
lr: 6.30373e-05
momentum: 0.611518", "Arm 15_0
mean_accuracy: 0.93 (SEM: None)
lr: 0.000407413
momentum: 0.66411", "Arm 16_0
mean_accuracy: 0.899 (SEM: None)
lr: 4.08289e-05
momentum: 0.735952", "Arm 17_0
mean_accuracy: 0.875667 (SEM: None)
lr: 1.56824e-05
momentum: 1", "Arm 18_0
mean_accuracy: 0.945167 (SEM: None)
lr: 0.000290357
momentum: 0.556927", "Arm 19_0
mean_accuracy: 0.934667 (SEM: None)
lr: 0.000623644
momentum: 0.476343", "Arm 20_0
mean_accuracy: 0.932333 (SEM: None)
lr: 0.0002016
momentum: 0.584793", "Arm 21_0
mean_accuracy: 0.935333 (SEM: None)
lr: 0.000461923
momentum: 0", "Arm 22_0
mean_accuracy: 0.957 (SEM: None)
lr: 0.000451056
momentum: 0.564712", "Arm 23_0
mean_accuracy: 0.946 (SEM: None)
lr: 0.00049588
momentum: 0.327785", "Arm 24_0
mean_accuracy: 0.101167 (SEM: None)
lr: 0.4
momentum: 1", "Arm 25_0
mean_accuracy: 0.1205 (SEM: None)
lr: 1e-06
momentum: 0.249746", "Arm 26_0
mean_accuracy: 0.948833 (SEM: None)
lr: 0.000413491
momentum: 0.432858" ], "type": "scatter", "x": [ 0.00044787229470350437, 2.6391451195406558e-06, 1.2354533434459913e-05, 0.015296960417382197, 0.12162408890342173, 1.215743554887908e-05, 0.006591135854943863, 7.002488983710606e-05, 0.000213762339959398, 0.00021049111156474346, 0.00011801470184262572, 0.00022337174734489087, 5.0416445411064314e-05, 0.00027480506015737147, 6.303725326478828e-05, 0.0004074132311899103, 4.082889596199312e-05, 1.5682433036902418e-05, 0.0002903571662715394, 0.0006236437694875057, 0.0002015995155383558, 0.0004619232279273696, 0.00045105615350493, 0.000495880474117641, 0.4, 1e-06, 0.0004134905141980337 ], "xaxis": "x", "y": [ 0.15957990288734436, 0.7961076777428389, 0.5383571069687605, 0.6770887486636639, 0.4075868986546993, 0.0, 0.0, 0.2272805876523356, 0.3751030785986231, 0.0, 0.6976055982708349, 0.21681163310728474, 1.0, 1.0, 0.6115180011527241, 0.6641099466681635, 0.7359515647638785, 1.0, 0.5569271584023326, 0.4763429053039613, 0.5847928294475059, 0.0, 0.5647121361805426, 0.3277848711979095, 1.0, 0.24974606892687617, 0.43285775076477323 ], "yaxis": "y" }, { "hoverinfo": "text", "legendgroup": "In-sample", "marker": { "color": "black", "opacity": 0.5, "symbol": 1 }, "mode": "markers", "name": "In-sample", "showlegend": false, "text": [ "Arm 0_0
mean_accuracy: 0.929667 (SEM: None)
lr: 0.000447872
momentum: 0.15958", "Arm 1_0
mean_accuracy: 0.5675 (SEM: None)
lr: 2.63915e-06
momentum: 0.796108", "Arm 2_0
mean_accuracy: 0.746 (SEM: None)
lr: 1.23545e-05
momentum: 0.538357", "Arm 3_0
mean_accuracy: 0.112667 (SEM: None)
lr: 0.015297
momentum: 0.677089", "Arm 4_0
mean_accuracy: 0.105333 (SEM: None)
lr: 0.121624
momentum: 0.407587", "Arm 5_0
mean_accuracy: 0.487333 (SEM: None)
lr: 1.21574e-05
momentum: 0", "Arm 6_0
mean_accuracy: 0.115667 (SEM: None)
lr: 0.00659114
momentum: 0", "Arm 7_0
mean_accuracy: 0.869 (SEM: None)
lr: 7.00249e-05
momentum: 0.227281", "Arm 8_0
mean_accuracy: 0.918167 (SEM: None)
lr: 0.000213762
momentum: 0.375103", "Arm 9_0
mean_accuracy: 0.906667 (SEM: None)
lr: 0.000210491
momentum: 0", "Arm 10_0
mean_accuracy: 0.931167 (SEM: None)
lr: 0.000118015
momentum: 0.697606", "Arm 11_0
mean_accuracy: 0.909167 (SEM: None)
lr: 0.000223372
momentum: 0.216812", "Arm 12_0
mean_accuracy: 0.849667 (SEM: None)
lr: 5.04164e-05
momentum: 1", "Arm 13_0
mean_accuracy: 0.268333 (SEM: None)
lr: 0.000274805
momentum: 1", "Arm 14_0
mean_accuracy: 0.9135 (SEM: None)
lr: 6.30373e-05
momentum: 0.611518", "Arm 15_0
mean_accuracy: 0.93 (SEM: None)
lr: 0.000407413
momentum: 0.66411", "Arm 16_0
mean_accuracy: 0.899 (SEM: None)
lr: 4.08289e-05
momentum: 0.735952", "Arm 17_0
mean_accuracy: 0.875667 (SEM: None)
lr: 1.56824e-05
momentum: 1", "Arm 18_0
mean_accuracy: 0.945167 (SEM: None)
lr: 0.000290357
momentum: 0.556927", "Arm 19_0
mean_accuracy: 0.934667 (SEM: None)
lr: 0.000623644
momentum: 0.476343", "Arm 20_0
mean_accuracy: 0.932333 (SEM: None)
lr: 0.0002016
momentum: 0.584793", "Arm 21_0
mean_accuracy: 0.935333 (SEM: None)
lr: 0.000461923
momentum: 0", "Arm 22_0
mean_accuracy: 0.957 (SEM: None)
lr: 0.000451056
momentum: 0.564712", "Arm 23_0
mean_accuracy: 0.946 (SEM: None)
lr: 0.00049588
momentum: 0.327785", "Arm 24_0
mean_accuracy: 0.101167 (SEM: None)
lr: 0.4
momentum: 1", "Arm 25_0
mean_accuracy: 0.1205 (SEM: None)
lr: 1e-06
momentum: 0.249746", "Arm 26_0
mean_accuracy: 0.948833 (SEM: None)
lr: 0.000413491
momentum: 0.432858" ], "type": "scatter", "x": [ 0.00044787229470350437, 2.6391451195406558e-06, 1.2354533434459913e-05, 0.015296960417382197, 0.12162408890342173, 1.215743554887908e-05, 0.006591135854943863, 7.002488983710606e-05, 0.000213762339959398, 0.00021049111156474346, 0.00011801470184262572, 0.00022337174734489087, 5.0416445411064314e-05, 0.00027480506015737147, 6.303725326478828e-05, 0.0004074132311899103, 4.082889596199312e-05, 1.5682433036902418e-05, 0.0002903571662715394, 0.0006236437694875057, 0.0002015995155383558, 0.0004619232279273696, 0.00045105615350493, 0.000495880474117641, 0.4, 1e-06, 0.0004134905141980337 ], "xaxis": "x2", "y": [ 0.15957990288734436, 0.7961076777428389, 0.5383571069687605, 0.6770887486636639, 0.4075868986546993, 0.0, 0.0, 0.2272805876523356, 0.3751030785986231, 0.0, 0.6976055982708349, 0.21681163310728474, 1.0, 1.0, 0.6115180011527241, 0.6641099466681635, 0.7359515647638785, 1.0, 0.5569271584023326, 0.4763429053039613, 0.5847928294475059, 0.0, 0.5647121361805426, 0.3277848711979095, 1.0, 0.24974606892687617, 0.43285775076477323 ], "yaxis": "y2" } ], "layout": { "annotations": [ { "font": { "size": 14 }, "showarrow": false, "text": "Mean", "x": 0.25, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 14 }, "showarrow": false, "text": "Standard Error", "x": 0.8, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" } ], "autosize": false, "height": 450, "hovermode": "closest", "legend": { "orientation": "h", "x": 0, "y": -0.25 }, "margin": { "b": 100, "l": 35, "pad": 0, "r": 35, "t": 35 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "sequentialminus": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "mean_accuracy" }, "width": 950, "xaxis": { "anchor": "y", "autorange": false, "domain": [ 0.05, 0.45 ], "exponentformat": "e", "range": [ -6.0, -0.39794000867203766 ], "tickfont": { "size": 11 }, "tickmode": "auto", "title": { "text": "lr" }, "type": "log" }, "xaxis2": { "anchor": "y2", "autorange": false, "domain": [ 0.6, 1 ], "exponentformat": "e", "range": [ -6.0, -0.39794000867203766 ], "tickfont": { "size": 11 }, "tickmode": "auto", "title": { "text": "lr" }, "type": "log" }, "yaxis": { "anchor": "x", "autorange": false, "domain": [ 0, 1 ], "exponentformat": "e", "range": [ 0.0, 1.0 ], "tickfont": { "size": 11 }, "tickmode": "auto", "title": { "text": "momentum" }, "type": "linear" }, "yaxis2": { "anchor": "x2", "autorange": false, "domain": [ 0, 1 ], "exponentformat": "e", "range": [ 0.0, 1.0 ], "tickfont": { "size": 11 }, "tickmode": "auto", "type": "linear" } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "render(\n", " plot_contour(\n", " model=ax.generation_strategy.model,\n", " param_x=\"lr\",\n", " param_y=\"momentum\",\n", " metric_name=\"mean_accuracy\",\n", " )\n", ")" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "execution": { "iopub.execute_input": "2022-12-29T21:37:40.716700Z", "iopub.status.busy": "2022-12-29T21:37:40.716251Z", "iopub.status.idle": "2022-12-29T21:37:40.776287Z", "shell.execute_reply": "2022-12-29T21:37:40.775118Z" }, "originalKey": "6dfd23ca-1c93-4846-8e85-4560f9e40304" }, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "linkText": "Export to plot.ly", "plotlyServerURL": "https://plot.ly", "showLink": false }, "data": [ { "hoverinfo": "none", "legendgroup": "", "line": { "width": 0 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30 ], "y": [ 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 94.51666666666667, 94.51666666666667, 94.51666666666667, 94.51666666666667, 95.7, 95.7, 95.7, 95.7, 95.7, 95.7, 95.7, 95.7 ] }, { "fill": "tonexty", "fillcolor": "rgba(128,177,211,0.3)", "legendgroup": "objective value", "line": { "color": "rgba(128,177,211,1)" }, "mode": "lines", "name": "objective value", "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30 ], "y": [ 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 94.51666666666667, 94.51666666666667, 94.51666666666667, 94.51666666666667, 95.7, 95.7, 95.7, 95.7, 95.7, 95.7, 95.7, 95.7 ] }, { "fill": "tonexty", "fillcolor": "rgba(128,177,211,0.3)", "hoverinfo": "none", "legendgroup": "", "line": { "width": 0 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30 ], "y": [ 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 92.96666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 93.11666666666667, 94.51666666666667, 94.51666666666667, 94.51666666666667, 94.51666666666667, 95.7, 95.7, 95.7, 95.7, 95.7, 95.7, 95.7, 95.7 ] } ], "layout": { "showlegend": true, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "sequentialminus": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "Model performance vs. # of iterations" }, "xaxis": { "title": { "text": "Iteration" } }, "yaxis": { "title": { "text": "Accuracy" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# `plot_single_method` expects a 2-d array of means, because it expects to average means from multiple\n", "# optimization runs, so we wrap out best objectives array in another array.\n", "best_objectives = np.array(\n", " [[trial.objective_mean * 100 for trial in ax.experiment.trials.values()]]\n", ")\n", "best_objective_plot = optimization_trace_single_method(\n", " y=np.maximum.accumulate(best_objectives, axis=1),\n", " title=\"Model performance vs. # of iterations\",\n", " ylabel=\"Accuracy\",\n", ")\n", "render(best_objective_plot)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.15" }, "last_base_url": "https://devvm3002.frc0.facebook.com:8090/", "last_kernel_id": "148e0817-7aae-4719-aff2-639ce2864738", "last_msg_id": "a1616ce6-ae3c4879620e9fb8029ca89e_240", "last_server_session_id": "3539ac9b-8cce-42fe-984a-f3a2a8a1dbde", "outputWidgetContext": {}, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": { "09b26a0adbe1420190bc680957da6928": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_7ff46f135fd44a8aaf3f3c6b1f7c9f9d", "IPY_MODEL_0dd97c4be3bc4c27a4c2352edaf4c0e7", "IPY_MODEL_72eeb05beeaa4bb89043580070102c29" ], "layout": "IPY_MODEL_d8b866d5da4a462dbcc7a80908161d48" } }, "0a34097995724b4a9bb3652959f6b2e7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_dded12729c124c23a0d21a43e11d7666", "IPY_MODEL_2eeca9f1ea0a4006a5731027b46d2980", "IPY_MODEL_9d1633a8589f43c6a10ff58522d65ea7" ], "layout": "IPY_MODEL_127c4f9429ed47f4b8ecc06e73e06948" } }, "0dd97c4be3bc4c27a4c2352edaf4c0e7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_b8a119d760f649b1a2114b969df92ab2", "max": 1648877.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_b5f8ad06c4fe4c73a272cf1b8cb837d5", "value": 1648877.0 } }, "127c4f9429ed47f4b8ecc06e73e06948": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "13ed8c79c4ae4893bc0030446a5e1571": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "154b9187c06248bd8b02a85df92c465d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a9cf40e443794c0280b162e065c5a03b", "placeholder": "​", "style": "IPY_MODEL_78df310470f4413b95547d6444c0231d", "value": " 28881/28881 [00:00<00:00, 2640847.91it/s]" } }, "167ac22827584baeba99b47702f6a43c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_54718f17d8a64b808d733f2bb9764aef", "IPY_MODEL_f37dcd6d9cec4f33872f7987244350c6", "IPY_MODEL_27f064c7fd8e4859a6cd740787bf380e" ], "layout": "IPY_MODEL_70c01fdcca8346adaa5703d8377f080f" } }, "27f064c7fd8e4859a6cd740787bf380e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_7ad867383da949a7a794061d804999b4", "placeholder": "​", "style": "IPY_MODEL_6d8ef0de93a747a2802c3170ef71296b", "value": " 4542/4542 [00:00<00:00, 469849.77it/s]" } }, "2eeca9f1ea0a4006a5731027b46d2980": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a084f97dc2d64c57af660dcbe901f6e3", "max": 9912422.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_890fd259e29b4e21aefaf2c319996c5d", "value": 9912422.0 } }, "4de3c7794f4443f29c8fc153d8553d2d": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4f3b1509e270477aa1daf7388772ee65": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "54718f17d8a64b808d733f2bb9764aef": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_6dd4fcb6b6da4ab1ad7b56b0a13b0b8b", "placeholder": "​", "style": "IPY_MODEL_9576f6a820db426a8eb7047441593326", "value": "100%" } }, "5a3e4d04ed0948c692748b622f2437a4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "5b76191b0efd41a4a0b2e284a0f230bd": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "5dad7c5423b543658fed29c1c11ac033": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "693d8cf5709c4f96852abd33d5f2c77d": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "6b855b845d1a4128afa5be47fc57160f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_99cc26a661604710a04992c6ad94dcf3", "max": 28881.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_5b76191b0efd41a4a0b2e284a0f230bd", "value": 28881.0 } }, "6d8ef0de93a747a2802c3170ef71296b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "6dd4fcb6b6da4ab1ad7b56b0a13b0b8b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "70c01fdcca8346adaa5703d8377f080f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "72eeb05beeaa4bb89043580070102c29": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5a3e4d04ed0948c692748b622f2437a4", "placeholder": "​", "style": "IPY_MODEL_9a07e0d7a9644c8e988a6e2f3827afba", "value": " 1648877/1648877 [00:00<00:00, 80774251.30it/s]" } }, "78df310470f4413b95547d6444c0231d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "7ad867383da949a7a794061d804999b4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7c6307ba834847e190dcc1c227eba043": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "7ff46f135fd44a8aaf3f3c6b1f7c9f9d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_bb19a81336f34e0f874f90b16ac32490", "placeholder": "​", "style": "IPY_MODEL_7c6307ba834847e190dcc1c227eba043", "value": "100%" } }, "890fd259e29b4e21aefaf2c319996c5d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "9576f6a820db426a8eb7047441593326": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "99cc26a661604710a04992c6ad94dcf3": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9a07e0d7a9644c8e988a6e2f3827afba": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9a9c6a3fc5504e2c8070b248456cdac6": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9d1633a8589f43c6a10ff58522d65ea7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_b6b40049a7824433a0d39b12644f6ccd", "placeholder": "​", "style": "IPY_MODEL_13ed8c79c4ae4893bc0030446a5e1571", "value": " 9912422/9912422 [00:00<00:00, 293686371.94it/s]" } }, "a084f97dc2d64c57af660dcbe901f6e3": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a9cf40e443794c0280b162e065c5a03b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ace805f9331746a3b054d025d0090116": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_db5e538cc9644cdc99d5407b466c50f5", "IPY_MODEL_6b855b845d1a4128afa5be47fc57160f", "IPY_MODEL_154b9187c06248bd8b02a85df92c465d" ], "layout": "IPY_MODEL_693d8cf5709c4f96852abd33d5f2c77d" } }, "b5f8ad06c4fe4c73a272cf1b8cb837d5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "b6b40049a7824433a0d39b12644f6ccd": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b8a119d760f649b1a2114b969df92ab2": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "bb19a81336f34e0f874f90b16ac32490": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c681328bd1fa4b52a764df652036df4b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "d8b866d5da4a462dbcc7a80908161d48": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "db5e538cc9644cdc99d5407b466c50f5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5dad7c5423b543658fed29c1c11ac033", "placeholder": "​", "style": "IPY_MODEL_4f3b1509e270477aa1daf7388772ee65", "value": "100%" } }, "dded12729c124c23a0d21a43e11d7666": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_4de3c7794f4443f29c8fc153d8553d2d", "placeholder": "​", "style": "IPY_MODEL_9a9c6a3fc5504e2c8070b248456cdac6", "value": "100%" } }, "e267ca9ad6e443c99074953d5e9b5ebb": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f37dcd6d9cec4f33872f7987244350c6": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e267ca9ad6e443c99074953d5e9b5ebb", "max": 4542.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_c681328bd1fa4b52a764df652036df4b", "value": 4542.0 } } }, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 2 }