{ "cells": [ { "cell_type": "markdown", "metadata": { "originalKey": "6dba2bea-d97e-4545-9803-4242850e1807" }, "source": [ "# Ax Service API with RayTune on PyTorch CNN\n", "\n", "Ax integrates easily with different scheduling frameworks and distributed training frameworks. In this example, Ax-driven optimization is executed in a distributed fashion using [RayTune](https://ray.readthedocs.io/en/latest/tune.html). \n", "\n", "RayTune is a scalable framework for hyperparameter tuning that provides many state-of-the-art hyperparameter tuning algorithms and seamlessly scales from laptop to distributed cluster with fault tolerance. RayTune leverages [Ray](https://ray.readthedocs.io/)'s Actor API to provide asynchronous parallel and distributed execution.\n", "\n", "Ray 'Actors' are a simple and clean abstraction for replicating your Python classes across multiple workers and nodes. Each hyperparameter evaluation is asynchronously executed on a separate Ray actor and reports intermediate training progress back to RayTune. Upon reporting, RayTune then uses this information to performs actions such as early termination, re-prioritization, or checkpointing." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2022-08-17T19:28:00.964546Z", "iopub.status.busy": "2022-08-17T19:28:00.963936Z", "iopub.status.idle": "2022-08-17T19:28:03.343020Z", "shell.execute_reply": "2022-08-17T19:28:03.342303Z" }, "originalKey": "fe7a9417-4bde-46d2-9de3-af1bc73bde45" }, "outputs": [], "source": [ "import logging\n", "\n", "from ray import tune\n", "from ray.tune import report\n", "from ray.tune.suggest.ax import AxSearch\n", "\n", "logger = logging.getLogger(tune.__name__)\n", "logger.setLevel(\n", " level=logging.CRITICAL\n", ") # Reduce the number of Ray warnings that are not relevant here." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2022-08-17T19:28:03.347109Z", "iopub.status.busy": "2022-08-17T19:28:03.346543Z", "iopub.status.idle": "2022-08-17T19:28:03.471480Z", "shell.execute_reply": "2022-08-17T19:28:03.470782Z" }, "originalKey": "19956234-25ae-4e72-9d72-dbcd1b90e530" }, "outputs": [ { "data": { "text/html": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:03] ax.utils.notebook.plotting: Injecting Plotly library into cell. Do not overwrite or delete cell.\n" ] }, { "data": { "text/html": [ " \n", " " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import torch\n", "from ax.plot.contour import plot_contour\n", "from ax.plot.trace import optimization_trace_single_method\n", "from ax.service.ax_client import AxClient\n", "from ax.utils.notebook.plotting import init_notebook_plotting, render\n", "from ax.utils.tutorials.cnn_utils import CNN, evaluate, load_mnist, train\n", "\n", "init_notebook_plotting()" ] }, { "cell_type": "markdown", "metadata": { "originalKey": "a26e18f8-caa7-411d-809a-61a9229cd6c6" }, "source": [ "## 1. Initialize client\n", "We specify `enforce_sequential_optimization` as False, because Ray runs many trials in parallel. With the sequential optimization enforcement, `AxClient` would expect the first few trials to be completed with data before generating more trials.\n", "\n", "When high parallelism is not required, it is best to enforce sequential optimization, as it allows for achieving optimal results in fewer (but sequential) trials. In cases where parallelism is important, such as with distributed training using Ray, we choose to forego minimizing resource utilization and run more trials in parallel." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2022-08-17T19:28:03.518240Z", "iopub.status.busy": "2022-08-17T19:28:03.517694Z", "iopub.status.idle": "2022-08-17T19:28:03.522360Z", "shell.execute_reply": "2022-08-17T19:28:03.521733Z" }, "originalKey": "a91e1cb2-999a-4b88-a2d2-85d0acaa8854" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:03] ax.service.ax_client: Starting optimization with verbose logging. To disable logging, set the `verbose_logging` argument to `False`. Note that float values in the logs are rounded to 6 decimal points.\n" ] } ], "source": [ "ax = AxClient(enforce_sequential_optimization=False)" ] }, { "cell_type": "markdown", "metadata": { "originalKey": "1766919c-fb6f-4271-a8e1-6f972eee78f3" }, "source": [ "## 2. Set up experiment\n", "Here we set up the search space and specify the objective; refer to the Ax API tutorials for more detail." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2022-08-17T19:28:03.525636Z", "iopub.status.busy": "2022-08-17T19:28:03.525405Z", "iopub.status.idle": "2022-08-17T19:28:03.528591Z", "shell.execute_reply": "2022-08-17T19:28:03.527864Z" }, "originalKey": "37e367d4-d09d-425b-98f7-c8849d9be4b7" }, "outputs": [], "source": [ "MINIMIZE = False # Whether we should be minimizing or maximizing the objective" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2022-08-17T19:28:03.531997Z", "iopub.status.busy": "2022-08-17T19:28:03.531761Z", "iopub.status.idle": "2022-08-17T19:28:03.539555Z", "shell.execute_reply": "2022-08-17T19:28:03.539021Z" }, "originalKey": "777c8d33-2cd1-4425-b45f-2a44922dce7d" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:03] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter lr. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:03] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter momentum. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:03] ax.service.utils.instantiation: Created search space: SearchSpace(parameters=[RangeParameter(name='lr', parameter_type=FLOAT, range=[1e-06, 0.4], log_scale=True), RangeParameter(name='momentum', parameter_type=FLOAT, range=[0.0, 1.0])], parameter_constraints=[]).\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:03] ax.modelbridge.dispatch_utils: Using Bayesian optimization since there are more ordered parameters than there are categories for the unordered categorical parameters.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:03] ax.modelbridge.dispatch_utils: Using Bayesian Optimization generation strategy: GenerationStrategy(name='Sobol+GPEI', steps=[Sobol for 5 trials, GPEI for subsequent trials]). Iterations after 5 will take longer to generate due to model-fitting.\n" ] } ], "source": [ "ax.create_experiment(\n", " name=\"mnist_experiment\",\n", " parameters=[\n", " {\"name\": \"lr\", \"type\": \"range\", \"bounds\": [1e-6, 0.4], \"log_scale\": True},\n", " {\"name\": \"momentum\", \"type\": \"range\", \"bounds\": [0.0, 1.0]},\n", " ],\n", " objective_name=\"mean_accuracy\",\n", " minimize=MINIMIZE,\n", ")" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2022-08-17T19:28:03.542412Z", "iopub.status.busy": "2022-08-17T19:28:03.542179Z", "iopub.status.idle": "2022-08-17T19:28:03.551819Z", "shell.execute_reply": "2022-08-17T19:28:03.551115Z" }, "originalKey": "589e4d80-02ae-461d-babc-0f96718f623e" }, "outputs": [ { "data": { "text/plain": [ "False" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ax.experiment.optimization_config.objective.minimize" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2022-08-17T19:28:03.554982Z", "iopub.status.busy": "2022-08-17T19:28:03.554405Z", "iopub.status.idle": "2022-08-17T19:28:04.833447Z", "shell.execute_reply": "2022-08-17T19:28:04.832697Z" }, "originalKey": "773a2c32-4ff3-4e92-8996-325504ce953e" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n", "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to /home/runner/.data/MNIST/raw/train-images-idx3-ubyte.gz\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "c108f244a15c4745b921dda4fa965b3a", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/9912422 [00:00,\n", " ,\n", " )" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "load_mnist(data_path=\"~/.data\") # Pre-load the dataset before the initial evaluations are executed." ] }, { "cell_type": "markdown", "metadata": { "originalKey": "5fec848a-3538-489c-bcdd-a74051f48140" }, "source": [ "## 3. Define how to evaluate trials\n", "Since we use the Ax Service API here, we evaluate the parameterizations that Ax suggests, using RayTune. The evaluation function follows its usual pattern, taking in a parameterization and outputting an objective value. For detail on evaluation functions, see [Trial Evaluation](https://ax.dev/docs/runner.html). " ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2022-08-17T19:28:04.837469Z", "iopub.status.busy": "2022-08-17T19:28:04.836991Z", "iopub.status.idle": "2022-08-17T19:28:04.842015Z", "shell.execute_reply": "2022-08-17T19:28:04.841317Z" }, "originalKey": "75fce84d-35bd-45b5-b55e-f52baf26db03" }, "outputs": [], "source": [ "def train_evaluate(parameterization):\n", " device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", " train_loader, valid_loader, test_loader = load_mnist(data_path=\"~/.data\")\n", " net = train(\n", " net=CNN(),\n", " train_loader=train_loader,\n", " parameters=parameterization,\n", " dtype=torch.float,\n", " device=device,\n", " )\n", " report(\n", " mean_accuracy=evaluate(\n", " net=net,\n", " data_loader=valid_loader,\n", " dtype=torch.float,\n", " device=device,\n", " )\n", " )" ] }, { "cell_type": "markdown", "metadata": { "originalKey": "dda3574c-5967-43ea-8d23-7a151dc59ec9" }, "source": [ "## 4. Run optimization\n", "Execute the Ax optimization and trial evaluation in RayTune using [AxSearch algorithm](https://ray.readthedocs.io/en/latest/tune-searchalg.html#ax-search):" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "code_folding": [], "execution": { "iopub.execute_input": "2022-08-17T19:28:04.845562Z", "iopub.status.busy": "2022-08-17T19:28:04.845202Z", "iopub.status.idle": "2022-08-17T19:30:46.009894Z", "shell.execute_reply": "2022-08-17T19:30:46.009184Z" }, "hidden_ranges": [], "originalKey": "1d768bb2-d46b-4c4c-879e-3242af7555f4" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:08] ax.service.ax_client: Generated new trial 0 with parameters {'lr': 3.8e-05, 'momentum': 0.446225}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:09] ax.service.ax_client: Generated new trial 1 with parameters {'lr': 2e-06, 'momentum': 0.859993}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:09] ax.service.ax_client: Generated new trial 2 with parameters {'lr': 2.7e-05, 'momentum': 0.247377}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:21] ax.service.ax_client: Completed trial 1 with data: {'mean_accuracy': (0.536167, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:21] ax.service.ax_client: Generated new trial 3 with parameters {'lr': 5.6e-05, 'momentum': 0.818812}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:21] ax.service.ax_client: Completed trial 0 with data: {'mean_accuracy': (0.873, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:21] ax.service.ax_client: Generated new trial 4 with parameters {'lr': 0.353442, 'momentum': 0.055097}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:30] ax.service.ax_client: Completed trial 2 with data: {'mean_accuracy': (0.7985, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:30] ax.service.ax_client: Generated new trial 5 with parameters {'lr': 0.00133, 'momentum': 0.441355}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:31] ax.service.ax_client: Completed trial 3 with data: {'mean_accuracy': (0.931167, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:32] ax.service.ax_client: Generated new trial 6 with parameters {'lr': 0.00043, 'momentum': 1.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:39] ax.service.ax_client: Completed trial 4 with data: {'mean_accuracy': (0.093167, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:40] ax.service.ax_client: Generated new trial 7 with parameters {'lr': 0.000168, 'momentum': 0.632889}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:41] ax.service.ax_client: Completed trial 5 with data: {'mean_accuracy': (0.965833, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:42] ax.service.ax_client: Generated new trial 8 with parameters {'lr': 0.00225, 'momentum': 0.83229}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:48] ax.service.ax_client: Completed trial 6 with data: {'mean_accuracy': (0.102667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:49] ax.service.ax_client: Generated new trial 9 with parameters {'lr': 1e-06, 'momentum': 0.493107}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:51] ax.service.ax_client: Completed trial 7 with data: {'mean_accuracy': (0.924833, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:51] ax.service.ax_client: Generated new trial 10 with parameters {'lr': 0.000432, 'momentum': 0.274466}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:58] ax.service.ax_client: Completed trial 8 with data: {'mean_accuracy': (0.1105, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:28:59] ax.service.ax_client: Generated new trial 11 with parameters {'lr': 2.9e-05, 'momentum': 0.687405}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:01] ax.service.ax_client: Completed trial 9 with data: {'mean_accuracy': (0.204333, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:02] ax.service.ax_client: Generated new trial 12 with parameters {'lr': 0.000631, 'momentum': 9.3e-05}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:07] ax.service.ax_client: Completed trial 10 with data: {'mean_accuracy': (0.931833, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:08] ax.service.ax_client: Generated new trial 13 with parameters {'lr': 0.000284, 'momentum': 0.43319}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:10] ax.service.ax_client: Completed trial 11 with data: {'mean_accuracy': (0.880667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:12] ax.service.ax_client: Generated new trial 14 with parameters {'lr': 0.002826, 'momentum': 0.281116}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:18] ax.service.ax_client: Completed trial 12 with data: {'mean_accuracy': (0.946667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:19] ax.service.ax_client: Generated new trial 15 with parameters {'lr': 0.000116, 'momentum': 0.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:21] ax.service.ax_client: Completed trial 13 with data: {'mean_accuracy': (0.915833, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:22] ax.service.ax_client: Generated new trial 16 with parameters {'lr': 0.003499, 'momentum': 0.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:26] ax.service.ax_client: Completed trial 14 with data: {'mean_accuracy': (0.957, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:30] ax.service.ax_client: Generated new trial 17 with parameters {'lr': 0.001275, 'momentum': 0.279215}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:32] ax.service.ax_client: Completed trial 15 with data: {'mean_accuracy': (0.885167, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:34] ax.service.ax_client: Generated new trial 18 with parameters {'lr': 1.4e-05, 'momentum': 1.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:38] ax.service.ax_client: Completed trial 16 with data: {'mean_accuracy': (0.973833, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:41] ax.service.ax_client: Generated new trial 19 with parameters {'lr': 0.001778, 'momentum': 0.104203}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:43] ax.service.ax_client: Completed trial 17 with data: {'mean_accuracy': (0.961, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:44] ax.service.ax_client: Generated new trial 20 with parameters {'lr': 0.001532, 'momentum': 0.36512}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:49] ax.service.ax_client: Completed trial 18 with data: {'mean_accuracy': (0.7945, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:52] ax.service.ax_client: Generated new trial 21 with parameters {'lr': 0.001805, 'momentum': 0.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:53] ax.service.ax_client: Completed trial 19 with data: {'mean_accuracy': (0.966833, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:55] ax.service.ax_client: Generated new trial 22 with parameters {'lr': 0.005163, 'momentum': 0.102548}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:29:59] ax.service.ax_client: Completed trial 20 with data: {'mean_accuracy': (0.958333, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:01] ax.service.ax_client: Generated new trial 23 with parameters {'lr': 0.010201, 'momentum': 0.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:03] ax.service.ax_client: Completed trial 21 with data: {'mean_accuracy': (0.9545, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:07] ax.service.ax_client: Generated new trial 24 with parameters {'lr': 0.009422, 'momentum': 0.135133}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:09] ax.service.ax_client: Completed trial 22 with data: {'mean_accuracy': (0.110667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:10] ax.service.ax_client: Generated new trial 25 with parameters {'lr': 0.000135, 'momentum': 0.311582}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:14] ax.service.ax_client: Completed trial 23 with data: {'mean_accuracy': (0.116667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:16] ax.service.ax_client: Generated new trial 26 with parameters {'lr': 0.000848, 'momentum': 0.160155}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:18] ax.service.ax_client: Completed trial 24 with data: {'mean_accuracy': (0.111833, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:20] ax.service.ax_client: Generated new trial 27 with parameters {'lr': 8.9e-05, 'momentum': 0.485283}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:25] ax.service.ax_client: Completed trial 25 with data: {'mean_accuracy': (0.888333, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:27] ax.modelbridge.base: Untransformed parameter 0.40000000000000013 greater than upper bound 0.4, clamping\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:28] ax.service.ax_client: Generated new trial 28 with parameters {'lr': 0.4, 'momentum': 1.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:30] ax.service.ax_client: Completed trial 26 with data: {'mean_accuracy': (0.951667, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:31] ax.service.ax_client: Generated new trial 29 with parameters {'lr': 9e-06, 'momentum': 0.0}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:36] ax.service.ax_client: Completed trial 27 with data: {'mean_accuracy': (0.911, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:39] ax.service.ax_client: Completed trial 28 with data: {'mean_accuracy': (0.092, None)}.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[INFO 08-17 19:30:45] ax.service.ax_client: Completed trial 29 with data: {'mean_accuracy': (0.569167, None)}.\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Set up AxSearcher in RayTune\n", "algo = AxSearch(ax_client=ax)\n", "# Wrap AxSearcher in a concurrently limiter, to ensure that Bayesian optimization receives the\n", "# data for completed trials before creating more trials\n", "algo = tune.suggest.ConcurrencyLimiter(algo, max_concurrent=3)\n", "tune.run(\n", " train_evaluate,\n", " num_samples=30,\n", " search_alg=algo,\n", " verbose=0, # Set this level to 1 to see status updates and to 2 to also see trial results.\n", " # To use GPU, specify: resources_per_trial={\"gpu\": 1}.\n", ")" ] }, { "cell_type": "markdown", "metadata": { "originalKey": "cb00f812-e9e5-4208-a680-adf6619d74c4" }, "source": [ "## 5. Retrieve the optimization results" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2022-08-17T19:30:46.013199Z", "iopub.status.busy": "2022-08-17T19:30:46.012963Z", "iopub.status.idle": "2022-08-17T19:30:46.448568Z", "shell.execute_reply": "2022-08-17T19:30:46.447848Z" }, "originalKey": "2ec54675-d0ad-4eac-aaf3-66b593037cce" }, "outputs": [ { "data": { "text/plain": [ "{'lr': 0.0008475328951433113, 'momentum': 0.16015488542720355}" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "best_parameters, values = ax.get_best_parameters()\n", "best_parameters" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "execution": { "iopub.execute_input": "2022-08-17T19:30:46.452522Z", "iopub.status.busy": "2022-08-17T19:30:46.451540Z", "iopub.status.idle": "2022-08-17T19:30:46.456343Z", "shell.execute_reply": "2022-08-17T19:30:46.455654Z" }, "originalKey": "50c764a6-a630-4935-9c07-ea84045e0ecc" }, "outputs": [ { "data": { "text/plain": [ "{'mean_accuracy': 1.0094649148812018}" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "means, covariances = values\n", "means" ] }, { "cell_type": "markdown", "metadata": { "originalKey": "12a87817-4409-4f07-a912-8d60eff71d68" }, "source": [ "## 6. Plot the response surface and optimization trace" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "execution": { "iopub.execute_input": "2022-08-17T19:30:46.459437Z", "iopub.status.busy": "2022-08-17T19:30:46.459207Z", "iopub.status.idle": "2022-08-17T19:30:47.069617Z", "shell.execute_reply": "2022-08-17T19:30:47.068988Z" }, "originalKey": "3742f35b-6b28-49ae-a606-a138459f4964", "scrolled": false }, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "linkText": "Export to plot.ly", "plotlyServerURL": "https://plot.ly", "showLink": false }, "data": [ { "autocolorscale": false, "autocontour": true, "colorbar": { "tickfont": { "size": 8 }, "ticksuffix": "", "x": 0.45, "y": 0.5 }, "colorscale": [ [ 0.0, "rgb(247,252,253)" ], [ 0.125, "rgb(229,245,249)" ], [ 0.25, "rgb(204,236,230)" ], [ 0.375, "rgb(153,216,201)" ], [ 0.5, "rgb(102,194,164)" ], [ 0.625, "rgb(65,174,118)" ], [ 0.75, "rgb(35,139,69)" ], [ 0.875, "rgb(0,109,44)" ], [ 1.0, "rgb(0,68,27)" ] ], "contours": { "coloring": "heatmap" }, "hoverinfo": "x+y+z", "ncontours": 25, "type": "contour", "x": [ 1e-06, 1.3011511650442548e-06, 1.692994354296022e-06, 2.2028415765056147e-06, 2.866229883678204e-06, 3.729398352432554e-06, 4.852511011181743e-06, 6.3138503555892e-06, 8.215273746089953e-06, 1.0689313005882424e-05, 1.390841207112662e-05, 1.809694657026198e-05, 2.354686311364001e-05, 3.063802837345029e-05, 3.986470631277378e-05, 5.1870009063012666e-05, 6.749072272319499e-05, 8.781563250096393e-05, 0.00011426141253772724, 0.00014867137004306603, 0.00019344392634026088, 0.0002516997901283655, 0.0003274994751669172, 0.0004261263236648159, 0.0005544547624925005, 0.0007214294601814526, 0.000938688782612345, 0.0012213760031100258, 0.0015891948094037057, 0.002067782677737912, 0.0026904978401970136, 0.0035007443993213955, 0.004554997653699184, 0.005926740503884541, 0.007711585311544345, 0.010033938212454078, 0.013055670395116691, 0.01698740074503987, 0.02210317627048227, 0.028759573555516536, 0.03742055263793628, 0.04868979566145066, 0.06335278435066323, 0.0824315491666629, 0.10725590623460621, 0.13955614735503497, 0.18158364372009145, 0.23626776957937787, 0.3074200836506151, 0.4 ], "xaxis": "x", "y": [ 0.0, 0.02040816326530612, 0.04081632653061224, 0.061224489795918366, 0.08163265306122448, 0.1020408163265306, 0.12244897959183673, 0.14285714285714285, 0.16326530612244897, 0.18367346938775508, 0.2040816326530612, 0.22448979591836732, 0.24489795918367346, 0.26530612244897955, 0.2857142857142857, 0.3061224489795918, 0.32653061224489793, 0.3469387755102041, 0.36734693877551017, 0.3877551020408163, 0.4081632653061224, 0.42857142857142855, 0.44897959183673464, 0.4693877551020408, 0.4897959183673469, 0.5102040816326531, 0.5306122448979591, 0.5510204081632653, 0.5714285714285714, 0.5918367346938775, 0.6122448979591836, 0.6326530612244897, 0.6530612244897959, 0.673469387755102, 0.6938775510204082, 0.7142857142857142, 0.7346938775510203, 0.7551020408163265, 0.7755102040816326, 0.7959183673469387, 0.8163265306122448, 0.836734693877551, 0.8571428571428571, 0.8775510204081632, 0.8979591836734693, 0.9183673469387754, 0.9387755102040816, 0.9591836734693877, 0.9795918367346939, 1.0 ], "yaxis": "y", "z": [ [ 0.2693250387993388, 0.288165941827061, 0.31233098192729414, 0.34141941961553113, 0.3748080421850189, 0.41172637449544897, 0.4513259188937241, 0.49272959319548726, 0.5350587413874378, 0.5774443615880446, 0.6190349614966912, 0.6590152921949206, 0.6966458756064887, 0.7313180696280588, 0.7625972746217131, 0.7902266951140792, 0.8141055574138844, 0.8342981460716907, 0.8511424520544015, 0.8655293503537552, 0.8790925667082645, 0.8936989533754638, 0.9107754014539046, 0.9307598453412627, 0.9525890498200198, 0.9731869422252912, 0.987037171005951, 0.9863872368620119, 0.9624917589404811, 0.9077298037033565, 0.8182155235796618, 0.6962128184096129, 0.55208162185169, 0.40308686367389907, 0.26568979174935986, 0.15035363614417896, 0.0615317093249661, -0.0009012465986905083, -0.03981997276733851, -0.0593048758908018, -0.06367399613363922, -0.05693781149798027, -0.042526806174208454, -0.023186712725958736, -0.0009730494709040416, 0.02269689633133798, 0.046957276725751784, 0.07138005647768253, 0.09583019302523277, 0.12027552928987473 ], [ 0.2648975578841458, 0.28424497397413323, 0.30912128551927315, 0.3390902005154043, 0.37348108287477055, 0.4114728798705684, 0.4521697724393776, 0.49465345597758525, 0.5380097981510842, 0.5813377183741216, 0.6237542245926895, 0.6644117818649797, 0.7025404107273406, 0.7375106230567686, 0.7688856142898496, 0.7964272874312242, 0.8200703019532595, 0.8399270775152771, 0.8563932556343731, 0.8704180295457054, 0.8836766881686535, 0.8980682265536144, 0.9150471731096163, 0.9350695483271859, 0.957066106648337, 0.9779128636697825, 0.9919821238837707, 0.9913102963268898, 0.9668603017768642, 0.9107550703459104, 0.819035624284118, 0.694210610649902, 0.5472308013294538, 0.39602509040560424, 0.25734212169928916, 0.14150210944187647, 0.05273384705431472, -0.009283162533858857, -0.04757630191055218, -0.0663347153744932, -0.06994962215371181, -0.06247875242338752, -0.04738142636573839, -0.027419208496589964, -0.004654244941873054, 0.01949616597174153, 0.044171135457558686, 0.06895167717615502, 0.09371561304963871, 0.11844611740885214 ], [ 0.2605273257155551, 0.28038679503342356, 0.3059833393210666, 0.3368423405733909, 0.37224272245389056, 0.41131052724724687, 0.4531014130048876, 0.4966557366588088, 0.541024549682774, 0.5852757169181416, 0.6284956554144774, 0.6698047735310447, 0.7084017424152217, 0.7436348421684174, 0.7750645480921518, 0.8024723150246281, 0.8258304642334582, 0.8453046305008785, 0.8613581561033556, 0.8750073244461389, 0.8879595016632736, 0.9021324897094372, 0.9190001942756324, 0.9390324999873534, 0.9611509776589658, 0.9821822792590003, 0.9963997714278503, 0.9956517433823253, 0.9706327244667707, 0.9132355934688622, 0.819452215312082, 0.6920673636530084, 0.542624753824939, 0.3896052251282874, 0.24994046522953023, 0.13376934629750825, 0.045113489014463326, -0.01650826727038024, -0.054243048264064786, -0.07236492683424822, -0.07532262573813053, -0.06721144684649794, -0.0515139998930112, -0.03100493911494495, -0.00775223153816007, 0.01682619906016336, 0.04187262713144424, 0.0669741558479412, 0.09201722563006975, 0.1169956138435253 ], [ 0.25624179508278205, 0.27661856495097464, 0.3029435127367816, 0.3347007545017974, 0.37111566940336305, 0.41125913132033776, 0.4541372681885555, 0.498749246338566, 0.5441122531821105, 0.5892644447739656, 0.6332629370703982, 0.6751967398354017, 0.7142326539985832, 0.7496954203834549, 0.7811422198980849, 0.8083746436770826, 0.8314042725310429, 0.8504542290587089, 0.8660640134752451, 0.8793224839422709, 0.8919625360796624, 0.9059088761965588, 0.9226470197306773, 0.9426572502362769, 0.9648492824011851, 0.9859995914129309, 1.0002952006400714, 0.9994212699876562, 0.9738337200066207, 0.9152283477590946, 0.8195752375082781, 0.6899550133306477, 0.5384660587666827, 0.38402522586626486, 0.24364960934894886, 0.12728206399368425, 0.038767095305107024, -0.02250352510500353, -0.05976506394766257, -0.07735372236371363, -0.07976111315489276, -0.07111131325774278, -0.05490534232007582, -0.03392868485998901, -0.010254664221376997, 0.01469731707073163, 0.040070736445960264, 0.06545571002455364, 0.09074289592743989, 0.11593105027371142 ], [ 0.2520707984767388, 0.272969786760532, 0.30003038821295397, 0.3326923468490121, 0.37012432861974365, 0.4113398815587629, 0.4552948328551182, 0.5009475970210765, 0.5472827485906226, 0.5933103963999704, 0.6380600293724402, 0.6805903960460508, 0.720036336903496, 0.7556979053375891, 0.7871283965428949, 0.8141498280147131, 0.8368138708289758, 0.8554042438122229, 0.8705425809122819, 0.8833929060299379, 0.8957096072830175, 0.9094146386068753, 0.9259982172929743, 0.9459486047016188, 0.9681620474798216, 0.989364637876239, 1.003668941201345, 1.0026235840143292, 0.976482444276229, 0.9167847397246659, 0.8195082406917406, 0.6880330323949432, 0.5349424126601668, 0.37946034568137893, 0.23860518979326872, 0.12213906692496723, 0.03376660937976117, -0.02721607796758785, -0.06410321697498034, -0.08127184716019431, -0.08324297935019986, -0.0741614714780926, -0.05754243139060555, -0.03618029861426597, -0.012153552885199148, 0.013115888002889875, 0.03877061658341807, 0.06440059571806367, 0.0898962314688001, 0.11525542092938945 ], [ 0.24804658158662418, 0.26947233282557287, 0.2972747680535078, 0.33084599050478736, 0.3692947507870081, 0.4115752705592625, 0.45659258981291584, 0.5032651280579197, 0.5505463994016595, 0.5974204260574817, 0.6428911127526511, 0.685988595123341, 0.725816201543129, 0.76164840099696, 0.7930340220375561, 0.8198154305853016, 0.8420842414705654, 0.8601862655779711, 0.8748283518222852, 0.8872501634843151, 0.8992253085683215, 0.9126658368935436, 0.9290610780839497, 0.9489062480798425, 0.9710841671296133, 0.9922707918402431, 1.0065142587099032, 1.0052544101287122, 0.978586456378016, 0.917940659468315, 0.8193313878876958, 0.6864244717590695, 0.5321996732904447, 0.3760413802691161, 0.23490100009164405, 0.11840472308336936, 0.0301558281922345, -0.030615465610913883, -0.06723575949901794, -0.08410343143337795, -0.0857564466846914, -0.07635308934001184, -0.05941863442271589, -0.03775485850482618, -0.013445368313311334, 0.012084248306722056, 0.037973524194071184, 0.06380903714848041, 0.08947645416048733, 0.11496755321897845 ], [ 0.24420381245930262, 0.2661604454194232, 0.2947096528400924, 0.32919247375491206, 0.36865454873680426, 0.41198899042013143, 0.45804990206962504, 0.5057168098772995, 0.5539140188222915, 0.6016016971096236, 0.64776054074281, 0.6913942352101242, 0.7315756891297317, 0.7675532493975735, 0.7988707167597673, 0.8253902315160256, 0.847241979633889, 0.8648333629028578, 0.8789566214939198, 0.8909260481889014, 0.9025333583245495, 0.9156759134983603, 0.9318382753531604, 0.9515232971325542, 0.9736025190132866, 0.994702562726482, 1.0088141415577405, 1.0072964636834798, 0.980136185509239, 0.9187075164851232, 0.8190873477996427, 0.6851975187203815, 0.5303156596932505, 0.373832629115057, 0.2325783333817369, 0.11610356835330216, 0.027947610256690014, -0.03269511593916863, -0.0691591796816392, -0.08584649484211215, -0.0873003697047745, -0.07768556841458918, -0.06053382021901299, -0.038652731145069286, -0.01413106935347741, 0.011600708966654771, 0.03767686822241301, 0.0636773569560597, 0.0894787639364567, 0.11506249858376816 ], [ 0.24057956115183293, 0.2630707063250845, 0.2923701858807873, 0.3277644102687945, 0.36823277566243384, 0.4126057928371959, 0.45968687311949086, 0.508318121830015, 0.5573967783225352, 0.6058616259795652, 0.652672804021271, 0.6968101907973558, 0.7373181011381764, 0.7734187148643598, 0.804650253351371, 0.8308933772819916, 0.8523140123899553, 0.869378430380428, 0.8829617283886424, 0.8944507359658787, 0.9056549092635797, 0.9184542108513268, 0.9343265130420931, 0.9537848402059668, 0.97569388990996, 0.9966327309710972, 1.0105379991469388, 1.0087157138648764, 0.9811003419283102, 0.9190652554606958, 0.8187717055882189, 0.6843546049006886, 0.5292843884375141, 0.37281873950753935, 0.23161978294032648, 0.11521678733544516, 0.027122270383831637, -0.03347298776643526, -0.06988848499673417, -0.08651307712994649, -0.0878842919698054, -0.07816656121514265, -0.060894351482884246, -0.03887954370155866, -0.01421605141748472, 0.011659642864949471, 0.03787436237839292, 0.06399826433194444, 0.08989489714537613, 0.11553210686345938 ], [ 0.23721324395823723, 0.26024196938411753, 0.29029355737549567, 0.32659610582024345, 0.36805975976853406, 0.4134513086277443, 0.46152417182322214, 0.5110849008915412, 0.5610060948586719, 0.6102078170547045, 0.65763250623017, 0.7022392786159235, 0.743046465498848, 0.7792506966497141, 0.8103840420145978, 0.8363435115748302, 0.857326323551057, 0.8738526469984691, 0.8868754422452668, 0.8978510890308595, 0.908606888857748, 0.9210044825832194, 0.9365152139111956, 0.955666571992795, 0.9773229800136628, 0.9980191690519193, 1.0116381410384663, 1.0094583478225616, 0.9814225526122495, 0.9189579428855787, 0.8183284524024173, 0.6838299716241092, 0.5290190511747854, 0.37290824788934807, 0.23194909446866435, 0.1156812188424613, 0.027627542667221672, -0.03299127382415534, -0.06945689138278266, -0.0861289854977273, -0.08752825087467475, -0.07781181823528427, -0.06051295754098118, -0.03844606590457822, -0.013710019439742593, 0.012251645813576184, 0.038556260732369396, 0.06476123324141869, 0.09071366950727944, 0.11636556968916645 ], [ 0.23414652544683923, 0.25771524907807253, 0.28851886117811976, 0.32572337496256526, 0.3681668897652813, 0.41455182257444173, 0.4635828187646056, 0.5140331580916617, 0.5647534924570342, 0.6146479825112985, 0.6626443470841258, 0.707684266624387, 0.7487634604719603, 0.7850544979999021, 0.8160826544296982, 0.8417579205053745, 0.8623027162664758, 0.8782840303767689, 0.8907254677675747, 0.9011490852837759, 0.9114004107303841, 0.9233234433525336, 0.938385298482209, 0.9571336501171155, 0.978440835358767, 0.9988019593867221, 1.012046503359353, 1.009448961185636, 0.9810198036793034, 0.9182924359224185, 0.8176504478177177, 0.6834947211257898, 0.529366314707152, 0.373947546538444, 0.23343700635327413, 0.11739208627860842, 0.029380347956859953, -0.031315133837153564, -0.06791492465798155, -0.0847331671424989, -0.08626233660398308, -0.07664486919229252, -0.0594084909688436, -0.03736800588340283, -0.012626791022477768, 0.013363760362134935, 0.039709651738339624, 0.06595291211715315, 0.0919214604056614, 0.11754989844633801 ], [ 0.23142317059227874, 0.2555335572747574, 0.2870868961521163, 0.3251833003331806, 0.36858634560485753, 0.4159339999012909, 0.4658839315547024, 0.5171788599793514, 0.568650433586662, 0.6191898387657627, 0.6677131016652785, 0.713147926844782, 0.7544714184371379, 0.7908346805609326, 0.8217554059542731, 0.8471517115252866, 0.8672636256703145, 0.882696062984495, 0.8945340338676315, 0.9043603560638886, 0.9140392780074434, 0.9253993983101423, 0.9399080992462484, 0.9581398787164044, 0.9789840286362052, 0.9989021858854414, 1.011673440914905, 1.0085904487835298, 0.9797831276515854, 0.9169404041996476, 0.8165843357597089, 0.6831671823423993, 0.5301228114113704, 0.3757367848324913, 0.2359113116893628, 0.12020931210553387, 0.032270265473439586, -0.028530517502726838, -0.06532897576477192, -0.08237673314239291, -0.08412602121259338, -0.07469654861820585, -0.05760557531289712, -0.035665726193701786, -0.01098403749105481, 0.014979749906337991, 0.04131878736767114, 0.06755752883093513, 0.09350263969582673, 0.11907034083803136 ], [ 0.22908883821329779, 0.2537416792019784, 0.28603990316060957, 0.3250139267965935, 0.3693507690763114, 0.41762456140166654, 0.4684484274464391, 0.5205376732407462, 0.5727081160459018, 0.6238409696498748, 0.6728435770515193, 0.7186331163067615, 0.7601724318280341, 0.7965950315237714, 0.8274099974271706, 0.8525370333446585, 0.8722249851557264, 0.887106367096117, 0.8983165392423669, 0.907492812363825, 0.9165185918234884, 0.927210996845256, 0.941044443836102, 0.9586272713195585, 0.9788747255805839, 0.9982231433676946, 1.0104097629856594, 1.0067657212172476, 0.9775804995400605, 0.914743586927041, 0.8149392092716017, 0.6826264892719567, 0.5310513374277888, 0.3780450283435021, 0.23916870209136526, 0.1239654007547405, 0.03616427310182746, -0.02474122771187015, -0.06177938497520841, -0.07912167605522502, -0.08116728273079143, -0.07200438116329566, -0.05513415432766566, -0.03336388850849081, -0.008802971921236358, 0.017080410536227997, 0.04336542857604009, 0.06955727356257835, 0.09543994386063492, 0.12091074363844634 ], [ 0.22719080571874795, 0.2523858785091007, 0.2854212277180135, 0.32525388232015806, 0.3704928694095188, 0.41964990528912316, 0.4712966828461439, 0.5241246718134149, 0.5769372321210378, 0.6286086462165845, 0.6780405208355538, 0.7241428419141118, 0.7658685757008654, 0.8023386574167041, 0.8330521895367454, 0.8579223355281417, 0.8771971511312581, 0.8915254136208288, 0.9020802255968599, 0.9105453389198954, 0.9188234796850706, 0.9287261700232156, 0.9417439376550245, 0.958525979672854, 0.9780215393941345, 0.9966529656904081, 1.0081302289439167, 1.0038409046695111, 0.9742615698044861, 0.9115219016350211, 0.8124983373525273, 0.6816279740240675, 0.5318963456308186, 0.3806241617977012, 0.2429865782254012, 0.12847374997657157, 0.04091220294159226, -0.020065436865998043, -0.05735815679636869, -0.07503933679062336, -0.07744155658192764, -0.06861184638119111, -0.05202895586169176, -0.030491037099392937, -0.0061079939210212375, 0.019643909016485495, 0.045829193638160626, 0.0719326533275837, 0.09771480826691692, 0.1230538687703665 ], [ 0.22577761380321, 0.25151351992669224, 0.28527489728827343, 0.32594191742595446, 0.37204495999858694, 0.4220356752864495, 0.4744481508693863, 0.5279540074782492, 0.5813476886351208, 0.6334995949827827, 0.6833084543215028, 0.7296802316189994, 0.7715621911189317, 0.8080681368582181, 0.8386854522644692, 0.8633116640908713, 0.8821839007172652, 0.8959552683732779, 0.9058228508189474, 0.9135065342496894, 0.920927969700233, 0.9299013364586458, 0.9419445035645242, 0.9577545181602807, 0.9763209696202149, 0.9940676922471737, 1.0046972461387362, 0.9996694149631526, 0.96966364031655, 0.9070837631110629, 0.8090334157920673, 0.6799195325492837, 0.5323983915658743, 0.3832212968715008, 0.24713388575222178, 0.13353678812251968, 0.046352428970845905, -0.014631891241959627, -0.05216642316429909, -0.07020868538849678, -0.0730105520438955, -0.06456754614661153, -0.04832888562686688, -0.02707913241381943, -0.002926301155492861, 0.022646136778779802, 0.04868790050845129, 0.07466281735936853, 0.10030766145403924, 0.12548166897443924 ], [ 0.22489861820191326, 0.25117259652171375, 0.28564510121030734, 0.32711635544227846, 0.3740384238719249, 0.42480627634919005, 0.47792093996486057, 0.5320385469040367, 0.5859482891379036, 0.6385197106623793, 0.6886514096864562, 0.7352483238095671, 0.7772558985442436, 0.8137854269615016, 0.8443105233562359, 0.8687039948716497, 0.8871815318980053, 0.9003884124784707, 0.9095313396614829, 0.9163534721304236, 0.9227940628501942, 0.9306809881313557, 0.9415723227702524, 0.956220180203332, 0.9736593306970001, 0.9903345938328347, 0.9999647156376023, 0.9940963119181037, 0.963618237302817, 0.90123753244939, 0.8043209814685361, 0.6772580321739382, 0.5323069773988102, 0.3855894621950364, 0.25138056096344247, 0.13895351054601013, 0.05231742507886017, -0.00857602578481631, -0.04631177654274632, -0.06471448472476893, -0.06794097472947269, -0.05992430008681271, -0.0440763675129352, -0.02316304676003611, 0.0007125225804088808, 0.02606107142480829, 0.05191789778177813, 0.0777258547996258, 0.10319818651774315, 0.1281755284885473 ], [ 0.2246034338916888, 0.2514111468575784, 0.28657556039049203, 0.3288144468369402, 0.3765031077615539, 0.42798434175994626, 0.4817313587695897, 0.5363894803564411, 0.5907463826711359, 0.6436737156168083, 0.6940725655923464, 0.7408496306836624, 0.7829519539406672, 0.8194912070896787, 0.8499248464869793, 0.8740926182950819, 0.8921781112467877, 0.9048067229855598, 0.913180419691483, 0.9190504602400136, 0.9243711050360732, 0.9309977770284968, 0.9405424389452158, 0.9538198105265185, 0.9699153136426055, 0.9853158791202663, 0.9937822846846018, 0.9869626926072133, 0.9559578671481308, 0.8938024288237105, 0.7981599023620142, 0.6734237509292664, 0.5313910989086739, 0.3874963871464743, 0.25550542585304997, 0.14452614074456782, 0.05863893823268229, -0.0020361784704368002, -0.03990558819694756, -0.0586454059368523, -0.06230319705594145, -0.05473819544317493, -0.03931664784326083, -0.01878003437331388, 0.00477693338970786, 0.02986113873826013, 0.05549438108451654, 0.08109906686500057, 0.10636555385415025, 0.13111647338409538 ], [ 0.2249412551880694, 0.25227654550006695, 0.28810877346350244, 0.3310716230026245, 0.37946664771486266, 0.43159015812148066, 0.4858934346765748, 0.5410159096032984, 0.595747486913736, 0.6489647752523722, 0.6995737953435373, 0.7464855133957986, 0.788651222381863, 0.8251838859557098, 0.8555219262443221, 0.879464602154066, 0.8971529237282319, 0.9091807518571532, 0.9167313953615007, 0.9215478412080498, 0.9255956285921748, 0.9307731961725987, 0.938760259492655, 0.950442290478057, 0.9649633912515252, 0.9788728840120692, 0.9860002697779923, 0.9781112863033725, 0.9465228794674725, 0.8846171312189106, 0.790382762356615, 0.6682298203470168, 0.5294469287592517, 0.38873127770593957, 0.25930250465351046, 0.15006576896039248, 0.06515260659392941, 0.004849947861554793, -0.03306041306894414, -0.052092160122062636, -0.05616991756754752, -0.04906761765869272, -0.03409708103078046, -0.013969187995728105, 0.009233858677714868, 0.034017569472385656, 0.059391693089386965, 0.08475921568497535, 0.10978862882139628, 0.1342853555204926 ], [ 0.22596003301333673, 0.2538146482780157, 0.29028512652676364, 0.33392064845436875, 0.3829537332814464, 0.4356410579650823, 0.4904184159756352, 0.5459244247655777, 0.6009548966371173, 0.6543940849871424, 0.7051551586546302, 0.7521554645505488, 0.7943521852660165, 0.8308586443805486, 0.8610906827282735, 0.8848003665593654, 0.9020761760725322, 0.9134694580817257, 0.920131586036214, 0.923781416874064, 0.9263918650857406, 0.9299188822050293, 0.9361239012851001, 0.9459725008115203, 0.9586779604729148, 0.9708707152548796, 0.9764752552268456, 0.9673937013019962, 0.9351683064815601, 0.8735453644204656, 0.7808565307864972, 0.6615255915718518, 0.5263025085235358, 0.3891096127883882, 0.2625858165901792, 0.15539691761052254, 0.0717019211009352, 0.011948469651602345, -0.02588756450108598, -0.04514570355631908, -0.04961484706752306, -0.04297228682122811, -0.028466413539121715, -0.008770893611844133, 0.01404912594212071, 0.038500746200614944, 0.06358360610683744, 0.08868275175665485, 0.11344615726055429, 0.1376630134764596 ], [ 0.2277054881689332, 0.25606877158538727, 0.2931418555964032, 0.3373906760748041, 0.3869853224562948, 0.44015079315569744, 0.49531426972518877, 0.5511186817893108, 0.6063692908804762, 0.6599604489405496, 0.7108143770653444, 0.7578563955671811, 0.800050093801244, 0.8365066370778894, 0.8666148864392335, 0.8900734031751896, 0.9069089882477219, 0.9176204895153339, 0.9233150292059558, 0.9256735743117053, 0.9266730396502448, 0.9283384835226975, 0.932527060788882, 0.9402953174277358, 0.9509377976083877, 0.9611831737610632, 0.9650760471974853, 0.9546776114966311, 0.9217703624775305, 0.8604792161092275, 0.7694807820689762, 0.6531962160772774, 0.5218198529764587, 0.38847612924930297, 0.2651927596929378, 0.16036105238826015, 0.0781414842458722, 0.019132083723264914, -0.018494922252312418, -0.03789556437244346, -0.04271145521155273, -0.03651232304618346, -0.022474081964592774, -0.0032262941998770778, 0.01918788175364894, 0.043280536324867835, 0.06804358659467324, 0.09284602162896893, 0.11731693128896992, 0.1412304132917268 ], [ 0.2302199367080549, 0.2590784830288118, 0.2967118560083048, 0.3415062184227487, 0.3915778256815361, 0.4451289058151615, 0.50058518956138, 0.5565989936818689, 0.6119883538030408, 0.6656598725074803, 0.7165463331880964, 0.7635819969420695, 0.8057362747766676, 0.842114358488735, 0.8720727220269077, 0.8952501605612252, 0.9116036821691547, 0.9215709949451361, 0.9262042238636943, 0.9271356509975823, 0.9263433549397393, 0.9259299705618277, 0.9278620857892348, 0.9332993854389542, 0.9416304225278868, 0.9496976851614753, 0.9516895593650094, 0.9398532278684403, 0.906232465759221, 0.8453413124284306, 0.7561866669806595, 0.6431610368984275, 0.5158951862925613, 0.3867062718298229, 0.26698624381254504, 0.16481910537200406, 0.08433956273359222, 0.02628236843464815, -0.010985017818172071, -0.030428328339204613, -0.03553180595447247, -0.02974736115432497, -0.01616953952931488, 0.002623227658736793, 0.0246149945639349, 0.04832660798317179, 0.07274504111112956, 0.09722545709898878, 0.1213799373320057, 0.14496877139926823 ], [ 0.23354090043706388, 0.26287817956384685, 0.30102234340078715, 0.34628605965222886, 0.39674228397703265, 0.45058011680770016, 0.5062311292750108, 0.5623619495410525, 0.6178064246605038, 0.6714851904990243, 0.7223426280590614, 0.7693222104278987, 0.8113975854839446, 0.8476631549092615, 0.8774364948125322, 0.9002901049763568, 0.9161043532568899, 0.9252488500724201, 0.9287122389093426, 0.92807048680559, 0.9253004190258685, 0.9225882341782452, 0.9220230518544784, 0.9248806209901987, 0.9306561207825893, 0.9363199316288309, 0.9362262877321497, 0.9228388114733113, 0.888490034635326, 0.8280864093136641, 0.740936391703296, 0.6313718971873801, 0.508458028856952, 0.3837064344345349, 0.26785575662120514, 0.16865310654733345, 0.09017996246853954, 0.033291614181833684, -0.00345342328591125, -0.022826310945857298, -0.02814550445941033, -0.022735731802647097, -0.009601623422917038, 0.008736539660800124, 0.03029543582983385, 0.05360872613700418, 0.0776615433213923, 0.10179774691457433, 0.1256144879870389, 0.14885966174051013 ], [ 0.2376994721411343, 0.26749543097757417, 0.30609339005712205, 0.35174214715078456, 0.40248357303744753, 0.4565037545649108, 0.5122473790136917, 0.5684000756347031, 0.6238141918203883, 0.6774257500933254, 0.7281912224281182, 0.7750628344337606, 0.8170160216601557, 0.8531288651589582, 0.8826724689162643, 0.905145960230736, 0.9203476973363065, 0.9285741589323033, 0.9307448566203816, 0.9283749024045955, 0.9234378669573468, 0.9182078250448041, 0.9149087407115886, 0.9149453913519408, 0.9179314691005325, 0.9209778903701877, 0.9186250960255016, 0.903585263826873, 0.8685133252855457, 0.8087024538873631, 0.7237228370171018, 0.6178115679955162, 0.4994696623020529, 0.37941331091242364, 0.26771755380789536, 0.1717670363551771, 0.09556327619597038, 0.04006419299821129, 0.004012545450393312, -0.015166433070779584, -0.020618772158555565, -0.01553372276514331, -0.002817973360013881, 0.0150732148154894, 0.03619463480592011, 0.05909702659198324, 0.08276704171856442, 0.10653999174482753, 0.1300003390095118, 0.1528851087280082 ], [ 0.24271840129006528, 0.2729490775019894, 0.31193639209722174, 0.3578785183237438, 0.40879967089845226, 0.4628932487508076, 0.51862420124681, 0.5747015523435566, 0.6299984443302096, 0.6834671641626842, 0.7340761792684344, 0.780785274648365, 0.8225684927840762, 0.8584815915355893, 0.8877408111323228, 0.9097641395602382, 0.9242640640114421, 0.9314609285699477, 0.9322026951646294, 0.9279421021629515, 0.9206480106014465, 0.9126857163260422, 0.9064254603642876, 0.9034133285181152, 0.9033922577467226, 0.9036250217437185, 0.8988569651694233, 0.8820802855572273, 0.8463097213829321, 0.7872110188068134, 0.704569025368672, 0.6024922266057884, 0.48892128434341275, 0.3737926258365115, 0.2665141561734139, 0.17408701718872666, 0.10040756765194603, 0.04651749210130718, 0.011334813016449563, -0.007519308539956304, -0.013013660913932745, -0.008194930524029642, 0.004135490461698921, 0.02159394567236217, 0.04227880322002031, 0.06476226614306069, 0.08803604777324014, 0.11142984300549974, 0.13451779246807904, 0.15702766744604624 ], [ 0.24860986168997695, 0.27924710955264576, 0.31855256868441567, 0.3646903330294167, 0.4156810302681274, 0.46973571357832766, 0.5253465430588441, 0.5812499996928149, 0.6363418925043728, 0.6895911462997905, 0.7399775170902182, 0.7864664456627275, 0.8280267859758523, 0.8636856623553151, 0.8925956419785323, 0.9140854046740841, 0.9277787118994012, 0.9338188640364282, 0.9329833302393866, 0.92666402674212, 0.9168244387233382, 0.9059239986575619, 0.8964896576525618, 0.8902197423263264, 0.8869957409530773, 0.8842424159185547, 0.876927224191799, 0.8583518352677805, 0.8219252676577691, 0.763666938680505, 0.6835272328611406, 0.5854538774917264, 0.4768320068923768, 0.3668374525029874, 0.2642133165027219, 0.17556095888810075, 0.10464856285564572, 0.05258244664501288, 0.018445228351642973, 5.145587579535871e-05, -0.005387411903434947, -0.0007697088801504881, 0.011215024017742659, 0.02826091318309254, 0.048515226900201824, 0.07057604745546753, 0.09344380429906518, 0.11644562600757113, 0.13914778692739493, 0.16127049225385737 ], [ 0.255372857756913, 0.28638447281172635, 0.32593164656446705, 0.37216309125031133, 0.4231100965546838, 0.4770116441883274, 0.5323938398720611, 0.5880243425423475, 0.6428230665406043, 0.6957754345762286, 0.7458711769579462, 0.7920788227383042, 0.8333577340476214, 0.8686999125036629, 0.8971854644382837, 0.918045791810108, 0.930813242319372, 0.9355552654945758, 0.9329834521818903, 0.9244336890517699, 0.911864550509152, 0.8978324491885739, 0.8850302869777831, 0.8753176159753853, 0.8687221887507672, 0.8628398113222258, 0.8528758538709464, 0.8324679291293129, 0.7954437078400798, 0.7381569613486901, 0.6606776359055729, 0.5667626324161006, 0.46324676016523014, 0.35856626460537083, 0.2608065933342532, 0.1761577647836774, 0.10823942077314486, 0.0582037133702169, 0.025285967377331797, 0.007489753901162133, 0.002208039884943891, 0.00669528194912361, 0.018379446878570627, 0.035038107012008046, 0.054872522254413636, 0.07651101770403457, 0.09896643392110593, 0.12156644782774118, 0.1438719753760349, 0.16559739477306656 ], [ 0.26299024709517244, 0.2943411910727917, 0.3340509185555461, 0.3802721133051688, 0.43106100809673475, 0.4846947462887467, 0.5397399233834856, 0.5949987643869759, 0.649416299538432, 0.7019938070944509, 0.7517291004012226, 0.7975906349312366, 0.8385235815678881, 0.8734783280614686, 0.9014541997503138, 0.9215777954146249, 0.933287181341414, 0.9365770243334955, 0.9321010988930356, 0.9211475352581441, 0.9056720454881627, 0.8883309889856854, 0.8719909442034876, 0.8586791862223858, 0.8485757588130706, 0.8394555310009084, 0.8267759188112797, 0.8045321354991941, 0.7669827029308697, 0.7107973998113897, 0.6361264729978909, 0.5465088049029743, 0.4482341230352437, 0.349020818952202, 0.25630764177604637, 0.17586619303296458, 0.11115015344152057, 0.06333953007093729, 0.03180967915849375, 0.014747277598602726, 0.009726418734541165, 0.014157425212144914, 0.025590458554237472, 0.04189159542571119, 0.061320856304243, 0.08254104039813659, 0.10458106765886832, 0.12677229027228265, 0.14867279151664015, 0.16999289209531332 ], [ 0.27142648035725064, 0.3030814572270443, 0.34287484866444046, 0.3889823428449036, 0.4394995060347392, 0.4927519143875885, 0.5473530434597395, 0.6021427562620305, 0.656091798658546, 0.7082161889873202, 0.7575194107767647, 0.8029661827205374, 0.8434825153002611, 0.8779709222486874, 0.9053424190995406, 0.9246117253461646, 0.9351196675116084, 0.9367927144840993, 0.9302380047101815, 0.9167078804265751, 0.8981594174772823, 0.8773520793622942, 0.8573317948975152, 0.8402971260554994, 0.8265847475159283, 0.8141555062187739, 0.7987307056068036, 0.7746782343537746, 0.7366889075937372, 0.681731027609481, 0.6100037804689886, 0.5248048110776781, 0.4318840849493, 0.3382639312562369, 0.25075030662121006, 0.1746934548227742, 0.1133667610391762, 0.06796130654232335, 0.03797940344933037, 0.02178363440037734, 0.017127325393947457, 0.021578398523394382, 0.03281288374689284, 0.04878974411733339, 0.06783212972988073, 0.0886413402144105, 0.11026595378184167, 0.13204408831285297, 0.15353350496920337, 0.17444224593257118 ], [ 0.2806289582357761, 0.3125541471947306, 0.35235531136433895, 0.3982485023684704, 0.44838306864658395, 0.5011433678768916, 0.5551960101687878, 0.6094212645877136, 0.6628158056735625, 0.7144088478377447, 0.7632066873406336, 0.8081662571228747, 0.8481893082502074, 0.8821246823109214, 0.908788538694304, 0.9270771229694569, 0.9362311900570186, 0.9361147626683554, 0.9273020998598668, 0.9110254755755106, 0.8892505138352249, 0.8648430965910834, 0.8410313080194264, 0.8201853544886497, 0.8028013071396215, 0.78703162776701, 0.7688702787665462, 0.7430654381562808, 0.7047332175949983, 0.6511235904857798, 0.5824608125242801, 0.5017828967599297, 0.4143057427265687, 0.32637718362947793, 0.24418658195770976, 0.17266361794124807, 0.11489014138463971, 0.07205299162394685, 0.04376828919279463, 0.02856630079913125, 0.02437632756559993, 0.028924329992203734, 0.04001484810965017, 0.05570338443326439, 0.07438012308338882, 0.09478862103949925, 0.11600054725592657, 0.1373637944067666, 0.15843826590025467, 0.17893149334678993 ], [ 0.29053112345243143, 0.32269451325089876, 0.3624324268084836, 0.4080155939890797, 0.45766127077020524, 0.5098229475879094, 0.5632264582910891, 0.6167949390454188, 0.6695508459059467, 0.7205346726157743, 0.768752319609896, 0.8131486345179249, 0.8525960261975489, 0.8858845053994165, 0.9117299447452514, 0.9289041430053795, 0.9365453146005243, 0.9344616594020729, 0.9232101785779672, 0.9040222690512687, 0.8788832283241702, 0.8507687068242572, 0.8230877833676008, 0.798379495995568, 0.7773007266331297, 0.7581996694462051, 0.7373479554951252, 0.7098742114312551, 0.6713065416334462, 0.6191602597943047, 0.5536672721826115, 0.4775927268523505, 0.39562493784912445, 0.3134585878842857, 0.236684484824721, 0.16981587225799744, 0.1157348267779611, 0.07561025868013094, 0.04915914396892174, 0.03507043185335901, 0.031444934846780015, 0.03616586555454615, 0.04716788749021705, 0.06260593245290413, 0.08094060696196204, 0.10096115778370307, 0.12176558026681039, 0.14271442916754795, 0.16337213957485275, 0.1834474696324896 ], [ 0.30105527963160345, 0.33342640728951883, 0.373035849954033, 0.4182197046783178, 0.467276355575544, 0.518738568978787, 0.5713972328065371, 0.6242204788694921, 0.6762560625609368, 0.7265535301493795, 0.7741149296973985, 0.8178686224148483, 0.8566527558924837, 0.8891940930220115, 0.914104032599203, 0.9300248423157619, 0.9359903316843403, 0.9317601431872126, 0.9178907145583304, 0.8956344295635189, 0.8670123987890095, 0.8351132248018356, 0.8035206338044885, 0.7749370037227339, 0.7501803677561557, 0.7277969825083715, 0.7043369381292317, 0.6753025883207748, 0.6366160813911002, 0.5860422236875067, 0.5238084651131252, 0.4523988775300817, 0.3759818420498847, 0.29962022014377715, 0.22832587806936427, 0.16620270335756349, 0.11592759458030688, 0.07863954884974478, 0.054143843865796515, 0.04127854651713958, 0.038310470646136796, 0.04327815484925934, 0.05424699477530648, 0.0694734612150304, 0.0874914174944248, 0.10713886285714269, 0.12754311447411426, 0.14808011891760442, 0.16832113132941773, 0.18797782388170825 ], [ 0.31211506261678007, 0.34466455176859445, 0.3840863335354491, 0.4287890535897891, 0.47716399559222633, 0.5278328225003899, 0.5796568902291508, 0.6316510733417601, 0.6828876317220725, 0.7324226927327177, 0.7792508521438725, 0.8222796357527444, 0.8603083262307423, 0.8919967920175121, 0.9158491497153428, 0.9303743434490228, 0.9345007676124005, 0.9279472582569643, 0.9112867218693229, 0.8858156964010163, 0.8536129742167784, 0.8178828603674823, 0.7823713508261751, 0.7499369499714443, 0.721558332038748, 0.6959800950580362, 0.6700271726077456, 0.6395628822672508, 0.6008820251450913, 0.5519834927594649, 0.49308245566660047, 0.42637826671261064, 0.3555284993970278, 0.2849858374342875, 0.21920426888046984, 0.16188801219279958, 0.1155059918703577, 0.08115700802603676, 0.058722632048071155, 0.047180108977677326, 0.04495585442535688, 0.050240763971659996, 0.061230609349143594, 0.07628472908305883, 0.09401249897729824, 0.1133033284970213, 0.1333165758114595, 0.1534461207274912, 0.1732722024767589, 0.19251102773143858 ], [ 0.3236176200172968, 0.3563166605514184, 0.39549740764134245, 0.4396452125299766, 0.4872542138431335, 0.5370437073623266, 0.587950307421885, 0.6390369298458551, 0.6893992517215639, 0.738097330482997, 0.784114664012946, 0.8263337907243082, 0.86351100317089, 0.8942363767274332, 0.9169054405516476, 0.92989186165563, 0.9320187079674923, 0.9229721620773756, 0.9033584268168985, 0.874541073068426, 0.8386834475666974, 0.7991076298553965, 0.7597040542822321, 0.7234794790160619, 0.6915719137069049, 0.662922290484171, 0.6346224181712491, 0.6028786859663372, 0.5643345501993626, 0.5172079188838806, 0.4616972669106707, 0.39971754712938795, 0.33442633158625745, 0.2696884845425374, 0.2094226039104803, 0.1569452121675824, 0.11451680915938289, 0.08318735006676714, 0.06290333264271508, 0.0527710258236328, 0.051369307979713175, 0.05703752408635698, 0.06810055484597755, 0.08302116779624413, 0.10048591589847178, 0.11943784639307398, 0.13907077279681113, 0.15879883562037578, 0.17821327767595652, 0.1970363777758548 ], [ 0.3354655503215751, 0.36828536730993783, 0.407177069280326, 0.45070443705131025, 0.4974724343462559, 0.5463054820344504, 0.5962193867422411, 0.6463258805981196, 0.6957426991397638, 0.7435310621258319, 0.7886597619362078, 0.8299825137659597, 0.8662091462134895, 0.8958577704485813, 0.9172156055843074, 0.9285216049414786, 0.9284949029850988, 0.9167975584214341, 0.8940853721460207, 0.8618104781898546, 0.8222491195266138, 0.778842578803827, 0.7356055197119926, 0.6956849152036311, 0.6603758753687268, 0.6288111949925046, 0.5983374803018687, 0.5654820660666624, 0.527211032931846, 0.48194638389540206, 0.42986813294712284, 0.37261047241868434, 0.31284361062988186, 0.2538680988572105, 0.19909107888553812, 0.15145533037712655, 0.11301453358879832, 0.08476267536907756, 0.06670050462453059, 0.05805307809882576, 0.05754399944866939, 0.0636563252251321, 0.0748419313278137, 0.08966683416910604, 0.10689583690600624, 0.12552740627496073, 0.1447918994500278, 0.16412581069136845, 0.18313324432527678, 0.2015439921162328 ], [ 0.34755864905869016, 0.3804699767819635, 0.41902942530478826, 0.4618790599050566, 0.5077406333070938, 0.5555496131979871, 0.6044038440260595, 0.6534640571270958, 0.7018684422430984, 0.7486765576075598, 0.7928389861924652, 0.8331771772239007, 0.8683518248911345, 0.8968077059539656, 0.9167256117246092, 0.9262135796923557, 0.9238896509409886, 0.9094006686073721, 0.8834675523585054, 0.8476506672385957, 0.8043636017125414, 0.7571679490093927, 0.7101846030481834, 0.6666925256267238, 0.6281405703187183, 0.5938463776564404, 0.5613955504086912, 0.5276108651670949, 0.48975337135329056, 0.44643409490823643, 0.39781478578686186, 0.34525523299789246, 0.2909529009155145, 0.23766912207143687, 0.18832497968115602, 0.14550513658891318, 0.11105980826441675, 0.08592127077699796, 0.07013455829475679, 0.06303330622740844, 0.0634776384101634, 0.07008886467487563, 0.08144296827402897, 0.09620832968475501, 0.11322849351849718, 0.13155867529827436, 0.15046752402702335, 0.1694157309559845, 0.1880219445630129, 0.206024801517793 ], [ 0.3597955061236276, 0.3927680720902244, 0.4309562693093374, 0.47307891550231534, 0.5179785665931187, 0.5647058040733626, 0.6124420639193393, 0.660396619725404, 0.7077263011908117, 0.7534861849417432, 0.7966052931662961, 0.8358697873077175, 0.8698894275253348, 0.8970353264081379, 0.915385432494743, 0.9229243572300554, 0.91817348823327, 0.9007737257715868, 0.871525401337844, 0.8321138984166818, 0.7851072208163333, 0.7341881292564072, 0.6835710418381782, 0.6366589505613647, 0.5950499287042014, 0.5582369540247258, 0.5240255996313802, 0.48950603387497416, 0.4522053315243695, 0.4109079149933191, 0.36575874069852926, 0.317851747075389, 0.26892847229234046, 0.22123813067221937, 0.17724257260470966, 0.1391853216379293, 0.10871792131075397, 0.08670641386454525, 0.07323085479514335, 0.06772336451590166, 0.06917203482453582, 0.0763303592155643, 0.08789484485099741, 0.10263469238951006, 0.11947211653488032, 0.13751996019441493, 0.156086564869931, 0.17465840179723402, 0.19287016048634775, 0.21047053564241847 ], [ 0.37207499531527977, 0.4050770160984636, 0.44285859702706654, 0.48421277986541156, 0.5281050538922117, 0.5737030825083564, 0.6202720052870208, 0.6670685274596093, 0.7132661428486058, 0.7579126918284895, 0.7999124751414152, 0.8380137555599197, 0.8707743796399859, 0.8964928048086322, 0.9131499427092743, 0.9186178739714187, 0.9113277476094702, 0.8909240619516958, 0.8582988146585547, 0.8152748093264064, 0.7645836131203705, 0.71002955840285, 0.6559136856022617, 0.6057563323890073, 0.5612993234117254, 0.5221991820019923, 0.4864597850519299, 0.4514089291501236, 0.4148098359418403, 0.37560365551006014, 0.333920531793914, 0.2905988849305961, 0.24694368916233966, 0.20472150230999842, 0.1659630636758368, 0.13258874589469438, 0.10605734571181391, 0.08716520194836641, 0.07601880694454888, 0.07213886051396079, 0.07463263381358831, 0.08237923013748594, 0.09419148386013398, 0.10893726554800931, 0.12561685319051485, 0.1434011542345459, 0.16163925473885143, 0.17984472292477172, 0.19766959322008215, 0.21487370482745016 ], [ 0.3842976940252033, 0.41729538553918216, 0.45463807885288915, 0.495189823269655, 0.5380393036635293, 0.5824709282868895, 0.6278321377467615, 0.6734253329348898, 0.7184385966476273, 0.7619099099862117, 0.8027159198588845, 0.8395647710101637, 0.8709621500038289, 0.8951364484903077, 0.9099800970974677, 0.913266344543286, 0.9033450728372765, 0.8798739211871817, 0.8438456086634607, 0.7972272115726246, 0.7429161232177129, 0.6848379599824513, 0.6273782673643383, 0.574170189448949, 0.5270933331560083, 0.485954042541282, 0.4489308422434043, 0.4135585337495445, 0.37780612024355115, 0.34075325639032766, 0.30251684194579226, 0.263691603637785, 0.22516838926016858, 0.18826314245905773, 0.15460464956063247, 0.12580877797010737, 0.10314834886442481, 0.08734742369293569, 0.07853099755722104, 0.07629869300270031, 0.07986803733624148, 0.08823676948697956, 0.1003293255650578, 0.1151095474728846, 0.1316546681287708, 0.14919367110100246, 0.16711709503275807, 0.18496665479085161, 0.20241283648374858, 0.21922757788190295 ], [ 0.39636726717228704, 0.42932437636569737, 0.46619851612652075, 0.5059210813807018, 0.5477022640654903, 0.5909404172988164, 0.6350623887425874, 0.6794139849440589, 0.723195776804065, 0.7654334673004382, 0.8049733929536564, 0.840481755549699, 0.8704125615120344, 0.8929284494777957, 0.9058444444163566, 0.9068513630329675, 0.894229994262587, 0.8676601475613606, 0.8282397845221062, 0.7780811937381754, 0.7202444571402772, 0.65877527689126, 0.5981448540314938, 0.542097090458747, 0.4926434224092533, 0.44972480439141743, 0.41166945705812424, 0.3761885778718949, 0.34142669732816366, 0.3065817814900668, 0.27175746664859235, 0.23731797843978586, 0.20376628136170605, 0.1720023048312489, 0.14328268589838755, 0.11893774345466601, 0.1000616888164575, 0.0873044888223321, 0.08080232931531284, 0.08022440081674709, 0.08488952279636897, 0.09390679539366042, 0.10630708729512739, 0.12114702681429712, 0.1375792312200732, 0.15489036776353604, 0.1725128013252486, 0.1900171784258451, 0.20709334531612333, 0.22352615637128115 ], [ 0.40819184349388715, 0.44106921842409763, 0.4774473123768567, 0.5163209542368483, 0.5570179821203465, 0.5990453572387796, 0.641905079044896, 0.684983621484659, 0.7274919955955519, 0.7684414905593253, 0.8066458172686886, 0.8407278471544181, 0.8690911932101434, 0.8898387008522424, 0.9007209145782242, 0.8993652586088736, 0.8839996718635662, 0.854333856982397, 0.8115698005897856, 0.7579604989708648, 0.6967215924128578, 0.6320165428315352, 0.5684051036726951, 0.5097421854052602, 0.45816555838907547, 0.41373457906530603, 0.3749016302939962, 0.33952458063381186, 0.30589409234347087, 0.27330415697687765, 0.24184204999769465, 0.21165614259924126, 0.18289241398294465, 0.15607154868978423, 0.132108001322494, 0.11206550286028827, 0.0968674122125085, 0.08708842918397619, 0.08286921822552562, 0.0839395331239472, 0.0897105675256944, 0.09939530364934901, 0.11212551433485718, 0.12704700739570918, 0.14338579517088623, 0.16048545842611295, 0.17782024163622978, 0.1949902496606628, 0.211705400619836, 0.22776414585975074 ], [ 0.41968540133768184, 0.4524406373517562, 0.4882969934990515, 0.5263087387654827, 0.5659149454121332, 0.6067233855476156, 0.6483058233304585, 0.6900863355133917, 0.7312844524908697, 0.770895280466757, 0.8076980174102735, 0.840271335960376, 0.8669706557263536, 0.8858464096730042, 0.8945987305034909, 0.8908127656427708, 0.8726848913034653, 0.8399601119346216, 0.7939368996867205, 0.7370000883270883, 0.6725108764636571, 0.6047467960112034, 0.5383594251654835, 0.47731664046043853, 0.4238777829407056, 0.37820387720615417, 0.3388460689609949, 0.30378087633718326, 0.2714173634286059, 0.24112158134587253, 0.2129565364480756, 0.18687119973547273, 0.16269079522529206, 0.14059488462289094, 0.12118538669095691, 0.1052781766849411, 0.0936337668680316, 0.08675098216623489, 0.08476884068951596, 0.08746905020734785, 0.09434638695431696, 0.10471012196579244, 0.11778712715627704, 0.13280842642627844, 0.1490710657310993, 0.1659744215465354, 0.18303436883483404, 0.19988074869490752, 0.21624407018524522, 0.23193692457308862 ], [ 0.43076915992162657, 0.4633564012103151, 0.4986668125541471, 0.5358101847951238, 0.5743273674988812, 0.6139169961094235, 0.6542143718734833, 0.6946778963139774, 0.7345338848884898, 0.7727599413353263, 0.8080993976973534, 0.8390864828471366, 0.8640316215987744, 0.8809413911882664, 0.8874802060649211, 0.8812130624870353, 0.8603313031842326, 0.8246174979611094, 0.7754534399690676, 0.7153438005274746, 0.6477832404405264, 0.5771580554635698, 0.5082140953467906, 0.4450350097194044, 0.3899977512070576, 0.3433481790778372, 0.303711653886294, 0.26915774345265253, 0.23818852886132386, 0.2102175373475459, 0.18526931700793015, 0.16311225983646116, 0.14329227127241984, 0.12568616492993434, 0.11061228810893065, 0.09865703337776977, 0.09042623952751072, 0.08634276521330375, 0.08653844225840202, 0.09083876223629062, 0.09881349313822607, 0.10986057255806614, 0.12329596955246047, 0.13843166961973774, 0.1546330671375361, 0.17135390184018984, 0.18815114852395465, 0.2046844259487201, 0.2207051668420636, 0.23604050993717113 ], [ 0.4413729331236952, 0.4737429858035266, 0.5084844745150674, 0.5447590283575171, 0.5821963618894805, 0.6205744581024274, 0.6595853697490642, 0.6987184105148588, 0.7372051680426924, 0.7740049505203005, 0.8078245268237396, 0.8371541699404466, 0.8602635516454635, 0.875124923675322, 0.8793820570615271, 0.8706020755034076, 0.8470005488006352, 0.8083973923232936, 0.7562411384507749, 0.6931420315379365, 0.6227144639837745, 0.5494463363619595, 0.4781783513379182, 0.41311255824679677, 0.3567402395721663, 0.30937552925522155, 0.2696950415618953, 0.2358387982084319, 0.20637920378060398, 0.18075336704358558, 0.15892722033910156, 0.14050986136048582, 0.12481278711602761, 0.11144777291764818, 0.10047772866263627, 0.0922775527959897, 0.08730672668009976, 0.0859125478888556, 0.08821471422663807, 0.09407481199295853, 0.10312927917796266, 0.11485714790333512, 0.1286573617014839, 0.14391838641638532, 0.16007100523136197, 0.1766216090652344, 0.19316748369508707, 0.20939784510402132, 0.22508520436957508, 0.24007152343460547 ], [ 0.4514363422384054, 0.4835373648064324, 0.51768799981435, 0.5530983927044214, 0.5894709351275438, 0.6266505900879012, 0.6643790116621168, 0.7021729085449214, 0.7392678542725755, 0.7746046570274991, 0.8068536115816534, 0.8344623563732646, 0.8556650903452138, 0.8684100660534829, 0.8703357861321084, 0.8590336795507468, 0.8327702644650143, 0.7914026930144508, 0.73642914433209, 0.6705493792346, 0.5974824412113792, 0.5218086628239773, 0.44846144573400104, 0.381762531560452, 0.3243146131072184, 0.2764841591923705, 0.23697845584342936, 0.20398882690141146, 0.17613795193550852, 0.15286387209440933, 0.13405208731691298, 0.11917411954749513, 0.10735214565710383, 0.09796965533998248, 0.09086147721700955, 0.08620867352887707, 0.08433284331985591, 0.08550662562091271, 0.08983324235929724, 0.09720320605301136, 0.10731163395365662, 0.11971120372806554, 0.13387766164566506, 0.1492713081495859, 0.16538513046542158, 0.18177621525392518, 0.19808113736617267, 0.21401832419853178, 0.22938135177218733, 0.24402715420642113 ], [ 0.4609097189688215, 0.4926887540889985, 0.5262275921293307, 0.5607818722671734, 0.59610872560162, 0.632107356479886, 0.6685615737566508, 0.705011844481124, 0.7406966443838084, 0.7745387042795949, 0.8051728529093999, 0.8310063350354104, 0.8502441314687827, 0.8608214054058422, 0.8603869068920299, 0.8465777041102213, 0.8177321048231104, 0.773745893788551, 0.7161519021428137, 0.6477222273988151, 0.5722644183336175, 0.4944400347409632, 0.41926963044699866, 0.3511933479842095, 0.29292222865585377, 0.24486013329814965, 0.20572771288153113, 0.1737521931094903, 0.14758886793077064, 0.12665538742576343, 0.11073939764253793, 0.09919389027586523, 0.09099334338608811, 0.08532872404383107, 0.0818334735281363, 0.08051222741066322, 0.0815573713258515, 0.08516829694501227, 0.09142803025614532, 0.10024939751913464, 0.11137859053486576, 0.12443467249600004, 0.13896403812397773, 0.15449407163101292, 0.17057660278075093, 0.18681725190805865, 0.202890654327476, 0.21854387558491783, 0.23359138649711508, 0.24790512180667268 ], [ 0.4697545146162511, 0.5011596061885512, 0.5340667624112931, 0.5677740902532584, 0.6020764291113241, 0.6369142626511819, 0.6721058084526654, 0.7072114995395828, 0.741471786787224, 0.7737923780379403, 0.8027746886127485, 0.8267888051208784, 0.8440175836621695, 0.8523942878931092, 0.8495931894884013, 0.8333162460810095, 0.801988387812536, 0.7555465963980078, 0.6955468240262263, 0.6248162760586528, 0.547234198755614, 0.4675303151234297, 0.3908030202421235, 0.3216056731488782, 0.2627537351849581, 0.21467500862321903, 0.17609050530324355, 0.14525187257419347, 0.12083155099231091, 0.1022076977444728, 0.08905952247535709, 0.0806370240355836, 0.07580249788993354, 0.07358862905749419, 0.07345350820131336, 0.07524255900274635, 0.07902784583346545, 0.08493744380209922, 0.09303109815903321, 0.10323792209012639, 0.11534801061310973, 0.1290397999173064, 0.14392425715727808, 0.1595910502583332, 0.1756473600860835, 0.19174500851976872, 0.20759528302381958, 0.22297314550918068, 0.23771364713595788, 0.2517036384956417 ], [ 0.47794312084645535, 0.5089251054701152, 0.5411818090945208, 0.574050618370019, 0.6073498861295831, 0.6410485382881406, 0.6749911941191034, 0.7087542815658805, 0.7415794012504862, 0.7723568860701651, 0.7996579371808397, 0.8218197904412867, 0.8370108893237461, 0.8431736577682303, 0.8380223802067158, 0.8193401606354846, 0.7856485145298281, 0.7369287090597181, 0.6747518450707047, 0.6019840572408084, 0.5225593416749027, 0.44126102702930886, 0.36325227910409913, 0.2931893205820576, 0.23398622032994743, 0.18608350179995248, 0.148194957180815, 0.11858907315198097, 0.09594214801403822, 0.07957712494903446, 0.0690603623310605, 0.06355151909087431, 0.06182931297322947, 0.06279988020285165, 0.06577114498648196, 0.0704463222512639, 0.07678627474189015, 0.08485021230687362, 0.09467215643346127, 0.1061920880310816, 0.11923730636297414, 0.13353890628738374, 0.1487664842660089, 0.1645671943830923, 0.18059999182579523, 0.19656043361624365, 0.21219489849832135, 0.22730535399854285, 0.2417469861126481, 0.25542137143240706 ], [ 0.48545816872634984, 0.5159716532339944, 0.5475602473874697, 0.5795973328484613, 0.6119138496078086, 0.6444951138945327, 0.6772040376075497, 0.7096289145416796, 0.7410117235983419, 0.7702295800599457, 0.7958278654774239, 0.8161164439888392, 0.8292573628920309, 0.8332126588628266, 0.825749811831986, 0.8047460137627296, 0.7688257157457166, 0.7180176097306018, 0.653902969482469, 0.5793725019678552, 0.498398413465687, 0.4158020883269717, 0.3367950745117729, 0.26611990869947927, 0.20678014587106608, 0.15922117817004944, 0.1221484575611097, 0.09384333162752712, 0.0729749455989378, 0.058799320730876126, 0.05077014026723914, 0.04796723413536186, 0.04910797341219064, 0.053000272634008194, 0.05882586294161041, 0.06616244140132488, 0.07486898353576399, 0.08493878945172828, 0.09637835152963314, 0.10913371950709028, 0.12306320027491102, 0.13794417380103208, 0.1534991037065191, 0.16942788232171635, 0.1854376188755742, 0.20126503936186974, 0.21668992721552727, 0.23154023568109644, 0.2456907228175929, 0.2590574051003049 ], [ 0.4922914909596837, 0.5222952016729897, 0.5531991416313056, 0.5844094134615813, 0.6157614901797156, 0.6472464094998237, 0.6787374336609204, 0.7098305133277578, 0.739767264162983, 0.7674141300330294, 0.7912962094206808, 0.8097027836319819, 0.820797412982821, 0.8225711400986542, 0.8128561484667659, 0.7896334700186568, 0.7516342440281913, 0.6989374877547329, 0.6331319237726233, 0.557120642242511, 0.474898384211625, 0.39130857315028406, 0.31159226910756943, 0.24055519229869116, 0.1812760280523773, 0.1342022259988377, 0.09803679061248094, 0.07107296384641337, 0.05196411310758131, 0.03989166940240452, 0.034199933163760354, 0.03389783586629469, 0.037658336798318204, 0.04421555716418746, 0.05264738828417603, 0.0624222190022955, 0.07330657505792215, 0.08523126951753723, 0.09817408098028557, 0.1120829517667935, 0.12684152272793203, 0.1422674603815136, 0.15813055565157208, 0.17417878305447942, 0.19016378076550344, 0.205860810587206, 0.22108127447284753, 0.2356779820888541, 0.24954459760268755, 0.26261120426972917 ], [ 0.4984429434072347, 0.527899736886411, 0.558103607212978, 0.5884901970843437, 0.6188937155798951, 0.6493019650087293, 0.6795910904149378, 0.7093605399589971, 0.7378508664586405, 0.7639206518867275, 0.7860811810365673, 0.8026093977648942, 0.8116776969526361, 0.81131416902638, 0.7994253659366011, 0.7741030752420042, 0.7341870057134777, 0.6798089902938615, 0.6125640210287633, 0.5353575386818399, 0.45219228877900086, 0.36791767038521495, 0.28778390440972473, 0.21663097548611276, 0.15759088957801914, 0.11111745613157764, 0.07592360074461502, 0.05031576987758135, 0.03292540208712669, 0.022855367021234918, 0.019345890475931604, 0.02134275855680756, 0.027487294106167903, 0.03646029030432074, 0.04725618099799722, 0.05924957077475146, 0.0721239918861567, 0.08575160258005621, 0.10008087294469148, 0.1150580758236941, 0.13058704638998564, 0.14652014002001013, 0.16266919181825812, 0.1788257313405862, 0.19478233100105335, 0.2103501189538895, 0.2253702550005076, 0.23971918592401326, 0.2533087270051389, 0.26608257777226213 ], [ 0.5039192246739201, 0.5327959442019194, 0.5622854951834289, 0.5918500279465111, 0.6213183811999049, 0.6506679469411879, 0.6797710343217391, 0.708226639503081, 0.7352736455818449, 0.7597657603246717, 0.7802074975823851, 0.7948731296198539, 0.8019502314273549, 0.7995106153982154, 0.7855429917715843, 0.7582543954751775, 0.7165936014161176, 0.6607472318688222, 0.5923163167232874, 0.5142005197825003, 0.43039728476355504, 0.34574609470384166, 0.2654852625187775, 0.19445650243985424, 0.13581473498854102, 0.09003275829356627, 0.05585025039639924, 0.031589936745228586, 0.015857746157610597, 0.007677243305105996, 0.006191173477086864, 0.010289055951155701, 0.018590193376032182, 0.02973880083828595, 0.042664039616399974, 0.05666136537352118, 0.0713406676180739, 0.08651961646607187, 0.10211732497618708, 0.11807542958920259, 0.13431335595228344, 0.15071296914298737, 0.1671231496661293, 0.18337461568570745, 0.19929734103677166, 0.21473564281056762, 0.2295585272437033, 0.24366478769645883, 0.25698356052056703, 0.26947164332885104 ] ], "zauto": true, "zmax": 1.012046503359353, "zmin": -1.012046503359353 }, { "autocolorscale": false, "autocontour": true, "colorbar": { "tickfont": { "size": 8 }, "ticksuffix": "", "x": 1, "y": 0.5 }, "colorscale": [ [ 0.0, "rgb(255,247,251)" ], [ 0.14285714285714285, "rgb(236,231,242)" ], [ 0.2857142857142857, "rgb(208,209,230)" ], [ 0.42857142857142855, "rgb(166,189,219)" ], [ 0.5714285714285714, "rgb(116,169,207)" ], [ 0.7142857142857143, "rgb(54,144,192)" ], [ 0.8571428571428571, "rgb(5,112,176)" ], [ 1.0, "rgb(3,78,123)" ] ], "contours": { "coloring": "heatmap" }, "hoverinfo": "x+y+z", "ncontours": 25, "type": "contour", "x": [ 1e-06, 1.3011511650442548e-06, 1.692994354296022e-06, 2.2028415765056147e-06, 2.866229883678204e-06, 3.729398352432554e-06, 4.852511011181743e-06, 6.3138503555892e-06, 8.215273746089953e-06, 1.0689313005882424e-05, 1.390841207112662e-05, 1.809694657026198e-05, 2.354686311364001e-05, 3.063802837345029e-05, 3.986470631277378e-05, 5.1870009063012666e-05, 6.749072272319499e-05, 8.781563250096393e-05, 0.00011426141253772724, 0.00014867137004306603, 0.00019344392634026088, 0.0002516997901283655, 0.0003274994751669172, 0.0004261263236648159, 0.0005544547624925005, 0.0007214294601814526, 0.000938688782612345, 0.0012213760031100258, 0.0015891948094037057, 0.002067782677737912, 0.0026904978401970136, 0.0035007443993213955, 0.004554997653699184, 0.005926740503884541, 0.007711585311544345, 0.010033938212454078, 0.013055670395116691, 0.01698740074503987, 0.02210317627048227, 0.028759573555516536, 0.03742055263793628, 0.04868979566145066, 0.06335278435066323, 0.0824315491666629, 0.10725590623460621, 0.13955614735503497, 0.18158364372009145, 0.23626776957937787, 0.3074200836506151, 0.4 ], "xaxis": "x2", "y": [ 0.0, 0.02040816326530612, 0.04081632653061224, 0.061224489795918366, 0.08163265306122448, 0.1020408163265306, 0.12244897959183673, 0.14285714285714285, 0.16326530612244897, 0.18367346938775508, 0.2040816326530612, 0.22448979591836732, 0.24489795918367346, 0.26530612244897955, 0.2857142857142857, 0.3061224489795918, 0.32653061224489793, 0.3469387755102041, 0.36734693877551017, 0.3877551020408163, 0.4081632653061224, 0.42857142857142855, 0.44897959183673464, 0.4693877551020408, 0.4897959183673469, 0.5102040816326531, 0.5306122448979591, 0.5510204081632653, 0.5714285714285714, 0.5918367346938775, 0.6122448979591836, 0.6326530612244897, 0.6530612244897959, 0.673469387755102, 0.6938775510204082, 0.7142857142857142, 0.7346938775510203, 0.7551020408163265, 0.7755102040816326, 0.7959183673469387, 0.8163265306122448, 0.836734693877551, 0.8571428571428571, 0.8775510204081632, 0.8979591836734693, 0.9183673469387754, 0.9387755102040816, 0.9591836734693877, 0.9795918367346939, 1.0 ], "yaxis": "y2", "z": [ [ 0.33847454326757576, 0.33649062603217705, 0.3354621098023293, 0.3342949090881483, 0.3317673361434805, 0.3267598379238369, 0.31844197048240674, 0.3064084679979821, 0.29077672101450935, 0.2722438122078249, 0.252054388141232, 0.23176896946157438, 0.21272766799160756, 0.19534741900312932, 0.1788426719069517, 0.1619682728134062, 0.144563768866548, 0.12891622116283416, 0.11937942439639837, 0.11826981694103982, 0.12186695023407287, 0.12414642552843885, 0.12156703224949211, 0.1141565756983501, 0.10476294955940986, 0.09700770564078157, 0.09197569207488274, 0.08808946619961086, 0.08463684865592125, 0.08280477635686917, 0.08308300581466056, 0.08452661174957668, 0.08621420405640033, 0.08771738892636116, 0.09100322187189254, 0.101313450858498, 0.12237788128215724, 0.15138495212828024, 0.1825462366827748, 0.2110253656335874, 0.23359282767870446, 0.24827118844715465, 0.25397199248251995, 0.250289663869208, 0.23747477039846626, 0.21663490303659813, 0.1902985808637725, 0.16357555015353903, 0.14553991280679596, 0.14660561797966853 ], [ 0.3317795667642394, 0.3299181009182161, 0.32921948790507244, 0.3284749475389942, 0.32633519643276626, 0.32156809748486964, 0.313260024939, 0.300957347536077, 0.2847700315518973, 0.26544431472692087, 0.24435760307259818, 0.2233063785384829, 0.20392707142293115, 0.18684029359487178, 0.17120593882720292, 0.1554902608720924, 0.139200875906802, 0.12439594503490166, 0.11534034571894085, 0.11438471540775244, 0.1179258355776034, 0.12003543767049152, 0.11723151826300503, 0.10957530304336147, 0.09993145996587348, 0.091961548474971, 0.08683535297599433, 0.08299461372973777, 0.0796874408798091, 0.07807453457742598, 0.07850517215193908, 0.07983728998788163, 0.08113711953492327, 0.08220584148713023, 0.08534277646410554, 0.09613526182365024, 0.11819865688847156, 0.14821700673366212, 0.18015455133687056, 0.2091650828366113, 0.23206230799928262, 0.2469032231576288, 0.25261566160388327, 0.2487923827805451, 0.23566399848928332, 0.21429901147935268, 0.18717721220422678, 0.15942287394641733, 0.14045866980288552, 0.14151375819289178 ], [ 0.32471519548562067, 0.32300340198025423, 0.32268959631960004, 0.3224376573142964, 0.3207584934487018, 0.3162991430851369, 0.30805823915334474, 0.2955305247955581, 0.2788141660235355, 0.258698636871968, 0.2366913260997638, 0.2148376869484677, 0.1951111830353036, 0.178386615856617, 0.16378302472386463, 0.1494478791832429, 0.13451712093566942, 0.12076948354968858, 0.11228592967601446, 0.11138876514866033, 0.11470764937336227, 0.11653886875111062, 0.11350548915916626, 0.10569764407670787, 0.09594337150962629, 0.08784814349857241, 0.08265790871029192, 0.07891835227187144, 0.07586142441507171, 0.07457042838935035, 0.07522648531019281, 0.07650728266913358, 0.07747384876241849, 0.078204618534768, 0.08133183388566421, 0.09265955535620302, 0.11556335985923828, 0.14632142790774658, 0.17877927018543605, 0.2081256033925073, 0.23122294067275595, 0.24615999449508807, 0.25187977278997165, 0.24797691147315834, 0.2346717766047071, 0.21301022703194403, 0.1854413116198386, 0.1570900439089706, 0.13757468998490469, 0.1386226141383402 ], [ 0.3172760971334143, 0.315743389927851, 0.31587407573633147, 0.3161903212260546, 0.3150499544293476, 0.3109710251387185, 0.3028606962472146, 0.29015977277026933, 0.27295060592646814, 0.25205881483464065, 0.2291153209718976, 0.20642242224745014, 0.1863302341082773, 0.17002362076527416, 0.15659527780752616, 0.14383629914657464, 0.13046637837964384, 0.11793814037693366, 0.11007498138686513, 0.10913974186672057, 0.11208953655268548, 0.11355086679197218, 0.11029182317876575, 0.10242974548683671, 0.09270799810557624, 0.08458738362243454, 0.07937142199813335, 0.07578272115516092, 0.07306541642383506, 0.07218606727549756, 0.07314574476465409, 0.07447767664273723, 0.07525209304614507, 0.07583046290205482, 0.07913279660894648, 0.09101607149720205, 0.11454159933070851, 0.1457301985197683, 0.17843584553011724, 0.2079159750379573, 0.2310817255249985, 0.24604868180805411, 0.2517734204231051, 0.24785644708951346, 0.23451912453796198, 0.2128046983064909, 0.18515730225204, 0.15670161942275732, 0.13708875969435028, 0.13813349786641446 ], [ 0.30945950591147725, 0.30813750560196745, 0.30877725829067626, 0.3097428118810749, 0.3092244964645144, 0.3056034377658984, 0.297692690084091, 0.28487807212261906, 0.2672228768809558, 0.24558105071836198, 0.22169691195805905, 0.1981308970882345, 0.17764580578667807, 0.16179695109087794, 0.1496659641733141, 0.13864215050272566, 0.126980535470237, 0.1157681851358663, 0.10852751205171726, 0.10746331071617676, 0.10992798220502482, 0.11095188366027321, 0.10748036106926706, 0.09965969145741944, 0.09010909086990757, 0.08206776688217406, 0.07686758133164322, 0.0734637276209185, 0.0711479946348561, 0.0707472261364645, 0.07208919288040364, 0.07361651511877267, 0.07442555635162049, 0.0751199676354011, 0.07881316489846078, 0.09124026164275297, 0.11513379806664098, 0.14643289536522036, 0.17911390377496694, 0.2085283189211194, 0.23163324719312137, 0.24656588070853383, 0.2522948753026441, 0.24843086784525875, 0.2352078641007611, 0.21368725025292876, 0.18633554612440267, 0.1582785105176943, 0.1390360752395854, 0.1400819969933824 ], [ 0.3012657722102282, 0.30018829276899445, 0.30140663477277124, 0.3031079208930781, 0.30329934460623825, 0.30021758647504415, 0.29258031318219946, 0.2797188809363864, 0.26167548851464023, 0.2393244759640001, 0.2145100964877976, 0.19004441767194266, 0.16913215757253405, 0.15376225549770306, 0.1430217576901252, 0.1338464022397889, 0.12397765069580655, 0.11410895945181293, 0.10745051396907956, 0.10617614191538874, 0.10807498663015667, 0.1086210968206236, 0.10496009775143225, 0.09727231431613198, 0.08802354165903585, 0.08016566984706093, 0.07501842234397384, 0.07180828274986868, 0.06992184787043124, 0.07004089868717077, 0.07184291786599459, 0.07374459595478482, 0.07488004823601459, 0.0760133612645755, 0.08032205988138344, 0.0932636231072748, 0.11727193729332448, 0.14837852508754312, 0.18077842458816137, 0.20993848091059542, 0.23286002908067835, 0.24769779230650127, 0.25343166935304384, 0.2496867236356838, 0.23672046712615258, 0.2156309047002324, 0.18892892724283217, 0.16173377780152196, 0.14327631951121092, 0.14432791661355904 ], [ 0.29269899719481884, 0.2919019962310253, 0.2937733751706817, 0.29630170448928395, 0.297294129006618, 0.29483600576973584, 0.28754996397701, 0.2747152645509637, 0.2563526026966519, 0.23334939763633955, 0.2076338431457355, 0.18225461570558066, 0.16087695024968796, 0.145986326299513, 0.13669387640117095, 0.12942756020135007, 0.12137123661782427, 0.11281114959240499, 0.10666131488185901, 0.10510706112694987, 0.10639334156957116, 0.10644817800244141, 0.10263084281380647, 0.09516332383895078, 0.08633829321406299, 0.07876331587807288, 0.07369433320341642, 0.07065448677522054, 0.06918892943258759, 0.06984785911690457, 0.07219009638753646, 0.07466850722749378, 0.07645349012717996, 0.07836202559340297, 0.08349813896513165, 0.09692684071847511, 0.12083023140081882, 0.1514813643609625, 0.1833726493431448, 0.21210756790333582, 0.23473344885003405, 0.2494208728837707, 0.25516118337345933, 0.2515979506276587, 0.23902121848801303, 0.21857930091464398, 0.19283894808989477, 0.1668893752719713, 0.14953127330517216, 0.15059299632647222 ], [ 0.28376776495375394, 0.2832892435531469, 0.2858929020900114, 0.28934383600232216, 0.29123094934816857, 0.2894823212140797, 0.28262778042997483, 0.2698989075937428, 0.2512964588240699, 0.22771501591788468, 0.20114940448863455, 0.1748614884777633, 0.1529808684611281, 0.13854732486469193, 0.1307184733168941, 0.12536464699883765, 0.11907906157506636, 0.11174193441760176, 0.10600508853358886, 0.10411298362455826, 0.10476879416169445, 0.10434287739004534, 0.10041278915672636, 0.09325054263975366, 0.0849626997522863, 0.07776156532549013, 0.07277869094040744, 0.06985047320597038, 0.06876371648978703, 0.0699708915091605, 0.07294190148095252, 0.07621036948857435, 0.0789635123224881, 0.08195821355770472, 0.08810507605082886, 0.102009055274235, 0.12564201590094373, 0.15562931777395367, 0.18682218388205435, 0.214984155078884, 0.23721511149592978, 0.2517028779282871, 0.25745167757157267, 0.2541272264000931, 0.2420585234985519, 0.22245153510451288, 0.19792759902160548, 0.1735068053985538, 0.15744747226938913, 0.15852353693267096 ], [ 0.2744859867306499, 0.27436581859980763, 0.277785513082403, 0.28225795041662005, 0.2851343910396903, 0.28418095038639785, 0.27783901221022417, 0.26529904514580266, 0.24654561971227715, 0.22247666962923449, 0.19513655349518688, 0.16796969990083005, 0.14555547892050563, 0.131533499951522, 0.12513588167966125, 0.12163948144815588, 0.11703022058286128, 0.1107953733354618, 0.10536550231533207, 0.10308872620757432, 0.10311812694660695, 0.10224159849824234, 0.09825311866824801, 0.09148122768206562, 0.08383541969163594, 0.07708596543395213, 0.07217654125261319, 0.0692689509848241, 0.06849285153347937, 0.07025552581439785, 0.07395688353496652, 0.07822765840238505, 0.0822344513580479, 0.08657362972758269, 0.0938757550900297, 0.10825993947266208, 0.13151811494948124, 0.16069310579806284, 0.19103967287643323, 0.21850689968485557, 0.24025854058096768, 0.25450420201491225, 0.2602636642356893, 0.2572278138410856, 0.24576803798142047, 0.22714852307795919, 0.20403194623210252, 0.1813206876856017, 0.1666555892669749, 0.16774970448450816 ], [ 0.26487387356264397, 0.2651535341057064, 0.269477042506665, 0.2750719591620293, 0.2790314733699155, 0.27895673729865866, 0.27320734905521904, 0.2609413647186571, 0.24213313948803275, 0.21768275705303622, 0.18966881018020804, 0.1616827826856753, 0.13871853851622223, 0.12503969834767342, 0.11998839927555242, 0.11823793237512197, 0.11516977106054858, 0.10989779373921983, 0.10466916536664994, 0.10197128184091166, 0.10139315682243051, 0.10011075723367027, 0.09612936938364018, 0.08983537475844396, 0.08292637358923255, 0.07668719173546798, 0.07181764737514475, 0.0688168356612928, 0.06826872947247087, 0.0706019674026171, 0.07514902522378791, 0.0806217231945508, 0.08611561364459305, 0.09199079828874447, 0.10054841699088578, 0.11542762700924018, 0.13826316217932852, 0.16653491457062994, 0.1959294858801105, 0.22260729927619496, 0.24381103708835447, 0.2577793998892097, 0.2635514988254488, 0.26084570337017654, 0.2500762256685397, 0.23255985698098883, 0.21097836931164507, 0.1900665386603893, 0.17681093489809982, 0.17792605500909553 ], [ 0.2549590549272357, 0.2556812079691099, 0.2609995454288635, 0.2678183045067828, 0.27295150536460505, 0.27383451549238147, 0.2687542279695922, 0.25684694316013296, 0.23808478934710334, 0.21337157939131193, 0.18480796476301525, 0.15609518417841073, 0.1325860209761258, 0.11916100000696626, 0.11531652438904498, 0.11515005057527648, 0.11346075189370687, 0.10900874245395213, 0.10388516986684372, 0.10073966104873085, 0.09958113930248412, 0.09794710358250493, 0.094049627892312, 0.0883253106684836, 0.08223490626683488, 0.07653918494418153, 0.07165754487299336, 0.06844069764935028, 0.06803592551552115, 0.0709684728678611, 0.07648688092250924, 0.08333680060536502, 0.0904879815942274, 0.09801992277504726, 0.10788833309541435, 0.12327937204532033, 0.14568807971869913, 0.17301567488189482, 0.20139200562792142, 0.22721237413037043, 0.24781556605946561, 0.26147877462769703, 0.26726506177387116, 0.26492185773291277, 0.2549039568838438, 0.23857026121994143, 0.21859433229755837, 0.19949914892464793, 0.18761358595899136, 0.1887517645746502 ], [ 0.24477786082457786, 0.24598574264992396, 0.25239197417103715, 0.2605341121552691, 0.2669258210558185, 0.26883859646430025, 0.2644981461077963, 0.25303129021399473, 0.23441749984235746, 0.2095684406061785, 0.1805984914065982, 0.15128269618943563, 0.1272605553128863, 0.1139831477953208, 0.11115396945118843, 0.11236923297320445, 0.11188378959032702, 0.10811832792512532, 0.10302089736664417, 0.09941126396805623, 0.09770213956558593, 0.09577535889846062, 0.09204972256603604, 0.08699187291991062, 0.08178509904469915, 0.07663729205581904, 0.07167926520383805, 0.06812883997576875, 0.06779167348233577, 0.07136783263254141, 0.07798608172963208, 0.08635222863733297, 0.09526193436118435, 0.1045033033387543, 0.11569614754617631, 0.13161168198615994, 0.1536182028446115, 0.18000063133388086, 0.20732727892445862, 0.232247115168315, 0.2522125569606348, 0.265549932754147, 0.2713514158386104, 0.26939438672264954, 0.26016983551232487, 0.2450650461401796, 0.22671680246408638, 0.20940208101132665, 0.19881383102350983, 0.1999761873905697 ], [ 0.23437678131001666, 0.23611329702231929, 0.24370080111834305, 0.2532611883097413, 0.26098736314155513, 0.2639921827158479, 0.26045400897238585, 0.24950357090687592, 0.23113818120813456, 0.20628337980973244, 0.17706269507367722, 0.1472926663740192, 0.12281697583179434, 0.10957031040554563, 0.10752234717797551, 0.10989078353151258, 0.11043472658212972, 0.10724174099886474, 0.10211484932376495, 0.0980353458548174, 0.09580380629558206, 0.09364359797883533, 0.09018763809730104, 0.0858973330045045, 0.08161871418633616, 0.07699560428735441, 0.07189345096833638, 0.06791011201487243, 0.06758164807581854, 0.07185839927801446, 0.0796967040342335, 0.08967037266082115, 0.10036996439718676, 0.11131242673968042, 0.12380743161764267, 0.14025233522584496, 0.16189747165276897, 0.18736321035461811, 0.21363793469065842, 0.23763660231295566, 0.2569415364289053, 0.26993922855691793, 0.27575634753602146, 0.2742005204716803, 0.2657930463999423, 0.2519342822106621, 0.23519740259683655, 0.21959068717599348, 0.21020948601716166, 0.21139625653040003 ], [ 0.22381410756555736, 0.22612052216553277, 0.23498051783736987, 0.24604579443503344, 0.25517008384836554, 0.2593167089604388, 0.2566325445107036, 0.24626607075334994, 0.22824306051165394, 0.20350988334210743, 0.17419751844008738, 0.1441362006207903, 0.11928761743701227, 0.10595251813281563, 0.10442694286813177, 0.10771034204254404, 0.10912082375667176, 0.10641165998972704, 0.10122691533356304, 0.09668379069830806, 0.09395387031147369, 0.09161689614628261, 0.08853557866771194, 0.08511524837770101, 0.08178618593530373, 0.0776424108797629, 0.07233500886490092, 0.06784889121750368, 0.06749152902495528, 0.0725305041896686, 0.08168663837240928, 0.09330248655711508, 0.10575771971176019, 0.11834202443487492, 0.13208804532456728, 0.1490580922811002, 0.1703896802487605, 0.19498739965024248, 0.22023137579368218, 0.2433077523625283, 0.2619425446532489, 0.27459304519137245, 0.28042572980168146, 0.27927829790213066, 0.27169562222633103, 0.2590756810218855, 0.2439049080328701, 0.2299112022461105, 0.22163979954473628, 0.2228504392427391 ], [ 0.21316173638377522, 0.2160758021569025, 0.22629390922895112, 0.23893812118604485, 0.24950813580159842, 0.2548311211807226, 0.253039812562808, 0.24331395092076769, 0.22571762599360923, 0.20122481255709485, 0.17197375164622766, 0.1417847173853911, 0.11665327560616275, 0.10311796332752723, 0.10185504815487326, 0.10582260817397185, 0.10795613956000366, 0.10566929813831304, 0.10042643769531437, 0.09543932655705413, 0.09223076398811915, 0.08976991305568804, 0.08717067287960749, 0.08471786057617642, 0.08233674136575325, 0.07861404081945016, 0.07305648123630427, 0.06803553760980245, 0.06763447950693341, 0.0734892596595794, 0.08402248289554706, 0.09725478863834343, 0.11137567366241781, 0.1255037257540944, 0.1404283381757424, 0.1579107345854969, 0.17897787727490044, 0.2027689442460692, 0.2270213131482906, 0.24919069547788447, 0.2671573145403044, 0.27945888411200404, 0.28530667112278757, 0.28456793202871256, 0.2778041169832356, 0.26639633600510476, 0.25272584816793736, 0.24023786714361514, 0.2329786147019429, 0.23421192048265663 ], [ 0.20250707571530205, 0.20606039123954423, 0.21771196218026817, 0.23199137573980924, 0.24403483563195685, 0.2505511114116736, 0.2496768359137949, 0.2406353166906689, 0.2235372014703222, 0.19938960668749317, 0.17033793315078957, 0.1401721949549106, 0.11484560759951302, 0.10101500677771523, 0.09977761070524127, 0.10422059518467851, 0.10695669557648631, 0.10505508973042035, 0.09977881484144194, 0.09438164129013933, 0.09071313123189088, 0.08817924021373386, 0.08616607789885895, 0.08476336353089171, 0.0833097627407146, 0.07994811555772954, 0.07411931341533066, 0.0685726071201426, 0.06813530392465593, 0.07483578015291051, 0.08675076637028711, 0.10151744020571972, 0.11717282871661758, 0.13272061802643284, 0.1487377189369641, 0.16671286972882862, 0.18756280359708644, 0.21061567825996924, 0.23392874141967923, 0.25521980542466505, 0.2725302139197195, 0.2844862542382684, 0.2903484403007009, 0.2900128501648161, 0.28405073371768375, 0.273813551352571, 0.2615638996538309, 0.25046932953201667, 0.24412802250756804, 0.24538225973774203 ], [ 0.19195490736199977, 0.1961692612170942, 0.2093132234342106, 0.22526039916973062, 0.23878140052118665, 0.24648833608130774, 0.24653937422303182, 0.23821159509204198, 0.22166809958475073, 0.19795262066436273, 0.1692166448273328, 0.13920259534003096, 0.11375885894793618, 0.09956222523108801, 0.09815369223191406, 0.1028953696676418, 0.10613599750682134, 0.10460031543726842, 0.09933241668963357, 0.09357279427888708, 0.0894696106186936, 0.0869163683702975, 0.0855843190553918, 0.08528966068214036, 0.0847295376062118, 0.08167730355388285, 0.07558456945915944, 0.0695607214863964, 0.06911310922121512, 0.07664990145693851, 0.0898858132845478, 0.10605900537562647, 0.12309308016361724, 0.1399232142622803, 0.15694019059499964, 0.17538415739029858, 0.1960609624938222, 0.21844727721480817, 0.24088246513799502, 0.2613344232890089, 0.2780089652363411, 0.28962736689867363, 0.2955031739553731, 0.29556043244150715, 0.29037398763125183, 0.2812549980238802, 0.27033860009033495, 0.26052501225082597, 0.2550129396894648, 0.25628596200705467 ], [ 0.18162892213704238, 0.18651135898486204, 0.2011823786818541, 0.2187997492322099, 0.23377548381705823, 0.24264965672634348, 0.24361785586928986, 0.23601819069822844, 0.22006923689863, 0.19685228917274222, 0.16852241029291928, 0.1387603789145346, 0.11326515225785534, 0.09866144372038431, 0.096936175105634, 0.1018359764211534, 0.10550137271608935, 0.10432104863945983, 0.09910926183809737, 0.09304585170843535, 0.08855072514365152, 0.08604193308749868, 0.08547345141038262, 0.0863148239858821, 0.08660374876136427, 0.08382428616985534, 0.0775042574127466, 0.07108658124233508, 0.07066434130001725, 0.07897732196073584, 0.09340803718503156, 0.11082687080110885, 0.12907415658748075, 0.14704687440669056, 0.1649709731887517, 0.1838581829462415, 0.20440267216881183, 0.22619465922581253, 0.24781927999680783, 0.2674793205989191, 0.283545165451313, 0.29483765211241536, 0.3007263856671268, 0.3011624868605787, 0.29671899753068326, 0.28865840919772107, 0.278983747393558, 0.27034177901426026, 0.265576659307733, 0.26686601680029 ], [ 0.17167241746736553, 0.17720882063676424, 0.193407800100834, 0.2126612264892186, 0.22903956967128417, 0.23903645262655315, 0.24089747593744587, 0.2340253664638731, 0.21869404906911472, 0.19602071343545027, 0.1681601782660619, 0.13872155586778806, 0.1132291800585187, 0.09820986059441629, 0.09607685480840254, 0.10102911140200829, 0.1050514121334062, 0.10421538849588978, 0.09910285481858813, 0.09280225364438358, 0.08798439829364588, 0.08560150204127158, 0.08586515480393332, 0.0878389965043794, 0.088924237468181, 0.08639812604149437, 0.07991378444069702, 0.07321105459573497, 0.07284763810689673, 0.0818227542410117, 0.0972677893093545, 0.11575145337959035, 0.13504852872973952, 0.15403048868140112, 0.17277412815970875, 0.19208000091048313, 0.2125302813510092, 0.2337992035814173, 0.25468390238574684, 0.27360494839411764, 0.2890946326499359, 0.30007611695650066, 0.3059773020517614, 0.30677550534258535, 0.303037496074774, 0.2959709859555321, 0.28744571599132296, 0.2798710191364777, 0.27577726853310897, 0.27708030262520916 ], [ 0.16224732223436392, 0.16839451035585107, 0.18607782991612637, 0.20689089402376645, 0.2245893255899227, 0.23564406360749082, 0.23835846186972773, 0.23219928228316997, 0.21749252750719386, 0.19538726178207308, 0.1680334499270473, 0.13896325614318478, 0.11351969654697644, 0.0981088181758939, 0.09552944802322154, 0.10045814830072022, 0.10477460730982238, 0.10426410069300754, 0.09928265386129523, 0.09281690365210965, 0.08777547128113251, 0.0856227402918457, 0.08677347199843928, 0.08984611782297548, 0.09166841814074808, 0.08939189334059464, 0.08282590458671028, 0.07595882179213947, 0.07567252199056276, 0.08514928228601433, 0.10139141989775233, 0.12075165402704552, 0.1409454756873039, 0.16081613033419587, 0.18030102044949917, 0.20000428374479604, 0.2203966248966212, 0.2412119024683285, 0.26142872495434955, 0.2796675159286327, 0.29461760652072777, 0.30530556869977615, 0.3112190533253211, 0.31236074537899433, 0.309287638630767, 0.30314864097696037, 0.29568182409947685, 0.28907616554942905, 0.285584785158245, 0.28689871020785024 ], [ 0.15353031954284882, 0.1602071036394737, 0.17927567110489828, 0.20152574364384024, 0.2204320503570743, 0.23246142595412, 0.23597650229361725, 0.230503119115679, 0.21641320949115164, 0.19488184483650847, 0.16804939711879854, 0.13937077238640044, 0.11401672693612956, 0.09826814571153734, 0.09524978879248396, 0.100101356989749, 0.1046491223560493, 0.1044339622769284, 0.09960196790568179, 0.09304652464471189, 0.08790843476611498, 0.08611353077727096, 0.08819335316161656, 0.09230498244964483, 0.09480050386802295, 0.0927813639871823, 0.08622667404631895, 0.07931271491578257, 0.07909622336593675, 0.0888830886155673, 0.10568804819603975, 0.1257400984015388, 0.14669359853768604, 0.1673493736941271, 0.18750943857790556, 0.20759398188022404, 0.22796373718804974, 0.24839251969688314, 0.2680134600152071, 0.2856289378930156, 0.30007882857488777, 0.31049272511150694, 0.3164187453368676, 0.31788417827968135, 0.3154336765900664, 0.310155170390749, 0.30365882191553717, 0.2979306114167481, 0.29497886918910715, 0.2963008415796981 ], [ 0.14570452009065876, 0.15278293638604293, 0.17307298271867858, 0.19659028460938072, 0.21656538509223552, 0.22947096323573318, 0.23372332960235834, 0.22889822062753246, 0.2154049851233112, 0.19443763814399775, 0.16812267101585582, 0.13984194359207597, 0.11461481181833058, 0.09860632137960003, 0.09519328199592791, 0.09992955844868123, 0.10464358553007493, 0.10468270581649172, 0.10000678733412073, 0.09343895316165274, 0.08835218021924258, 0.08706067745193144, 0.09009887227229219, 0.0951697921796586, 0.09827243260700615, 0.09652473941239971, 0.09007389470972148, 0.08321403785977925, 0.08303083849613574, 0.09292176427110944, 0.11005607549222605, 0.1306276133377193, 0.15222331961668933, 0.17358001624437577, 0.19436320483944763, 0.2148193965672997, 0.2352018095613544, 0.25530879922655575, 0.2744047175855677, 0.29145668336072905, 0.3054475255708962, 0.31560823274398186, 0.3215474377615883, 0.32331633947526556, 0.32144554674503, 0.316961414003213, 0.31135153049297704, 0.3064159730898658, 0.3039469871179981, 0.3052741629392747 ], [ 0.13894523547081503, 0.14624419360538626, 0.16752264356627347, 0.1920934506985643, 0.21297646824614577, 0.2266487857033852, 0.23156744315825248, 0.2273451954123726, 0.21441862708912682, 0.1939931432765361, 0.1681778990203578, 0.14028934764355983, 0.1152235183992576, 0.09904783209943485, 0.09531037159509244, 0.09990399279790801, 0.10471881392564235, 0.10496450983051334, 0.10044412130654705, 0.09394208116287014, 0.08906586857880881, 0.08842940140996544, 0.09244190474897418, 0.09838088591750896, 0.10202479004999197, 0.10056346166060465, 0.09429828607649973, 0.08756716503800997, 0.08735447093671245, 0.0971439151299379, 0.1143889574461486, 0.13532686971445873, 0.15746914901472422, 0.1794630096288841, 0.20083213142692566, 0.22165757352432883, 0.24208836241196421, 0.2619357443683161, 0.2805755517424883, 0.29712355379956906, 0.3106973166838473, 0.3206266116964081, 0.3265800500180547, 0.3286321101913889, 0.3272984154611324, 0.3235444418321517, 0.3187416376563395, 0.3145206450691105, 0.3124829305468915, 0.31381251302043145 ], [ 0.1333994294012263, 0.14068394512951257, 0.16265160993934016, 0.18802630104278312, 0.20964170352613948, 0.22396523672396382, 0.22947495747608165, 0.22580493826350279, 0.21340799341542221, 0.1934935762944933, 0.1681510526568744, 0.14064101445685864, 0.11576676219032854, 0.09952016520030642, 0.09554165030967206, 0.0999756502697336, 0.10483042245166839, 0.10523526933236912, 0.10086905321793518, 0.09451163284695985, 0.0900054283739341, 0.09016558088238534, 0.0951535256003958, 0.10186642594829798, 0.10598810850034289, 0.10482419191416643, 0.0988073678452333, 0.09224650749658546, 0.09192255929511142, 0.10141831210011903, 0.11858012510857267, 0.13975533789879271, 0.16237166808826378, 0.18495947075679597, 0.20689220520241244, 0.22809193550231202, 0.24860759599213308, 0.2682549736692957, 0.2865049980734768, 0.30260741189631735, 0.3158060618976662, 0.3255261428334554, 0.3314952130994009, 0.3338104538731744, 0.3329722055984781, 0.32988678905764307, 0.3258166446360917, 0.3222385973181882, 0.3205856124506999, 0.3219148902111054 ], [ 0.12916076501098456, 0.13615015807915268, 0.15845520883466338, 0.18436098924827152, 0.20652726981810615, 0.2213858037320637, 0.227410558269416, 0.2242395458933267, 0.21233089239881286, 0.19289163759447858, 0.1679899383542281, 0.14084033299646623, 0.11618224685531899, 0.09995338036277776, 0.0958173571110584, 0.10008725979839465, 0.10493223322603021, 0.10545719072533064, 0.10125014376914021, 0.09511731935350136, 0.09113021349900466, 0.09220305087920018, 0.09815014367056575, 0.10554537279624612, 0.11008476815843403, 0.10922187807067532, 0.10349168226837889, 0.09710709058655927, 0.0965794893985386, 0.10561188126788935, 0.1225271680952197, 0.1438377674433259, 0.16687926824956442, 0.19003769701536882, 0.21252590907064542, 0.23411208275373027, 0.2547498821698622, 0.2742541497990411, 0.29217761634335476, 0.3078908778938808, 0.3207556661397218, 0.3302887109319557, 0.33627508225880914, 0.33883412544106756, 0.3384511255658511, 0.3359757507345621, 0.33256895097825045, 0.32956837227475333, 0.32825808206681334, 0.3295844593232588 ], [ 0.12624542028618324, 0.1326327926744255, 0.15489434827807497, 0.18105136759829984, 0.20359043755208628, 0.21887238691293862, 0.22533854861867988, 0.2226131186974413, 0.21114962914380947, 0.19214775012052032, 0.16765404610409418, 0.14084562192871333, 0.1164217206438367, 0.10028354473449377, 0.09606459791364695, 0.10017816716381102, 0.10498027666108903, 0.10560248332500806, 0.10157305165440762, 0.0957470878257036, 0.09240897483025194, 0.09447338034174936, 0.10134219303620035, 0.10933137828151464, 0.11423147874522568, 0.1136636374152646, 0.10823243479110622, 0.10199792313085886, 0.1011701633439462, 0.10959650531103221, 0.12613548105571848, 0.14750839013140724, 0.17094971366213135, 0.19467414441248984, 0.21772260872213717, 0.23971370054353336, 0.26051136029329774, 0.27992647178442404, 0.29758304588441575, 0.31296100578013175, 0.3255318509625213, 0.33489961491893255, 0.34090512256520433, 0.34368936698416125, 0.3437232133433102, 0.34180274056587223, 0.33899506298313514, 0.33651224608365576, 0.33550671323448583, 0.3368277327458854 ], [ 0.1245787566728117, 0.13005911533799053, 0.15189678061873632, 0.17803538523480822, 0.20078167705918137, 0.21638489453774584, 0.2232239675854401, 0.22089245421799983, 0.2098312744234799, 0.1912298621868716, 0.16711392847637316, 0.14062957699419232, 0.11645205736065917, 0.10045847312374263, 0.09621699539467878, 0.10019105349085021, 0.1049370314665726, 0.10565605023933225, 0.10184232616464464, 0.09640919749192693, 0.09382402286857887, 0.09691378702327315, 0.104641659044768, 0.11313679186007448, 0.11834211413781962, 0.11805303180694625, 0.11290925815408888, 0.1067735490109665, 0.10554975533694963, 0.11325477049290736, 0.1293215681369854, 0.15071299991171855, 0.17455159226943526, 0.19885434524211254, 0.22247894874385812, 0.24489852162128803, 0.2658936025390357, 0.2852702175996952, 0.30271557638756624, 0.31780894805282695, 0.33012390318243534, 0.33934735431870816, 0.3453738768300667, 0.34836559986390475, 0.3487799034704872, 0.34736271362140486, 0.34509491073639725, 0.3430755250604336, 0.3423405315732169, 0.34365389128673024 ], [ 0.12400528610144888, 0.1283004302717044, 0.14936271797990935, 0.17523917821286547, 0.19804746635907972, 0.21388311284118133, 0.22103376422978288, 0.21904764926840578, 0.20834771064242985, 0.1901129082633847, 0.166350211233672, 0.14017858213476586, 0.1162566841187713, 0.10044352450846725, 0.09622321076296148, 0.10007892675386358, 0.10477548650798624, 0.10561714717969563, 0.10208131886734499, 0.09713181456429772, 0.09537262069151436, 0.09947145141768153, 0.10796686766601264, 0.11687604436089141, 0.12233058854644217, 0.12229428556504979, 0.11740700858418608, 0.1113022071441891, 0.10959076249094311, 0.11648478015930203, 0.13201615232550856, 0.15341101031336396, 0.1776656977638234, 0.20257374608178416, 0.2267992117509881, 0.24967429878179428, 0.270903317040574, 0.290288322644856, 0.30757373344819955, 0.3224296149118575, 0.3345244078446317, 0.3436233992720112, 0.349672723313435, 0.3528551203419403, 0.3536156215262139, 0.35265365027362666, 0.3508712596800348, 0.34926595388702036, 0.348770653862424, 0.3500742179941185 ], [ 0.12431569205749823, 0.12718882686238428, 0.14717407723291973, 0.17258250887226778, 0.19533364371787304, 0.21132878307279698, 0.21873801106788365, 0.21705263623727675, 0.20667551914052779, 0.18877801663105526, 0.16535228393355955, 0.1394917237094842, 0.1158366970155717, 0.10022612261361818, 0.09605346224350882, 0.09981118630062007, 0.10448266622689628, 0.10550002382721846, 0.10233015415274878, 0.0979598333528052, 0.09706512759506025, 0.10210418175416203, 0.11124454436635053, 0.12046804282325457, 0.12611349989815024, 0.1262960880163462, 0.12162107783353494, 0.11547110801255471, 0.11318772079433297, 0.11920417790228006, 0.134167188069156, 0.15557754095727414, 0.1802863553525484, 0.2058384414835825, 0.2306955990652089, 0.2540547483932899, 0.27555206036203467, 0.2949879795144601, 0.31215987614075347, 0.3268213315567076, 0.338728971202472, 0.34772195002262596, 0.3537956289191936, 0.35715280365350155, 0.35822740820697274, 0.3576760972064611, 0.35632920403073326, 0.35509321654686454, 0.3548098189745071, 0.3561016242521934 ], [ 0.12527885035778308, 0.1265390169485496, 0.14520588712542937, 0.16998506071197286, 0.19258911465338627, 0.2086878099759159, 0.2163111423516856, 0.21488568519499407, 0.20479578025728315, 0.187211554246873, 0.1641167004708068, 0.13857929388352844, 0.11521104127572669, 0.09981838388085897, 0.09570374993946902, 0.09937818210733214, 0.10406240602840604, 0.10533362107350236, 0.10264178756407273, 0.09894879833457851, 0.09891997391932221, 0.10477813469190275, 0.11440977687760025, 0.12383758267582894, 0.1296123752697994, 0.12997477712159858, 0.1254612128518239, 0.11918966356554478, 0.11626032825884862, 0.12135357612045848, 0.1357428424039018, 0.15720554141209153, 0.18242267069981186, 0.2086657669771868, 0.2341883928153873, 0.25805943042651686, 0.27985593438449063, 0.2993802453169907, 0.31647980282974375, 0.33098549572830016, 0.34273593806566643, 0.3516396905684006, 0.3577389022403465, 0.3612558198050516, 0.362614573481402, 0.36243276071167607, 0.3614757308530376, 0.36056851463373324, 0.36047199421409376, 0.3617502519533055 ], [ 0.12667111755651267, 0.12617007680228887, 0.14333821475775066, 0.16737306521138673, 0.1897697161766473, 0.20593252387198674, 0.21373320383851233, 0.21252990739630379, 0.20269386384894075, 0.18540410964597379, 0.16264534128087, 0.1374606087666181, 0.11441525844216699, 0.09925751090824755, 0.09519772545794286, 0.09879414670737008, 0.10353731795657566, 0.10516047579753075, 0.10307644773540908, 0.10015618485899556, 0.10095609268944895, 0.107463715064301, 0.11740472820436178, 0.12691601926589202, 0.13275547765544932, 0.13325685430963216, 0.12885409830945171, 0.12239140695903016, 0.11875563061103861, 0.12289961163094446, 0.13673448618597542, 0.15830791804872688, 0.18409964498843245, 0.2110846996882042, 0.23730595869239052, 0.2617135346136368, 0.2838352464257784, 0.30347964441766034, 0.3205423611751255, 0.3349262366525628, 0.346546106858867, 0.3553755402168184, 0.3615009497618922, 0.36516336317062187, 0.36677838024282944, 0.3669281473296986, 0.3663193450599504, 0.3657042105612525, 0.3657720443098177, 0.3670351389516977 ], [ 0.12829838029869087, 0.12592370031800246, 0.14146731012780345, 0.164685813326977, 0.1868420567548454, 0.203043920586263, 0.21099110141836522, 0.20997379772333832, 0.20035929344442124, 0.18334953308513866, 0.1609434473584047, 0.1361610938502248, 0.11349846302723639, 0.0986036837231058, 0.09458621300957616, 0.09809860297314336, 0.10294998203707524, 0.10503505990481717, 0.10369521754486669, 0.10163093894291846, 0.10318401205553694, 0.11013103718905105, 0.12017695179709678, 0.1296415374380768, 0.13547923500967085, 0.13608089539453821, 0.13174502549694925, 0.12503513728499044, 0.12064973024962834, 0.12383782620787226, 0.13715970533787522, 0.15891957886286595, 0.18535905887788365, 0.2131359984011237, 0.24008454763959008, 0.2650475465567441, 0.28751411551588096, 0.30730375668697496, 0.3243590586874844, 0.338650076034152, 0.3501624450068572, 0.3589304060408893, 0.3650820377209145, 0.36887639709825637, 0.37072175623695014, 0.37116824702377943, 0.37086974695999486, 0.3705135254987622, 0.3707254528786275, 0.3719719375803573 ], [ 0.130009861818147, 0.12567768739086263, 0.13951524380335456, 0.16188173598370392, 0.1837871762476509, 0.200013806302124, 0.20807983531012297, 0.2072118533414942, 0.19778577093844157, 0.1810441744867341, 0.15901771851355115, 0.13470879927558943, 0.1125184279179391, 0.09793523295730938, 0.09394440908320115, 0.09735640263341837, 0.10236339472686679, 0.10502180543899367, 0.10455400752383023, 0.10340339043653315, 0.10559778987097104, 0.11274641197505318, 0.12267804309222771, 0.13195935888835977, 0.13772941685051493, 0.1383989891262602, 0.13409893954823474, 0.12710562631594438, 0.12194928391507354, 0.1241954948766924, 0.1370652797780327, 0.15909924175651083, 0.18625998377262748, 0.21487199992465486, 0.24256785531085806, 0.26809677258539877, 0.2909200123228944, 0.31087278402853935, 0.3279436710871456, 0.34216559162676463, 0.3535898067840387, 0.36230693866781816, 0.36848406151384616, 0.37239741413471167, 0.3744490326834069, 0.37516025437081246, 0.37513755516553426, 0.3750102836855894, 0.3753480881492735, 0.37657667798137656 ], [ 0.13170494239938527, 0.1253549005288265, 0.13743783348089159, 0.15894386295625793, 0.18060389053783488, 0.19684677414533874, 0.20500370349461147, 0.20424530277530203, 0.19497144888408216, 0.17848647675697474, 0.15687475219135555, 0.1331307832781623, 0.11153502138003692, 0.09734104825199567, 0.09336691047047092, 0.0966564840085681, 0.1018605906113469, 0.10519299590251145, 0.10569932503841553, 0.10547986684600295, 0.10817162133155174, 0.11527105843007412, 0.12486315944240957, 0.13382217407363506, 0.13946221264581046, 0.14017786176110533, 0.13590109029248706, 0.12861405768699022, 0.12269287049717235, 0.12403439170990044, 0.13652995964600048, 0.15893075382181138, 0.1868787322009255, 0.21635597862295738, 0.24480630205393467, 0.2709007085779424, 0.29408322537223386, 0.3142090910317784, 0.3313118469884717, 0.34548308408532297, 0.3568346554909027, 0.36550929341985006, 0.371710324088523, 0.3757302120673031, 0.37796570784678357, 0.37891232362693184, 0.37913406871765465, 0.37920869623123094, 0.37965600624780727, 0.3808655695164514 ], [ 0.13333494056736508, 0.12492856698669236, 0.13523094650245981, 0.15588453449187553, 0.17731168584441367, 0.19356193199447028, 0.2017774513368626, 0.2010829727957508, 0.19191953202252462, 0.17567708700665266, 0.1545201555538174, 0.13145008166609307, 0.11060293191324638, 0.09691059282838227, 0.09296107464481972, 0.09610928134542093, 0.10154314503891188, 0.10562652805209939, 0.10716669848055187, 0.10784528905278987, 0.1108636360323958, 0.11766252601241428, 0.12669168973636816, 0.13519100901881598, 0.14064536914436118, 0.14139984621201107, 0.13715744103053426, 0.12959822974569604, 0.12295211736162484, 0.12345326958600218, 0.13566666801883934, 0.1585235391578904, 0.18730801706155722, 0.21766097562358006, 0.24685600529599316, 0.2735022463071066, 0.2970362519583171, 0.3173367189397856, 0.33448070894141424, 0.34861424820025233, 0.3599047917031583, 0.36854289853378275, 0.3747653244283571, 0.37887968571440617, 0.3812782337780576, 0.3824333539298471, 0.3828710631772372, 0.38312317867385015, 0.3836652865283389, 0.38485483470350995 ], [ 0.13490147275973666, 0.12442472940593374, 0.1329352687808553, 0.15274920685079674, 0.17395299595713065, 0.19019428355024454, 0.19842733245730776, 0.19774230856021385, 0.18863927676099232, 0.17261963532305136, 0.15195866420663173, 0.12968413149135946, 0.10976579211809685, 0.09672326454078264, 0.09283971030624477, 0.09584249585685689, 0.10152802782027098, 0.10640333081380658, 0.10898158119502017, 0.1104708524942245, 0.11362409730663167, 0.11987843397899192, 0.1281291082182886, 0.13603665908219134, 0.1412595312449188, 0.14206383871970207, 0.13789492378059753, 0.1301224283332235, 0.1228322445466247, 0.12258950494042184, 0.13462345750547272, 0.1580116423053255, 0.1876550661967972, 0.21886801924437754, 0.24877743201013738, 0.27594672136750875, 0.29981311843615527, 0.320280875605693, 0.33746845249417873, 0.3515718501425073, 0.36280908933447575, 0.3714142319840619, 0.37765455699775313, 0.3818516342300385, 0.38439382449447135, 0.385732801301481, 0.3863606161948731, 0.38676819750725416, 0.38739189436096655, 0.38856057105241676 ], [ 0.1364523041893178, 0.12392212918498376, 0.13063934903034277, 0.1496190393174887, 0.17059462585185847, 0.1867956355710201, 0.19499202723668133, 0.194250543459067, 0.18514743517745066, 0.169322298035085, 0.1491955317580728, 0.12784538279596863, 0.10905449856021325, 0.09684289752975607, 0.09311420906549799, 0.0959947267792519, 0.10194213909330739, 0.10760408971429053, 0.11116172711707821, 0.11332261648095981, 0.11640419350296102, 0.12188158472521111, 0.12914986039409618, 0.13634175625638334, 0.1412999108985524, 0.14218636033243284, 0.1381615681619951, 0.13027675518691545, 0.12247140633562371, 0.12161890362201913, 0.1335821592411737, 0.15755070435331645, 0.18803845688589432, 0.22006369446652418, 0.2506337421548674, 0.27828081921206627, 0.302448641179303, 0.3230674075645896, 0.3402939465707786, 0.354369412930616, 0.365557241316872, 0.37413060829705924, 0.38038432284652274, 0.38465258359188564, 0.38732028396056956, 0.3888205144956462, 0.38961495872047186, 0.3901581416462612, 0.3908515675272976, 0.3919986369092511 ], [ 0.13807493712513225, 0.12354909408466032, 0.12848019271668004, 0.14661167426598626, 0.16732798169173224, 0.18343486487772115, 0.19152333848956218, 0.19064598657236217, 0.181470156879224, 0.16580021180034782, 0.14623932560533554, 0.1259443863652798, 0.108490665020377, 0.0973198745281958, 0.09388853405384737, 0.09670644517311008, 0.1029139990104159, 0.10930498900131236, 0.11371983096692162, 0.11636897885896004, 0.11916365871138762, 0.12364547765089523, 0.12974103279324323, 0.13610349066422053, 0.1407783843294735, 0.14180280474026435, 0.13802646860750986, 0.13017558327842338, 0.12203790415825568, 0.12075213236118097, 0.13275328492692373, 0.15731218393382254, 0.18858352289189503, 0.22133708074308941, 0.25248886401602866, 0.2805513695846279, 0.3049776460236258, 0.32572226356886347, 0.342976339962896, 0.35702091289340354, 0.3681595167918176, 0.37669997664625116, 0.38296155294540196, 0.3872896238922309, 0.3900658523525287, 0.3917065920864772, 0.3926463485627761, 0.3933072154200573, 0.3940597229676835, 0.39518455801050983 ], [ 0.13988776725655963, 0.12347633204106011, 0.12663998371456145, 0.14387926475440346, 0.16426765286830478, 0.18019733575644237, 0.18808654834254976, 0.18697935439445093, 0.17764531459141633, 0.16207873970038347, 0.14310610768115145, 0.12399508206072384, 0.10809332831426957, 0.09819549272020117, 0.09525361776908242, 0.09810822209956499, 0.10456263920563603, 0.11157250310506586, 0.116665535037515, 0.11958619657849472, 0.12187661000264847, 0.12515951653285912, 0.1299065411141245, 0.13533697749497733, 0.13972608673710793, 0.14096890214330854, 0.13757948410838527, 0.12995472325036067, 0.12172412169412888, 0.12022584570730128, 0.13236561974035693, 0.1574743450645581, 0.18941636888656188, 0.22277616296524283, 0.25440537681865205, 0.28280407346519487, 0.30743416996547357, 0.32827096169407144, 0.34553467999859866, 0.3595404903769106, 0.37062653178251426, 0.3791307314374888, 0.38539364422741496, 0.38977026102950313, 0.3926390692159388, 0.3944012585096309, 0.39546696347400734, 0.3962293501919795, 0.3970313811066554, 0.3981334519294539 ], [ 0.14202859141731658, 0.12390422292077895, 0.1253369048310767, 0.14160249783125725, 0.16154781839694746, 0.17718322583681256, 0.18476027799904363, 0.1833150162814224, 0.1737251522081277, 0.1581975107426221, 0.13982482757310852, 0.12202159385068972, 0.10788649262409918, 0.09950587417625688, 0.0972817345100634, 0.10030708690411588, 0.10698474250647948, 0.11445778042865838, 0.12000640144192709, 0.12296171701943506, 0.12453545644165831, 0.12643348346769115, 0.1296715886801225, 0.1340792331086703, 0.13819652455695314, 0.13976234878140054, 0.13693048064310218, 0.12976687051462998, 0.12173612593100108, 0.1202867703370545, 0.13264947672441108, 0.15821011118903747, 0.19065680474242477, 0.22446392066413867, 0.2564423103225568, 0.28508221885225804, 0.3098506736377253, 0.33073807418503737, 0.3479875503638086, 0.3619421782738521, 0.372969035351457, 0.38143153651164274, 0.3876883087266864, 0.3921022823911659, 0.3950486522532594, 0.3969147570487047, 0.3980888113282342, 0.3989381321247043, 0.39978110537897743, 0.40085996799179807 ], [ 0.14464063891590753, 0.12504367900623314, 0.12480803860091667, 0.13997938273869448, 0.15931600473058244, 0.1745045201017647, 0.1816356490916575, 0.17973194799883083, 0.1697790585037427, 0.154215050029639, 0.13644361784387227, 0.12006565345913031, 0.10790587972984168, 0.10128466102113248, 0.10002130172651298, 0.10337307141332085, 0.11024224070760372, 0.11799167317434557, 0.12374790361153958, 0.12649545794312436, 0.12715293848795087, 0.12750101052092533, 0.12908714296074836, 0.13239367057003573, 0.13626913629994183, 0.13828442636450572, 0.13620782926948388, 0.12977502457920775, 0.1222786203993546, 0.1211683711733076, 0.13381417254527483, 0.1596728575084874, 0.19241084159508046, 0.2264743964305329, 0.2586529991633112, 0.2874254503518328, 0.31225729602674596, 0.3331467453641196, 0.3503527355281842, 0.36423965209407094, 0.3751977132122202, 0.38361116399363476, 0.38985343612645107, 0.39429363611671586, 0.3973033905944568, 0.39925725800468775, 0.40052365529468487, 0.40144674395945384, 0.4023229549068127, 0.4033782405657549 ], [ 0.14785709668716734, 0.12709126889770714, 0.1252836931252948, 0.13920832418567858, 0.1577240115508171, 0.17228050052608762, 0.17881451510327098, 0.17632409744240135, 0.1658961349887705, 0.15021366433565522, 0.13303652462822152, 0.11819370991158328, 0.10820382166960503, 0.10356425379800217, 0.10349350826310871, 0.1073289458287474, 0.11435362510675451, 0.12218169418990513, 0.127892787924462, 0.13019936921427538, 0.12976241214494813, 0.12842180038860906, 0.1282340989145511, 0.13037490209665606, 0.1340530684821966, 0.13666124699987464, 0.13555577717678527, 0.1301438861627077, 0.12353654238611227, 0.12306252573985188, 0.1360229987141366, 0.1619823482008827, 0.19476367441174752, 0.22886910975810604, 0.26108314464894206, 0.2898686607827275, 0.3146811834919541, 0.3355182580047978, 0.3526469192063038, 0.36644600521576515, 0.377323010643344, 0.38567834869149503, 0.391896969943423, 0.39635232352587213, 0.3994120515087834, 0.4014387804985239, 0.40278295218838267, 0.40376791897012887, 0.40467044855275836, 0.4057018539061575 ], [ 0.15178608034354368, 0.1302019235240879, 0.12695564532062434, 0.1394667758403811, 0.15691641458248687, 0.1706317250337374, 0.17640653643250206, 0.17319977379115398, 0.16218704524239586, 0.14630400922757641, 0.129709958892454, 0.11650266605879385, 0.10885150540485596, 0.10637573078784576, 0.10769231463862171, 0.1121479389251413, 0.11929266644848135, 0.12701187557436244, 0.1324402555674497, 0.13409567723531335, 0.13241647490639555, 0.12928226472783153, 0.12722660145767825, 0.12815338404268045, 0.13169064580377848, 0.13504400073355532, 0.13513025423589609, 0.1310297397461219, 0.12565667506315953, 0.1260927539248646, 0.13937221178213552, 0.1652137566785949, 0.19777418313763312, 0.23169419105089506, 0.2637692329710257, 0.2924410695328961, 0.317145923089148, 0.33787166250660516, 0.3548854237456584, 0.36857355264851605, 0.37935497633685217, 0.38764165879020135, 0.3938267974811783, 0.39828630428726824, 0.4013832996052375, 0.4034691265386335, 0.40487780241803395, 0.40591390549880135, 0.40683653880458587, 0.40784381696469885 ], [ 0.15649868538030345, 0.1344654183774953, 0.12994622842616124, 0.14088923537416298, 0.15701788145834658, 0.16967276689576236, 0.1745249432174719, 0.17047960595828776, 0.1587844153010249, 0.14262840142879854, 0.12660772985370672, 0.11512292026320849, 0.10993795950811291, 0.10974778773170606, 0.11258830366221224, 0.11776328350103694, 0.12499593468881319, 0.13244567009078229, 0.13738533836100147, 0.13821420887891792, 0.13518400645852105, 0.13019412307579983, 0.12621366647446816, 0.12589899269534383, 0.1293595610421912, 0.13360728456223944, 0.13509273389320628, 0.1325699075400978, 0.12873388107523603, 0.130300418530787, 0.1438819531076138, 0.16939238452574235, 0.2014718201451282, 0.23497854757718512, 0.26673743390204413, 0.29516554183733046, 0.3196711054602781, 0.3402234810206696, 0.3570819962951744, 0.37063366610789117, 0.38130312850489134, 0.38950938339074276, 0.39565065357811563, 0.4001034138847312, 0.40322562764334513, 0.40535782613945187, 0.40681891015167626, 0.4078964406861352, 0.4088335941436038, 0.4098165467763583 ], [ 0.1620225359183877, 0.13989482424478694, 0.13428943157922352, 0.14355013310115278, 0.1581213172138451, 0.16950434126668776, 0.17328099442300485, 0.16829263038540782, 0.1558408511103827, 0.13936145574610384, 0.12391288534688139, 0.11421704034800047, 0.11156541960480956, 0.11370505866443813, 0.11813521078828877, 0.12408376252964627, 0.1313756383831242, 0.13843107794453272, 0.1427186635196322, 0.14258915913849704, 0.13814571947223248, 0.13129042405344288, 0.1253778106751124, 0.12382194579672127, 0.12727219973735193, 0.13254435929249483, 0.135602020635941, 0.1348732969078078, 0.13280557573352256, 0.1356501335880524, 0.14950226189184743, 0.17449516737466186, 0.2058563233633068, 0.23873324423702671, 0.2700030601333136, 0.2980581862927514, 0.3222720354635494, 0.34258749551242157, 0.3592486461352523, 0.37263664247366957, 0.38317634417435276, 0.39128943721730786, 0.39737603805561833, 0.4018112929108034, 0.4049472981350113, 0.4071140924067711, 0.4086165524898812, 0.40972673218570216, 0.41067338871598824, 0.4116318592021567 ], [ 0.16834188939754982, 0.14643186328802416, 0.1399327774591573, 0.14745654089762597, 0.16027915966494305, 0.1702057815732439, 0.17277740713985193, 0.16677023062519888, 0.15352356968978764, 0.1367061405015017, 0.12184387114257154, 0.11397223708501418, 0.11384116181913864, 0.11826611400702194, 0.12427706500951857, 0.13100814274613767, 0.13833271160368377, 0.14490664289336141, 0.14842661418871358, 0.14725563106327622, 0.14138841722066423, 0.13271853323552618, 0.12492904032088745, 0.12216869777381027, 0.12566991249750864, 0.1320582275695916, 0.13680433603740924, 0.13801361603285073, 0.13785514536830612, 0.14204739568662733, 0.15612963058888313, 0.18045803805548927, 0.2109001344239155, 0.24295211448009155, 0.2735706111561499, 0.3011282467224114, 0.3249595999350984, 0.3449746248597984, 0.3613955357452655, 0.3745916068218798, 0.38498277213776194, 0.3929892825606716, 0.3990101466409914, 0.4034173276810923, 0.4065562949635294, 0.4087467856139888, 0.4102805565791171, 0.41141544680019987, 0.4123670982701653, 0.41330096595536114 ], [ 0.1754034415688454, 0.15396445686554378, 0.14675609595301192, 0.15255230761949556, 0.16349964607515619, 0.17182899840956994, 0.1731013577564357, 0.1660380186356914, 0.15200490140985298, 0.1348832033238893, 0.1206422188965876, 0.11458534559448919, 0.11686660898678088, 0.1234413840200844, 0.13095421108814617, 0.1384358896871748, 0.14576720379979738, 0.15180705023367025, 0.15449176625755956, 0.15224624596885378, 0.1449983362739001, 0.13463000515146104, 0.12509261137855587, 0.12120996607187717, 0.12480990489556694, 0.13234804160903255, 0.13882275104262043, 0.14202638075086532, 0.14382210675256762, 0.14936019132905112, 0.16362709669620473, 0.18718678985305096, 0.2165529053135446, 0.24761345108614505, 0.27743436678438366, 0.3043782817188635, 0.32774029239787295, 0.34739289180288396, 0.3635309261797757, 0.37650645023992957, 0.38672976952182725, 0.3946158682526844, 0.40055981500099314, 0.40492860161992533, 0.40806028427920477, 0.4102643853832904, 0.41182028371839224, 0.4129727051046359, 0.41392530144735723, 0.41483447696602604 ], [ 0.18312559765672382, 0.16234779272886635, 0.15459678825121387, 0.1587311892556257, 0.16774864398498437, 0.17439492786529184, 0.17431794897078484, 0.16620631043397668, 0.15144871577108193, 0.1341127130051464, 0.12054997204490828, 0.11624077699852405, 0.12072536095666325, 0.1292312586678754, 0.13810741311735306, 0.14627379841763766, 0.15358517537006655, 0.15906754166939172, 0.16089344149270066, 0.15758809197447549, 0.14905417364341036, 0.13716797736182285, 0.1260899543590824, 0.12121874208315507, 0.12494347570784632, 0.13359171522513044, 0.1417476195378533, 0.14690999054210455, 0.1506154191345905, 0.1574390383831955, 0.17184270042396918, 0.19456879525977594, 0.2227472001831127, 0.2526825014571443, 0.2815794443087159, 0.30780460385624986, 0.33061638517270836, 0.3498474762569207, 0.36566117526987346, 0.3783878016168575, 0.3884238614093511, 0.39617558518632595, 0.40203147537996353, 0.406351856819959, 0.40946658395455504, 0.41167497016020255, 0.413244619614101, 0.4144080812302151, 0.41535798561988596, 0.41624240725036277 ], [ 0.19140872881030316, 0.17142315801418773, 0.1632736221913675, 0.1658542131582549, 0.17295619547027044, 0.17789303201592482, 0.1764651594685803, 0.1673604720154963, 0.1519941782988033, 0.13458875706937382, 0.12177817908836393, 0.11908480572611739, 0.12547243226232455, 0.13562463438173022, 0.1456800425868509, 0.15443919722078028, 0.1617024464193553, 0.16662688710977408, 0.1676082445053954, 0.1633002392314786, 0.15362057837858373, 0.14045366550294344, 0.12811437608357523, 0.12243891795571417, 0.12628721724080727, 0.13592748290923262, 0.14562985942273438, 0.15263023779841542, 0.15812663000426944, 0.1661324056609693, 0.18062368598011674, 0.20248355332683257, 0.2294044842034818, 0.25811443070961, 0.2859831978429688, 0.31139793335935495, 0.3335862311561594, 0.35234084754185174, 0.36779078521182806, 0.3802410316118921, 0.3900707224339055, 0.3976742376202813, 0.4034311251970587, 0.4076934651286747, 0.4107821408980314, 0.41298620323158347, 0.4145619700242142, 0.41573060707597304, 0.4166745565653747, 0.41753418754935656 ], [ 0.20014455035792064, 0.18103206220921833, 0.17260492820220197, 0.17376666603853066, 0.17902590598693782, 0.18228378158416972, 0.1795511415555339, 0.16955283353416842, 0.15373977145056703, 0.1364519395807488, 0.12447252811957783, 0.12320244700079695, 0.1311269940562831, 0.14259816009663293, 0.15361878719993488, 0.16286072383302552, 0.17004597610469094, 0.17442902066858884, 0.17461051026074653, 0.16939199192896334, 0.15874293360412636, 0.14457432484353475, 0.13130618798367877, 0.12504984803197294, 0.1289930472991662, 0.13943863461309403, 0.1504784683293499, 0.15912699103737013, 0.16624074253566956, 0.1752967923100869, 0.18982588870546305, 0.2108109862038562, 0.23644067561587123, 0.2638574202108689, 0.2906168225449648, 0.3151442097682979, 0.33664467126535225, 0.35487296478357744, 0.3699224943882348, 0.38207028610493876, 0.3916751787871605, 0.3991170292487648, 0.40476430682560627, 0.4089594080661163, 0.41201351553601645, 0.4142053245884994, 0.4157802611063057, 0.41694878029563437, 0.41788385134672484, 0.418718678087311 ] ] }, { "hoverinfo": "text", "legendgroup": "In-sample", "marker": { "color": "black", "opacity": 0.5, "symbol": 1 }, "mode": "markers", "name": "In-sample", "text": [ "Arm 0_0
mean_accuracy: 0.873 (SEM: None)
lr: 3.77048e-05
momentum: 0.446225", "Arm 10_0
mean_accuracy: 0.931833 (SEM: None)
lr: 0.000432165
momentum: 0.274466", "Arm 11_0
mean_accuracy: 0.880667 (SEM: None)
lr: 2.89393e-05
momentum: 0.687405", "Arm 12_0
mean_accuracy: 0.946667 (SEM: None)
lr: 0.000631163
momentum: 9.34713e-05", "Arm 13_0
mean_accuracy: 0.915833 (SEM: None)
lr: 0.000283762
momentum: 0.43319", "Arm 14_0
mean_accuracy: 0.957 (SEM: None)
lr: 0.00282559
momentum: 0.281116", "Arm 15_0
mean_accuracy: 0.885167 (SEM: None)
lr: 0.000115583
momentum: 0", "Arm 16_0
mean_accuracy: 0.973833 (SEM: None)
lr: 0.00349935
momentum: 0", "Arm 17_0
mean_accuracy: 0.961 (SEM: None)
lr: 0.00127536
momentum: 0.279215", "Arm 18_0
mean_accuracy: 0.7945 (SEM: None)
lr: 1.44105e-05
momentum: 1", "Arm 19_0
mean_accuracy: 0.966833 (SEM: None)
lr: 0.00177763
momentum: 0.104203", "Arm 1_0
mean_accuracy: 0.536167 (SEM: None)
lr: 1.50007e-06
momentum: 0.859993", "Arm 20_0
mean_accuracy: 0.958333 (SEM: None)
lr: 0.00153168
momentum: 0.36512", "Arm 21_0
mean_accuracy: 0.9545 (SEM: None)
lr: 0.00180488
momentum: 0", "Arm 22_0
mean_accuracy: 0.110667 (SEM: None)
lr: 0.00516279
momentum: 0.102548", "Arm 23_0
mean_accuracy: 0.116667 (SEM: None)
lr: 0.010201
momentum: 0", "Arm 24_0
mean_accuracy: 0.111833 (SEM: None)
lr: 0.00942208
momentum: 0.135133", "Arm 25_0
mean_accuracy: 0.888333 (SEM: None)
lr: 0.000135261
momentum: 0.311582", "Arm 26_0
mean_accuracy: 0.951667 (SEM: None)
lr: 0.000847533
momentum: 0.160155", "Arm 2_0
mean_accuracy: 0.7985 (SEM: None)
lr: 2.65622e-05
momentum: 0.247377", "Arm 3_0
mean_accuracy: 0.931167 (SEM: None)
lr: 5.61769e-05
momentum: 0.818812", "Arm 4_0
mean_accuracy: 0.0931667 (SEM: None)
lr: 0.353442
momentum: 0.0550975", "Arm 5_0
mean_accuracy: 0.965833 (SEM: None)
lr: 0.00133006
momentum: 0.441355", "Arm 6_0
mean_accuracy: 0.102667 (SEM: None)
lr: 0.000429886
momentum: 1", "Arm 7_0
mean_accuracy: 0.924833 (SEM: None)
lr: 0.000168348
momentum: 0.632889", "Arm 8_0
mean_accuracy: 0.1105 (SEM: None)
lr: 0.00224961
momentum: 0.83229", "Arm 9_0
mean_accuracy: 0.204333 (SEM: None)
lr: 1e-06
momentum: 0.493107" ], "type": "scatter", "x": [ 3.770478314941209e-05, 0.00043216540546621934, 2.893928031624302e-05, 0.0006311630696854857, 0.00028376216822593883, 0.0028255895325504246, 0.00011558254490459247, 0.003499350987498055, 0.0012753591554555425, 1.4410478955993728e-05, 0.0017776312510395422, 1.5000705860135848e-06, 0.0015316810490992995, 0.0018048791350339288, 0.005162793679334533, 0.010200974476274442, 0.00942208485482061, 0.0001352613243432586, 0.0008475328951433113, 2.6562208898820995e-05, 5.617694542454495e-05, 0.35344232890633676, 0.0013300560007633963, 0.0004298857159271888, 0.0001683479355878444, 0.002249609867271813, 1e-06 ], "xaxis": "x", "y": [ 0.4462248980998993, 0.2744658616618241, 0.687404610446525, 9.347126188592805e-05, 0.43319045420051433, 0.2811155816297643, 0.0, 0.0, 0.27921473636842903, 1.0, 0.10420324217137585, 0.8599934121593833, 0.3651204901643246, 0.0, 0.10254845447365886, 0.0, 0.13513306632554634, 0.3115821445870835, 0.16015488542720355, 0.24737721495330334, 0.8188121551647782, 0.05509748309850693, 0.4413554677802919, 1.0, 0.6328890805509166, 0.8322899366209464, 0.49310670658150574 ], "yaxis": "y" }, { "hoverinfo": "text", "legendgroup": "In-sample", "marker": { "color": "black", "opacity": 0.5, "symbol": 1 }, "mode": "markers", "name": "In-sample", "showlegend": false, "text": [ "Arm 0_0
mean_accuracy: 0.873 (SEM: None)
lr: 3.77048e-05
momentum: 0.446225", "Arm 10_0
mean_accuracy: 0.931833 (SEM: None)
lr: 0.000432165
momentum: 0.274466", "Arm 11_0
mean_accuracy: 0.880667 (SEM: None)
lr: 2.89393e-05
momentum: 0.687405", "Arm 12_0
mean_accuracy: 0.946667 (SEM: None)
lr: 0.000631163
momentum: 9.34713e-05", "Arm 13_0
mean_accuracy: 0.915833 (SEM: None)
lr: 0.000283762
momentum: 0.43319", "Arm 14_0
mean_accuracy: 0.957 (SEM: None)
lr: 0.00282559
momentum: 0.281116", "Arm 15_0
mean_accuracy: 0.885167 (SEM: None)
lr: 0.000115583
momentum: 0", "Arm 16_0
mean_accuracy: 0.973833 (SEM: None)
lr: 0.00349935
momentum: 0", "Arm 17_0
mean_accuracy: 0.961 (SEM: None)
lr: 0.00127536
momentum: 0.279215", "Arm 18_0
mean_accuracy: 0.7945 (SEM: None)
lr: 1.44105e-05
momentum: 1", "Arm 19_0
mean_accuracy: 0.966833 (SEM: None)
lr: 0.00177763
momentum: 0.104203", "Arm 1_0
mean_accuracy: 0.536167 (SEM: None)
lr: 1.50007e-06
momentum: 0.859993", "Arm 20_0
mean_accuracy: 0.958333 (SEM: None)
lr: 0.00153168
momentum: 0.36512", "Arm 21_0
mean_accuracy: 0.9545 (SEM: None)
lr: 0.00180488
momentum: 0", "Arm 22_0
mean_accuracy: 0.110667 (SEM: None)
lr: 0.00516279
momentum: 0.102548", "Arm 23_0
mean_accuracy: 0.116667 (SEM: None)
lr: 0.010201
momentum: 0", "Arm 24_0
mean_accuracy: 0.111833 (SEM: None)
lr: 0.00942208
momentum: 0.135133", "Arm 25_0
mean_accuracy: 0.888333 (SEM: None)
lr: 0.000135261
momentum: 0.311582", "Arm 26_0
mean_accuracy: 0.951667 (SEM: None)
lr: 0.000847533
momentum: 0.160155", "Arm 2_0
mean_accuracy: 0.7985 (SEM: None)
lr: 2.65622e-05
momentum: 0.247377", "Arm 3_0
mean_accuracy: 0.931167 (SEM: None)
lr: 5.61769e-05
momentum: 0.818812", "Arm 4_0
mean_accuracy: 0.0931667 (SEM: None)
lr: 0.353442
momentum: 0.0550975", "Arm 5_0
mean_accuracy: 0.965833 (SEM: None)
lr: 0.00133006
momentum: 0.441355", "Arm 6_0
mean_accuracy: 0.102667 (SEM: None)
lr: 0.000429886
momentum: 1", "Arm 7_0
mean_accuracy: 0.924833 (SEM: None)
lr: 0.000168348
momentum: 0.632889", "Arm 8_0
mean_accuracy: 0.1105 (SEM: None)
lr: 0.00224961
momentum: 0.83229", "Arm 9_0
mean_accuracy: 0.204333 (SEM: None)
lr: 1e-06
momentum: 0.493107" ], "type": "scatter", "x": [ 3.770478314941209e-05, 0.00043216540546621934, 2.893928031624302e-05, 0.0006311630696854857, 0.00028376216822593883, 0.0028255895325504246, 0.00011558254490459247, 0.003499350987498055, 0.0012753591554555425, 1.4410478955993728e-05, 0.0017776312510395422, 1.5000705860135848e-06, 0.0015316810490992995, 0.0018048791350339288, 0.005162793679334533, 0.010200974476274442, 0.00942208485482061, 0.0001352613243432586, 0.0008475328951433113, 2.6562208898820995e-05, 5.617694542454495e-05, 0.35344232890633676, 0.0013300560007633963, 0.0004298857159271888, 0.0001683479355878444, 0.002249609867271813, 1e-06 ], "xaxis": "x2", "y": [ 0.4462248980998993, 0.2744658616618241, 0.687404610446525, 9.347126188592805e-05, 0.43319045420051433, 0.2811155816297643, 0.0, 0.0, 0.27921473636842903, 1.0, 0.10420324217137585, 0.8599934121593833, 0.3651204901643246, 0.0, 0.10254845447365886, 0.0, 0.13513306632554634, 0.3115821445870835, 0.16015488542720355, 0.24737721495330334, 0.8188121551647782, 0.05509748309850693, 0.4413554677802919, 1.0, 0.6328890805509166, 0.8322899366209464, 0.49310670658150574 ], "yaxis": "y2" } ], "layout": { "annotations": [ { "font": { "size": 14 }, "showarrow": false, "text": "Mean", "x": 0.25, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": { "size": 14 }, "showarrow": false, "text": "Standard Error", "x": 0.8, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" } ], "autosize": false, "height": 450, "hovermode": "closest", "legend": { "orientation": "h", "x": 0, "y": -0.25 }, "margin": { "b": 100, "l": 35, "pad": 0, "r": 35, "t": 35 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "sequentialminus": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "width": 950, "xaxis": { "anchor": "y", "autorange": false, "domain": [ 0.05, 0.45 ], "exponentformat": "e", "range": [ -6.0, -0.3979400086720376 ], "tickfont": { "size": 11 }, "tickmode": "auto", "title": { "text": "lr" }, "type": "log" }, "xaxis2": { "anchor": "y2", "autorange": false, "domain": [ 0.6, 1 ], "exponentformat": "e", "range": [ -6.0, -0.3979400086720376 ], "tickfont": { "size": 11 }, "tickmode": "auto", "title": { "text": "lr" }, "type": "log" }, "yaxis": { "anchor": "x", "autorange": false, "domain": [ 0, 1 ], "exponentformat": "e", "range": [ 0.0, 1.0 ], "tickfont": { "size": 11 }, "tickmode": "auto", "title": { "text": "momentum" }, "type": "linear" }, "yaxis2": { "anchor": "x2", "autorange": false, "domain": [ 0, 1 ], "exponentformat": "e", "range": [ 0.0, 1.0 ], "tickfont": { "size": 11 }, "tickmode": "auto", "type": "linear" } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "render(\n", " plot_contour(\n", " model=ax.generation_strategy.model,\n", " param_x=\"lr\",\n", " param_y=\"momentum\",\n", " metric_name=\"mean_accuracy\",\n", " )\n", ")" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "execution": { "iopub.execute_input": "2022-08-17T19:30:47.077897Z", "iopub.status.busy": "2022-08-17T19:30:47.077626Z", "iopub.status.idle": "2022-08-17T19:30:47.144169Z", "shell.execute_reply": "2022-08-17T19:30:47.143423Z" }, "originalKey": "6dfd23ca-1c93-4846-8e85-4560f9e40304" }, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "linkText": "Export to plot.ly", "plotlyServerURL": "https://plot.ly", "showLink": false }, "data": [ { "hoverinfo": "none", "legendgroup": "", "line": { "width": 0 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30 ], "y": [ 87.3, 87.3, 87.3, 93.11666666666667, 93.11666666666667, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333 ] }, { "fill": "tonexty", "fillcolor": "rgba(128,177,211,0.3)", "legendgroup": "objective value", "line": { "color": "rgba(128,177,211,1)" }, "mode": "lines", "name": "objective value", "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30 ], "y": [ 87.3, 87.3, 87.3, 93.11666666666667, 93.11666666666667, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333 ] }, { "fill": "tonexty", "fillcolor": "rgba(128,177,211,0.3)", "hoverinfo": "none", "legendgroup": "", "line": { "width": 0 }, "mode": "lines", "showlegend": false, "type": "scatter", "x": [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30 ], "y": [ 87.3, 87.3, 87.3, 93.11666666666667, 93.11666666666667, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 96.58333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333, 97.38333333333333 ] } ], "layout": { "showlegend": true, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ], "sequentialminus": [ [ 0.0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1.0, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "title": { "text": "Model performance vs. # of iterations" }, "xaxis": { "title": { "text": "Iteration" } }, "yaxis": { "title": { "text": "Accuracy" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# `plot_single_method` expects a 2-d array of means, because it expects to average means from multiple\n", "# optimization runs, so we wrap out best objectives array in another array.\n", "best_objectives = np.array(\n", " [[trial.objective_mean * 100 for trial in ax.experiment.trials.values()]]\n", ")\n", "best_objective_plot = optimization_trace_single_method(\n", " y=np.maximum.accumulate(best_objectives, axis=1),\n", " title=\"Model performance vs. # of iterations\",\n", " ylabel=\"Accuracy\",\n", ")\n", "render(best_objective_plot)" ] } ], "metadata": { "kernelspec": { "display_name": "python3", "language": "python", "metadata": { "fbpkg_supported": true, "is_prebuilt": true, "kernel_name": "bento_kernel_default", "nightly_builds": true }, "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.13" }, "last_base_url": "https://devvm3002.frc0.facebook.com:8090/", "last_kernel_id": "148e0817-7aae-4719-aff2-639ce2864738", "last_msg_id": "a1616ce6-ae3c4879620e9fb8029ca89e_240", "last_server_session_id": "3539ac9b-8cce-42fe-984a-f3a2a8a1dbde", "outputWidgetContext": {}, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": { "0137cda85ae44d10aeaeea0863da9267": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0200140be03941619fe2a4b3eb0b3dcb": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0aa753e977c744bdbdba274421c97187": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_2f58bcd0044d4705851b96408035715b", "IPY_MODEL_24b446ee195542e4877873b92374ac78", "IPY_MODEL_0bc3bc796b494d6b97541439343cd049" ], "layout": "IPY_MODEL_341df3f261c34f6788e724a047b5077a" } }, "0adf92707a70448c9f9e214d42e6894e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_3e8e9cea5f734c519b1dcead56368302", "IPY_MODEL_90cb532b39984f9a81ea96fd22a02bc1", "IPY_MODEL_dcfeda74ff104a63b3b303871ad8aa5e" ], "layout": "IPY_MODEL_141a41da71d149a1979518b1bd7dd040" } }, "0bc3bc796b494d6b97541439343cd049": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_88f8ccaa39964477b53d4ca93dd5230a", "placeholder": "​", "style": "IPY_MODEL_bfb8ebe41d5d468aa2674812ee6164a3", "value": " 1648877/1648877 [00:00<00:00, 14433151.99it/s]" } }, "0ca5551e857644dbb83a1f85c4206fff": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "141a41da71d149a1979518b1bd7dd040": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "1c2edbadbca540a28657f29540b9f97b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "24b446ee195542e4877873b92374ac78": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_97638502a58e46498fcd96f38b70db2f", "max": 1648877.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_4b3b0ab02ed54b29a5735a163a4d0e03", "value": 1648877.0 } }, "2f58bcd0044d4705851b96408035715b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a28e5c3ca1024d6ca80ad718bfbe123a", "placeholder": "​", "style": "IPY_MODEL_35d688e964e340a2a48ef121d2675867", "value": "100%" } }, "33c8493819d24d5a89c646e5478d526f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "341df3f261c34f6788e724a047b5077a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "35d688e964e340a2a48ef121d2675867": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "3a58a420629b4593baeedd3901c2b682": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_d67ece3f6ef54359bb2ba227bf3574cd", "IPY_MODEL_cc36030f52944175a75b9fe77d4b8932", "IPY_MODEL_467ad5c90ec84b14ac9431d143555dec" ], "layout": "IPY_MODEL_0ca5551e857644dbb83a1f85c4206fff" } }, "3e8e9cea5f734c519b1dcead56368302": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_bbbc36e2cab24e59b308b67bdd63d411", "placeholder": "​", "style": "IPY_MODEL_5a8ada2c006547faa4edc1c005194d88", "value": "100%" } }, "43b2596d4d814c0aae266a3575e9da14": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "467ad5c90ec84b14ac9431d143555dec": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e7246ebc80634da0b324f7f502392add", "placeholder": "​", "style": "IPY_MODEL_b5e3071dd5dc4828a389ea1c8f5d775a", "value": " 4542/4542 [00:00<00:00, 194768.78it/s]" } }, "488453da3909464e9fd7d2eec8064792": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_f55c3a6529a34a63b88b779c9fb301db", "placeholder": "​", "style": "IPY_MODEL_e5600876e1b94c07aef774e9d2bcd51f", "value": "100%" } }, "4b3b0ab02ed54b29a5735a163a4d0e03": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "5a8ada2c006547faa4edc1c005194d88": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "5fd474388e144cfd9318ce1cf9fa4208": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0137cda85ae44d10aeaeea0863da9267", "placeholder": "​", "style": "IPY_MODEL_bb88b2990875453ca4b4d67a77471247", "value": " 9912422/9912422 [00:00<00:00, 12436403.85it/s]" } }, "669b9b0387d842aa91e2996599ce84f4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "757ac4396f8b4f61a874e5948f74985c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a755d702b68d49d8a27eadf8522479e5", "max": 9912422.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_981e1f6671834834abac9bc336d37d72", "value": 9912422.0 } }, "88f8ccaa39964477b53d4ca93dd5230a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "90cb532b39984f9a81ea96fd22a02bc1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_1c2edbadbca540a28657f29540b9f97b", "max": 28881.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_e2255ebdaee14fdda01c5b995b01e2fb", "value": 28881.0 } }, "97638502a58e46498fcd96f38b70db2f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "981e1f6671834834abac9bc336d37d72": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "a28e5c3ca1024d6ca80ad718bfbe123a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a2de93922a094246b21ba8a76371dfd7": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a755d702b68d49d8a27eadf8522479e5": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a7a7a62292064b64b60044715671a9bc": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b5e3071dd5dc4828a389ea1c8f5d775a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "bb88b2990875453ca4b4d67a77471247": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "bbbc36e2cab24e59b308b67bdd63d411": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "bfb8ebe41d5d468aa2674812ee6164a3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "c108f244a15c4745b921dda4fa965b3a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_488453da3909464e9fd7d2eec8064792", "IPY_MODEL_757ac4396f8b4f61a874e5948f74985c", "IPY_MODEL_5fd474388e144cfd9318ce1cf9fa4208" ], "layout": "IPY_MODEL_a2de93922a094246b21ba8a76371dfd7" } }, "cc36030f52944175a75b9fe77d4b8932": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0200140be03941619fe2a4b3eb0b3dcb", "max": 4542.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_e3ab192376694974bbb71c45933677d6", "value": 4542.0 } }, "d67ece3f6ef54359bb2ba227bf3574cd": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a7a7a62292064b64b60044715671a9bc", "placeholder": "​", "style": "IPY_MODEL_43b2596d4d814c0aae266a3575e9da14", "value": "100%" } }, "dcfeda74ff104a63b3b303871ad8aa5e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_669b9b0387d842aa91e2996599ce84f4", "placeholder": "​", "style": "IPY_MODEL_33c8493819d24d5a89c646e5478d526f", "value": " 28881/28881 [00:00<00:00, 1243463.16it/s]" } }, "e2255ebdaee14fdda01c5b995b01e2fb": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "e3ab192376694974bbb71c45933677d6": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "e5600876e1b94c07aef774e9d2bcd51f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "e7246ebc80634da0b324f7f502392add": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f55c3a6529a34a63b88b779c9fb301db": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } } }, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 2 }