This tutorial illustrates how to add a trial-level early stopping strategy to an Ax hyper-parameter optimization (HPO) loop. The goal of trial-level early stopping is to monitor the results of expensive evaluations and terminate those that are unlikely to produce promising results, freeing up resources to explore more configurations.
Most of this tutorial is adapted from the PyTorch Ax Multiobjective NAS Tutorial. The training job is different from the original in that we do not optimize batch_size or epochs. This was done for illustrative purposes, as each validation curve now has the same number of points. The companion training file mnist_train_nas.py has also been altered to log to Tensorboard during training.
NOTE: Although the original NAS tutorial is for a multi-objective problem, this tutorial focuses on a single objective (validation accuracy) problem. Early stopping currently does not support "true" multi-objective stopping, although one can use logical compositions of early stopping strategies to target multiple objectives separately. Early stopping for the multi-objective case is currently a work in progress.
import os
import tempfile
from pathlib import Path
import torchx
from ax.core import Experiment, Objective, ParameterType, RangeParameter, SearchSpace
from ax.core.optimization_config import OptimizationConfig
from ax.early_stopping.strategies import PercentileEarlyStoppingStrategy
from ax.metrics.tensorboard import TensorboardCurveMetric
from ax.modelbridge.dispatch_utils import choose_generation_strategy
from ax.runners.torchx import TorchXRunner
from ax.service.scheduler import Scheduler, SchedulerOptions
from ax.service.utils.report_utils import exp_to_df
from torchx import specs
from torchx.components import utils
from matplotlib import pyplot as plt
%matplotlib inline
SMOKE_TEST = os.environ.get("SMOKE_TEST")
Our goal is to optimize the PyTorch Lightning training job defined in mnist_train_nas.py. To do this using TorchX, we write a helper function that takes in the values of the architcture and hyperparameters of the training job and creates a TorchX AppDef with the appropriate settings.
if SMOKE_TEST:
epochs = 3
else:
epochs = 10
def trainer(
log_path: str,
hidden_size_1: int,
hidden_size_2: int,
learning_rate: float,
dropout: float,
trial_idx: int = -1,
) -> specs.AppDef:
# define the log path so we can pass it to the TorchX AppDef
if trial_idx >= 0:
log_path = Path(log_path).joinpath(str(trial_idx)).absolute().as_posix()
batch_size = 32
return utils.python(
# command line args to the training script
"--log_path",
log_path,
"--hidden_size_1",
str(hidden_size_1),
"--hidden_size_2",
str(hidden_size_2),
"--learning_rate",
str(learning_rate),
"--epochs",
str(epochs),
"--dropout",
str(dropout),
"--batch_size",
str(batch_size),
# other config options
name="trainer",
script="tutorials/early_stopping/mnist_train_nas.py",
image=torchx.version.TORCHX_IMAGE,
)
Ax’s Runner
abstraction allows writing interfaces to various backends.
Ax already comes with Runner for TorchX, so we just need to
configure it. For the purpose of this tutorial, we run jobs locally
in a fully asynchronous fashion. In order to launch them on a cluster, you can instead specify a
different TorchX scheduler and adjust the configuration appropriately.
For example, if you have a Kubernetes cluster, you just need to change the
scheduler from local_cwd to kubernetes.
The training job launched by this runner will log partial results to Tensorboard, which will then be monitored by the early stopping strategy. We will show how this is done using an Ax TensorboardCurveMetric below.
# Make a temporary dir to log our results into
log_dir = tempfile.mkdtemp()
ax_runner = TorchXRunner(
tracker_base="/tmp/",
component=trainer,
# NOTE: To launch this job on a cluster instead of locally you can
# specify a different scheduler and adjust args appropriately.
scheduler="local_cwd",
component_const_params={"log_path": log_dir},
cfg={},
)
First, we define our search space. Ax supports both range parameters of type integer and float as well as choice parameters which can have non-numerical types such as strings. We will tune the hidden sizes, learning rate, and dropout parameters.
parameters = [
# NOTE: In a real-world setting, hidden_size_1 and hidden_size_2
# should probably be powers of 2, but in our simple example this
# would mean that num_params can't take on that many values, which
# in turn makes the Pareto frontier look pretty weird.
RangeParameter(
name="hidden_size_1",
lower=16,
upper=128,
parameter_type=ParameterType.INT,
log_scale=True,
),
RangeParameter(
name="hidden_size_2",
lower=16,
upper=128,
parameter_type=ParameterType.INT,
log_scale=True,
),
RangeParameter(
name="learning_rate",
lower=1e-4,
upper=1e-2,
parameter_type=ParameterType.FLOAT,
log_scale=True,
),
RangeParameter(
name="dropout",
lower=0.0,
upper=0.5,
parameter_type=ParameterType.FLOAT,
),
]
search_space = SearchSpace(
parameters=parameters,
# NOTE: In practice, it may make sense to add a constraint
# hidden_size_2 <= hidden_size_1
parameter_constraints=[],
)
Ax has the concept of a Metric that defines properties of outcomes and how observations are obtained for these outcomes. This allows e.g. encodig how data is fetched from some distributed execution backend and post-processed before being passed as input to Ax.
We will optimize the validation accuracy, which is a TensorboardCurveMetric that points to the logging directory assigned above. Note that we have set is_available_while_running, allowing for the metric to be queried as the trial progresses. This is critical for the early stopping strategy to monitor partial results.
class MyTensorboardMetric(TensorboardCurveMetric):
# NOTE: We need to tell the new Tensorboard metric how to get the id /
# file handle for the tensorboard logs from a trial. In this case
# our convention is to just save a separate file per trial in
# the pre-specified log dir.
@classmethod
def get_ids_from_trials(cls, trials):
return {
trial.index: Path(log_dir).joinpath(str(trial.index)).as_posix()
for trial in trials
}
# This indicates whether the metric is queryable while the trial is
# still running. This is required for early stopping to monitor the
# progress of the running trial.ArithmeticError
@classmethod
def is_available_while_running(cls):
return True
val_acc = MyTensorboardMetric(
name="val_acc",
curve_name="val_acc",
lower_is_better=False,
)
The OptimizationConfig specifies the objective for Ax to optimize.
opt_config = OptimizationConfig(
objective=Objective(
metric=val_acc,
minimize=False,
)
)
A PercentileEarlyStoppingStrategy is a simple method that stops a trial if its performance falls below a certain percentile of other trials at the same step (e.g., when percentile_threshold is 50, at a given point in time, if a trial ranks in the bottom 50% of trials, it is stopped).
normalize_progressions which normalizes the progression column (e.g. timestamp, epochs, training data used) to be in [0, 1]. This is useful because one doesn't need to know the maximum progression values of the curve (which might be, e.g., the total number of data points in the training dataset).min_progression parameter specifies that trials should only be considered for stopping if the latest progression value is greater than this threshold.min_curves parameter specifies the minimum number of completed curves (i.e., fully completed training jobs) before early stopping will be considered. This should be larger than zero if normalize_progression is used. In general, we want a few completed curves to have a baseline for comparison.Note that PercentileEarlyStoppingStrategy does not make use of learning curve modeling or prediction. More sophisticated model-based methods will be available in future versions of Ax.
percentile_early_stopping_strategy = PercentileEarlyStoppingStrategy(
# stop if in bottom 70% of runs at the same progression
percentile_threshold=70,
# the trial must have passed `min_progression` steps before early stopping is initiated
# note that we are using `normalize_progressions`, so this is on a scale of [0, 1]
min_progression=0.3,
# there must be `min_curves` completed trials and `min_curves` trials reporting data in
# order for early stopping to be applicable
min_curves=5,
# specify, e.g., [0, 1] if the first two trials should never be stopped
trial_indices_to_ignore=None,
# check for new data every 10 seconds
seconds_between_polls=10,
normalize_progressions=True,
)
In Ax, the Experiment object is the object that stores all the information about the problem setup.
experiment = Experiment(
name="torchx_mnist",
search_space=search_space,
optimization_config=opt_config,
runner=ax_runner,
)
A GenerationStrategy is the abstract representation of how we would like to perform the optimization. While this can be customized (if you’d like to do so, see this tutorial), in most cases Ax can automatically determine an appropriate strategy based on the search space, optimization config, and the total number of trials we want to run.
Typically, Ax chooses to evaluate a number of random configurations before starting a model-based Bayesian Optimization strategy.
We remark that in Ax, generation strategies and early stopping strategies are separate, a design decision motivated by ease-of-use. However, we should acknowledge that jointly considering generation and stopping using a single strategy would likely be the "proper" formulation.
if SMOKE_TEST:
total_trials = 6
else:
total_trials = 15 # total evaluation budget
gs = choose_generation_strategy(
search_space=experiment.search_space,
optimization_config=experiment.optimization_config,
num_trials=total_trials,
)
[INFO 05-02 23:43:38] ax.modelbridge.dispatch_utils: Using Models.BOTORCH_MODULAR since there is at least one ordered parameter and there are no unordered categorical parameters.
[INFO 05-02 23:43:38] ax.modelbridge.dispatch_utils: Calculating the number of remaining initialization trials based on num_initialization_trials=None max_initialization_trials=None num_tunable_parameters=4 num_trials=15 use_batch_trials=False
[INFO 05-02 23:43:38] ax.modelbridge.dispatch_utils: calculated num_initialization_trials=5
[INFO 05-02 23:43:38] ax.modelbridge.dispatch_utils: num_completed_initialization_trials=0 num_remaining_initialization_trials=5
[INFO 05-02 23:43:38] ax.modelbridge.dispatch_utils: `verbose`, `disable_progbar`, and `jit_compile` are not yet supported when using `choose_generation_strategy` with ModularBoTorchModel, dropping these arguments.
[INFO 05-02 23:43:38] ax.modelbridge.dispatch_utils: Using Bayesian Optimization generation strategy: GenerationStrategy(name='Sobol+BoTorch', steps=[Sobol for 5 trials, BoTorch for subsequent trials]). Iterations after 5 will take longer to generate due to model-fitting.
The Scheduler acts as the loop control for the optimization.
It communicates with the backend to launch trials, check their status, retrieve (partial) results, and importantly for this tutorial, calls the early stopping strategy. If the early stopping strategy suggests a trial to be the stopped, the Scheduler communicates with the backend to terminate the trial.
The Scheduler requires the Experiment and the GenerationStrategy.
A set of options can be passed in via SchedulerOptions. Here, we
configure the number of total evaluations as well as max_pending_trials,
the maximum number of trials that should run concurrently. In our
local setting, this is the number of training jobs running as individual
processes, while in a remote execution setting, this would be the number
of machines you want to use in parallel.
scheduler = Scheduler(
experiment=experiment,
generation_strategy=gs,
options=SchedulerOptions(
total_trials=total_trials,
max_pending_trials=5,
early_stopping_strategy=percentile_early_stopping_strategy,
),
)
[INFO 05-02 23:43:38] Scheduler: `Scheduler` requires experiment to have immutable search space and optimization config. Setting property immutable_search_space_and_opt_config to `True` on experiment.
%%time
scheduler.run_all_trials()
[INFO 05-02 23:43:38] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:43:38] ax.early_stopping.strategies.base: PercentileEarlyStoppingStrategy received empty data. Not stopping any trials.
[INFO 05-02 23:43:38] Scheduler: Running trials [0]...
[INFO 05-02 23:43:39] Scheduler: Running trials [1]...
[INFO 05-02 23:43:40] Scheduler: Running trials [2]...
[INFO 05-02 23:43:42] Scheduler: Running trials [3]...
[INFO 05-02 23:43:42] Scheduler: Running trials [4]...
[WARNING 05-02 23:43:43] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-02 23:43:43] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:43:43] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
[INFO 05-02 23:43:43] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/0. Returning without this metric.
[INFO 05-02 23:43:43] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/1. Returning without this metric.
[INFO 05-02 23:43:43] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/2. Returning without this metric.
[INFO 05-02 23:43:43] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/3. Returning without this metric.
[INFO 05-02 23:43:43] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/4. Returning without this metric.
[INFO 05-02 23:43:43] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:43:43] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:43:43] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:43:43] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:43:43] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-02 23:43:43] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:43:43] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:43:43] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:43:43] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:43:43] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-02 23:43:43] Scheduler: Failed to fetch val_acc for trial 0, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:43:43] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 0 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:43:43] Scheduler: Failed to fetch val_acc for trial 1, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:43:43] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 1 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:43:43] Scheduler: Failed to fetch val_acc for trial 2, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:43:43] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 2 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:43:43] Scheduler: Failed to fetch val_acc for trial 3, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:43:43] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 3 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:43:43] Scheduler: Failed to fetch val_acc for trial 4, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:43:43] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 4 is still RUNNING continuing the experiment and retrying on next poll...
[INFO 05-02 23:43:43] ax.early_stopping.strategies.base: PercentileEarlyStoppingStrategy received empty data. Not stopping any trials.
[INFO 05-02 23:43:43] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:43:53] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:43:53] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
[INFO 05-02 23:43:53] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/0. Returning without this metric.
[INFO 05-02 23:43:53] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/1. Returning without this metric.
[INFO 05-02 23:43:53] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/2. Returning without this metric.
[INFO 05-02 23:43:53] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/3. Returning without this metric.
[INFO 05-02 23:43:53] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/4. Returning without this metric.
[INFO 05-02 23:43:53] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:43:53] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:43:53] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:43:53] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:43:53] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-02 23:43:53] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:43:53] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:43:53] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:43:53] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:43:53] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-02 23:43:53] Scheduler: Failed to fetch val_acc for trial 0, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:43:53] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 0 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:43:53] Scheduler: Failed to fetch val_acc for trial 1, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:43:53] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 1 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:43:53] Scheduler: Failed to fetch val_acc for trial 2, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:43:53] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 2 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:43:53] Scheduler: Failed to fetch val_acc for trial 3, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:43:53] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 3 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:43:53] Scheduler: Failed to fetch val_acc for trial 4, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:43:53] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 4 is still RUNNING continuing the experiment and retrying on next poll...
[INFO 05-02 23:43:53] ax.early_stopping.strategies.base: PercentileEarlyStoppingStrategy received empty data. Not stopping any trials.
[INFO 05-02 23:43:53] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:44:03] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:44:03] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
[INFO 05-02 23:44:03] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/0. Returning without this metric.
[INFO 05-02 23:44:03] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/1. Returning without this metric.
[INFO 05-02 23:44:03] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/2. Returning without this metric.
[INFO 05-02 23:44:03] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/3. Returning without this metric.
[INFO 05-02 23:44:03] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/4. Returning without this metric.
[INFO 05-02 23:44:03] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:03] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:03] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:03] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:03] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-02 23:44:03] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:03] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:03] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:03] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:03] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-02 23:44:03] Scheduler: Failed to fetch val_acc for trial 0, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:03] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 0 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:03] Scheduler: Failed to fetch val_acc for trial 1, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:03] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 1 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:03] Scheduler: Failed to fetch val_acc for trial 2, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:03] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 2 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:03] Scheduler: Failed to fetch val_acc for trial 3, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:03] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 3 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:03] Scheduler: Failed to fetch val_acc for trial 4, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:03] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 4 is still RUNNING continuing the experiment and retrying on next poll...
[INFO 05-02 23:44:03] ax.early_stopping.strategies.base: PercentileEarlyStoppingStrategy received empty data. Not stopping any trials.
[INFO 05-02 23:44:03] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:44:13] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:44:13] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
[INFO 05-02 23:44:13] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/0. Returning without this metric.
[INFO 05-02 23:44:13] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/1. Returning without this metric.
[INFO 05-02 23:44:13] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/2. Returning without this metric.
[INFO 05-02 23:44:13] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/3. Returning without this metric.
[INFO 05-02 23:44:13] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/4. Returning without this metric.
[INFO 05-02 23:44:13] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:13] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:13] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:13] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:13] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-02 23:44:13] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:13] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:13] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:13] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:13] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-02 23:44:13] Scheduler: Failed to fetch val_acc for trial 0, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:13] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 0 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:13] Scheduler: Failed to fetch val_acc for trial 1, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:13] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 1 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:13] Scheduler: Failed to fetch val_acc for trial 2, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:13] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 2 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:13] Scheduler: Failed to fetch val_acc for trial 3, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:13] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 3 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:13] Scheduler: Failed to fetch val_acc for trial 4, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:13] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 4 is still RUNNING continuing the experiment and retrying on next poll...
[INFO 05-02 23:44:13] ax.early_stopping.strategies.base: PercentileEarlyStoppingStrategy received empty data. Not stopping any trials.
[INFO 05-02 23:44:13] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:44:23] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:44:23] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
[INFO 05-02 23:44:23] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/0. Returning without this metric.
[INFO 05-02 23:44:23] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/1. Returning without this metric.
[INFO 05-02 23:44:23] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/2. Returning without this metric.
[INFO 05-02 23:44:23] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/3. Returning without this metric.
[INFO 05-02 23:44:23] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/4. Returning without this metric.
[INFO 05-02 23:44:23] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:23] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:23] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:23] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:23] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-02 23:44:23] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:23] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:23] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:23] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:23] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-02 23:44:23] Scheduler: Failed to fetch val_acc for trial 0, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:23] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 0 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:23] Scheduler: Failed to fetch val_acc for trial 1, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:23] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 1 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:23] Scheduler: Failed to fetch val_acc for trial 2, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:23] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 2 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:23] Scheduler: Failed to fetch val_acc for trial 3, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:23] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 3 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:23] Scheduler: Failed to fetch val_acc for trial 4, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:23] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 4 is still RUNNING continuing the experiment and retrying on next poll...
[INFO 05-02 23:44:23] ax.early_stopping.strategies.base: PercentileEarlyStoppingStrategy received empty data. Not stopping any trials.
[INFO 05-02 23:44:23] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:44:33] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:44:33] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
[INFO 05-02 23:44:33] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/0. Returning without this metric.
[INFO 05-02 23:44:33] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/1. Returning without this metric.
[INFO 05-02 23:44:33] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/2. Returning without this metric.
[INFO 05-02 23:44:33] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/3. Returning without this metric.
[INFO 05-02 23:44:33] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/4. Returning without this metric.
[INFO 05-02 23:44:33] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:33] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:33] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:33] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:33] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-02 23:44:33] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:33] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:33] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:33] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:33] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-02 23:44:33] Scheduler: Failed to fetch val_acc for trial 0, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:33] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 0 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:33] Scheduler: Failed to fetch val_acc for trial 1, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:33] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 1 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:33] Scheduler: Failed to fetch val_acc for trial 2, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:33] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 2 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:33] Scheduler: Failed to fetch val_acc for trial 3, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:33] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 3 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:33] Scheduler: Failed to fetch val_acc for trial 4, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:33] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 4 is still RUNNING continuing the experiment and retrying on next poll...
[INFO 05-02 23:44:33] ax.early_stopping.strategies.base: PercentileEarlyStoppingStrategy received empty data. Not stopping any trials.
[INFO 05-02 23:44:33] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:44:43] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:44:43] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
[INFO 05-02 23:44:43] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/0. Returning without this metric.
[INFO 05-02 23:44:43] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/1. Returning without this metric.
[INFO 05-02 23:44:43] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/2. Returning without this metric.
[INFO 05-02 23:44:43] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/3. Returning without this metric.
[INFO 05-02 23:44:43] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/4. Returning without this metric.
[INFO 05-02 23:44:43] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:43] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:43] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:43] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:43] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-02 23:44:43] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:43] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:43] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:43] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:43] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-02 23:44:43] Scheduler: Failed to fetch val_acc for trial 0, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:43] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 0 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:43] Scheduler: Failed to fetch val_acc for trial 1, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:43] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 1 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:43] Scheduler: Failed to fetch val_acc for trial 2, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:43] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 2 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:43] Scheduler: Failed to fetch val_acc for trial 3, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:43] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 3 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:43] Scheduler: Failed to fetch val_acc for trial 4, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:43] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 4 is still RUNNING continuing the experiment and retrying on next poll...
[INFO 05-02 23:44:43] ax.early_stopping.strategies.base: PercentileEarlyStoppingStrategy received empty data. Not stopping any trials.
[INFO 05-02 23:44:43] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:44:53] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:44:53] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
[INFO 05-02 23:44:54] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/0. Returning without this metric.
[INFO 05-02 23:44:54] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/1. Returning without this metric.
[INFO 05-02 23:44:54] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/2. Returning without this metric.
[INFO 05-02 23:44:54] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/3. Returning without this metric.
[INFO 05-02 23:44:54] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/4. Returning without this metric.
[INFO 05-02 23:44:54] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:54] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:54] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:54] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:44:54] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-02 23:44:54] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:54] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:54] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:54] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:44:54] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-02 23:44:54] Scheduler: Failed to fetch val_acc for trial 0, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:54] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 0 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:54] Scheduler: Failed to fetch val_acc for trial 1, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:54] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 1 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:54] Scheduler: Failed to fetch val_acc for trial 2, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:54] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 2 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:54] Scheduler: Failed to fetch val_acc for trial 3, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:54] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 3 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:44:54] Scheduler: Failed to fetch val_acc for trial 4, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:44:54] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 4 is still RUNNING continuing the experiment and retrying on next poll...
[INFO 05-02 23:44:54] ax.early_stopping.strategies.base: PercentileEarlyStoppingStrategy received empty data. Not stopping any trials.
[INFO 05-02 23:44:54] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:45:04] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:45:04] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
[INFO 05-02 23:45:04] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/0. Returning without this metric.
[INFO 05-02 23:45:04] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/1. Returning without this metric.
[INFO 05-02 23:45:04] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/2. Returning without this metric.
[INFO 05-02 23:45:04] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/3. Returning without this metric.
[INFO 05-02 23:45:04] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/4. Returning without this metric.
[INFO 05-02 23:45:04] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:45:04] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:45:04] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:45:04] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:45:04] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-02 23:45:04] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:45:04] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:45:04] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:45:04] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:45:04] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-02 23:45:04] Scheduler: Failed to fetch val_acc for trial 0, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:04] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 0 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:45:04] Scheduler: Failed to fetch val_acc for trial 1, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:04] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 1 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:45:04] Scheduler: Failed to fetch val_acc for trial 2, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:04] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 2 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:45:04] Scheduler: Failed to fetch val_acc for trial 3, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:04] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 3 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:45:04] Scheduler: Failed to fetch val_acc for trial 4, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:04] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 4 is still RUNNING continuing the experiment and retrying on next poll...
[INFO 05-02 23:45:04] ax.early_stopping.strategies.base: PercentileEarlyStoppingStrategy received empty data. Not stopping any trials.
[INFO 05-02 23:45:04] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:45:14] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:45:14] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
[INFO 05-02 23:45:14] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/0. Returning without this metric.
[INFO 05-02 23:45:14] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/1. Returning without this metric.
[INFO 05-02 23:45:14] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/2. Returning without this metric.
[INFO 05-02 23:45:14] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/3. Returning without this metric.
[INFO 05-02 23:45:14] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/4. Returning without this metric.
[INFO 05-02 23:45:14] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:45:14] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:45:14] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:45:14] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:45:14] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-02 23:45:14] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:45:14] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:45:14] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:45:14] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:45:14] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-02 23:45:14] Scheduler: Failed to fetch val_acc for trial 0, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:14] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 0 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:45:14] Scheduler: Failed to fetch val_acc for trial 1, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:14] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 1 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:45:14] Scheduler: Failed to fetch val_acc for trial 2, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:14] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 2 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:45:14] Scheduler: Failed to fetch val_acc for trial 3, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:14] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 3 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:45:14] Scheduler: Failed to fetch val_acc for trial 4, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:14] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 4 is still RUNNING continuing the experiment and retrying on next poll...
[INFO 05-02 23:45:14] ax.early_stopping.strategies.base: PercentileEarlyStoppingStrategy received empty data. Not stopping any trials.
[INFO 05-02 23:45:14] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:45:24] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:45:24] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
[INFO 05-02 23:45:24] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/0. Returning without this metric.
[INFO 05-02 23:45:24] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/1. Returning without this metric.
[INFO 05-02 23:45:24] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/2. Returning without this metric.
[INFO 05-02 23:45:24] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/3. Returning without this metric.
[INFO 05-02 23:45:24] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/4. Returning without this metric.
[INFO 05-02 23:45:24] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:45:24] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:45:24] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:45:24] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-02 23:45:24] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-02 23:45:24] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:45:24] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:45:24] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:45:24] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-02 23:45:24] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-02 23:45:24] Scheduler: Failed to fetch val_acc for trial 0, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:24] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 0 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:45:24] Scheduler: Failed to fetch val_acc for trial 1, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:24] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 1 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:45:24] Scheduler: Failed to fetch val_acc for trial 2, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:24] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 2 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:45:24] Scheduler: Failed to fetch val_acc for trial 3, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:24] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 3 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-02 23:45:24] Scheduler: Failed to fetch val_acc for trial 4, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-02 23:45:24] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 4 is still RUNNING continuing the experiment and retrying on next poll...
[INFO 05-02 23:45:24] ax.early_stopping.strategies.base: PercentileEarlyStoppingStrategy received empty data. Not stopping any trials.
[INFO 05-02 23:45:24] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:45:34] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:45:34] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
[INFO 05-02 23:45:34] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/1. Returning without this metric.
[INFO 05-02 23:45:34] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/3. Returning without this metric.
[INFO 05-02 23:45:34] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/4. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
[INFO 05-02 23:45:34] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:45:34] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:45:44] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:45:44] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:45:44] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:45:44] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:45:54] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:45:54] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:45:55] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:45:55] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:46:05] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:46:05] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:46:05] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:46:05] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:46:15] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:46:15] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:46:15] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:46:15] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:46:25] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:46:25] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:46:26] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:46:26] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:46:36] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:46:36] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:46:36] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:46:36] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:46:46] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:46:46] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:46:46] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:46:46] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:46:56] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:46:56] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:46:57] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:46:57] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:47:07] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:47:07] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:47:07] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:47:07] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:47:17] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:47:17] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:47:17] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:47:17] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:47:27] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:47:27] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:47:28] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:47:28] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:47:38] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:47:38] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:47:38] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:47:38] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:47:48] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:47:48] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:47:48] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:47:48] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:47:58] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:47:58] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:47:58] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:47:58] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:48:08] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:48:08] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:48:09] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:48:09] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:48:19] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:48:19] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:48:19] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:48:19] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:48:29] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:48:29] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:48:29] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:48:29] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:48:39] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:48:39] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:48:39] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:48:39] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:48:49] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:48:49] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:48:50] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:48:50] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:49:00] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:49:00] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:49:00] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:49:00] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:49:10] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:49:10] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:49:10] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:49:10] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:49:20] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:49:20] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:49:21] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:49:21] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:49:31] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:49:31] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:49:31] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:49:31] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:49:41] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:49:41] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:49:41] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:49:41] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:49:51] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:49:51] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:49:51] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:49:51] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:50:01] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:50:01] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:50:02] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:50:02] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:50:12] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:50:12] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:50:12] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:50:12] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:50:22] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:50:22] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:50:22] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:50:22] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:50:32] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:50:32] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:50:33] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:50:33] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:50:43] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:50:43] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:50:43] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:50:43] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:50:53] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:50:53] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:50:53] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:50:53] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:51:03] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:51:03] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:51:03] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:51:04] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:51:14] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:51:14] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:51:14] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:51:14] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:51:24] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:51:24] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:51:24] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:51:24] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:51:34] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:51:34] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:51:34] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:51:34] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:51:44] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:51:44] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:51:45] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:51:45] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:51:55] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:51:55] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:51:55] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:51:55] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:52:05] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:52:05] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:52:05] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:52:05] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:52:15] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:52:15] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:52:15] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:52:15] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:52:26] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:52:26] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:52:26] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:52:26] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:52:36] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:52:36] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:52:36] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:52:36] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:52:46] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:52:46] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:52:46] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:52:46] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:52:56] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:52:56] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:52:57] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:52:57] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:53:07] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:53:07] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:53:07] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:53:07] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:53:17] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:53:17] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:53:17] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:53:17] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:53:27] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:53:27] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:53:28] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:53:28] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:53:38] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:53:38] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:53:38] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:53:38] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:53:48] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:53:48] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:53:48] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:53:48] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:53:58] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:53:58] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:53:58] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:53:58] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:54:08] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:54:08] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
[INFO 05-02 23:54:09] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:54:09] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:54:19] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:54:19] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:54:19] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:54:19] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:54:29] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:54:29] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:54:29] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:54:29] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:54:39] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:54:39] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:54:40] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:54:40] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:54:50] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:54:50] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:54:50] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:54:50] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:55:00] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:55:00] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:55:00] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:55:00] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:55:10] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:55:10] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:55:10] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:55:10] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:55:20] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:55:20] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:55:21] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:55:21] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:55:31] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:55:31] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:55:31] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:55:31] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:55:41] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:55:41] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:55:41] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:55:41] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:55:51] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:55:51] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:55:51] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:55:51] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:56:01] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:56:02] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:56:02] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:56:02] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:56:12] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:56:12] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:56:12] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:56:12] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:56:22] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:56:22] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:56:22] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:56:22] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:56:32] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:56:32] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:56:33] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:56:33] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:56:43] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:56:43] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:56:43] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:56:43] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:56:53] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:56:53] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:56:53] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:56:53] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:57:03] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:57:03] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:57:04] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:57:04] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:57:14] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:57:14] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:57:14] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:57:14] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:57:24] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:57:24] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:57:24] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:57:24] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:57:34] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:57:34] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:57:34] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:57:34] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:57:44] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:57:44] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:57:45] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:57:45] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:57:55] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:57:55] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:57:55] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:57:55] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:58:05] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:58:05] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:58:05] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:58:05] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:58:15] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:58:15] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:58:16] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:58:16] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:58:26] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:58:26] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:58:26] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:58:26] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:58:36] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:58:36] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:58:36] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:58:36] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:58:46] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:58:46] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:58:47] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:58:47] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:58:57] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:58:57] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:58:57] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:58:57] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:59:07] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:59:07] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:59:07] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:59:07] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:59:17] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:59:17] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:59:17] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[WARNING 05-02 23:59:17] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-02 23:59:17] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:59:17] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:59:18] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:59:18] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:59:28] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:59:28] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:59:28] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:59:28] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:59:38] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:59:38] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:59:38] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:59:38] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:59:48] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:59:48] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:59:49] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:59:49] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-02 23:59:59] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-02 23:59:59] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-02 23:59:59] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-02 23:59:59] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-03 00:00:09] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:00:09] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:00:09] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-03 00:00:09] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-03 00:00:19] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:00:19] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:00:19] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-03 00:00:20] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-03 00:00:30] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:00:30] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:00:30] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-03 00:00:30] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-03 00:00:40] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:00:40] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:00:40] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-03 00:00:40] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-03 00:00:50] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:00:50] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:00:50] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-03 00:00:50] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-03 00:01:00] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:01:00] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:01:01] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-03 00:01:01] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-03 00:01:11] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:01:11] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:01:11] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-03 00:01:11] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-03 00:01:21] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:01:21] Scheduler: Fetching data for trials: 0 - 4 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:01:21] ax.early_stopping.strategies.base: The number of completed trials (0) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-03 00:01:21] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 5).
[INFO 05-03 00:01:32] Scheduler: Fetching data for newly completed trials: [3].
[INFO 05-03 00:01:32] Scheduler: Fetching data for trials: [0, 1, 2, 4] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:01:32] Scheduler: Retrieved COMPLETED trials: [3].
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:01:32] ax.early_stopping.strategies.base: The number of completed trials (1) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[WARNING 05-03 00:01:32] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-03 00:01:32] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:01:32] Scheduler: Fetching data for trials: [0, 1, 2, 4] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:01:32] ax.early_stopping.strategies.base: The number of completed trials (1) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-03 00:01:32] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 4).
[INFO 05-03 00:01:42] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:01:42] Scheduler: Fetching data for trials: [0, 1, 2, 4] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
[INFO 05-03 00:01:42] ax.early_stopping.strategies.base: The number of completed trials (1) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-03 00:01:42] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 4).
[INFO 05-03 00:01:52] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:01:52] Scheduler: Fetching data for trials: [0, 1, 2, 4] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:01:52] ax.early_stopping.strategies.base: The number of completed trials (1) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-03 00:01:52] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 4).
[INFO 05-03 00:02:02] Scheduler: Fetching data for newly completed trials: [4].
[INFO 05-03 00:02:02] Scheduler: Fetching data for trials: 0 - 2 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:02:02] Scheduler: Retrieved COMPLETED trials: [4].
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:02:03] ax.early_stopping.strategies.base: The number of completed trials (2) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[WARNING 05-03 00:02:03] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-03 00:02:03] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:02:03] Scheduler: Fetching data for trials: 0 - 2 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
[INFO 05-03 00:02:03] ax.early_stopping.strategies.base: The number of completed trials (2) is less than the minimum number of curves needed for early stopping (5). Not early stopping.
[INFO 05-03 00:02:03] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:02:13] Scheduler: Fetching data for newly completed trials: 0 - 2.
[INFO 05-03 00:02:13] Scheduler: Retrieved COMPLETED trials: 0 - 2.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:02:13] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.
[INFO 05-03 00:02:13] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
[INFO 05-03 00:02:14] Scheduler: Running trials [5]...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
[INFO 05-03 00:02:15] Scheduler: Running trials [6]...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
[INFO 05-03 00:02:17] Scheduler: Running trials [7]...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:02:18] Scheduler: Generated all trials that can be generated currently. Max parallelism currently reached.
[WARNING 05-03 00:02:19] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-03 00:02:19] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:02:19] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:02:19] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/5. Returning without this metric.
[INFO 05-03 00:02:19] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/6. Returning without this metric.
[INFO 05-03 00:02:19] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/7. Returning without this metric.
[INFO 05-03 00:02:19] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-03 00:02:19] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-03 00:02:19] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-03 00:02:19] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-03 00:02:19] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-03 00:02:19] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-03 00:02:19] Scheduler: Failed to fetch val_acc for trial 5, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:19] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 5 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-03 00:02:19] Scheduler: Failed to fetch val_acc for trial 6, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:19] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 6 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-03 00:02:19] Scheduler: Failed to fetch val_acc for trial 7, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:19] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 7 is still RUNNING continuing the experiment and retrying on next poll...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:02:19] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.
[INFO 05-03 00:02:19] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:02:19] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:02:29] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:02:29] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:02:29] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/5. Returning without this metric.
[INFO 05-03 00:02:29] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/6. Returning without this metric.
[INFO 05-03 00:02:29] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/7. Returning without this metric.
[INFO 05-03 00:02:29] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-03 00:02:29] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-03 00:02:29] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-03 00:02:29] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-03 00:02:29] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-03 00:02:29] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-03 00:02:29] Scheduler: Failed to fetch val_acc for trial 5, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:29] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 5 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-03 00:02:29] Scheduler: Failed to fetch val_acc for trial 6, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:29] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 6 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-03 00:02:29] Scheduler: Failed to fetch val_acc for trial 7, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:29] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 7 is still RUNNING continuing the experiment and retrying on next poll...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:02:29] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.
[INFO 05-03 00:02:29] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:02:29] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:02:39] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:02:39] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:02:39] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/5. Returning without this metric.
[INFO 05-03 00:02:39] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/6. Returning without this metric.
[INFO 05-03 00:02:39] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/7. Returning without this metric.
[INFO 05-03 00:02:39] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-03 00:02:39] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-03 00:02:39] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-03 00:02:39] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-03 00:02:39] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-03 00:02:39] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-03 00:02:39] Scheduler: Failed to fetch val_acc for trial 5, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:39] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 5 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-03 00:02:39] Scheduler: Failed to fetch val_acc for trial 6, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:39] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 6 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-03 00:02:39] Scheduler: Failed to fetch val_acc for trial 7, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:39] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 7 is still RUNNING continuing the experiment and retrying on next poll...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:02:39] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.
[INFO 05-03 00:02:39] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:02:39] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:02:49] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:02:49] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:02:49] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/5. Returning without this metric.
[INFO 05-03 00:02:49] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/6. Returning without this metric.
[INFO 05-03 00:02:49] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/7. Returning without this metric.
[INFO 05-03 00:02:49] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-03 00:02:49] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-03 00:02:49] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-03 00:02:49] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-03 00:02:49] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-03 00:02:49] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-03 00:02:49] Scheduler: Failed to fetch val_acc for trial 5, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:49] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 5 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-03 00:02:49] Scheduler: Failed to fetch val_acc for trial 6, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:49] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 6 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-03 00:02:49] Scheduler: Failed to fetch val_acc for trial 7, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:49] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 7 is still RUNNING continuing the experiment and retrying on next poll...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:02:49] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.
[INFO 05-03 00:02:49] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:02:49] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:02:59] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:02:59] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:02:59] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/5. Returning without this metric.
[INFO 05-03 00:02:59] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/6. Returning without this metric.
[INFO 05-03 00:02:59] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/7. Returning without this metric.
[INFO 05-03 00:02:59] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-03 00:02:59] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[INFO 05-03 00:02:59] ax.core.metric: MetricFetchE INFO: Initialized MetricFetchE(message="DataFrame from curve series is empty")
[ERROR 05-03 00:02:59] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-03 00:02:59] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[ERROR 05-03 00:02:59] ax.core.experiment: Discovered Metric fetching Err while attaching data MetricFetchE(message="DataFrame from curve series is empty"). Ignoring for now -- will retry query on next call to fetch.
[WARNING 05-03 00:02:59] Scheduler: Failed to fetch val_acc for trial 5, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:59] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 5 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-03 00:02:59] Scheduler: Failed to fetch val_acc for trial 6, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:59] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 6 is still RUNNING continuing the experiment and retrying on next poll...
[WARNING 05-03 00:02:59] Scheduler: Failed to fetch val_acc for trial 7, found MetricFetchE(message="DataFrame from curve series is empty").
[INFO 05-03 00:02:59] Scheduler: MetricFetchE INFO: Because val_acc is available_while_running and trial 7 is still RUNNING continuing the experiment and retrying on next poll...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:02:59] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.
[INFO 05-03 00:02:59] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:02:59] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:03:09] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:03:09] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:03:09] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.09994666666666667.
[INFO 05-03 00:03:09] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:03:09] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:03:19] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:03:19] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:03:19] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.09994666666666667.
[INFO 05-03 00:03:19] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:03:19] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:03:29] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:03:29] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:03:29] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.09994666666666667.
[INFO 05-03 00:03:29] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:03:29] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:03:39] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:03:39] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:03:39] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.19994666666666666.
[INFO 05-03 00:03:39] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:03:39] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:03:49] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:03:49] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:03:50] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.19994666666666666.
[INFO 05-03 00:03:50] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:03:50] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:04:00] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:04:00] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:04:00] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.19994666666666666.
[INFO 05-03 00:04:00] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:04:00] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:04:10] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:04:10] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:04:10] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.19994666666666666.
[INFO 05-03 00:04:10] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:04:10] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:04:20] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:04:20] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:04:20] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.19994666666666666.
[INFO 05-03 00:04:20] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:04:20] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:04:30] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:04:30] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:04:30] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.29994666666666664.
[INFO 05-03 00:04:30] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:04:30] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:04:40] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:04:40] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:04:40] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.29994666666666664.
[INFO 05-03 00:04:40] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:04:40] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:04:50] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:04:50] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:04:51] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.29994666666666664.
[INFO 05-03 00:04:51] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:04:51] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:05:01] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:05:01] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:05:01] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.29994666666666664.
[INFO 05-03 00:05:01] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:05:01] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:05:11] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:05:11] Scheduler: Fetching data for trials: 5 - 7 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:05:11] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.3999466666666667.
[INFO 05-03 00:05:11] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:05:11] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.29994666666666664.
[INFO 05-03 00:05:11] ax.early_stopping.strategies.base: Trial 5's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:05:11] ax.early_stopping.strategies.percentile: Considering trial 6 for early stopping.
[INFO 05-03 00:05:11] ax.early_stopping.strategies.base: Last progression of Trial 6 is 0.3999466666666667.
[INFO 05-03 00:05:11] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 6 0.924415 7 0.943446 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:05:11] ax.early_stopping.strategies.percentile: Early stopping decision for 6: True. Reason: Trial objective value 0.9244154095649719 is worse than 70.0-th percentile (0.9247362375259399) across comparable trials.
[INFO 05-03 00:05:11] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:05:11] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.3999466666666667.
[INFO 05-03 00:05:11] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 6 0.924415 7 0.943446 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:05:11] ax.early_stopping.strategies.percentile: Early stopping decision for 7: False. Reason: Trial objective value 0.9434463977813721 is better than 70.0-th percentile (0.9247362375259399) across comparable trials.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
[WARNING 05-03 00:05:13] ax.models.torch.botorch_modular.acquisition: Encountered Xs pending for some Surrogates but observed for others. Considering these points to be pending.
[INFO 05-03 00:05:19] Scheduler: Running trials [8]...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:05:20] Scheduler: Generated all trials that can be generated currently. Max parallelism currently reached.
[WARNING 05-03 00:05:20] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-03 00:05:20] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:05:20] Scheduler: Fetching data for trials: [5, 7, 8] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:05:20] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/8. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:05:20] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.3999466666666667.
[INFO 05-03 00:05:20] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:05:20] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.3999466666666667.
[INFO 05-03 00:05:20] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:05:20] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9384358525276184 is better than 70.0-th percentile (0.925859135389328) across comparable trials.
[INFO 05-03 00:05:20] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:05:20] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.3999466666666667.
[INFO 05-03 00:05:20] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:05:20] ax.early_stopping.strategies.percentile: Early stopping decision for 7: False. Reason: Trial objective value 0.9434463977813721 is better than 70.0-th percentile (0.925859135389328) across comparable trials.
[INFO 05-03 00:05:20] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:05:20] ax.early_stopping.strategies.base: There is not yet any data associated with trial 8 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:05:20] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:05:30] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:05:30] Scheduler: Fetching data for trials: [5, 7, 8] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:05:30] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/8. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:05:30] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.3999466666666667.
[INFO 05-03 00:05:30] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:05:30] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.3999466666666667.
[INFO 05-03 00:05:30] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:05:30] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9384358525276184 is better than 70.0-th percentile (0.925859135389328) across comparable trials.
[INFO 05-03 00:05:30] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:05:30] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.3999466666666667.
[INFO 05-03 00:05:30] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:05:30] ax.early_stopping.strategies.percentile: Early stopping decision for 7: False. Reason: Trial objective value 0.9434463977813721 is better than 70.0-th percentile (0.925859135389328) across comparable trials.
[INFO 05-03 00:05:30] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:05:30] ax.early_stopping.strategies.base: There is not yet any data associated with trial 8 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:05:30] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:05:40] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:05:40] Scheduler: Fetching data for trials: [5, 7, 8] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:05:40] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/8. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:05:40] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.3999466666666667.
[INFO 05-03 00:05:40] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:05:40] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.3999466666666667.
[INFO 05-03 00:05:40] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:05:40] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9384358525276184 is better than 70.0-th percentile (0.925859135389328) across comparable trials.
[INFO 05-03 00:05:40] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:05:40] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.3999466666666667.
[INFO 05-03 00:05:40] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:05:40] ax.early_stopping.strategies.percentile: Early stopping decision for 7: False. Reason: Trial objective value 0.9434463977813721 is better than 70.0-th percentile (0.925859135389328) across comparable trials.
[INFO 05-03 00:05:40] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:05:40] ax.early_stopping.strategies.base: There is not yet any data associated with trial 8 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:05:40] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:05:50] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:05:50] Scheduler: Fetching data for trials: [5, 7, 8] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:05:50] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/8. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:05:50] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.49994666666666665.
[INFO 05-03 00:05:50] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:05:50] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.3999466666666667.
[INFO 05-03 00:05:50] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:05:50] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9384358525276184 is better than 70.0-th percentile (0.925859135389328) across comparable trials.
[INFO 05-03 00:05:50] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:05:50] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.49994666666666665.
[INFO 05-03 00:05:50] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 7 0.931275 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:05:50] ax.early_stopping.strategies.percentile: Early stopping decision for 7: False. Reason: Trial objective value 0.9312747120857239 is better than 70.0-th percentile (0.9220441579818726) across comparable trials.
[INFO 05-03 00:05:50] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:05:50] ax.early_stopping.strategies.base: There is not yet any data associated with trial 8 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:05:50] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:06:00] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:06:00] Scheduler: Fetching data for trials: [5, 7, 8] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:06:00] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/8. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:06:00] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.49994666666666665.
[INFO 05-03 00:06:00] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:06:00] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.49994666666666665.
[INFO 05-03 00:06:00] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:06:00] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9425516724586487 is better than 70.0-th percentile (0.927650511264801) across comparable trials.
[INFO 05-03 00:06:00] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:06:00] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.49994666666666665.
[INFO 05-03 00:06:00] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:06:00] ax.early_stopping.strategies.percentile: Early stopping decision for 7: False. Reason: Trial objective value 0.9312747120857239 is better than 70.0-th percentile (0.927650511264801) across comparable trials.
[INFO 05-03 00:06:00] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:06:00] ax.early_stopping.strategies.base: There is not yet any data associated with trial 8 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:06:00] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:06:11] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:06:11] Scheduler: Fetching data for trials: [5, 7, 8] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:06:11] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/8. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:06:11] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.49994666666666665.
[INFO 05-03 00:06:11] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:06:11] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.49994666666666665.
[INFO 05-03 00:06:11] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:06:11] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9425516724586487 is better than 70.0-th percentile (0.927650511264801) across comparable trials.
[INFO 05-03 00:06:11] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:06:11] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.49994666666666665.
[INFO 05-03 00:06:11] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:06:11] ax.early_stopping.strategies.percentile: Early stopping decision for 7: False. Reason: Trial objective value 0.9312747120857239 is better than 70.0-th percentile (0.927650511264801) across comparable trials.
[INFO 05-03 00:06:11] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:06:11] ax.early_stopping.strategies.base: There is not yet any data associated with trial 8 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:06:11] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:06:21] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:06:21] Scheduler: Fetching data for trials: [5, 7, 8] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:06:21] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:06:21] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:06:21] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:06:21] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.49994666666666665.
[INFO 05-03 00:06:21] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:06:21] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9425516724586487 is better than 70.0-th percentile (0.927650511264801) across comparable trials.
[INFO 05-03 00:06:21] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:06:21] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.5999466666666666.
[INFO 05-03 00:06:21] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 7 0.940188 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:06:21] ax.early_stopping.strategies.percentile: Early stopping decision for 7: False. Reason: Trial objective value 0.9401884078979492 is better than 70.0-th percentile (0.9281519949436188) across comparable trials.
[INFO 05-03 00:06:21] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:06:21] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.09994666666666667.
[INFO 05-03 00:06:21] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:06:21] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:06:31] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:06:31] Scheduler: Fetching data for trials: [5, 7, 8] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:06:31] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:06:31] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:06:31] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:06:31] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.49994666666666665.
[INFO 05-03 00:06:31] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:06:31] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9425516724586487 is better than 70.0-th percentile (0.927650511264801) across comparable trials.
[INFO 05-03 00:06:31] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:06:31] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.5999466666666666.
[INFO 05-03 00:06:31] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 7 0.940188 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:06:31] ax.early_stopping.strategies.percentile: Early stopping decision for 7: False. Reason: Trial objective value 0.9401884078979492 is better than 70.0-th percentile (0.9281519949436188) across comparable trials.
[INFO 05-03 00:06:31] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:06:31] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.09994666666666667.
[INFO 05-03 00:06:31] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:06:31] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:06:41] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:06:41] Scheduler: Fetching data for trials: [5, 7, 8] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:06:41] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:06:41] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:06:41] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:06:41] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.5999466666666666.
[INFO 05-03 00:06:41] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:06:41] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9446216821670532 is better than 70.0-th percentile (0.9337283611297608) across comparable trials.
[INFO 05-03 00:06:41] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:06:41] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.5999466666666666.
[INFO 05-03 00:06:41] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:06:41] ax.early_stopping.strategies.percentile: Early stopping decision for 7: False. Reason: Trial objective value 0.9401884078979492 is better than 70.0-th percentile (0.9337283611297608) across comparable trials.
[INFO 05-03 00:06:41] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:06:41] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.09994666666666667.
[INFO 05-03 00:06:41] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:06:41] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:06:51] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:06:51] Scheduler: Fetching data for trials: [5, 7, 8] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:06:51] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:06:51] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:06:51] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:06:52] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.5999466666666666.
[INFO 05-03 00:06:52] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:06:52] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9446216821670532 is better than 70.0-th percentile (0.9337283611297608) across comparable trials.
[INFO 05-03 00:06:52] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:06:52] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.5999466666666666.
[INFO 05-03 00:06:52] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:06:52] ax.early_stopping.strategies.percentile: Early stopping decision for 7: False. Reason: Trial objective value 0.9401884078979492 is better than 70.0-th percentile (0.9337283611297608) across comparable trials.
[INFO 05-03 00:06:52] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:06:52] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.09994666666666667.
[INFO 05-03 00:06:52] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:06:52] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:07:02] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:07:02] Scheduler: Fetching data for trials: [5, 7, 8] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:07:02] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:07:02] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:07:02] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.5999466666666666.
[INFO 05-03 00:07:02] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:07:02] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9446216821670532 is better than 70.0-th percentile (0.9337283611297608) across comparable trials.
[INFO 05-03 00:07:02] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:07:02] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.5999466666666666.
[INFO 05-03 00:07:02] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:07:02] ax.early_stopping.strategies.percentile: Early stopping decision for 7: False. Reason: Trial objective value 0.9401884078979492 is better than 70.0-th percentile (0.9337283611297608) across comparable trials.
[INFO 05-03 00:07:02] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:07:02] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.19994666666666666.
[INFO 05-03 00:07:02] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:07:02] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:07:12] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:07:12] Scheduler: Fetching data for trials: [5, 7, 8] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:07:12] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:07:12] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:07:12] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.5999466666666666.
[INFO 05-03 00:07:12] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:07:12] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9446216821670532 is better than 70.0-th percentile (0.9337283611297608) across comparable trials.
[INFO 05-03 00:07:12] ax.early_stopping.strategies.percentile: Considering trial 7 for early stopping.
[INFO 05-03 00:07:12] ax.early_stopping.strategies.base: Last progression of Trial 7 is 0.6999466666666667.
[INFO 05-03 00:07:12] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 7 0.918427 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:07:12] ax.early_stopping.strategies.percentile: Early stopping decision for 7: True. Reason: Trial objective value 0.918427050113678 is worse than 70.0-th percentile (0.9247997105121613) across comparable trials.
[INFO 05-03 00:07:12] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:07:12] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.19994666666666666.
[INFO 05-03 00:07:12] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [WARNING 05-03 00:07:13] ax.models.torch.botorch_modular.acquisition: Encountered Xs pending for some Surrogates but observed for others. Considering these points to be pending.
[INFO 05-03 00:07:14] Scheduler: Running trials [9]...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:07:15] Scheduler: Generated all trials that can be generated currently. Max parallelism currently reached.
[WARNING 05-03 00:07:15] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-03 00:07:15] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:07:15] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:07:15] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/9. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:07:15] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:07:15] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:07:15] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.6999466666666667.
[INFO 05-03 00:07:15] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:07:15] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9471415281295776 is better than 70.0-th percentile (0.9313710808753968) across comparable trials.
[INFO 05-03 00:07:15] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:07:15] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.19994666666666666.
[INFO 05-03 00:07:15] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:07:15] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:07:15] ax.early_stopping.strategies.base: There is not yet any data associated with trial 9 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:07:15] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:07:25] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:07:25] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:07:25] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/9. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:07:26] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:07:26] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:07:26] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.6999466666666667.
[INFO 05-03 00:07:26] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:07:26] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9471415281295776 is better than 70.0-th percentile (0.9313710808753968) across comparable trials.
[INFO 05-03 00:07:26] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:07:26] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.19994666666666666.
[INFO 05-03 00:07:26] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:07:26] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:07:26] ax.early_stopping.strategies.base: There is not yet any data associated with trial 9 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:07:26] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:07:36] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:07:36] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:07:36] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/9. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:07:36] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:07:36] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:07:36] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.6999466666666667.
[INFO 05-03 00:07:36] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:07:36] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9471415281295776 is better than 70.0-th percentile (0.9313710808753968) across comparable trials.
[INFO 05-03 00:07:36] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:07:36] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.29994666666666664.
[INFO 05-03 00:07:36] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:07:36] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:07:36] ax.early_stopping.strategies.base: There is not yet any data associated with trial 9 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:07:36] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:07:46] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:07:46] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:07:46] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/9. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:07:46] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:07:46] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:07:46] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.6999466666666667.
[INFO 05-03 00:07:46] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:07:46] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9471415281295776 is better than 70.0-th percentile (0.9313710808753968) across comparable trials.
[INFO 05-03 00:07:46] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:07:46] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.29994666666666664.
[INFO 05-03 00:07:46] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:07:46] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:07:46] ax.early_stopping.strategies.base: There is not yet any data associated with trial 9 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:07:46] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:07:56] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:07:56] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:07:56] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/9. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:07:56] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:07:56] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:07:56] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.7999466666666667.
[INFO 05-03 00:07:56] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:07:56] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9425002932548523 is better than 70.0-th percentile (0.927400678396225) across comparable trials.
[INFO 05-03 00:07:56] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:07:56] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.29994666666666664.
[INFO 05-03 00:07:56] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:07:56] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:07:56] ax.early_stopping.strategies.base: There is not yet any data associated with trial 9 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:07:56] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:08:06] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:08:06] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:08:06] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:08:06] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:08:06] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:08:06] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.7999466666666667.
[INFO 05-03 00:08:06] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:08:06] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9425002932548523 is better than 70.0-th percentile (0.927400678396225) across comparable trials.
[INFO 05-03 00:08:06] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:08:06] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.29994666666666664.
[INFO 05-03 00:08:06] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:08:06] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:08:06] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.09994666666666667.
[INFO 05-03 00:08:06] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:08:06] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:08:16] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:08:16] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:08:16] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:08:16] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:08:16] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:08:16] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.7999466666666667.
[INFO 05-03 00:08:16] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:08:16] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9425002932548523 is better than 70.0-th percentile (0.927400678396225) across comparable trials.
[INFO 05-03 00:08:16] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:08:16] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.29994666666666664.
[INFO 05-03 00:08:16] ax.early_stopping.strategies.base: Trial 8's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:08:16] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:08:16] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.09994666666666667.
[INFO 05-03 00:08:16] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:08:16] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:08:26] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:08:26] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:08:27] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:08:27] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:08:27] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:08:27] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.7999466666666667.
[INFO 05-03 00:08:27] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:08:27] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9425002932548523 is better than 70.0-th percentile (0.927400678396225) across comparable trials.
[INFO 05-03 00:08:27] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:08:27] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.3999466666666667.
[INFO 05-03 00:08:27] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:08:27] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9482073187828064 is better than 70.0-th percentile (0.9334693312644958) across comparable trials.
[INFO 05-03 00:08:27] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:08:27] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.09994666666666667.
[INFO 05-03 00:08:27] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:08:27] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:08:37] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:08:37] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:08:37] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:08:37] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:08:37] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:08:37] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.7999466666666667.
[INFO 05-03 00:08:37] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:08:37] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9425002932548523 is better than 70.0-th percentile (0.927400678396225) across comparable trials.
[INFO 05-03 00:08:37] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:08:37] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.3999466666666667.
[INFO 05-03 00:08:37] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:08:37] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9482073187828064 is better than 70.0-th percentile (0.9334693312644958) across comparable trials.
[INFO 05-03 00:08:37] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:08:37] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.09994666666666667.
[INFO 05-03 00:08:37] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:08:37] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:08:47] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:08:47] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:08:47] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:08:47] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:08:47] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.8999466666666667.
[INFO 05-03 00:08:47] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:08:47] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9475665092468262 is better than 70.0-th percentile (0.9321170449256897) across comparable trials.
[INFO 05-03 00:08:47] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:08:47] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.3999466666666667.
[INFO 05-03 00:08:47] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:08:47] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9482073187828064 is better than 70.0-th percentile (0.9334693312644958) across comparable trials.
[INFO 05-03 00:08:47] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:08:47] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.19994666666666666.
[INFO 05-03 00:08:47] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:08:47] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:08:57] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:08:57] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:08:57] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:08:57] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:08:57] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.8999466666666667.
[INFO 05-03 00:08:57] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:08:57] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9475665092468262 is better than 70.0-th percentile (0.9321170449256897) across comparable trials.
[INFO 05-03 00:08:57] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:08:57] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.3999466666666667.
[INFO 05-03 00:08:57] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:08:57] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9482073187828064 is better than 70.0-th percentile (0.9334693312644958) across comparable trials.
[INFO 05-03 00:08:57] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:08:57] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.19994666666666666.
[INFO 05-03 00:08:57] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:08:57] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:09:07] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:09:07] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:09:07] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:09:07] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:09:07] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.8999466666666667.
[INFO 05-03 00:09:07] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:09:07] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9475665092468262 is better than 70.0-th percentile (0.9321170449256897) across comparable trials.
[INFO 05-03 00:09:07] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:09:07] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.49994666666666665.
[INFO 05-03 00:09:07] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:09:07] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9498466849327087 is better than 70.0-th percentile (0.9308216869831085) across comparable trials.
[INFO 05-03 00:09:07] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:09:07] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.19994666666666666.
[INFO 05-03 00:09:07] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:09:07] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:09:17] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:09:17] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:09:18] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:09:18] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:09:18] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.8999466666666667.
[INFO 05-03 00:09:18] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:09:18] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9475665092468262 is better than 70.0-th percentile (0.9321170449256897) across comparable trials.
[INFO 05-03 00:09:18] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:09:18] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.49994666666666665.
[INFO 05-03 00:09:18] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:09:18] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9498466849327087 is better than 70.0-th percentile (0.9308216869831085) across comparable trials.
[INFO 05-03 00:09:18] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:09:18] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.19994666666666666.
[INFO 05-03 00:09:18] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:09:18] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:09:28] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:09:28] Scheduler: Fetching data for trials: [5, 8, 9] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:09:28] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.9999466666666667.
[INFO 05-03 00:09:28] ax.early_stopping.strategies.percentile: Considering trial 5 for early stopping.
[INFO 05-03 00:09:28] ax.early_stopping.strategies.base: Last progression of Trial 5 is 0.9999466666666667.
[INFO 05-03 00:09:28] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935098 1 0.919584 2 0.923828 3 0.904117 4 0.864199 5 0.949224 Name: 0.9999466666666667, dtype: float64.
[INFO 05-03 00:09:28] ax.early_stopping.strategies.percentile: Early stopping decision for 5: False. Reason: Trial objective value 0.9492241144180298 is better than 70.0-th percentile (0.929463267326355) across comparable trials.
[INFO 05-03 00:09:28] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:09:28] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.49994666666666665.
[INFO 05-03 00:09:28] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:09:28] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9498466849327087 is better than 70.0-th percentile (0.9308216869831085) across comparable trials.
[INFO 05-03 00:09:28] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:09:28] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.29994666666666664.
[INFO 05-03 00:09:28] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:09:28] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:09:38] Scheduler: Fetching data for newly completed trials: [5].
[INFO 05-03 00:09:38] Scheduler: Fetching data for trials: 8 - 9 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:09:38] Scheduler: Retrieved COMPLETED trials: [5].
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:09:38] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:09:38] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:09:38] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.5999466666666666.
[INFO 05-03 00:09:38] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:09:38] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9623385667800903 is better than 70.0-th percentile (0.9393809020519257) across comparable trials.
[INFO 05-03 00:09:38] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:09:38] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.29994666666666664.
[INFO 05-03 00:09:38] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
[WARNING 05-03 00:09:38] ax.models.torch.botorch_modular.acquisition: Encountered Xs pending for some Surrogates but observed for others. Considering these points to be pending.
[INFO 05-03 00:09:40] Scheduler: Running trials [10]...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:09:41] Scheduler: Generated all trials that can be generated currently. Max parallelism currently reached.
[WARNING 05-03 00:09:41] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-03 00:09:41] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:09:41] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:09:41] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/10. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:09:41] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:09:41] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:09:41] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.5999466666666666.
[INFO 05-03 00:09:41] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:09:41] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9623385667800903 is better than 70.0-th percentile (0.9393809020519257) across comparable trials.
[INFO 05-03 00:09:41] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:09:41] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.29994666666666664.
[INFO 05-03 00:09:41] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:09:41] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:09:41] ax.early_stopping.strategies.base: There is not yet any data associated with trial 10 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:09:41] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:09:51] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:09:51] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:09:51] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/10. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:09:51] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:09:51] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:09:51] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.5999466666666666.
[INFO 05-03 00:09:51] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:09:51] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9623385667800903 is better than 70.0-th percentile (0.9393809020519257) across comparable trials.
[INFO 05-03 00:09:51] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:09:51] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.29994666666666664.
[INFO 05-03 00:09:51] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:09:51] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:09:51] ax.early_stopping.strategies.base: There is not yet any data associated with trial 10 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:09:51] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:10:01] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:10:01] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:10:01] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/10. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:10:01] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:10:01] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:10:01] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.5999466666666666.
[INFO 05-03 00:10:01] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:10:01] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9623385667800903 is better than 70.0-th percentile (0.9393809020519257) across comparable trials.
[INFO 05-03 00:10:01] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:10:01] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.29994666666666664.
[INFO 05-03 00:10:01] ax.early_stopping.strategies.base: Trial 9's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:10:01] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:10:01] ax.early_stopping.strategies.base: There is not yet any data associated with trial 10 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:10:01] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:10:11] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:10:11] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:10:11] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/10. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:10:11] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:10:11] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:10:11] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.6999466666666667.
[INFO 05-03 00:10:11] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:10:11] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9592832922935486 is better than 70.0-th percentile (0.9320665657520294) across comparable trials.
[INFO 05-03 00:10:11] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:10:11] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.3999466666666667.
[INFO 05-03 00:10:11] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:10:11] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9517658352851868 is better than 70.0-th percentile (0.9399390161037445) across comparable trials.
[INFO 05-03 00:10:11] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:10:11] ax.early_stopping.strategies.base: There is not yet any data associated with trial 10 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:10:11] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:10:21] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:10:21] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:10:21] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/10. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:10:21] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:10:21] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:10:21] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.6999466666666667.
[INFO 05-03 00:10:21] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:10:21] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9592832922935486 is better than 70.0-th percentile (0.9320665657520294) across comparable trials.
[INFO 05-03 00:10:21] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:10:21] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.3999466666666667.
[INFO 05-03 00:10:21] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:10:21] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9517658352851868 is better than 70.0-th percentile (0.9399390161037445) across comparable trials.
[INFO 05-03 00:10:21] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:10:21] ax.early_stopping.strategies.base: There is not yet any data associated with trial 10 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:10:22] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:10:32] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:10:32] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:10:32] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:10:32] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:10:32] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:10:32] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.6999466666666667.
[INFO 05-03 00:10:32] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:10:32] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9592832922935486 is better than 70.0-th percentile (0.9320665657520294) across comparable trials.
[INFO 05-03 00:10:32] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:10:32] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.3999466666666667.
[INFO 05-03 00:10:32] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:10:32] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9517658352851868 is better than 70.0-th percentile (0.9399390161037445) across comparable trials.
[INFO 05-03 00:10:32] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:10:32] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.09994666666666667.
[INFO 05-03 00:10:32] ax.early_stopping.strategies.base: Trial 10's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:10:32] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:10:42] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:10:42] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:10:42] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:10:42] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:10:42] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:10:42] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.6999466666666667.
[INFO 05-03 00:10:42] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:10:42] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9592832922935486 is better than 70.0-th percentile (0.9320665657520294) across comparable trials.
[INFO 05-03 00:10:42] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:10:42] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.3999466666666667.
[INFO 05-03 00:10:42] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:10:42] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9517658352851868 is better than 70.0-th percentile (0.9399390161037445) across comparable trials.
[INFO 05-03 00:10:42] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:10:42] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.09994666666666667.
[INFO 05-03 00:10:42] ax.early_stopping.strategies.base: Trial 10's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:10:42] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:10:52] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:10:52] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:10:52] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:10:52] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:10:52] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:10:52] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.7999466666666667.
[INFO 05-03 00:10:52] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:10:52] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9596459269523621 is better than 70.0-th percentile (0.9366804480552673) across comparable trials.
[INFO 05-03 00:10:52] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:10:52] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.49994666666666665.
[INFO 05-03 00:10:52] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:10:52] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9494989514350891 is better than 70.0-th percentile (0.9380408883094787) across comparable trials.
[INFO 05-03 00:10:52] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:10:52] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.09994666666666667.
[INFO 05-03 00:10:52] ax.early_stopping.strategies.base: Trial 10's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:10:52] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:11:02] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:11:02] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:11:02] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:11:02] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:11:02] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:11:02] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.7999466666666667.
[INFO 05-03 00:11:02] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:11:02] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9596459269523621 is better than 70.0-th percentile (0.9366804480552673) across comparable trials.
[INFO 05-03 00:11:02] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:11:02] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.49994666666666665.
[INFO 05-03 00:11:02] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:11:02] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9494989514350891 is better than 70.0-th percentile (0.9380408883094787) across comparable trials.
[INFO 05-03 00:11:02] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:11:02] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.09994666666666667.
[INFO 05-03 00:11:02] ax.early_stopping.strategies.base: Trial 10's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:11:02] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:11:12] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:11:12] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:11:13] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:11:13] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:11:13] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.7999466666666667.
[INFO 05-03 00:11:13] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:11:13] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9596459269523621 is better than 70.0-th percentile (0.9366804480552673) across comparable trials.
[INFO 05-03 00:11:13] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:11:13] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.49994666666666665.
[INFO 05-03 00:11:13] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:11:13] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9494989514350891 is better than 70.0-th percentile (0.9380408883094787) across comparable trials.
[INFO 05-03 00:11:13] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:11:13] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.19994666666666666.
[INFO 05-03 00:11:13] ax.early_stopping.strategies.base: Trial 10's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:11:13] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:11:23] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:11:23] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:11:23] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:11:23] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:11:23] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.7999466666666667.
[INFO 05-03 00:11:23] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:11:23] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9596459269523621 is better than 70.0-th percentile (0.9366804480552673) across comparable trials.
[INFO 05-03 00:11:23] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:11:23] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.49994666666666665.
[INFO 05-03 00:11:23] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:11:23] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9494989514350891 is better than 70.0-th percentile (0.9380408883094787) across comparable trials.
[INFO 05-03 00:11:23] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:11:23] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.19994666666666666.
[INFO 05-03 00:11:23] ax.early_stopping.strategies.base: Trial 10's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:11:23] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:11:33] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:11:33] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:11:33] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:11:33] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:11:33] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.8999466666666667.
[INFO 05-03 00:11:33] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:11:33] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9589828252792358 is better than 70.0-th percentile (0.9386385440826416) across comparable trials.
[INFO 05-03 00:11:33] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:11:33] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.49994666666666665.
[INFO 05-03 00:11:33] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:11:33] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9494989514350891 is better than 70.0-th percentile (0.9380408883094787) across comparable trials.
[INFO 05-03 00:11:33] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:11:33] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.19994666666666666.
[INFO 05-03 00:11:33] ax.early_stopping.strategies.base: Trial 10's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:11:33] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:11:43] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:11:43] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:11:43] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:11:43] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:11:43] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.8999466666666667.
[INFO 05-03 00:11:43] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:11:43] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9589828252792358 is better than 70.0-th percentile (0.9386385440826416) across comparable trials.
[INFO 05-03 00:11:43] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:11:43] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.5999466666666666.
[INFO 05-03 00:11:43] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 9 0.953896 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:11:43] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9538964629173279 is better than 70.0-th percentile (0.9428483724594117) across comparable trials.
[INFO 05-03 00:11:43] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:11:43] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.19994666666666666.
[INFO 05-03 00:11:43] ax.early_stopping.strategies.base: Trial 10's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:11:43] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:11:53] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:11:53] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:11:53] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:11:53] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:11:53] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.8999466666666667.
[INFO 05-03 00:11:53] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:11:53] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9589828252792358 is better than 70.0-th percentile (0.9386385440826416) across comparable trials.
[INFO 05-03 00:11:53] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:11:53] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.5999466666666666.
[INFO 05-03 00:11:53] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 9 0.953896 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:11:53] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9538964629173279 is better than 70.0-th percentile (0.9428483724594117) across comparable trials.
[INFO 05-03 00:11:53] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:11:53] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.29994666666666664.
[INFO 05-03 00:11:53] ax.early_stopping.strategies.base: Trial 10's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:11:53] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:12:03] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:12:03] Scheduler: Fetching data for trials: 8 - 10 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:12:03] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:12:04] ax.early_stopping.strategies.percentile: Considering trial 8 for early stopping.
[INFO 05-03 00:12:04] ax.early_stopping.strategies.base: Last progression of Trial 8 is 0.8999466666666667.
[INFO 05-03 00:12:04] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:12:04] ax.early_stopping.strategies.percentile: Early stopping decision for 8: False. Reason: Trial objective value 0.9589828252792358 is better than 70.0-th percentile (0.9386385440826416) across comparable trials.
[INFO 05-03 00:12:04] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:12:04] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.5999466666666666.
[INFO 05-03 00:12:04] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 9 0.953896 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:12:04] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9538964629173279 is better than 70.0-th percentile (0.9428483724594117) across comparable trials.
[INFO 05-03 00:12:04] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:12:04] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.29994666666666664.
[INFO 05-03 00:12:04] ax.early_stopping.strategies.base: Trial 10's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:12:04] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:12:14] Scheduler: Fetching data for newly completed trials: [8].
[INFO 05-03 00:12:14] Scheduler: Fetching data for trials: 9 - 10 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:12:14] Scheduler: Retrieved COMPLETED trials: [8].
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:12:14] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:12:14] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:12:14] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.6999466666666667.
[INFO 05-03 00:12:14] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:12:14] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9533754587173462 is better than 70.0-th percentile (0.9411512851715088) across comparable trials.
[INFO 05-03 00:12:14] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:12:14] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.29994666666666664.
[INFO 05-03 00:12:14] ax.early_stopping.strategies.base: Trial 10's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
[WARNING 05-03 00:12:14] ax.models.torch.botorch_modular.acquisition: Encountered Xs pending for some Surrogates but observed for others. Considering these points to be pending.
[INFO 05-03 00:12:26] Scheduler: Running trials [11]...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:12:27] Scheduler: Generated all trials that can be generated currently. Max parallelism currently reached.
[WARNING 05-03 00:12:27] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-03 00:12:27] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:12:27] Scheduler: Fetching data for trials: 9 - 11 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:12:27] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/11. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:12:27] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:12:27] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:12:27] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.6999466666666667.
[INFO 05-03 00:12:27] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:12:27] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9533754587173462 is better than 70.0-th percentile (0.9411512851715088) across comparable trials.
[INFO 05-03 00:12:27] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:12:27] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.3999466666666667.
[INFO 05-03 00:12:27] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 10 0.949104 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:12:27] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9491037130355835 is better than 70.0-th percentile (0.9434463977813721) across comparable trials.
[INFO 05-03 00:12:27] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:12:27] ax.early_stopping.strategies.base: There is not yet any data associated with trial 11 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:12:27] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:12:37] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:12:37] Scheduler: Fetching data for trials: 9 - 11 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:12:37] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/11. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:12:37] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:12:37] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:12:37] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.6999466666666667.
[INFO 05-03 00:12:37] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:12:37] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9533754587173462 is better than 70.0-th percentile (0.9411512851715088) across comparable trials.
[INFO 05-03 00:12:37] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:12:37] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.3999466666666667.
[INFO 05-03 00:12:37] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 10 0.949104 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:12:37] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9491037130355835 is better than 70.0-th percentile (0.9434463977813721) across comparable trials.
[INFO 05-03 00:12:37] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:12:37] ax.early_stopping.strategies.base: There is not yet any data associated with trial 11 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:12:37] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:12:47] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:12:47] Scheduler: Fetching data for trials: 9 - 11 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:12:47] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/11. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:12:47] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:12:47] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:12:47] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.6999466666666667.
[INFO 05-03 00:12:47] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:12:47] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9533754587173462 is better than 70.0-th percentile (0.9411512851715088) across comparable trials.
[INFO 05-03 00:12:47] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:12:47] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.3999466666666667.
[INFO 05-03 00:12:47] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 10 0.949104 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:12:47] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9491037130355835 is better than 70.0-th percentile (0.9434463977813721) across comparable trials.
[INFO 05-03 00:12:47] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:12:47] ax.early_stopping.strategies.base: There is not yet any data associated with trial 11 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:12:47] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:12:57] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:12:57] Scheduler: Fetching data for trials: 9 - 11 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:12:57] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/11. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:12:57] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:12:57] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:12:57] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.7999466666666667.
[INFO 05-03 00:12:57] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:12:57] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9513505697250366 is better than 70.0-th percentile (0.9417728126049042) across comparable trials.
[INFO 05-03 00:12:57] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:12:57] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.3999466666666667.
[INFO 05-03 00:12:57] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 10 0.949104 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:12:57] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9491037130355835 is better than 70.0-th percentile (0.9434463977813721) across comparable trials.
[INFO 05-03 00:12:57] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:12:57] ax.early_stopping.strategies.base: There is not yet any data associated with trial 11 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:12:57] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:13:07] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:13:07] Scheduler: Fetching data for trials: 9 - 11 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:13:07] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/11. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:13:08] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:13:08] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:13:08] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.7999466666666667.
[INFO 05-03 00:13:08] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:13:08] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9513505697250366 is better than 70.0-th percentile (0.9417728126049042) across comparable trials.
[INFO 05-03 00:13:08] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:13:08] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.49994666666666665.
[INFO 05-03 00:13:08] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 10 0.956445 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:13:08] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.956444501876831 is better than 70.0-th percentile (0.9446358561515809) across comparable trials.
[INFO 05-03 00:13:08] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:13:08] ax.early_stopping.strategies.base: There is not yet any data associated with trial 11 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:13:08] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:13:18] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:13:18] Scheduler: Fetching data for trials: 9 - 11 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:13:18] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:13:18] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:13:18] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:13:18] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.7999466666666667.
[INFO 05-03 00:13:18] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:13:18] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9513505697250366 is better than 70.0-th percentile (0.9417728126049042) across comparable trials.
[INFO 05-03 00:13:18] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:13:18] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.49994666666666665.
[INFO 05-03 00:13:18] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 10 0.956445 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:13:18] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.956444501876831 is better than 70.0-th percentile (0.9446358561515809) across comparable trials.
[INFO 05-03 00:13:18] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:13:18] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.09994666666666667.
[INFO 05-03 00:13:18] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:13:18] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:13:28] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:13:28] Scheduler: Fetching data for trials: 9 - 11 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:13:28] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:13:28] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:13:28] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:13:28] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.7999466666666667.
[INFO 05-03 00:13:28] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:13:28] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9513505697250366 is better than 70.0-th percentile (0.9417728126049042) across comparable trials.
[INFO 05-03 00:13:28] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:13:28] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.49994666666666665.
[INFO 05-03 00:13:28] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 10 0.956445 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:13:28] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.956444501876831 is better than 70.0-th percentile (0.9446358561515809) across comparable trials.
[INFO 05-03 00:13:28] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:13:28] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.09994666666666667.
[INFO 05-03 00:13:28] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:13:28] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:13:38] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:13:38] Scheduler: Fetching data for trials: 9 - 11 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:13:38] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:13:38] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:13:38] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:13:38] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.8999466666666667.
[INFO 05-03 00:13:38] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 9 0.955601 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:13:38] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9556013941764832 is better than 70.0-th percentile (0.9464505136013031) across comparable trials.
[INFO 05-03 00:13:38] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:13:38] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.49994666666666665.
[INFO 05-03 00:13:38] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 10 0.956445 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:13:38] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.956444501876831 is better than 70.0-th percentile (0.9446358561515809) across comparable trials.
[INFO 05-03 00:13:38] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:13:38] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.09994666666666667.
[INFO 05-03 00:13:38] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:13:38] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:13:48] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:13:48] Scheduler: Fetching data for trials: 9 - 11 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:13:48] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:13:49] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:13:49] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:13:49] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.8999466666666667.
[INFO 05-03 00:13:49] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 9 0.955601 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:13:49] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9556013941764832 is better than 70.0-th percentile (0.9464505136013031) across comparable trials.
[INFO 05-03 00:13:49] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:13:49] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.49994666666666665.
[INFO 05-03 00:13:49] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 10 0.956445 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:13:49] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.956444501876831 is better than 70.0-th percentile (0.9446358561515809) across comparable trials.
[INFO 05-03 00:13:49] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:13:49] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.09994666666666667.
[INFO 05-03 00:13:49] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:13:49] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:13:59] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:13:59] Scheduler: Fetching data for trials: 9 - 11 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:13:59] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:13:59] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:13:59] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.8999466666666667.
[INFO 05-03 00:13:59] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 9 0.955601 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:13:59] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9556013941764832 is better than 70.0-th percentile (0.9464505136013031) across comparable trials.
[INFO 05-03 00:13:59] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:13:59] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.5999466666666666.
[INFO 05-03 00:13:59] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 9 0.953896 10 0.953981 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:13:59] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9539807438850403 is better than 70.0-th percentile (0.9474041163921356) across comparable trials.
[INFO 05-03 00:13:59] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:13:59] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.19994666666666666.
[INFO 05-03 00:13:59] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:13:59] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:14:09] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:14:09] Scheduler: Fetching data for trials: 9 - 11 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:14:09] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.9999466666666667.
[INFO 05-03 00:14:09] ax.early_stopping.strategies.percentile: Considering trial 9 for early stopping.
[INFO 05-03 00:14:09] ax.early_stopping.strategies.base: Last progression of Trial 9 is 0.9999466666666667.
[INFO 05-03 00:14:09] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935098 1 0.919584 2 0.923828 3 0.904117 4 0.864199 5 0.949224 8 0.957670 9 0.949123 Name: 0.9999466666666667, dtype: float64.
[INFO 05-03 00:14:09] ax.early_stopping.strategies.percentile: Early stopping decision for 9: False. Reason: Trial objective value 0.9491230249404907 is better than 70.0-th percentile (0.9477205634117126) across comparable trials.
[INFO 05-03 00:14:09] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:14:09] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.5999466666666666.
[INFO 05-03 00:14:09] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 9 0.953896 10 0.953981 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:14:09] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9539807438850403 is better than 70.0-th percentile (0.9474041163921356) across comparable trials.
[INFO 05-03 00:14:09] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:14:09] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.19994666666666666.
[INFO 05-03 00:14:09] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:14:09] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:14:19] Scheduler: Fetching data for newly completed trials: [9].
[INFO 05-03 00:14:19] Scheduler: Fetching data for trials: 10 - 11 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:14:19] Scheduler: Retrieved COMPLETED trials: [9].
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:14:19] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:14:19] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:14:19] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.5999466666666666.
[INFO 05-03 00:14:19] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 9 0.953896 10 0.953981 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:14:19] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9539807438850403 is better than 70.0-th percentile (0.9474041163921356) across comparable trials.
[INFO 05-03 00:14:19] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:14:19] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.19994666666666666.
[INFO 05-03 00:14:19] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
[WARNING 05-03 00:14:19] ax.models.torch.botorch_modular.acquisition: Encountered Xs pending for some Surrogates but observed for others. Considering these points to be pending.
[INFO 05-03 00:14:23] Scheduler: Running trials [12]...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:14:24] Scheduler: Generated all trials that can be generated currently. Max parallelism currently reached.
[WARNING 05-03 00:14:24] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-03 00:14:24] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:14:24] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:14:24] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/12. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:14:24] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:14:24] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:14:24] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.5999466666666666.
[INFO 05-03 00:14:24] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 9 0.953896 10 0.953981 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:14:24] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9539807438850403 is better than 70.0-th percentile (0.9474041163921356) across comparable trials.
[INFO 05-03 00:14:24] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:14:24] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.19994666666666666.
[INFO 05-03 00:14:24] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:14:24] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:14:24] ax.early_stopping.strategies.base: There is not yet any data associated with trial 12 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:14:24] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:14:34] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:14:34] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:14:34] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/12. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:14:34] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:14:34] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:14:34] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.6999466666666667.
[INFO 05-03 00:14:34] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 10 0.955318 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:14:34] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9553176760673523 is better than 70.0-th percentile (0.9490117073059082) across comparable trials.
[INFO 05-03 00:14:34] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:14:34] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.29994666666666664.
[INFO 05-03 00:14:34] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:14:34] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:14:34] ax.early_stopping.strategies.base: There is not yet any data associated with trial 12 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:14:34] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:14:44] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:14:44] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:14:44] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/12. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:14:44] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:14:44] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:14:44] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.6999466666666667.
[INFO 05-03 00:14:44] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 10 0.955318 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:14:44] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9553176760673523 is better than 70.0-th percentile (0.9490117073059082) across comparable trials.
[INFO 05-03 00:14:44] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:14:44] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.29994666666666664.
[INFO 05-03 00:14:44] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:14:44] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:14:44] ax.early_stopping.strategies.base: There is not yet any data associated with trial 12 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:14:44] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:14:54] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:14:54] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:14:54] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/12. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:14:54] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:14:54] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:14:54] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.6999466666666667.
[INFO 05-03 00:14:54] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 10 0.955318 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:14:54] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9553176760673523 is better than 70.0-th percentile (0.9490117073059082) across comparable trials.
[INFO 05-03 00:14:54] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:14:54] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.29994666666666664.
[INFO 05-03 00:14:54] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:14:54] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:14:54] ax.early_stopping.strategies.base: There is not yet any data associated with trial 12 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:14:54] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:15:04] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:15:04] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:15:04] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/12. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:15:04] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:15:05] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:15:05] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.7999466666666667.
[INFO 05-03 00:15:05] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 10 0.959402 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:15:05] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.959402322769165 is better than 70.0-th percentile (0.9478104591369629) across comparable trials.
[INFO 05-03 00:15:05] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:15:05] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.29994666666666664.
[INFO 05-03 00:15:05] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:15:05] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:15:05] ax.early_stopping.strategies.base: There is not yet any data associated with trial 12 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:15:05] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:15:15] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:15:15] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:15:15] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:15:15] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:15:15] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:15:15] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.7999466666666667.
[INFO 05-03 00:15:15] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 10 0.959402 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:15:15] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.959402322769165 is better than 70.0-th percentile (0.9478104591369629) across comparable trials.
[INFO 05-03 00:15:15] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:15:15] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.29994666666666664.
[INFO 05-03 00:15:15] ax.early_stopping.strategies.base: Trial 11's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:15:15] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:15:15] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.09994666666666667.
[INFO 05-03 00:15:15] ax.early_stopping.strategies.base: Trial 12's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:15:15] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:15:25] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:15:25] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:15:25] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:15:25] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:15:25] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:15:25] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.7999466666666667.
[INFO 05-03 00:15:25] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 10 0.959402 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:15:25] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.959402322769165 is better than 70.0-th percentile (0.9478104591369629) across comparable trials.
[INFO 05-03 00:15:25] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:15:25] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.3999466666666667.
[INFO 05-03 00:15:25] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 10 0.949104 11 0.952827 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:15:25] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9528272747993469 is better than 70.0-th percentile (0.9467790424823761) across comparable trials.
[INFO 05-03 00:15:25] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:15:25] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.09994666666666667.
[INFO 05-03 00:15:25] ax.early_stopping.strategies.base: Trial 12's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:15:25] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:15:35] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:15:35] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:15:35] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:15:35] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:15:35] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:15:35] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.7999466666666667.
[INFO 05-03 00:15:35] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 10 0.959402 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:15:35] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.959402322769165 is better than 70.0-th percentile (0.9478104591369629) across comparable trials.
[INFO 05-03 00:15:35] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:15:35] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.3999466666666667.
[INFO 05-03 00:15:35] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 10 0.949104 11 0.952827 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:15:35] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9528272747993469 is better than 70.0-th percentile (0.9467790424823761) across comparable trials.
[INFO 05-03 00:15:35] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:15:35] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.09994666666666667.
[INFO 05-03 00:15:35] ax.early_stopping.strategies.base: Trial 12's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:15:35] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:15:45] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:15:45] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:15:45] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:15:45] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:15:45] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:15:45] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.7999466666666667.
[INFO 05-03 00:15:45] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 10 0.959402 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:15:45] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.959402322769165 is better than 70.0-th percentile (0.9478104591369629) across comparable trials.
[INFO 05-03 00:15:45] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:15:45] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.3999466666666667.
[INFO 05-03 00:15:45] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 10 0.949104 11 0.952827 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:15:45] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9528272747993469 is better than 70.0-th percentile (0.9467790424823761) across comparable trials.
[INFO 05-03 00:15:45] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:15:45] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.09994666666666667.
[INFO 05-03 00:15:45] ax.early_stopping.strategies.base: Trial 12's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:15:45] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:15:55] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:15:55] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:15:56] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:15:56] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:15:56] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.8999466666666667.
[INFO 05-03 00:15:56] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 9 0.955601 10 0.962233 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:15:56] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9622330069541931 is better than 70.0-th percentile (0.9523874402046204) across comparable trials.
[INFO 05-03 00:15:56] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:15:56] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.3999466666666667.
[INFO 05-03 00:15:56] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 10 0.949104 11 0.952827 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:15:56] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9528272747993469 is better than 70.0-th percentile (0.9467790424823761) across comparable trials.
[INFO 05-03 00:15:56] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:15:56] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.19994666666666666.
[INFO 05-03 00:15:56] ax.early_stopping.strategies.base: Trial 12's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:15:56] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:16:06] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:16:06] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:16:06] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:16:06] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:16:06] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.8999466666666667.
[INFO 05-03 00:16:06] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 9 0.955601 10 0.962233 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:16:06] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9622330069541931 is better than 70.0-th percentile (0.9523874402046204) across comparable trials.
[INFO 05-03 00:16:06] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:16:06] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.49994666666666665.
[INFO 05-03 00:16:06] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 10 0.956445 11 0.954861 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:16:06] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9548612236976624 is better than 70.0-th percentile (0.9494989514350891) across comparable trials.
[INFO 05-03 00:16:06] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:16:06] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.19994666666666666.
[INFO 05-03 00:16:06] ax.early_stopping.strategies.base: Trial 12's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:16:06] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:16:16] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:16:16] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:16:16] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:16:16] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:16:16] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.8999466666666667.
[INFO 05-03 00:16:16] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 9 0.955601 10 0.962233 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:16:16] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9622330069541931 is better than 70.0-th percentile (0.9523874402046204) across comparable trials.
[INFO 05-03 00:16:16] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:16:16] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.49994666666666665.
[INFO 05-03 00:16:16] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 10 0.956445 11 0.954861 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:16:16] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9548612236976624 is better than 70.0-th percentile (0.9494989514350891) across comparable trials.
[INFO 05-03 00:16:16] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:16:16] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.19994666666666666.
[INFO 05-03 00:16:16] ax.early_stopping.strategies.base: Trial 12's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:16:16] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:16:26] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:16:26] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:16:26] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:16:26] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:16:27] ax.early_stopping.strategies.base: Last progression of Trial 10 is 0.8999466666666667.
[INFO 05-03 00:16:27] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 9 0.955601 10 0.962233 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:16:27] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9622330069541931 is better than 70.0-th percentile (0.9523874402046204) across comparable trials.
[INFO 05-03 00:16:27] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:16:27] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.49994666666666665.
[INFO 05-03 00:16:27] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 10 0.956445 11 0.954861 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:16:27] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9548612236976624 is better than 70.0-th percentile (0.9494989514350891) across comparable trials.
[INFO 05-03 00:16:27] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:16:27] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.19994666666666666.
[INFO 05-03 00:16:27] ax.early_stopping.strategies.base: Trial 12's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:16:27] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:16:37] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:16:37] Scheduler: Fetching data for trials: 10 - 12 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:16:37] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 1.0.
[INFO 05-03 00:16:37] ax.early_stopping.strategies.percentile: Considering trial 10 for early stopping.
[INFO 05-03 00:16:37] ax.early_stopping.strategies.base: Last progression of Trial 10 is 1.0.
[INFO 05-03 00:16:37] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935098 1 0.919584 2 0.923828 3 0.904117 4 0.864199 5 0.949224 8 0.957670 9 0.949123 10 0.962607 Name: 1.0, dtype: float64.
[INFO 05-03 00:16:37] ax.early_stopping.strategies.percentile: Early stopping decision for 10: False. Reason: Trial objective value 0.9626068472862244 is better than 70.0-th percentile (0.9491836786270141) across comparable trials.
[INFO 05-03 00:16:37] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:16:37] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.5999466666666666.
[INFO 05-03 00:16:37] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 9 0.953896 10 0.953981 11 0.954557 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:16:37] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9545567631721497 is better than 70.0-th percentile (0.9538964629173279) across comparable trials.
[INFO 05-03 00:16:37] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:16:37] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.29994666666666664.
[INFO 05-03 00:16:37] ax.early_stopping.strategies.base: Trial 12's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:16:37] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:16:47] Scheduler: Fetching data for newly completed trials: [10].
[INFO 05-03 00:16:47] Scheduler: Fetching data for trials: 11 - 12 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:16:47] Scheduler: Retrieved COMPLETED trials: [10].
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:16:47] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:16:47] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:16:47] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.5999466666666666.
[INFO 05-03 00:16:47] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 9 0.953896 10 0.953981 11 0.954557 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:16:47] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9545567631721497 is better than 70.0-th percentile (0.9538964629173279) across comparable trials.
[INFO 05-03 00:16:47] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:16:47] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.29994666666666664.
[INFO 05-03 00:16:47] ax.early_stopping.strategies.base: Trial 12's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
[WARNING 05-03 00:16:47] ax.models.torch.botorch_modular.acquisition: Encountered Xs pending for some Surrogates but observed for others. Considering these points to be pending.
[INFO 05-03 00:16:49] Scheduler: Running trials [13]...
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:16:50] Scheduler: Generated all trials that can be generated currently. Max parallelism currently reached.
[WARNING 05-03 00:16:50] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-03 00:16:50] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:16:50] Scheduler: Fetching data for trials: 11 - 13 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:16:50] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/13. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:16:50] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:16:50] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:16:50] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.5999466666666666.
[INFO 05-03 00:16:50] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 9 0.953896 10 0.953981 11 0.954557 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:16:50] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9545567631721497 is better than 70.0-th percentile (0.9538964629173279) across comparable trials.
[INFO 05-03 00:16:50] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:16:50] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.29994666666666664.
[INFO 05-03 00:16:50] ax.early_stopping.strategies.base: Trial 12's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:16:50] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:16:50] ax.early_stopping.strategies.base: There is not yet any data associated with trial 13 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:16:50] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:17:00] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:17:00] Scheduler: Fetching data for trials: 11 - 13 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:17:00] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/13. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:17:00] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:17:00] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:17:00] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.5999466666666666.
[INFO 05-03 00:17:00] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 9 0.953896 10 0.953981 11 0.954557 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:17:00] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9545567631721497 is better than 70.0-th percentile (0.9538964629173279) across comparable trials.
[INFO 05-03 00:17:00] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:17:00] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.29994666666666664.
[INFO 05-03 00:17:00] ax.early_stopping.strategies.base: Trial 12's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:17:00] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:17:00] ax.early_stopping.strategies.base: There is not yet any data associated with trial 13 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:17:00] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:17:10] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:17:10] Scheduler: Fetching data for trials: 11 - 13 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:17:10] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/13. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:17:10] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:17:10] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:17:10] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.6999466666666667.
[INFO 05-03 00:17:10] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 10 0.955318 11 0.953431 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:17:10] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9534313678741455 is better than 70.0-th percentile (0.9533754587173462) across comparable trials.
[INFO 05-03 00:17:10] ax.early_stopping.strategies.percentile: Considering trial 12 for early stopping.
[INFO 05-03 00:17:10] ax.early_stopping.strategies.base: Last progression of Trial 12 is 0.3999466666666667.
[INFO 05-03 00:17:11] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 10 0.949104 11 0.952827 12 0.944046 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:17:11] ax.early_stopping.strategies.percentile: Early stopping decision for 12: True. Reason: Trial objective value 0.9440456032752991 is worse than 70.0-th percentile (0.945710289478302) across comparable trials.
[INFO 05-03 00:17:11] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:17:11] ax.early_stopping.strategies.base: There is not yet any data associated with trial 13 and metric val_acc. Not early stopping this trial.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [WARNING 05-03 00:17:12] ax.models.torch.botorch_modular.acquisition: Encountered Xs pending for some Surrogates but observed for others. Considering these points to be pending.
[INFO 05-03 00:17:20] Scheduler: Running trials [14]...
[WARNING 05-03 00:17:20] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-03 00:17:20] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:17:20] Scheduler: Fetching data for trials: [11, 13, 14] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:17:20] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/13. Returning without this metric.
[INFO 05-03 00:17:20] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/14. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:17:20] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:17:20] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:17:20] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.6999466666666667.
[INFO 05-03 00:17:20] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 10 0.955318 11 0.953431 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:17:20] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9534313678741455 is better than 70.0-th percentile (0.9533754587173462) across comparable trials.
[INFO 05-03 00:17:20] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:17:20] ax.early_stopping.strategies.base: There is not yet any data associated with trial 13 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:17:20] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:17:20] ax.early_stopping.strategies.base: There is not yet any data associated with trial 14 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:17:20] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:17:30] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:17:30] Scheduler: Fetching data for trials: [11, 13, 14] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:17:30] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/13. Returning without this metric.
[INFO 05-03 00:17:30] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/14. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:17:30] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:17:30] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:17:30] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.6999466666666667.
[INFO 05-03 00:17:30] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 10 0.955318 11 0.953431 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:17:30] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9534313678741455 is better than 70.0-th percentile (0.9533754587173462) across comparable trials.
[INFO 05-03 00:17:30] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:17:30] ax.early_stopping.strategies.base: There is not yet any data associated with trial 13 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:17:30] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:17:30] ax.early_stopping.strategies.base: There is not yet any data associated with trial 14 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:17:30] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:17:40] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:17:40] Scheduler: Fetching data for trials: [11, 13, 14] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:17:40] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/14. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:17:40] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:17:40] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:17:40] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:17:40] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.6999466666666667.
[INFO 05-03 00:17:40] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 10 0.955318 11 0.953431 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:17:40] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9534313678741455 is better than 70.0-th percentile (0.9533754587173462) across comparable trials.
[INFO 05-03 00:17:40] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:17:40] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.09994666666666667.
[INFO 05-03 00:17:40] ax.early_stopping.strategies.base: Trial 13's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:17:40] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:17:40] ax.early_stopping.strategies.base: There is not yet any data associated with trial 14 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:17:40] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:17:50] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:17:50] Scheduler: Fetching data for trials: [11, 13, 14] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:17:50] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/14. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:17:50] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:17:50] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:17:50] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:17:50] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.7999466666666667.
[INFO 05-03 00:17:50] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 10 0.959402 11 0.955646 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:17:50] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9556459188461304 is better than 70.0-th percentile (0.9526391744613647) across comparable trials.
[INFO 05-03 00:17:50] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:17:50] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.09994666666666667.
[INFO 05-03 00:17:50] ax.early_stopping.strategies.base: Trial 13's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:17:50] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:17:50] ax.early_stopping.strategies.base: There is not yet any data associated with trial 14 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:17:50] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:18:00] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:18:00] Scheduler: Fetching data for trials: [11, 13, 14] because some metrics on experiment are available while trials are running.
[INFO 05-03 00:18:00] ax.metrics.curve: val_acc not yet present in curves from /tmp/tmpmal164i2/14. Returning without this metric.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:18:00] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:18:01] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:18:01] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:18:01] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.7999466666666667.
[INFO 05-03 00:18:01] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 10 0.959402 11 0.955646 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:18:01] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9556459188461304 is better than 70.0-th percentile (0.9526391744613647) across comparable trials.
[INFO 05-03 00:18:01] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:18:01] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.09994666666666667.
[INFO 05-03 00:18:01] ax.early_stopping.strategies.base: Trial 13's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:18:01] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:18:01] ax.early_stopping.strategies.base: There is not yet any data associated with trial 14 and metric val_acc. Not early stopping this trial.
[INFO 05-03 00:18:01] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:18:11] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:18:11] Scheduler: Fetching data for trials: [11, 13, 14] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:18:11] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:18:11] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:18:11] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:18:11] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:18:11] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.7999466666666667.
[INFO 05-03 00:18:11] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 10 0.959402 11 0.955646 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:18:11] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9556459188461304 is better than 70.0-th percentile (0.9526391744613647) across comparable trials.
[INFO 05-03 00:18:11] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:18:11] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.09994666666666667.
[INFO 05-03 00:18:11] ax.early_stopping.strategies.base: Trial 13's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:18:11] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:18:11] ax.early_stopping.strategies.base: Last progression of Trial 14 is 0.09994666666666667.
[INFO 05-03 00:18:11] ax.early_stopping.strategies.base: Trial 14's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:18:11] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:18:21] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:18:21] Scheduler: Fetching data for trials: [11, 13, 14] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:18:21] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:18:21] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:18:21] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:18:21] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.7999466666666667.
[INFO 05-03 00:18:21] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 10 0.959402 11 0.955646 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:18:21] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9556459188461304 is better than 70.0-th percentile (0.9526391744613647) across comparable trials.
[INFO 05-03 00:18:21] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:18:21] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.19994666666666666.
[INFO 05-03 00:18:21] ax.early_stopping.strategies.base: Trial 13's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:18:21] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:18:21] ax.early_stopping.strategies.base: Last progression of Trial 14 is 0.09994666666666667.
[INFO 05-03 00:18:21] ax.early_stopping.strategies.base: Trial 14's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:18:21] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:18:31] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:18:31] Scheduler: Fetching data for trials: [11, 13, 14] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:18:31] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:18:31] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:18:31] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:18:31] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.8999466666666667.
[INFO 05-03 00:18:31] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 9 0.955601 10 0.962233 11 0.962643 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:18:31] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9626431465148926 is better than 70.0-th percentile (0.956615823507309) across comparable trials.
[INFO 05-03 00:18:31] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:18:31] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.19994666666666666.
[INFO 05-03 00:18:31] ax.early_stopping.strategies.base: Trial 13's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:18:31] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:18:31] ax.early_stopping.strategies.base: Last progression of Trial 14 is 0.09994666666666667.
[INFO 05-03 00:18:31] ax.early_stopping.strategies.base: Trial 14's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:18:31] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:18:41] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:18:41] Scheduler: Fetching data for trials: [11, 13, 14] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:18:41] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:18:41] ax.early_stopping.utils: Got exception `x and y arrays must have at least 2 entries` during interpolation. Using uninterpolated values instead.
[INFO 05-03 00:18:41] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:18:41] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.8999466666666667.
[INFO 05-03 00:18:42] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 9 0.955601 10 0.962233 11 0.962643 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:18:42] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9626431465148926 is better than 70.0-th percentile (0.956615823507309) across comparable trials.
[INFO 05-03 00:18:42] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:18:42] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.19994666666666666.
[INFO 05-03 00:18:42] ax.early_stopping.strategies.base: Trial 13's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:18:42] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:18:42] ax.early_stopping.strategies.base: Last progression of Trial 14 is 0.09994666666666667.
[INFO 05-03 00:18:42] ax.early_stopping.strategies.base: Trial 14's most recent progression (0.09994666666666667) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:18:42] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:18:52] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:18:52] Scheduler: Fetching data for trials: [11, 13, 14] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:18:52] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:18:52] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:18:52] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.8999466666666667.
[INFO 05-03 00:18:52] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 9 0.955601 10 0.962233 11 0.962643 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:18:52] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9626431465148926 is better than 70.0-th percentile (0.956615823507309) across comparable trials.
[INFO 05-03 00:18:52] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:18:52] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.19994666666666666.
[INFO 05-03 00:18:52] ax.early_stopping.strategies.base: Trial 13's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:18:52] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:18:52] ax.early_stopping.strategies.base: Last progression of Trial 14 is 0.19994666666666666.
[INFO 05-03 00:18:52] ax.early_stopping.strategies.base: Trial 14's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:18:52] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:19:02] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:19:02] Scheduler: Fetching data for trials: [11, 13, 14] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:19:02] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:19:02] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:19:02] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.8999466666666667.
[INFO 05-03 00:19:02] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 9 0.955601 10 0.962233 11 0.962643 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:19:02] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9626431465148926 is better than 70.0-th percentile (0.956615823507309) across comparable trials.
[INFO 05-03 00:19:02] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:19:02] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.29994666666666664.
[INFO 05-03 00:19:02] ax.early_stopping.strategies.base: Trial 13's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:19:02] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:19:02] ax.early_stopping.strategies.base: Last progression of Trial 14 is 0.19994666666666666.
[INFO 05-03 00:19:02] ax.early_stopping.strategies.base: Trial 14's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:19:02] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:19:12] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:19:12] Scheduler: Fetching data for trials: [11, 13, 14] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:19:12] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.9999466666666667.
[INFO 05-03 00:19:12] ax.early_stopping.strategies.percentile: Considering trial 11 for early stopping.
[INFO 05-03 00:19:12] ax.early_stopping.strategies.base: Last progression of Trial 11 is 0.9999466666666667.
[INFO 05-03 00:19:12] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935098 1 0.919584 2 0.923828 3 0.904117 4 0.864199 5 0.949224 8 0.957670 9 0.949123 10 0.962607 11 0.959875 Name: 0.9999466666666667, dtype: float64.
[INFO 05-03 00:19:12] ax.early_stopping.strategies.percentile: Early stopping decision for 11: False. Reason: Trial objective value 0.9598748087882996 is better than 70.0-th percentile (0.9517579436302185) across comparable trials.
[INFO 05-03 00:19:12] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:19:12] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.29994666666666664.
[INFO 05-03 00:19:12] ax.early_stopping.strategies.base: Trial 13's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:19:12] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:19:12] ax.early_stopping.strategies.base: Last progression of Trial 14 is 0.19994666666666666.
[INFO 05-03 00:19:12] ax.early_stopping.strategies.base: Trial 14's most recent progression (0.19994666666666666) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:19:12] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 3).
[INFO 05-03 00:19:22] Scheduler: Fetching data for newly completed trials: [11].
[INFO 05-03 00:19:22] Scheduler: Fetching data for trials: 13 - 14 because some metrics on experiment are available while trials are running.
[INFO 05-03 00:19:22] Scheduler: Retrieved COMPLETED trials: [11].
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:19:22] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.29994666666666664.
[INFO 05-03 00:19:22] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:19:22] Scheduler: Done submitting trials, waiting for remaining 2 running trials...
[WARNING 05-03 00:19:22] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-03 00:19:22] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:19:22] Scheduler: Fetching data for trials: 13 - 14 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:19:22] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.29994666666666664.
[INFO 05-03 00:19:22] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[INFO 05-03 00:19:22] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 2).
[INFO 05-03 00:19:32] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:19:32] Scheduler: Fetching data for trials: 13 - 14 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:19:33] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.3999466666666667.
[INFO 05-03 00:19:33] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:19:33] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.3999466666666667.
[INFO 05-03 00:19:33] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 10 0.949104 11 0.952827 12 0.944046 13 0.956569 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:19:33] ax.early_stopping.strategies.percentile: Early stopping decision for 13: False. Reason: Trial objective value 0.9565685391426086 is better than 70.0-th percentile (0.9482969582080841) across comparable trials.
[INFO 05-03 00:19:33] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:19:33] ax.early_stopping.strategies.base: Last progression of Trial 14 is 0.29994666666666664.
[INFO 05-03 00:19:33] ax.early_stopping.strategies.base: Trial 14's most recent progression (0.29994666666666664) that is available for metric val_acc falls out of the min/max_progression range (0.3, None). Not early stopping this trial.
[INFO 05-03 00:19:33] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 2).
[INFO 05-03 00:19:43] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:19:43] Scheduler: Fetching data for trials: 13 - 14 because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:19:43] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.49994666666666665.
[INFO 05-03 00:19:43] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:19:43] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.49994666666666665.
[INFO 05-03 00:19:43] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 10 0.956445 11 0.954861 13 0.961859 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:19:43] ax.early_stopping.strategies.percentile: Early stopping decision for 13: False. Reason: Trial objective value 0.9618589878082275 is better than 70.0-th percentile (0.9497423648834229) across comparable trials.
[INFO 05-03 00:19:43] ax.early_stopping.strategies.percentile: Considering trial 14 for early stopping.
[INFO 05-03 00:19:43] ax.early_stopping.strategies.base: Last progression of Trial 14 is 0.3999466666666667.
[INFO 05-03 00:19:43] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.922330 1 0.910567 2 0.926020 3 0.884824 4 0.852351 5 0.938436 6 0.924415 7 0.943446 8 0.948207 9 0.951766 10 0.949104 11 0.952827 12 0.944046 13 0.956569 14 0.940916 Name: 0.3999466666666667, dtype: float64.
[INFO 05-03 00:19:43] ax.early_stopping.strategies.percentile: Early stopping decision for 14: True. Reason: Trial objective value 0.9409162402153015 is worse than 70.0-th percentile (0.9473749756813049) across comparable trials.
[WARNING 05-03 00:19:43] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
[INFO 05-03 00:19:43] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:19:43] Scheduler: Fetching data for trials: [13] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:19:43] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.49994666666666665.
[INFO 05-03 00:19:43] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:19:43] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.49994666666666665.
[INFO 05-03 00:19:43] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 10 0.956445 11 0.954861 13 0.961859 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:19:43] ax.early_stopping.strategies.percentile: Early stopping decision for 13: False. Reason: Trial objective value 0.9618589878082275 is better than 70.0-th percentile (0.9497423648834229) across comparable trials.
[INFO 05-03 00:19:43] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 1).
[INFO 05-03 00:19:53] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:19:53] Scheduler: Fetching data for trials: [13] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:19:53] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.49994666666666665.
[INFO 05-03 00:19:53] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:19:53] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.49994666666666665.
[INFO 05-03 00:19:53] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.926744 1 0.917344 2 0.911218 3 0.890555 4 0.864412 5 0.942552 7 0.931275 8 0.949847 9 0.949499 10 0.956445 11 0.954861 13 0.961859 Name: 0.49994666666666665, dtype: float64.
[INFO 05-03 00:19:53] ax.early_stopping.strategies.percentile: Early stopping decision for 13: False. Reason: Trial objective value 0.9618589878082275 is better than 70.0-th percentile (0.9497423648834229) across comparable trials.
[INFO 05-03 00:19:53] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 1).
[INFO 05-03 00:20:04] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:20:04] Scheduler: Fetching data for trials: [13] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:20:04] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.5999466666666666.
[INFO 05-03 00:20:04] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:20:04] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.5999466666666666.
[INFO 05-03 00:20:04] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932113 1 0.913559 2 0.924191 3 0.892993 4 0.880571 5 0.944622 7 0.940188 8 0.962339 9 0.953896 10 0.953981 11 0.954557 13 0.959957 Name: 0.5999466666666666, dtype: float64.
[INFO 05-03 00:20:04] ax.early_stopping.strategies.percentile: Early stopping decision for 13: False. Reason: Trial objective value 0.9599568247795105 is better than 70.0-th percentile (0.9539554595947266) across comparable trials.
[INFO 05-03 00:20:04] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 1).
[INFO 05-03 00:20:14] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:20:14] Scheduler: Fetching data for trials: [13] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:20:14] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:20:14] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:20:14] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.6999466666666667.
[INFO 05-03 00:20:14] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 10 0.955318 11 0.953431 13 0.961527 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:20:14] ax.early_stopping.strategies.percentile: Early stopping decision for 13: False. Reason: Trial objective value 0.9615271687507629 is better than 70.0-th percentile (0.9534145951271057) across comparable trials.
[INFO 05-03 00:20:14] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 1).
[INFO 05-03 00:20:24] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:20:24] Scheduler: Fetching data for trials: [13] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:20:24] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.6999466666666667.
[INFO 05-03 00:20:24] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:20:24] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.6999466666666667.
[INFO 05-03 00:20:24] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.932166 1 0.917778 2 0.931172 3 0.896606 4 0.871559 5 0.947142 7 0.918427 8 0.959283 9 0.953375 10 0.955318 11 0.953431 13 0.961527 Name: 0.6999466666666667, dtype: float64.
[INFO 05-03 00:20:24] ax.early_stopping.strategies.percentile: Early stopping decision for 13: False. Reason: Trial objective value 0.9615271687507629 is better than 70.0-th percentile (0.9534145951271057) across comparable trials.
[INFO 05-03 00:20:24] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 1).
[INFO 05-03 00:20:34] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:20:34] Scheduler: Fetching data for trials: [13] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:20:34] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.7999466666666667.
[INFO 05-03 00:20:34] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:20:34] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.7999466666666667.
[INFO 05-03 00:20:34] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935225 1 0.913272 2 0.919576 3 0.900533 4 0.867889 5 0.942500 8 0.959646 9 0.951351 10 0.959402 11 0.955646 13 0.959523 Name: 0.7999466666666667, dtype: float64.
[INFO 05-03 00:20:34] ax.early_stopping.strategies.percentile: Early stopping decision for 13: False. Reason: Trial objective value 0.9595229625701904 is better than 70.0-th percentile (0.9556459188461304) across comparable trials.
[INFO 05-03 00:20:34] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 1).
[INFO 05-03 00:20:44] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:20:44] Scheduler: Fetching data for trials: [13] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:20:44] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.8999466666666667.
[INFO 05-03 00:20:44] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:20:44] ax.early_stopping.strategies.base: Last progression of Trial 13 is 0.8999466666666667.
[INFO 05-03 00:20:44] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.936407 1 0.914466 2 0.927828 3 0.903227 4 0.872152 5 0.947567 8 0.958983 9 0.955601 10 0.962233 11 0.962643 13 0.962669 Name: 0.8999466666666667, dtype: float64.
[INFO 05-03 00:20:44] ax.early_stopping.strategies.percentile: Early stopping decision for 13: False. Reason: Trial objective value 0.9626692533493042 is better than 70.0-th percentile (0.9589828252792358) across comparable trials.
[INFO 05-03 00:20:44] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 1).
[INFO 05-03 00:20:54] Scheduler: Fetching data for newly completed trials: [].
[INFO 05-03 00:20:54] Scheduler: Fetching data for trials: [13] because some metrics on experiment are available while trials are running.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:20:54] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 1.0.
[INFO 05-03 00:20:54] ax.early_stopping.strategies.percentile: Considering trial 13 for early stopping.
[INFO 05-03 00:20:54] ax.early_stopping.strategies.base: Last progression of Trial 13 is 1.0.
[INFO 05-03 00:20:54] ax.early_stopping.strategies.percentile: Early stopping objective at last progression is: 0 0.935098 1 0.919584 2 0.923828 3 0.904117 4 0.864199 5 0.949224 8 0.957670 9 0.949123 10 0.962607 11 0.959875 13 0.964476 Name: 1.0, dtype: float64.
[INFO 05-03 00:20:54] ax.early_stopping.strategies.percentile: Early stopping decision for 13: False. Reason: Trial objective value 0.9644763469696045 is better than 70.0-th percentile (0.9576702117919922) across comparable trials.
[INFO 05-03 00:20:54] Scheduler: Waiting for completed trials (for 10 sec, currently running trials: 1).
[INFO 05-03 00:21:04] Scheduler: Fetching data for newly completed trials: [13].
[INFO 05-03 00:21:04] Scheduler: Retrieved COMPLETED trials: [13].
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( /tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [INFO 05-03 00:21:04] ax.early_stopping.strategies.base: Last progression of any candidate for trial stopping is 0.
[INFO 05-03 00:21:04] ax.early_stopping.strategies.base: No trials have reached 0.3. Not stopping any trials.
[WARNING 05-03 00:21:04] Scheduler: Both `init_seconds_between_polls` and `early_stopping_strategy supplied. `init_seconds_between_polls=1` will be overrridden by `early_stopping_strategy.seconds_between_polls=10` and polling will take place at a constant rate.
CPU times: user 1min 7s, sys: 1.64 s, total: 1min 8s Wall time: 37min 25s
OptimizationResult()
First, we examine the data stored on the experiment. This shows that each trial is associated with an entire learning curve, represented by the column "steps".
experiment.lookup_data().map_df.head(n=10)
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
| arm_name | metric_name | mean | sem | trial_index | steps | |
|---|---|---|---|---|---|---|
| 0 | 0_0 | val_acc | 0.890804 | NaN | 0 | 1874.0 |
| 1 | 0_0 | val_acc | 0.906278 | NaN | 0 | 3749.0 |
| 2 | 0_0 | val_acc | 0.921300 | NaN | 0 | 5624.0 |
| 3 | 0_0 | val_acc | 0.922330 | NaN | 0 | 7499.0 |
| 4 | 0_0 | val_acc | 0.926744 | NaN | 0 | 9374.0 |
| 5 | 0_0 | val_acc | 0.932113 | NaN | 0 | 11249.0 |
| 6 | 0_0 | val_acc | 0.932166 | NaN | 0 | 13124.0 |
| 7 | 0_0 | val_acc | 0.935225 | NaN | 0 | 14999.0 |
| 8 | 0_0 | val_acc | 0.936407 | NaN | 0 | 16874.0 |
| 9 | 0_0 | val_acc | 0.935098 | NaN | 0 | 18749.0 |
Below is a summary of the experiment, showing that a portion of trials have been early stopped.
exp_to_df(experiment)
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [WARNING 05-03 00:21:05] ax.service.utils.report_utils: Column reason missing for all trials. Not appending column.
| trial_index | arm_name | trial_status | generation_method | val_acc | hidden_size_1 | hidden_size_2 | learning_rate | dropout | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0_0 | COMPLETED | Sobol | 0.935098 | 28 | 32 | 0.000401 | 0.368782 |
| 1 | 1 | 1_0 | COMPLETED | Sobol | 0.919584 | 19 | 34 | 0.003596 | 0.191825 |
| 2 | 2 | 2_0 | COMPLETED | Sobol | 0.923828 | 33 | 33 | 0.004869 | 0.153733 |
| 3 | 3 | 3_0 | COMPLETED | Sobol | 0.904117 | 19 | 19 | 0.000153 | 0.350804 |
| 4 | 4 | 4_0 | COMPLETED | Sobol | 0.864199 | 23 | 22 | 0.006478 | 0.360394 |
| 5 | 5 | 5_0 | COMPLETED | BoTorch | 0.949224 | 34 | 46 | 0.000518 | 0.275200 |
| 6 | 6 | 6_0 | EARLY_STOPPED | BoTorch | 0.924415 | 35 | 42 | 0.000300 | 0.474984 |
| 7 | 7 | 7_0 | EARLY_STOPPED | BoTorch | 0.918427 | 25 | 43 | 0.002749 | 0.012497 |
| 8 | 8 | 8_0 | COMPLETED | BoTorch | 0.957670 | 31 | 43 | 0.001155 | 0.077080 |
| 9 | 9 | 9_0 | COMPLETED | BoTorch | 0.949123 | 36 | 39 | 0.000740 | 0.174787 |
| 10 | 10 | 10_0 | COMPLETED | BoTorch | 0.962607 | 28 | 58 | 0.000750 | 0.118605 |
| 11 | 11 | 11_0 | COMPLETED | BoTorch | 0.959875 | 42 | 52 | 0.001160 | 0.134887 |
| 12 | 12 | 12_0 | EARLY_STOPPED | BoTorch | 0.944046 | 25 | 39 | 0.000709 | 0.111734 |
| 13 | 13 | 13_0 | COMPLETED | BoTorch | 0.964476 | 37 | 63 | 0.000650 | 0.068455 |
| 14 | 14 | 14_0 | EARLY_STOPPED | BoTorch | 0.940916 | 28 | 63 | 0.001054 | 0.183375 |
We can give a very rough estimate of the amount of computational savings due to early stopping, by looking at the total number of steps used when early stopping is used versus the number of steps used if we ran all trials to completion. Note to do a true comparison, one should run full HPO loops with and without early stopping (as early stopping will influence the model and future points selected by the generation strategy).
map_df = experiment.lookup_data().map_df
trial_to_max_steps = map_df.groupby("trial_index")["steps"].max()
completed_trial_steps = trial_to_max_steps.iloc[0]
savings = 1.0 - trial_to_max_steps.sum() / (
completed_trial_steps * len(trial_to_max_steps)
)
# TODO format nicer
print(f"A rough estimate of the computational savings is {100 * savings}%.")
A rough estimate of the computational savings is 14.001422222222226%.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
Finally, we show a visualization of learning curves versus actual elapsed wall time. This helps to illustrate that stopped trials make room for additional trials to be run.
# helper function for getting trial start times
def time_started(row):
trial_index = row["trial_index"]
return experiment.trials[trial_index].time_run_started
# helper function for getting trial completion times
def time_completed(row):
trial_index = row["trial_index"]
return experiment.trials[trial_index].time_completed
# helper function for getting relevant data from experiment
# with early stopping into useful dfs
def early_stopping_exp_to_df(experiment):
trials_df = exp_to_df(experiment)
curve_df = experiment.lookup_data().map_df
training_row_df = (
curve_df.groupby("trial_index").max().reset_index()[["trial_index", "steps"]]
)
trials_df = trials_df.merge(training_row_df, on="trial_index")
trials_df["time_started"] = trials_df.apply(func=time_started, axis=1)
trials_df["time_completed"] = trials_df.apply(func=time_completed, axis=1)
start_time = trials_df["time_started"].min()
trials_df["time_started_rel"] = (
trials_df["time_started"] - start_time
).dt.total_seconds()
trials_df["time_completed_rel"] = (
trials_df["time_completed"] - start_time
).dt.total_seconds()
return trials_df, curve_df
def plot_curves_by_wall_time(trials_df, curve_df):
trials = set(curve_df["trial_index"])
fig, ax = plt.subplots(1, 1, figsize=(10, 6))
ax.set(xlabel="seconds since start", ylabel="validation accuracy")
for trial_index in trials:
this_trial_df = curve_df[curve_df["trial_index"] == trial_index]
start_time_rel = trials_df["time_started_rel"].iloc[trial_index]
completed_time_rel = trials_df["time_completed_rel"].iloc[trial_index]
total_steps = trials_df.loc[trial_index, "steps"]
smoothed_curve = this_trial_df["mean"].rolling(window=3).mean()
x = (
start_time_rel
+ (completed_time_rel - start_time_rel)
/ total_steps
* this_trial_df["steps"]
)
ax.plot(
x,
smoothed_curve,
label=f"trial #{trial_index}" if trial_index % 2 == 1 else None,
)
ax.legend()
# wrap in try/except in case of flaky I/O issues
try:
trials_df, curve_df = early_stopping_exp_to_df(experiment)
plot_curves_by_wall_time(trials_df, curve_df)
except Exception as e:
print(f"Encountered exception while plotting results: {e}")
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat( [WARNING 05-03 00:21:06] ax.service.utils.report_utils: Column reason missing for all trials. Not appending column.
/tmp/tmp.J4Ip05By14/Ax-main/ax/core/map_data.py:190: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. df = pd.concat(
Total runtime of script: 37 minutes, 35.9 seconds.