ax.telemetry¶

AxClient¶

class ax.telemetry.ax_client.AxClientCompletedRecord(experiment_completed_record: ExperimentCompletedRecord, best_point_quality: float, model_fit_quality: float, model_std_quality: float, model_fit_generalization: float, model_std_generalization: float)[source]¶

Bases: object

Record of the AxClient completion event. This will have information only available after the optimization has completed.

best_point_quality: float¶

experiment_completed_record: ExperimentCompletedRecord¶

flatten() → dict[str, Any][source]¶

Flatten into an appropriate format for logging to a tabular database.

classmethod from_ax_client(ax_client: AxClient) → AxClientCompletedRecord[source]¶

model_fit_generalization: float¶

model_fit_quality: float¶

model_std_generalization: float¶

model_std_quality: float¶

class ax.telemetry.ax_client.AxClientCreatedRecord(experiment_created_record: ExperimentCreatedRecord, generation_strategy_created_record: GenerationStrategyCreatedRecord, arms_per_trial: int, early_stopping_strategy_cls: str | None, global_stopping_strategy_cls: str | None, transformed_dimensionality: int | None)[source]¶

Bases: object

Record of the AxClient creation event. This can be used for telemetry in settings where many AxClients are being created either manually or programatically. In order to facilitate easy serialization only include simple types: numbers, strings, bools, and None.

arms_per_trial: int¶

early_stopping_strategy_cls: str | None¶

experiment_created_record: ExperimentCreatedRecord¶

flatten() → dict[str, Any][source]¶

Flatten into an appropriate format for logging to a tabular database.

classmethod from_ax_client(ax_client: AxClient) → AxClientCreatedRecord[source]¶

generation_strategy_created_record: GenerationStrategyCreatedRecord¶

global_stopping_strategy_cls: str | None¶

transformed_dimensionality: int | None¶

Common¶

ax.telemetry.common.get_unique_identifier(experiment: Experiment) → str[source]¶

Return a unique identifier for an experiment so creation and completion events can be joined.

Experiment¶

class ax.telemetry.experiment.ExperimentCompletedRecord(num_initialization_trials: int, num_bayesopt_trials: int, num_other_trials: int, num_completed_trials: int, num_failed_trials: int, num_abandoned_trials: int, num_early_stopped_trials: int, total_fit_time: int, total_gen_time: int, num_objectives: int, num_tracking_metrics: int, num_outcome_constraints: int)[source]¶

Bases: object

Record of the Experiment completion event. This can be used for telemetry in settings where many Experiments are being created either manually or programatically. In order to facilitate easy serialization only include simple types: numbers, strings, bools, and None.

classmethod from_experiment(experiment: Experiment) → ExperimentCompletedRecord[source]¶

num_abandoned_trials: int¶

num_bayesopt_trials: int¶

num_completed_trials: int¶

num_early_stopped_trials: int¶

num_failed_trials: int¶

num_initialization_trials: int¶

num_objectives: int¶

num_other_trials: int¶

num_outcome_constraints: int¶

num_tracking_metrics: int¶

total_fit_time: int¶

total_gen_time: int¶

class ax.telemetry.experiment.ExperimentCreatedRecord(experiment_name: str | None, experiment_type: str | None, num_continuous_range_parameters: int, num_int_range_parameters_small: int, num_int_range_parameters_medium: int, num_int_range_parameters_large: int, num_log_scale_range_parameters: int, num_unordered_choice_parameters_small: int, num_unordered_choice_parameters_medium: int, num_unordered_choice_parameters_large: int, num_fixed_parameters: int, dimensionality: int, hierarchical_tree_height: int, num_parameter_constraints: int, num_objectives: int, num_tracking_metrics: int, num_outcome_constraints: int, num_map_metrics: int, metric_cls_to_quantity: dict[str, int], runner_cls: str)[source]¶

Bases: object

Record of the Experiment creation event. This can be used for telemetry in settings where many Experiments are being created either manually or programatically. In order to facilitate easy serialization only include simple types: numbers, strings, bools, and None.

dimensionality: int¶

experiment_name: str | None¶

experiment_type: str | None¶

classmethod from_experiment(experiment: Experiment) → ExperimentCreatedRecord[source]¶

hierarchical_tree_height: int¶

metric_cls_to_quantity: dict[str, int]¶

num_continuous_range_parameters: int¶

num_fixed_parameters: int¶

num_int_range_parameters_large: int¶

num_int_range_parameters_medium: int¶

num_int_range_parameters_small: int¶

num_log_scale_range_parameters: int¶

num_map_metrics: int¶

num_objectives: int¶

num_outcome_constraints: int¶

num_parameter_constraints: int¶

num_tracking_metrics: int¶

num_unordered_choice_parameters_large: int¶

num_unordered_choice_parameters_medium: int¶

num_unordered_choice_parameters_small: int¶

runner_cls: str¶

Generation Strategy¶

class ax.telemetry.generation_strategy.GenerationStrategyCreatedRecord(generation_strategy_name: str, num_requested_initialization_trials: None | int, num_requested_bayesopt_trials: int | None, num_requested_other_trials: int | None, max_parallelism: int | None)[source]¶

Bases: object

Record of the GenerationStrategy creation event. This can be used for telemetry in settings where many GenerationStrategy are being created either manually or programatically. In order to facilitate easy serialization only include simple types: numbers, strings, bools, and None.

classmethod from_generation_strategy(generation_strategy: GenerationStrategy) → GenerationStrategyCreatedRecord[source]¶

generation_strategy_name: str¶

max_parallelism: int | None¶

num_requested_bayesopt_trials: int | None¶

num_requested_initialization_trials: None | int¶

num_requested_other_trials: int | None¶

Optimization¶

class ax.telemetry.optimization.OptimizationCompletedRecord(unique_identifier: str, num_initialization_trials: int, num_bayesopt_trials: int, num_other_trials: int, num_completed_trials: int, num_failed_trials: int, num_abandoned_trials: int, num_early_stopped_trials: int, total_fit_time: int, total_gen_time: int, best_point_quality: float, model_fit_quality: float, model_std_quality: float, model_fit_generalization: float, model_std_generalization: float, improvement_over_baseline: float, num_metric_fetch_e_encountered: int, num_trials_bad_due_to_err: int, deployed_job_id: int | None, estimated_early_stopping_savings: float, estimated_global_stopping_savings: float, num_objectives: int, num_tracking_metrics: int, num_outcome_constraints: int)[source]¶

Bases: object

Record of the “Optimization” completion event. This can come from either an AxClient or a Scheduler. This Record is especially useful for logging Ax-backed optimization results to a tabular database (i.e. one row per Record)

best_point_quality: float¶

deployed_job_id: int | None¶

estimated_early_stopping_savings: float¶

estimated_global_stopping_savings: float¶

classmethod from_ax_client(ax_client: AxClient, unique_identifier: str, deployed_job_id: int | None, estimated_early_stopping_savings: float, estimated_global_stopping_savings: float) → OptimizationCompletedRecord[source]¶

classmethod from_scheduler(scheduler: Scheduler, unique_identifier: str, deployed_job_id: int | None, estimated_early_stopping_savings: float, estimated_global_stopping_savings: float) → OptimizationCompletedRecord[source]¶

improvement_over_baseline: float¶

model_fit_generalization: float¶

model_fit_quality: float¶

model_std_generalization: float¶

model_std_quality: float¶

num_abandoned_trials: int¶

num_bayesopt_trials: int¶

num_completed_trials: int¶

num_early_stopped_trials: int¶

num_failed_trials: int¶

num_initialization_trials: int¶

num_metric_fetch_e_encountered: int¶

num_objectives: int¶

num_other_trials: int¶

num_outcome_constraints: int¶

num_tracking_metrics: int¶

num_trials_bad_due_to_err: int¶

total_fit_time: int¶

total_gen_time: int¶

unique_identifier: str¶

class ax.telemetry.optimization.OptimizationCreatedRecord(unique_identifier: str, owner: str, experiment_name: str | None, experiment_type: str | None, num_continuous_range_parameters: int, num_int_range_parameters_small: int, num_int_range_parameters_medium: int, num_int_range_parameters_large: int, num_log_scale_range_parameters: int, num_unordered_choice_parameters_small: int, num_unordered_choice_parameters_medium: int, num_unordered_choice_parameters_large: int, num_fixed_parameters: int, dimensionality: int, hierarchical_tree_height: int, num_parameter_constraints: int, num_objectives: int, num_tracking_metrics: int, num_outcome_constraints: int, num_map_metrics: int, metric_cls_to_quantity: dict[str, int], runner_cls: str, generation_strategy_name: str | None, num_requested_initialization_trials: int | None, num_requested_bayesopt_trials: int | None, num_requested_other_trials: int | None, max_parallelism: int | None, early_stopping_strategy_cls: str | None, global_stopping_strategy_cls: str | None, transformed_dimensionality: int | None, scheduler_total_trials: int | None, scheduler_max_pending_trials: int, arms_per_trial: int, product_surface: str, launch_surface: str, deployed_job_id: int, trial_evaluation_identifier: str | None, is_manual_generation_strategy: bool, warm_started_from: str | None, num_custom_trials: int, support_tier: str)[source]¶

Bases: object

Record of the “Optimization” creation event. This can come from either an AxClient or a Scheduler. This Record is especially useful for logging Ax-backed optimization results to a tabular database (i.e. one row per Record).

arms_per_trial: int¶

deployed_job_id: int¶

dimensionality: int¶

early_stopping_strategy_cls: str | None¶

experiment_name: str | None¶

experiment_type: str | None¶

classmethod from_ax_client(ax_client: AxClient, unique_identifier: str, owner: str, product_surface: str, launch_surface: str, deployed_job_id: int, trial_evaluation_identifier: str | None, is_manual_generation_strategy: bool, warm_started_from: str | None, num_custom_trials: int) → OptimizationCreatedRecord[source]¶

classmethod from_experiment(experiment: Experiment, generation_strategy: GenerationStrategy | None, unique_identifier: str, owner: str, product_surface: str, launch_surface: str, deployed_job_id: int, is_manual_generation_strategy: bool, num_custom_trials: int, warm_started_from: str | None = None, arms_per_trial: int | None = None, trial_evaluation_identifier: str | None = None) → OptimizationCreatedRecord[source]¶

classmethod from_scheduler(scheduler: Scheduler, unique_identifier: str, owner: str, product_surface: str, launch_surface: str, deployed_job_id: int, trial_evaluation_identifier: str | None, is_manual_generation_strategy: bool, warm_started_from: str | None, num_custom_trials: int, support_tier: str) → OptimizationCreatedRecord[source]¶

generation_strategy_name: str | None¶

global_stopping_strategy_cls: str | None¶

hierarchical_tree_height: int¶

is_manual_generation_strategy: bool¶

launch_surface: str¶

max_parallelism: int | None¶

metric_cls_to_quantity: dict[str, int]¶

num_continuous_range_parameters: int¶

num_custom_trials: int¶

num_fixed_parameters: int¶

num_int_range_parameters_large: int¶

num_int_range_parameters_medium: int¶

num_int_range_parameters_small: int¶

num_log_scale_range_parameters: int¶

num_map_metrics: int¶

num_objectives: int¶

num_outcome_constraints: int¶

num_parameter_constraints: int¶

num_requested_bayesopt_trials: int | None¶

num_requested_initialization_trials: int | None¶

num_requested_other_trials: int | None¶

num_tracking_metrics: int¶

num_unordered_choice_parameters_large: int¶

num_unordered_choice_parameters_medium: int¶

num_unordered_choice_parameters_small: int¶

owner: str¶

product_surface: str¶

runner_cls: str¶

scheduler_max_pending_trials: int¶

scheduler_total_trials: int | None¶

support_tier: str¶

transformed_dimensionality: int | None¶

trial_evaluation_identifier: str | None¶

unique_identifier: str¶

warm_started_from: str | None¶

Scheduler¶

class ax.telemetry.scheduler.SchedulerCompletedRecord(experiment_completed_record: ExperimentCompletedRecord, best_point_quality: float, model_fit_quality: float | None, model_std_quality: float | None, model_fit_generalization: float | None, model_std_generalization: float | None, improvement_over_baseline: float, num_metric_fetch_e_encountered: int, num_trials_bad_due_to_err: int)[source]¶

Bases: object

Record of the Scheduler completion event. This will have information only available after the optimization has completed.

best_point_quality: float¶

experiment_completed_record: ExperimentCompletedRecord¶

flatten() → dict[str, Any][source]¶

Flatten into an appropriate format for logging to a tabular database.

classmethod from_scheduler(scheduler: Scheduler) → SchedulerCompletedRecord[source]¶

improvement_over_baseline: float¶

model_fit_generalization: float | None¶

model_fit_quality: float | None¶

model_std_generalization: float | None¶

model_std_quality: float | None¶

num_metric_fetch_e_encountered: int¶

num_trials_bad_due_to_err: int¶

class ax.telemetry.scheduler.SchedulerCreatedRecord(experiment_created_record: ExperimentCreatedRecord, generation_strategy_created_record: GenerationStrategyCreatedRecord, scheduler_total_trials: int | None, scheduler_max_pending_trials: int, arms_per_trial: int, early_stopping_strategy_cls: str | None, global_stopping_strategy_cls: str | None, transformed_dimensionality: int | None)[source]¶

Bases: object

Record of the Scheduler creation event. This can be used for telemetry in settings where many Schedulers are being created either manually or programatically. In order to facilitate easy serialization only include simple types: numbers, strings, bools, and None.

arms_per_trial: int¶

early_stopping_strategy_cls: str | None¶

experiment_created_record: ExperimentCreatedRecord¶

flatten() → dict[str, Any][source]¶

Flatten into an appropriate format for logging to a tabular database.

classmethod from_scheduler(scheduler: Scheduler) → SchedulerCreatedRecord[source]¶

generation_strategy_created_record: GenerationStrategyCreatedRecord¶

global_stopping_strategy_cls: str | None¶

scheduler_max_pending_trials: int¶

scheduler_total_trials: int | None¶

transformed_dimensionality: int | None¶