Skip to content

Feature Idea: Pre-Tuning Architectural Validation #1

@davidrsch

Description

@davidrsch

When using tune to search over architectural parameters (e.g., num_dense), it's easy to create a grid with combinations that are architecturally impossible (e.g., a conv2d layer following a dense layer without reshaping). Currently, tune_grid() attempts to fit these models, fails with an error, and moves on. This is computationally wasteful and clutters the console with errors.

This should ben addressed by introducing a new set of functions that allow for a "pre-flight check" of the tuning grid.

How it would work:

  1. compile_keras_grid(): A new function that iterates through a tuning grid and performs a "dry run" of model construction and compilation for each hyperparameter combination. It returns a tibble with the compiled model, a summary of the model's architecture, and any errors that occurred during compilation.

  2. extract_valid_grid(): A helper function that takes the results of compile_keras_grid() and returns a new grid containing only the hyperparameter combinations that compiled successfully.

  3. inform_errors(): A helper function that takes the results of compile_keras_grid() and prints a formatted summary of any compilation errors that occurred.

The current alternative is to let tune_grid() fail on invalid architectures, which is inefficient.

Acceptance Criteria

  • Create and export the compile_keras_grid() function.
  • Create and export the extract_valid_grid() and inform_errors() helper functions.
  • Ensure compile_keras_grid() correctly identifies and reports invalid parameter sets.
  • The extract_valid_grid() function must return a cleaned grid of only valid combinations.
  • The inform_errors() function must return a report of the removed sets and the corresponding errors.
  • Add unit tests for the new functionality.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions