Would it be better to return the best validation result in the objective function?

[pomelyu]

Hello,

I am new to optuna, and I am confused about the return value of the objective function.
In this pytorch example

def objective():
    for epoch in range(EPOCHS):
        ... # training step
        ... # validation step
        trial.report(accuracy, epoch)

        # Handle pruning based on the intermediate value.
        if trial.should_prune():
            raise optuna.exceptions.TrialPruned()

    return accuracy

The program executes trial.report in every epoch to determine whether this trial should be pruned or not. It returns the last accuracy value upon completion of the trial. However, there is confusion regarding the return value. As far as I understand, the validation metric may fluctuate or even bounce back during training. It would be more logical to return the best validation metric instead of the last one. Is there a specific reason for choosing the last result over the best validation result or is there something I am missing?

Thank you

[nzw0301]

Your understanding is right to use the last value rather than the best value. This one needs to be discussed #3542

[pomelyu]

No, I mean it would be better to use best value than the last value.

[pomelyu]

Thanks for reply. I also notice that Sampler use the value return by objective function(stored in trial.value) to choose the next hyper-parameter. It is reasonable to use these potentially suboptimal value to suggest the next trial?

[nzw0301]

Sorry for my previous unclear response. I meant your understanding of the implementation. Personally, I agree that using the best value instead of the last value, but I don't know why this rule has been used.