Skip to content

Evaluation with different thresholds give the same results #23

Open
@mikel-brostrom

Description

@mikel-brostrom

This piece of code :

for x in np.arange(0.6, 0.9, 0.1):
    print('Eval with TH:', x)
    metrics = [
        cdp.utils.metrics.Fscore(activation='argmax2d', threshold=x),
        cdp.utils.metrics.Precision(activation='argmax2d', threshold=x),
        cdp.utils.metrics.Recall(activation='argmax2d', threshold=x),
    ]

    valid_epoch = cdp.utils.train.ValidEpoch(
        model,
        loss=loss,
        metrics=metrics,
        device=DEVICE,
        verbose=True,
    )

    valid_logs = valid_epoch.run(valid_loader)
    print(valid_logs)

Give me the following result:

Eval with TH: 0.6
valid: 100%|█████████████████████████████████████████████████████████████| 505/505 [01:12<00:00,  6.98it/s, cross_entropy_loss - 0.08708, fscore - 0.8799, precision - 0.8946, recall - 0.8789]
{'cross_entropy_loss': 0.0870812193864016, 'fscore': 0.8798528309538921, 'precision': 0.8946225793644936, 'recall': 0.8789094516579565}

Eval with TH: 0.7
valid: 100%|█████████████████████████████████████████████████████████████| 505/505 [01:12<00:00,  6.99it/s, cross_entropy_loss - 0.08708, fscore - 0.8799, precision - 0.8946, recall - 0.8789]
{'cross_entropy_loss': 0.08708121913835626, 'fscore': 0.8798528309538921, 'precision': 0.8946225793644936, 'recall': 0.8789094516579565}

Eval with TH: 0.7999999999999999
valid: 100%|█████████████████████████████████████████████████████████████| 505/505 [01:11<00:00,  7.02it/s, cross_entropy_loss - 0.08708, fscore - 0.8799, precision - 0.8946, recall - 0.8789]
{'cross_entropy_loss': 0.08708121978843793, 'fscore': 0.8798528309538921, 'precision': 0.8946225793644936, 'recall': 0.8789094516579565}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions