Fix pdf() function

[klemens-floege]

#796 highlighted that we might have a problem with the pdf() function that returns an inverse probability as opposed to the actual probability.

This PR now fixes the pdf() function, the function is not used in our repo, since it is external API we could just leave it in?

TabPFN) klemensfloege@Klemenss-MacBook-Pro TabPFN % python3 reproduce_pdf_bug.py
forward() (NLL = -log p(y)): 1.092203
Buggy pdf()  = exp(NLL):     2.980833
Correct pdf  = exp(-NLL):    0.335477

buggy_pdf * correct_pdf = 1.000000  (should be 1.0 if they are inverses)

my script:

"""Reproduce the pdf() bug in FullSupportBarDistribution.

The forward() method returns the negative log density (NLL loss), i.e. -log(p(y)).
Therefore:
    pdf() = exp(forward(logits, y)) = exp(-log(p(y))) = 1/p(y)   <-- WRONG
    pdf() = exp(-forward(logits, y)) = exp(log(p(y))) = p(y)      <-- CORRECT
"""

import torch
from tabpfn.architectures.base.bar_distribution import FullSupportBarDistribution

# Create a simple FullSupportBarDistribution with uniform-width buckets
borders = torch.linspace(-3.0, 3.0, 102)  # 101 buckets
dist = FullSupportBarDistribution(borders)

# Create fake logits (batch of 1) and a target value
logits = torch.randn(1, dist.num_bars)
y = torch.tensor([0.5])

# Get the NLL from forward()
nll = dist.forward(logits, y)
print(f"forward() (NLL = -log p(y)): {nll.item():.6f}")

# Current (buggy) pdf: exp(NLL) = exp(-log p(y)) = 1/p(y)
buggy_pdf = dist.pdf(logits, y)
print(f"Buggy pdf()  = exp(NLL):     {buggy_pdf.item():.6f}")

# Correct pdf: exp(-NLL) = exp(log p(y)) = p(y)
correct_pdf = torch.exp(-nll)
print(f"Correct pdf  = exp(-NLL):    {correct_pdf.item():.6f}")

# Verify: buggy_pdf * correct_pdf should equal 1 (they are inverses)
product = buggy_pdf * correct_pdf
print(f"\nbuggy_pdf * correct_pdf = {product.item():.6f}  (should be 1.0 if they are inverses)")

# Sanity check: a density value > 100 or < 1e-6 is suspicious but possible;
# however the product being 1 proves they are exact inverses.
assert torch.allclose(product, torch.tensor(1.0), atol=1e-5), (
    "Bug confirmed: pdf() returns 1/p(y) instead of p(y)"
)
print("\nBug confirmed: pdf() and correct_pdf are inverses of each other.")
print("The fix: change `return torch.exp(self.forward(logits, y))`")
print("     to: `return torch.exp(-self.forward(logits, y))`")

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[priorjulien]

LGTM!