Inference value is different from train value

[unolife]

Describe the bug
when I train model, I got this kind of prediction:

      0_probability  1_probability  prediction
0          0.849047       0.150953           0
1          0.847665       0.152334           0
2          0.848109       0.151891           0
3          0.849097       0.150903           0
4          0.843085       0.156915           0
...             ...            ...         ...
4399       0.819843       0.180157           0
4400       0.656908       0.343092           0
4401       0.831712       0.168288           0
4402       0.778614       0.221386           0
4403       0.848845       0.151155           0

but when I do inference, I got this kind of result:

tensor([[ 1.2595, -0.8684],
        [ 2.6755, -1.3752],
        [ 2.2348, -1.2487],
        ...,
        [ 3.6170, -1.6117],
        [ 1.2994, -0.9014],
        [ 1.6549, -1.0630]], grad_fn=<AddmmBackward0>)

why suddenly - value(<0) poped up??

here is my train code

continuous_cols = train_x.select_dtypes(include=['float64', 'int64']).columns.tolist()
categorical_cols = train_x.select_dtypes(include=['object', 'category']).columns.tolist()
target_column = params.get('target_column')
data_config = DataConfig(
            target=[target_column],
            continuous_cols=continuous_cols,
            categorical_cols=categorical_cols,
        )
optimizer = params.get('optimizer')
if optimizer is None:
     optimizer = 'Adam'
optimizer_config = OptimizerConfig(
            optimizer=optimizer,
            lr_scheduler=params.get('lr_scheduler'),
        )
trainer_config = TrainerConfig(
            auto_lr_find=params.get('auto_lr_find'),
            batch_size=params.get('batch_size'),
            max_epochs=params.get('max_epochs'),
            seed=params.get('random_state'),
            early_stopping_patience=params.get('early_stopping_patience'),
        )
tabular_model = TabularModel(
            data_config=data_config,
            model_config=model_config,
            optimizer_config=optimizer_config,
            trainer_config=trainer_config,
        )
train = pd.concat([train_x, pd.DataFrame(train_y, columns=[target_column])], axis=1)
valid = pd.concat([valid_x, pd.DataFrame(valid_y, columns=[target_column])], axis=1)
tabular_model.fit(train=train, validation=valid)
res = tabular_model.predict(valid)
print(res)
 tabular_model.save_model_for_inference(path=f"./model.pt")

---

and here is my inference code

prediction_model = torch.load('./model.pt')
prediction_model.eval()
continuous_cols = before_fe_x.select_dtypes(include=['float64', 'int64']).columns.tolist()
categorical_cols = before_fe_x.select_dtypes(include=['object', 'category']).columns.tolist()
x = {"continuous": torch.tensor(before_fe_x[continuous_cols].values, dtype=torch.float32), "categorical": torch.tensor(before_fe_x[categorical_cols].values, dtype=torch.float32)}
res = prediction_model.predict(x)
print(res)

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

[manujosephv]

The barebones PyTorch model outputs the logits of the model, without the final activation. I this case, a softmax. I guess we should make it clear in the save_inference method. I'll add the documentation update. In your case, can you try adding a softmax and see if that works out?

Another thing I noticed is that you aren't using the datamodule during inference. In that case, all the transformations (like normalization) which happens to the input data isn't done and the output of the network would be different.

[manujosephv]

Follow up comment:
here is my fixed code to activate softmax

model = torch.load('./model.pt')
model.eval()
continuous_cols = df.select_dtypes(include=['float64', 'int64']).columns.tolist()
categorical_cols = df.select_dtypes(include=['object', 'category']).columns.tolist()
x = {
    "continuous": torch.tensor(df[continuous_cols].values, dtype=torch.float32), 
    "categorical": torch.tensor(df[categorical_cols].values, dtype=torch.float32)
    }
logits = model.predict(x)
res = F.softmax(logits, dim=1)

But Idk how can I apply datamodule in this code, can you give me some advice please??

[manujosephv]

so, instead of constructing a dictionary directly from the dataframe, you should be using the datamodule.prepare_inference_dataloader to create a dataloader with desired batch size, and then iterate over that dataloader to do predictions.
Here is the simplified code which is used in the TabularModel.predict

point_predictions = []
for batch in progress_bar(inference_dataloader):
    for k, v in batch.items():
        if isinstance(v, list) and (len(v) == 0):
            continue  # Skipping empty list
        batch[k] = v.to(model.device)
        y_hat=F.softmax(model.predict(batch), dim=1)
        point_predictions.append(y_hat.detach().cpu())
point_predictions = torch.cat(point_predictions, dim=0)

[manujosephv]

closing the issue... feel free to re-open if its not solved