Repository navigation
Replies: 2 comments
|
Your learning rate is almost certainly the issue — Things to try, in order:
Try the LR change first — that's the single most likely fix. |
|
I don't think the input shape is the problem. The first thing I would change is your learning rate: optimizer = torch.optim.AdamW(model_0.parameters(), lr=1e-3)
I would also normalize/standardize both your input features and target values before training. Your inputs are very small (around You also don't need: X_train.requires_grad_(True)
X_test.requires_grad_(True)Gradients are needed for the model parameters, not your input data, so you can remove those lines. I would start with a simpler model too: model_0 = ForexModelV0(
input_size=3,
hidden_size=32,
output_size=1,
layers=1
)
optimizer = torch.optim.AdamW(model_0.parameters(), lr=1e-3)Then train it for more than 3 epochs and watch whether the loss is actually decreasing. One useful test is to try to overfit a very small amount of data, for example one batch. If the model cannot learn one batch after many iterations, then there is likely still a problem with the training setup or data. If it can, then the model itself is probably fine and you can look at scaling/data quality next. So I would try, in order: lower the learning rate, normalize the data, remove If this resolves your issue, please mark it as the answer. 🙂 |
Uh oh!
There was an error while loading. Please reload this page.
I have an LSTM model: Problem is the evaluation returns stuck values.
Context: I am running on Jupyter Lab
First this is the Dataset that I made.
Data loader and data setup
Now for the model:
And these are the loss, accuracy and optimizer:
Then the training loop:
Now after:
No matter what the input, it is stuck on a constant output:
Change it to 4, 6? Sure:

And yes, I do run the cell lol.
And no matter what I do, it's still stuck, much like me in this project.
And yes, I do check the value of X_sample_1 and X_sample_2
Typically, the shape of either were (30, 3) but the input required by my LSTM model is (batch, 30, 3), that's why the unsqueeze on the 0th index. On training, its data's shape was (32, 30, 3) (I like it more when it's batch first, feats last).
Solutions I've tried
You might wonder:
All reactions