New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Getting all negative predictions when fine-tune my data #154
Comments
Do you shuffle your data? |
Thanks for reminding. I forgot to shuffle the data. I'll update the result once the model's trained on the new dataset. Thanks again! |
The model performs perfectly after shuffling the dataset. Thanks! I am closing it. INFO:tensorflow: eval_accuracy = 0.97306395 |
Even after shuffling my model predicting only positive i,e 2 column has high probability, My data set has question and answer pairs with their label, i,e label is 1 if question and answer matches ,else 0. Am doing anything wrong here,tired with different sequence lengths |
@yanfan0531 Does shuffling the dataset mean that all the examples of a class not be in a serial order ..i.e.all class o examples followed by all class 1 examples and so on. Instead we should have examples in train.tsv mixed e.g. few of class 0 and then of class 1 ,then few of class 0 and so on. Actually, my bert classifier is always predicting one particular class for all examples in test dataset and my training dataset examples are in a serial order, not mixed order |
@anubhavpnp Hi, you are right about the shuffling. You should try mixing the order of labels in training data, and it solved my problem. |
Thanks for your help @yanfan0531 ..Shuffling solved my issue and bert is now giving correct predictions. I read a bit more and shuffling is usually a pre requisite for training a model in neural networks. |
How do we get probabilities [0.0,1.0] ? Do we need to add an additional softmax as per #322? |
Are the probabilities returned for each class in the range [0.0,1.0]? Not even the example colab returns normalised probabilities. |
yes..it is between 0 and 1 |
Thanks for confirming. Is your code the same as on the example colab ? |
I am using code which is here..https://github.com/winstarwang/rasa_nlu_bert |
I don't see any difference with the official google bert model function, so I'm not too sure as to why probabilities are not in the [0,1] range in the example colab... And in my code. Relevant bits: |
Hi, I'm following the fine-tuning codes on my own dataset, which is a sentence pair classification task. All the parameters are the same as the example code. However, I get all negative predictions when doing the evaluation. Any ideas of what happened?
The code:
The evaluation result:
INFO:tensorflow:***** Eval results *****
INFO:tensorflow: eval_accuracy = 0.64309764
INFO:tensorflow: eval_fn = 106.0
INFO:tensorflow: eval_fp = 0.0
INFO:tensorflow: eval_loss = 1.2086438
INFO:tensorflow: eval_precision = 0.0
INFO:tensorflow: eval_recall = 0.0
INFO:tensorflow: eval_tn = 191.0
INFO:tensorflow: eval_tp = 0.0
INFO:tensorflow: global_step = 83
INFO:tensorflow: loss = 1.1816607
The text was updated successfully, but these errors were encountered: