Some weights of the model checkpoint at ckpt/pretrain.pt were not used when initializing RobertaModel: ['lm_head.layer_norm.bias', 'lm_head.layer_norm.weight', 'lm_head.bias', 'lm_head.dense.bias', 'lm_head.dense.weight']
- This IS expected if you are initializing RobertaModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of RobertaModel were not initialized from the model checkpoint at ckpt/pretrain.pt and are newly initialized: ['roberta.pooler.dense.weight', 'roberta.pooler.dense.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'RobertaTokenizer'. 
The class this function is called from is 'PolymerSmilesTokenizer'.
{'CV_flag': False, 'add_vocab_flag': True, 'LLRD_flag': False, 'aug_flag': False, 'aug_special_flag': False, 'model_indicator': 'pretrain', 'aug_indicator': None, 'vocab_sup_file': 'data/vocab/vocab_sup_PE_I.csv', 'train_file': 'data/train_PE_I.csv', 'test_file': 'data/test_PE_I.csv', 'model_path': 'ckpt/pretrain.pt', 'save_path': 'ckpt/PE_I_train.pt', 'best_model_path': 'ckpt/PE_I_best_model.pt', 'k': 5, 'blocksize': 411, 'batch_size': 32, 'num_epochs': 20, 'warmup_ratio': 0.05, 'drop_rate': 0.1, 'lr_rate': 5e-05, 'lr_rate_reg': 0.0001, 'weight_decay': 0.01, 'hidden_dropout_prob': 0.1, 'attention_probs_dropout_prob': 0.1, 'tolerance': 5, 'num_workers': 8}
Use the pretrained model
Train Test Split
(2, 1) (2, 1)
Traceback (most recent call last):
  File "/workspace/cxliang/TransPolymer/Downstream.py", line 459, in <module>
    main(finetune_config)
  File "/workspace/cxliang/TransPolymer/Downstream.py", line 364, in main
    train_data.iloc[:, 1] = scaler.fit_transform(train_data.iloc[:, 1].values.reshape(-1, 1))
  File "/workspace/cxliang/conda/envs/TransPolymer/lib/python3.9/site-packages/pandas/core/indexing.py", line 1147, in __getitem__
    return self._getitem_tuple(key)
  File "/workspace/cxliang/conda/envs/TransPolymer/lib/python3.9/site-packages/pandas/core/indexing.py", line 1652, in _getitem_tuple
    tup = self._validate_tuple_indexer(tup)
  File "/workspace/cxliang/conda/envs/TransPolymer/lib/python3.9/site-packages/pandas/core/indexing.py", line 940, in _validate_tuple_indexer
    self._validate_key(k, i)
  File "/workspace/cxliang/conda/envs/TransPolymer/lib/python3.9/site-packages/pandas/core/indexing.py", line 1554, in _validate_key
    self._validate_integer(key, axis)
  File "/workspace/cxliang/conda/envs/TransPolymer/lib/python3.9/site-packages/pandas/core/indexing.py", line 1647, in _validate_integer
    raise IndexError("single positional indexer is out-of-bounds")
IndexError: single positional indexer is out-of-bounds