Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kobert hidden states vector 추출 에러 #91

Open
shinebin0501 opened this issue Sep 11, 2022 · 0 comments
Open

kobert hidden states vector 추출 에러 #91

shinebin0501 opened this issue Sep 11, 2022 · 0 comments
Labels
bug Something isn't working

Comments

@shinebin0501
Copy link

kobert를 활용하여 분류 학습 후 파이토치 모델로 저장하였습니다
fine-tuning 후 새로운 데이터를 입력으로 주어 모델에 output중 hidden_states 값을 사용하려고합니다.
transformer BertForSequenceClassification.from_pretrained()를 사용하여 fine_tuning을 진행하였고,
옵션으로 output_hidden_states = True로 설정한 뒤 모델을 다시 로드하여 데이터셋을 주고, output을 출력하면
1 12 512 768가 아닌 1 12 512 512으로 768차원에 embedding vector을 출력하지 못하고 있습니다.
kobert fine_tuning model의 hidden_states를 사용하여 문장들의 CLS token vector를 얻고 싶은데,
output을 어떻게 설정하고 가져와야하는 것인지 설명해 주시면 정말 감사드리겠습니다.

@shinebin0501 shinebin0501 added the bug Something isn't working label Sep 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant