Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

from_pretrained should just work for DecisionGPT2LMHeadModel #2

Open
thejaminator opened this issue Feb 26, 2023 · 0 comments
Open
Labels
good first issue Good for newcomers

Comments

@thejaminator
Copy link
Owner

thejaminator commented Feb 26, 2023

Currently to create a model that accept the scalar reward, you need to do this if you want to use pretrained gpt2.

from transformers import GPT2LMHeadModel
from conditionme import DecisionGPT2LMHeadModel

loaded_model = GPT2LMHeadModel.from_pretrained("gpt2")
decision_model = DecisionGPT2LMHeadModel.from_loaded_pretrained_model(loaded_model)

rather than just

decision_model = DecisionGPT2LMHeadModel.from_loaded_pretrained_model("gpt2")

We should allow users to just pass "gpt2". And under the hood, we'll detect that its a pretrained gpt2 model.

Case 1 -
Its a DecisionGPT2LMHeadModel that got saved.
We should retrain the original behavior and load all weights

Case 2 -
Its a gpt2 pretrained model that doesn't have our embed_return layer.
We'll log that we detected that, and we'll randomly initialize our self.embed_return layer.
Write tests for this behavior :)

Case 3 -
Neither of these scenarios.
We should throw an exception here.
Write tests for this behavior :)

@thejaminator thejaminator added the good first issue Good for newcomers label Feb 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant