Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What should I do if I want a blank, untrained ColBRET? #206

Closed
jmnian opened this issue Apr 24, 2024 · 1 comment
Closed

What should I do if I want a blank, untrained ColBRET? #206

jmnian opened this issue Apr 24, 2024 · 1 comment
Labels
question Further information is requested

Comments

@jmnian
Copy link

jmnian commented Apr 24, 2024

I have some data. My goal is to train from a vanilla version of ColBERT that uses the bert-base-uncased as the encoder, and the linear layer parameters are randomly initialized. I would love to leverage RAGatouille's utilities for the ease of training. Thank you for pointing me to the right direction.
When initializing the trainer I'm using this:
trainer = RAGTrainer(model_name=model_name, pretrained_model_name = "colbert-ir/colbertv1.9", language_code='en', n_usable_gpus=1 )

Does colbertv1.9 mean I am using an untrained ColBERT? Is there any way I can use RAGatouille to accomplish my goal?

@bclavie
Copy link
Owner

bclavie commented May 2, 2024

Hey! The way to start the trainer from a completely untrained ColBERT would be just specifying whatever base model you want to use as an argument to pretrained_model_name. So in your case, you'd run it as pretrained_model_name='bert-base-uncased'.

@bclavie bclavie closed this as completed May 2, 2024
@bclavie bclavie added the question Further information is requested label May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants