c-btm inference #50
c-btm inference #50
Conversation
inference code for c-btm
|
Hi @NourFahmy,
The script should load the models, run inference on the input data and save the results. This would allow us to evaluate the performance of the method using perplexity and also on downstream tasks. It would be very helpful if the PR would solve the end to end. It's possible to break this up into a few PR's, if you prefer. |
|
Hi @mrcabbage972 - thank you for your feedback! Will update accordingly by Wednesday. |
|
kindly note, still needs to be tested! as I understand the sequence of tasks that need to be implemented by for c-btm inference are:
cc: @mrcabbage972 |
|
@NourFahmy @mrcabbage972 |
|
Where are we on this? @NourFahmy @kenhktsui @mrcabbage972 ? |
pass in embedder for prompt
| tokenizers = [] | ||
|
|
||
| for model_name in model_names: | ||
| model = AutoModelForCausalLM.from_pretrained(model_name) |
There was a problem hiding this comment.
some issues with loading the models and maintaining HF credentials -- had to load models and tokenizers outside of the function
There was a problem hiding this comment.
Ok. good to know. Strange that you can't load. I made all models public now
scripts/c-btmInference.py
Outdated
| inputs = tokenizer(prompt) | ||
| print(inputs['input_ids']) | ||
| sizeOfInputs = len(inputs['input_ids']) | ||
| outputs = model(**inputs, max_new_tokens=1, |
There was a problem hiding this comment.
max_new_tokens not a parameter of geoptx -- how can I limit the number of tokens
|
I merged it. you can keep adding to it as another PR. |
inference code for c-btm - replicating formulas 2 & 3 from c-BTM paper, and tested locally.
kindly inform if anything else is needed!
link to #40