Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

failed to load bart model: unexpected EOF #24

Closed
ior308 opened this issue Jun 27, 2023 · 20 comments
Closed

failed to load bart model: unexpected EOF #24

ior308 opened this issue Jun 27, 2023 · 20 comments

Comments

@ior308
Copy link

ior308 commented Jun 27, 2023

Dear Matteo,

i try to build abstract q&a but I got this error:

...
...
{"level":"debug","time":"2023-06-27T19:58:00+02:00","message":"1.45 GiB of 1.51 GiB (95%) downloaded"}
{"level":"debug","time":"2023-06-27T19:58:03+02:00","message":"1.46 GiB of 1.51 GiB (96%) downloaded"}
{"level":"debug","time":"2023-06-27T19:58:06+02:00","message":"1.47 GiB of 1.51 GiB (97%) downloaded"}
{"level":"debug","time":"2023-06-27T19:58:09+02:00","message":"1.48 GiB of 1.51 GiB (97%) downloaded"}
{"level":"debug","time":"2023-06-27T19:58:12+02:00","message":"1.49 GiB of 1.51 GiB (98%) downloaded"}
{"level":"debug","time":"2023-06-27T19:58:15+02:00","message":"1.50 GiB of 1.51 GiB (99%) downloaded"}
{"level":"debug","time":"2023-06-27T19:58:18+02:00","message":"1.51 GiB of 1.51 GiB (99%) downloaded"}
{"level":"debug","time":"2023-06-27T19:58:18+02:00","message":"1.51 GiB (100%) downloaded"}
{"level":"debug","url":"https://huggingface.co/vblagoje/bart_lfqa/resolve/main/vocab.json","destination":"models/vblagoje/bart_lfqa/vocab.json","time":"2023-06-27T19:58:18+02:00","message":"downloading"}
{"level":"debug","time":"2023-06-27T19:58:19+02:00","message":"877.76 KiB (100%) downloaded"}
{"level":"debug","url":"https://huggingface.co/vblagoje/bart_lfqa/resolve/main/merges.txt","destination":"models/vblagoje/bart_lfqa/merges.txt","time":"2023-06-27T19:58:19+02:00","message":"downloading"}
{"level":"debug","time":"2023-06-27T19:58:20+02:00","message":"445.62 KiB (100%) downloaded"}
{"level":"trace","time":"2023-06-27T19:58:32+02:00","message":"Reporting possible conversion mapping anomalies"}
{"level":"trace","parameter":"model.decoder.layer_norm.bias","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"model.decoder.layer_norm.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"classification_head.dense.bias","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"classification_head.out_proj.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"classification_head.dense.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"model.encoder.layer_norm.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"classification_head.out_proj.bias","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"model.encoder.layer_norm.bias","time":"2023-06-27T19:58:32+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"model.encoder.embed_positions.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not mapped"}
{"level":"trace","parameter":"lm_head.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not mapped"}
{"level":"trace","parameter":"model.encoder.embed_tokens.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not mapped"}
{"level":"trace","parameter":"model.decoder.embed_tokens.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not mapped"}
{"level":"trace","parameter":"model.decoder.embed_positions.weight","time":"2023-06-27T19:58:32+02:00","message":"parameter not mapped"}
Serializing model to "models/vblagoje/bart_lfqa/spago_model.bin"... signal: killed

I tried again, and I got:

[root@minix8 cybertron@v0.1.2]# GOARCH=amd64 CYBERTRON_MODEL=Helsinki-NLP/opus-mt-en-it CYBERTRON_MODELS_DIR=models go run ./examples/abstractivequestionasnwering/main.go
{"level":"debug","file":"models/vblagoje/bart_lfqa/config.json","time":"2023-06-27T19:59:17+02:00","message":"model file already exists, skipping download"}
{"level":"debug","file":"models/vblagoje/bart_lfqa/pytorch_model.bin","time":"2023-06-27T19:59:17+02:00","message":"model file already exists, skipping download"}
{"level":"debug","file":"models/vblagoje/bart_lfqa/vocab.json","time":"2023-06-27T19:59:17+02:00","message":"model file already exists, skipping download"}
{"level":"debug","file":"models/vblagoje/bart_lfqa/merges.txt","time":"2023-06-27T19:59:17+02:00","message":"model file already exists, skipping download"}
{"level":"info","model":"models/vblagoje/bart_lfqa/spago_model.bin","time":"2023-06-27T19:59:17+02:00","message":"model file already exists, skipping conversion"}
{"level":"fatal","error":"failed to load bart model: unexpected EOF","time":"2023-06-27T19:59:17+02:00"}
exit status 1

Any ideas ?

Thank you.

@mooijtech
Copy link

Please see the tags of a model on HuggingFace before using:
out
This model supports textgeneration:
CYBERTRON_MODELS_DIR=models/ CYBERTRON_MODEL=vblagoje/bart_lfqa go run examples/textgeneration/main.go

"signal: killed" indicates you have run out of memory while converting the model.

It seems the config.json of this model is missing max_length tokens, so it will not work.

@ior308
Copy link
Author

ior308 commented Jun 28, 2023

Yes, I launch the binary and it starts downloading the model. The config.json is present in models/vblagoje/bart_lfqa/config.json
-rw-r--r-- 1 root root 1321 Jun 28 10:15 models/vblagoje/bart_lfqa/config.json

and "max_length" is 142.

My VM has 8GB but maybe it needs more..

{"level":"debug","time":"2023-06-28T10:22:39+02:00","message":"1.51 GiB of 1.51 GiB (99%) downloaded"}
{"level":"debug","time":"2023-06-28T10:22:41+02:00","message":"1.51 GiB (100%) downloaded"}
{"level":"debug","url":"https://huggingface.co/vblagoje/bart_lfqa/resolve/main/vocab.json","destination":"models/vblagoje/bart_lfqa/vocab.json","time":"2023-06-28T10:22:41+02:00","message":"downloading"}
{"level":"debug","time":"2023-06-28T10:22:43+02:00","message":"877.76 KiB (100%) downloaded"}
{"level":"debug","url":"https://huggingface.co/vblagoje/bart_lfqa/resolve/main/merges.txt","destination":"models/vblagoje/bart_lfqa/merges.txt","time":"2023-06-28T10:22:43+02:00","message":"downloading"}
{"level":"debug","time":"2023-06-28T10:22:44+02:00","message":"445.62 KiB (100%) downloaded"}
{"level":"trace","time":"2023-06-28T10:23:21+02:00","message":"Reporting possible conversion mapping anomalies"}
{"level":"trace","parameter":"model.encoder.layer_norm.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"classification_head.dense.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"classification_head.out_proj.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"classification_head.dense.bias","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"model.decoder.layer_norm.bias","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"classification_head.out_proj.bias","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"model.decoder.layer_norm.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"model.encoder.layer_norm.bias","time":"2023-06-28T10:23:21+02:00","message":"parameter not initialized"}
{"level":"trace","parameter":"lm_head.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not mapped"}
{"level":"trace","parameter":"model.decoder.embed_positions.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not mapped"}
{"level":"trace","parameter":"model.decoder.embed_tokens.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not mapped"}
{"level":"trace","parameter":"model.encoder.embed_positions.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not mapped"}
{"level":"trace","parameter":"model.encoder.embed_tokens.weight","time":"2023-06-28T10:23:21+02:00","message":"parameter not mapped"}
Serializing model to "models/vblagoje/bart_lfqa/spago_model.bin"... Killed
[root@minix8 cybertron@v0.1.2]# GOARCH=amd64 CYBERTRON_MODELS_DIR=models go build examples/abstractivequestionasnwering/main.go

@ior308
Copy link
Author

ior308 commented Jun 28, 2023

I increased RAM to 16GB and the spago serializing succeed, but then I got this error message:

{"level":"trace","parameter":"model.decoder.embed_positions.weight","time":"2023-06-28T19:11:39+02:00","message":"parameter not mapped"}
{"level":"trace","parameter":"model.decoder.embed_tokens.weight","time":"2023-06-28T19:11:39+02:00","message":"parameter not mapped"}
Serializing model to "models/vblagoje/bart_lfqa/spago_model.bin"... Done.

panic: input sequence too long: 145 > 0

goroutine 1 [running]:
main.main()
/root/go/pkg/mod/github.com/nlpodyssey/cybertron@v0.1.2/examples/abstractivequestionasnwering/main.go:52 +0x50e

If I launch again, the same error message:

[root@minix8 cybertron@v0.1.2]# ./main
{"level":"debug","file":"models/vblagoje/bart_lfqa/config.json","time":"2023-06-28T19:14:11+02:00","message":"model file already exists, skipping download"}
{"level":"debug","file":"models/vblagoje/bart_lfqa/pytorch_model.bin","time":"2023-06-28T19:14:11+02:00","message":"model file already exists, skipping download"}
{"level":"debug","file":"models/vblagoje/bart_lfqa/vocab.json","time":"2023-06-28T19:14:11+02:00","message":"model file already exists, skipping download"}
{"level":"debug","file":"models/vblagoje/bart_lfqa/merges.txt","time":"2023-06-28T19:14:11+02:00","message":"model file already exists, skipping download"}
{"level":"info","model":"models/vblagoje/bart_lfqa/spago_model.bin","time":"2023-06-28T19:14:11+02:00","message":"model file already exists, skipping conversion"}
panic: input sequence too long: 145 > 0

goroutine 1 [running]:
main.main()
/root/go/pkg/mod/github.com/nlpodyssey/cybertron@v0.1.2/examples/abstractivequestionasnwering/main.go:52 +0x50e

@mooijtech
Copy link

Yes that's a bug due to config.json not specifying max_length in the root.
I set those values manually but then I get an error about slice length.

@ior308
Copy link
Author

ior308 commented Jun 28, 2023

I'm using this model:

./models/vblagoje/bart_lfqa/config.json

and config.json contains max_length=142

@mooijtech
Copy link

I know, but the code doesn't look at task_specific_params.summarization.max_length.

@ior308
Copy link
Author

ior308 commented Jun 29, 2023

Ok, so what do I need to do to make this example work ?

Do you know which model to use for the Q&A example ?

@ior308
Copy link
Author

ior308 commented Jun 29, 2023

I downloaded models/deepset/bert-base-cased-squad2/ and now Q&A example works, BUT I don't understand the output. I supposed it should be able to answer to my question based on paragraph content..

./main
{"level":"debug","file":"models/deepset/bert-base-cased-squad2/config.json","time":"2023-06-29T07:08:52+02:00","message":"model file already exists, skipping download"}
{"level":"debug","file":"models/deepset/bert-base-cased-squad2/pytorch_model.bin","time":"2023-06-29T07:08:52+02:00","message":"model file already exists, skipping download"}
{"level":"debug","file":"models/deepset/bert-base-cased-squad2/vocab.txt","time":"2023-06-29T07:08:52+02:00","message":"model file already exists, skipping download"}
{"level":"debug","file":"models/deepset/bert-base-cased-squad2/tokenizer_config.json","time":"2023-06-29T07:08:52+02:00","message":"model file already exists, skipping download"}
{"level":"info","model":"models/deepset/bert-base-cased-squad2/spago_model.bin","time":"2023-06-29T07:08:52+02:00","message":"model file already exists, skipping conversion"}
paragraph.txt

what is artificial intelligence ?
0.249498236
{
"Answers": [
{
"Text": "txt",
"Start": 10,
"End": 13,
"Score": 0.4198876741953599
},
{
"Text": "paragraph.txt",
"Start": 0,
"End": 13,
"Score": 0.2049869292790566
},
{
"Text": ".txt",
"Start": 9,
"End": 13,
"Score": 0.15559880539241716
}
]
}

@mooijtech
Copy link

The t5 models may be more suited but are currently unsupported.

@ior308
Copy link
Author

ior308 commented Jun 30, 2023

Ok, so I don't understand if this project is still maintained. What I should do to test Abstract Q&A examples over a text ??? :)

@mooijtech
Copy link

Try: https://github.com/nlpodyssey/verbaflow
But I don't think you have enough memory.

@ior308
Copy link
Author

ior308 commented Jun 30, 2023 via email

@mooijtech
Copy link

Sure, if you have time you can contribute the changes yourself, it's open source after all.
Work is also being done on a new serialization format: https://github.com/nlpodyssey/safetensors

@mooijtech
Copy link

BART currently has no Question Answering support

@ior308
Copy link
Author

ior308 commented Jul 9, 2023 via email

@mooijtech
Copy link

Only BERT supports Q&A so you will have to find (or train?) a model based on BERT.

@ior308
Copy link
Author

ior308 commented Jul 13, 2023 via email

@matteo-grella
Copy link
Member

Sorry guys I've been a bit far from the project due to other compelling priorities. I'll be back asap and provide you support.

@ior308
Copy link
Author

ior308 commented Aug 10, 2023 via email

@ior308
Copy link
Author

ior308 commented Aug 20, 2023

Dear, now Q&A works for me. Instead using an external paragraph file as in gopher examples, I put the content into a variable:

[root@minix8 cybertron@v0.1.2]# sh exa1.sh
{"level":"debug","file":"models/deepset/bert-base-cased-squad2/config.json","time":"2023-08-20T17:04:17+02:00","message":"model file already exists, skipping download"}
{"level":"debug","file":"models/deepset/bert-base-cased-squad2/pytorch_model.bin","time":"2023-08-20T17:04:17+02:00","message":"model file already exists, skipping download"}
{"level":"debug","file":"models/deepset/bert-base-cased-squad2/vocab.txt","time":"2023-08-20T17:04:17+02:00","message":"model file already exists, skipping download"}
{"level":"debug","file":"models/deepset/bert-base-cased-squad2/tokenizer_config.json","time":"2023-08-20T17:04:17+02:00","message":"model file already exists, skipping download"}
{"level":"info","model":"models/deepset/bert-base-cased-squad2/spago_model.bin","time":"2023-08-20T17:04:17+02:00","message":"model file already exists, skipping conversion"}
Go is a programming language designed at Google in 2007 by Robert Griesemer, Rob Pike, and Ken Thompson to address criticism of other languages.

when go has been designed ?
0.93027006
{
"Answers": [
{
"Text": "2007",
"Start": 51,
"End": 55,
"Score": 0.9280015216017522
}
]
}

@ior308 ior308 closed this as completed Aug 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants