Filter auto_transformer kwargs based on forward signature #3329
Conversation
tgaddair
added
bug
| "hf-internal-testing/tiny-random-GPTJModel", | ||
| ], | ||
| ) | ||
| def test_hf_ludwig_model_auto_transformers(tmpdir, csv_filename, pretrained_model_name_or_path): |
There was a problem hiding this comment.
Could you mark this with @pytest.mark.slow?
There was a problem hiding this comment.
I think this should only take 30s, so I would want to run it on every commit if we can get away with it.
| input_features = [ | ||
| text_feature( | ||
| preprocessing={ | ||
| "max_sequence_length": 10, | ||
| }, | ||
| encoder={ | ||
| "vocab_size": 30, | ||
| "min_len": 1, | ||
| "type": "auto_transformer", | ||
| "pretrained_model_name_or_path": pretrained_model_name_or_path, | ||
| "use_pretrained": True, | ||
| }, | ||
| ) | ||
| ] | ||
| output_features = [category_feature(decoder={"vocab_size": 2})] | ||
| rel_path = generate_data(input_features, output_features, csv_filename) | ||
|
|
||
| config = { | ||
| "input_features": input_features, | ||
| "output_features": output_features, | ||
| TRAINER: {"train_steps": 1}, | ||
| } | ||
| model = LudwigModel(config=config, backend=LocalTestBackend()) | ||
|
|
||
| # Validates that the defaults associated with the encoder are compatible with Ludwig training. | ||
| with mock.patch( | ||
| "ludwig.encoders.text_encoders.load_pretrained_hf_model_with_hub_fallback", | ||
| side_effect=_load_pretrained_hf_model_no_weights, | ||
| ): | ||
| model.train(dataset=rel_path, output_directory=tmpdir) | ||
|
|
||
|
|
There was a problem hiding this comment.
Code here is a duplicate of that in test_hf_ludwig_model_reduce_options and test_hf_ludwig_model_e2e. Perhaps separate into a separate function and reuse.
|
Hey guys, I am working in the domain of large language models. Thanks for fixing this issue. I would love to contribute in the open source. It would be a matter of pride to be the part of this. |
|
Also this issue is resolved.after trying the latest code I got the following error: AttributeError: 'BloomModel' object has no attribute 'n_head' |
Fixes #3328.