Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Change BloomConfig docstring #19336

Merged
merged 4 commits into from Oct 5, 2022
Merged

Change BloomConfig docstring #19336

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d99d89f
change `BloomConfig` docstring
younesbelkada Oct 5, 2022
671ae11
Update src/transformers/models/bloom/configuration_bloom.py
younesbelkada Oct 5, 2022
b3478f0
Update src/transformers/models/bloom/configuration_bloom.py
younesbelkada Oct 5, 2022
137f156
make style
younesbelkada Oct 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 7 additions & 5 deletions src/transformers/models/bloom/configuration_bloom.py
View file
Open in desktop
Expand Up @@ -53,14 +53,16 @@ class BloomConfig(PretrainedConfig):


Args:
vocab_size (`int`, *optional*, defaults to 50257):
vocab_size (`int`, *optional*, defaults to 250880):
Vocabulary size of the Bloom model. Defines the number of different tokens that can be represented by the
younesbelkada marked this conversation as resolved.
Show resolved Hide resolved
`inputs_ids` passed when calling [`BloomModel`].
hidden_size (`int`, *optional*, defaults to 768):
`inputs_ids` passed when calling [`BloomModel`]. Check
https://huggingface.co/bigscience/bloom/discussions/120#633d28389addb8530b406c2a on how the `vocab_size`
has been defined.
younesbelkada marked this conversation as resolved.
Show resolved Hide resolved
hidden_size (`int`, *optional*, defaults to 64):
Dimensionality of the embeddings and hidden states.
n_layer (`int`, *optional*, defaults to 12):
n_layer (`int`, *optional*, defaults to 2):
Number of hidden layers in the Transformer encoder.
n_head (`int`, *optional*, defaults to 12):
n_head (`int`, *optional*, defaults to 8):
Number of attention heads for each attention layer in the Transformer encoder.
layer_norm_epsilon (`float`, *optional*, defaults to 1e-5):
The epsilon to use in the layer normalization layers.
Expand Down