Skip to content

Commit

Permalink
Merge pull request #492 from QData/doc-fix
Browse files Browse the repository at this point in the history
fix document errors and inconsistency
  • Loading branch information
qiyanjun committed Jul 27, 2021
2 parents 8f36926 + 4f66ad0 commit 00d5f83
Show file tree
Hide file tree
Showing 41 changed files with 47 additions and 130 deletions.
15 changes: 8 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -296,14 +296,15 @@ textattack attack --model bert-base-uncased-sst2 --recipe textfooler --num-examp
### Augmenting Text: `textattack augment`

Many of the components of TextAttack are useful for data augmentation. The `textattack.Augmenter` class
uses a transformation and a list of constraints to augment data. We also offer five built-in recipes
uses a transformation and a list of constraints to augment data. We also offer built-in recipes
for data augmentation:
- `textattack.WordNetAugmenter` augments text by replacing words with WordNet synonyms
- `textattack.EmbeddingAugmenter` augments text by replacing words with neighbors in the counter-fitted embedding space, with a constraint to ensure their cosine similarity is at least 0.8
- `textattack.CharSwapAugmenter` augments text by substituting, deleting, inserting, and swapping adjacent characters
- `textattack.EasyDataAugmenter` augments text with a combination of word insertions, substitutions and deletions.
- `textattack.CheckListAugmenter` augments text by contraction/extension and by substituting names, locations, numbers.
- `textattack.CLAREAugmenter` augments text by replacing, inserting, and merging with a pre-trained masked language model.
- `wordnet` augments text by replacing words with WordNet synonyms
- `embedding` augments text by replacing words with neighbors in the counter-fitted embedding space, with a constraint to ensure their cosine similarity is at least 0.8
- `charswap` augments text by substituting, deleting, inserting, and swapping adjacent characters
- `eda` augments text with a combination of word insertions, substitutions and deletions.
- `checklist` augments text by contraction/extension and by substituting names, locations, numbers.
- `clare` augments text by replacing, inserting, and merging with a pre-trained masked language model.


#### Augmentation Command-Line Interface
The easiest way to use our data augmentation tools is with `textattack augment <args>`. `textattack augment`
Expand Down
13 changes: 8 additions & 5 deletions README_ZH.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,11 +292,14 @@ textattack attack --model bert-base-uncased-sst2 --recipe textfooler --num-examp
### 增强文本数据:`textattack augment`

TextAttack 的组件中,有很多易用的数据增强工具。`textattack.Augmenter` 类使用 *变换* 与一系列的 *约束* 进行数据增强。我们提供了 5 中内置的数据增强策略:
- `textattack.WordNetAugmenter` 通过基于 WordNet 同义词替换的方式增强文本
- `textattack.EmbeddingAugmenter` 通过邻近词替换的方式增强文本,使用 counter-fitted 词嵌入空间中的邻近词进行替换,约束二者的 cosine 相似度不低于 0.8
- `textattack.CharSwapAugmenter` 通过字符的增删改,以及临近字符交换的方式增强文本
- `textattack.EasyDataAugmenter` 通过对词的增删改来增强文本
- `textattack.CheckListAugmenter` 通过简写,扩写以及对实体、地点、数字的替换来增强文本
- `wordnet` 通过基于 WordNet 同义词替换的方式增强文本
- `embedding` 通过邻近词替换的方式增强文本,使用 counter-fitted 词嵌入空间中的邻近词进行替换,约束二者的 cosine 相似度不低于 0.8
- `charswap` 通过字符的增删改,以及临近字符交换的方式增强文本
- `eda` 通过对词的增删改来增强文本
- `checklist` 通过简写,扩写以及对实体、地点、数字的替换来增强文本
- `clare` 使用 pre-trained masked language model, 通过对词的增删改来增强文本



#### 数据增强的命令行接口
使用 textattack 来进行数据增强,最快捷的方法是通过 `textattack augment <args>` 命令行接口。 `textattack augment` 使用 CSV 文件作为输入,在参数中设置需要增强的文本列,每个样本允许改变的比例,以及对于每个输入样本生成多少个增强样本。输出的结果保存为与输入文件格式一致的 CSV 文件,结果文件中为对指定的文本列生成的增强样本。
Expand Down
36 changes: 19 additions & 17 deletions docs/3recipes/augmenter_recipes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,25 +7,27 @@ In addition to the command-line interface, you can augment text dynamically by i
`Augmenter` in your own code. All `Augmenter` objects implement `augment` and `augment_many` to generate augmentations
of a string or a list of strings. Here's an example of how to use the `EmbeddingAugmenter` in a python script:

```python
>>> from textattack.augmentation import EmbeddingAugmenter
>>> augmenter = EmbeddingAugmenter()
>>> s = 'What I cannot create, I do not understand.'
>>> augmenter.augment(s)
['What I notable create, I do not understand.', 'What I significant create, I do not understand.', 'What I cannot engender, I do not understand.', 'What I cannot creating, I do not understand.', 'What I cannot creations, I do not understand.', 'What I cannot create, I do not comprehend.', 'What I cannot create, I do not fathom.', 'What I cannot create, I do not understanding.', 'What I cannot create, I do not understands.', 'What I cannot create, I do not understood.', 'What I cannot create, I do not realise.']
```
.. code-block:: python
>>> from textattack.augmentation import EmbeddingAugmenter
>>> augmenter = EmbeddingAugmenter()
>>> s = 'What I cannot create, I do not understand.'
>>> augmenter.augment(s)
['What I notable create, I do not understand.', 'What I significant create, I do not understand.', 'What I cannot engender, I do not understand.', 'What I cannot creating, I do not understand.', 'What I cannot creations, I do not understand.', 'What I cannot create, I do not comprehend.', 'What I cannot create, I do not fathom.', 'What I cannot create, I do not understanding.', 'What I cannot create, I do not understands.', 'What I cannot create, I do not understood.', 'What I cannot create, I do not realise.']
You can also create your own augmenter from scratch by importing transformations/constraints from `textattack.transformations` and `textattack.constraints`. Here's an example that generates augmentations of a string using `WordSwapRandomCharacterDeletion`:

```python
>>> from textattack.transformations import WordSwapRandomCharacterDeletion
>>> from textattack.transformations import CompositeTransformation
>>> from textattack.augmentation import Augmenter
>>> transformation = CompositeTransformation([WordSwapRandomCharacterDeletion()])
>>> augmenter = Augmenter(transformation=transformation, transformations_per_example=5)
>>> s = 'What I cannot create, I do not understand.'
>>> augmenter.augment(s)
['What I cannot creae, I do not understand.', 'What I cannot creat, I do not understand.', 'What I cannot create, I do not nderstand.', 'What I cannot create, I do nt understand.', 'Wht I cannot create, I do not understand.']
```
.. code-block:: python
>>> from textattack.transformations import WordSwapRandomCharacterDeletion
>>> from textattack.transformations import CompositeTransformation
>>> from textattack.augmentation import Augmenter
>>> transformation = CompositeTransformation([WordSwapRandomCharacterDeletion()])
>>> augmenter = Augmenter(transformation=transformation, transformations_per_example=5)
>>> s = 'What I cannot create, I do not understand.'
>>> augmenter.augment(s)
['What I cannot creae, I do not understand.', 'What I cannot creat, I do not understand.', 'What I cannot create, I do not nderstand.', 'What I cannot create, I do nt understand.', 'Wht I cannot create, I do not understand.']
Here is a list of recipes for NLP data augmentations
Expand Down
17 changes: 9 additions & 8 deletions docs/3recipes/augmenter_recipes_cmd.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,18 @@ The [`examples/`](https://github.com/QData/TextAttack/tree/master/examples) fold
The [documentation website](https://textattack.readthedocs.io/en/latest) contains walkthroughs explaining basic usage of TextAttack, including building a custom transformation and a custom constraint..


## Augmenting Text: `textattack augment`
### Augmenting Text: `textattack augment`

Many of the components of TextAttack are useful for data augmentation. The `textattack.Augmenter` class
uses a transformation and a list of constraints to augment data. We also offer five built-in recipes
uses a transformation and a list of constraints to augment data. We also offer built-in recipes
for data augmentation:
- `textattack.WordNetAugmenter` augments text by replacing words with WordNet synonyms
- `textattack.EmbeddingAugmenter` augments text by replacing words with neighbors in the counter-fitted embedding space, with a constraint to ensure their cosine similarity is at least 0.8
- `textattack.CharSwapAugmenter` augments text by substituting, deleting, inserting, and swapping adjacent characters
- `textattack.EasyDataAugmenter` augments text with a combination of word insertions, substitutions and deletions.
- `textattack.CheckListAugmenter` augments text by contraction/extension and by substituting names, locations, numbers.
- `textattack.CLAREAugmenter` augments text by replacing, inserting, and merging with a pre-trained masked language model.
- `wordnet` augments text by replacing words with WordNet synonyms
- `embedding` augments text by replacing words with neighbors in the counter-fitted embedding space, with a constraint to ensure their cosine similarity is at least 0.8
- `charswap` augments text by substituting, deleting, inserting, and swapping adjacent characters
- `eda` augments text with a combination of word insertions, substitutions and deletions.
- `checklist` augments text by contraction/extension and by substituting names, locations, numbers.
- `clare` augments text by replacing, inserting, and merging with a pre-trained masked language model.


### Augmentation Command-Line Interface
The easiest way to use our data augmentation tools is with `textattack augment <args>`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ Constraints API Reference

Constraint
------------
.. automodule:: textattack.constraints.Constraint
.. autoclass:: textattack.constraints.Constraint
:members:

PreTransformationConstraint
-----------------------------
.. automodule:: textattack.constraints.PreTransformationConstraint
.. autoclass:: textattack.constraints.PreTransformationConstraint
:members:

2 changes: 0 additions & 2 deletions docs/apidoc/textattack.attack_recipes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.attack\_recipes package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.attack_recipes.attack_recipe
Expand Down
2 changes: 0 additions & 2 deletions docs/apidoc/textattack.attack_results.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.attack\_results package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.attack_results.attack_result
Expand Down
2 changes: 0 additions & 2 deletions docs/apidoc/textattack.augmentation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.augmentation package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.augmentation.augmenter
Expand Down
2 changes: 0 additions & 2 deletions docs/apidoc/textattack.commands.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.commands package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.commands.attack_command
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.constraints.grammaticality.language\_models.google\_language\_model p
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.constraints.grammaticality.language_models.google_language_model.alzantot_goog_lm
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.constraints.grammaticality.language\_models.learning\_to\_write packa
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.constraints.grammaticality.language_models.learning_to_write.adaptive_softmax
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,13 @@ textattack.constraints.grammaticality.language\_models package
:undoc-members:
:show-inheritance:

Subpackages
-----------

.. toctree::
:maxdepth: 6

textattack.constraints.grammaticality.language_models.google_language_model
textattack.constraints.grammaticality.language_models.learning_to_write

Submodules
----------


.. automodule:: textattack.constraints.grammaticality.language_models.gpt2
Expand Down
4 changes: 0 additions & 4 deletions docs/apidoc/textattack.constraints.grammaticality.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,12 @@ textattack.constraints.grammaticality package
:undoc-members:
:show-inheritance:

Subpackages
-----------

.. toctree::
:maxdepth: 6

textattack.constraints.grammaticality.language_models

Submodules
----------


.. automodule:: textattack.constraints.grammaticality.cola
Expand Down
2 changes: 0 additions & 2 deletions docs/apidoc/textattack.constraints.overlap.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.constraints.overlap package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.constraints.overlap.bleu_score
Expand Down
2 changes: 0 additions & 2 deletions docs/apidoc/textattack.constraints.pre_transformation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.constraints.pre\_transformation package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.constraints.pre_transformation.input_column_modification
Expand Down
4 changes: 0 additions & 4 deletions docs/apidoc/textattack.constraints.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.constraints package
:undoc-members:
:show-inheritance:

Subpackages
-----------

.. toctree::
:maxdepth: 6
Expand All @@ -17,8 +15,6 @@ Subpackages
textattack.constraints.pre_transformation
textattack.constraints.semantics

Submodules
----------


.. automodule:: textattack.constraints.constraint
Expand Down
4 changes: 0 additions & 4 deletions docs/apidoc/textattack.constraints.semantics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,12 @@ textattack.constraints.semantics package
:undoc-members:
:show-inheritance:

Subpackages
-----------

.. toctree::
:maxdepth: 6

textattack.constraints.semantics.sentence_encoders

Submodules
----------


.. automodule:: textattack.constraints.semantics.bert_score
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.constraints.semantics.sentence\_encoders.bert package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.constraints.semantics.sentence_encoders.bert.bert
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.constraints.semantics.sentence\_encoders.infer\_sent package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.constraints.semantics.sentence_encoders.infer_sent.infer_sent
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.constraints.semantics.sentence\_encoders package
:undoc-members:
:show-inheritance:

Subpackages
-----------

.. toctree::
:maxdepth: 6
Expand All @@ -16,8 +14,6 @@ Subpackages
textattack.constraints.semantics.sentence_encoders.infer_sent
textattack.constraints.semantics.sentence_encoders.universal_sentence_encoder

Submodules
----------


.. automodule:: textattack.constraints.semantics.sentence_encoders.sentence_encoder
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.constraints.semantics.sentence\_encoders.universal\_sentence\_encoder
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.constraints.semantics.sentence_encoders.universal_sentence_encoder.multilingual_universal_sentence_encoder
Expand Down
2 changes: 0 additions & 2 deletions docs/apidoc/textattack.datasets.helpers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.datasets.helpers package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.datasets.helpers.ted_multi
Expand Down
4 changes: 0 additions & 4 deletions docs/apidoc/textattack.datasets.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,12 @@ textattack.datasets package
:undoc-members:
:show-inheritance:

Subpackages
-----------

.. toctree::
:maxdepth: 6

textattack.datasets.helpers

Submodules
----------


.. automodule:: textattack.datasets.dataset
Expand Down
2 changes: 0 additions & 2 deletions docs/apidoc/textattack.goal_function_results.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.goal\_function\_results package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.goal_function_results.classification_goal_function_result
Expand Down
2 changes: 0 additions & 2 deletions docs/apidoc/textattack.goal_functions.classification.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.goal\_functions.classification package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.goal_functions.classification.classification_goal_function
Expand Down
4 changes: 0 additions & 4 deletions docs/apidoc/textattack.goal_functions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,13 @@ textattack.goal\_functions package
:undoc-members:
:show-inheritance:

Subpackages
-----------

.. toctree::
:maxdepth: 6

textattack.goal_functions.classification
textattack.goal_functions.text

Submodules
----------


.. automodule:: textattack.goal_functions.goal_function
Expand Down
2 changes: 0 additions & 2 deletions docs/apidoc/textattack.goal_functions.text.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.goal\_functions.text package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.goal_functions.text.minimize_bleu
Expand Down
2 changes: 0 additions & 2 deletions docs/apidoc/textattack.loggers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ textattack.loggers package
:undoc-members:
:show-inheritance:

Submodules
----------


.. automodule:: textattack.loggers.attack_log_manager
Expand Down

0 comments on commit 00d5f83

Please sign in to comment.