Merge branch 'dev' into setup-api

.github/CONTRIBUTING.md

@@ -88,6 +88,13 @@ Also, you should run `python setup.py develop` to build the project and be able
 python setup.py develop
 ```
 
+Once the packages are installed inside your activated environment, you can run the following command to install the pre-commit
+hooks.
+
+``` shell
+pre-commit install
+```
+
 ## Use a Consistent Coding Style
 
 All of the code is formatted using [black](https://black.readthedocs.io) with the
@@ -110,6 +117,15 @@ pylint deepparse
 pylint tests
 ```
 
+### Pre-commit hooks
+These last commands will automatically be run along with others verifications when committing your code change using pre-commit hooks.
+
+You can also run them locally with the following command:
+
+``` shell 
+pre-commit run --all-files colors always
+```
+
 ## Tests
 
 If your pull request introduces a new feature, please deliver it with tests that ensure correct behavior. All of the

.pre-commit-config.yaml

@@ -0,0 +1,75 @@
+# https://pre-commit.com
+default_install_hook_types: [commit-msg, pre-commit]
+default_stages: [commit, manual]
+fail_fast: true
+repos:
+  - repo: meta
+    hooks:
+      - id: check-useless-excludes
+  - repo: https://github.com/pre-commit/pygrep-hooks
+    rev: v1.9.0
+    hooks:
+      - id: python-check-mock-methods
+      - id: python-use-type-annotations
+      - id: rst-backticks
+      - id: rst-directive-colons
+      - id: rst-inline-touching-normal
+      - id: text-unicode-replacement-char
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.3.0
+    hooks:
+      - id: check-added-large-files
+      - id: check-ast
+      - id: check-builtin-literals
+      - id: check-case-conflict
+      - id: check-docstring-first
+      - id: check-json
+      - id: check-merge-conflict
+      - id: check-shebang-scripts-are-executable
+      - id: check-symlinks
+      - id: check-toml
+      - id: check-vcs-permalinks
+      - id: check-xml
+      - id: check-yaml
+      - id: debug-statements
+      - id: destroyed-symlinks
+      - id: detect-private-key
+      - id: end-of-file-fixer
+        types: [ python ]
+      - id: fix-byte-order-marker
+      - id: mixed-line-ending
+      - id: name-tests-test
+        args: [ --pytest-test-first]
+        exclude: |
+          (?x)^(
+                tests/embeddings_models/integration/tools.py|
+                tests/parser/integration/base_predict.py|
+                tests/parser/integration/base_retrain.py|
+                tests/base_capture_output.py|
+                tests/tools.py|
+                tests/base_file_exist.py|
+                tests/network/base.py|
+                tests/parser/base.py|
+                tests/network/integration/base.py|
+                tests/network/base.py
+            )$
+      - id: trailing-whitespace
+        types: [ python ]
+  - repo: local
+    hooks:
+      - id: black
+        name: black
+        entry: black
+        require_serial: true
+        language: system
+        types: [ python ]
+      - id: pylint
+        name: pylint
+        entry:  pylint
+        language: system
+        types: [ python ]
+        files: ^deepparse/|tests/
+        args:
+          [
+            "--rcfile=.pylintrc", # Link to your config file<
+        ]

deepparse/cli/tools.py

@@ -191,7 +191,7 @@ def data_container_factory(
 
 
 # pylint: disable=pointless-string-statement
-"""
+PYPYR_COPYRIGHT_APACHE_LICENSE = """
 The code below was copied from the pypyr project, and has been modified for the purpose of this package.
 
 COPYRIGHT

deepparse/download_tools.py

@@ -202,7 +202,7 @@ def latest_version(model: str, cache_path: str, verbose: bool) -> bool:
 
 
 # pylint: disable=pointless-string-statement
-"""
+FASTTEXT_COPYRIGHT_MIT_LICENSE = """
 The code below was copied from the fastText project, and has been modified for the purpose of this package.
 
 COPYRIGHT

docs/source/examples/retrain_with_new_prediction_tags.rst

@@ -72,7 +72,7 @@ Now let's test our fine-tuned model using the best checkpoint (default parameter
 
 
 Now let's see how we can reload our new AddressParser.
-When you retrain a model, at the end, we create a retrained checkpoint using the best checkpoint and also include some metadata for the reloading (the tags, the dimension, etc.). By defaults, the checkpoint is named `"retrain_modeltype_parser.ckpt"`.
+When you retrain a model, at the end, we create a retrained checkpoint using the best checkpoint and also include some metadata for the reloading (the tags, the dimension, etc.). By defaults, the checkpoint is named ``"retrain_modeltype_parser.ckpt"``.
 
 .. code-block:: python
 

docs/source/index.rst

@@ -753,7 +753,7 @@ look at our `contributing guidelines <https://github.com/GRAAL-Research/deeppars
 details on this matter.
 
 License
-=======
+========
 Deepparse is LGPLv3 licensed, as found in the `LICENSE file <https://github.com/GRAAL-Research/deepparse/blob/main/LICENSE>`_.
 
 

docs/source/parser.rst

@@ -110,9 +110,9 @@ are a little bit lower for the trained country (around ~2%) but are similar for
 
 Thus, the more address is, the faster each address can be processed. You can also improve performance by using more
 workers for the data loader created with your data within the call. But note that this performance improvement is not linear.
-Furthermore, as of version `0.9.6`, we now use Torch 2.0 and many other tricks to improve
+Furthermore, as of version ``0.9.6``, we now use Torch 2.0 and many other tricks to improve
 processing performance. Here a few: if the parser uses a GPU, it will pin the memory in the Dataloader and reduce some
-operations (e.g. useless `.to(device)`).
+operations (e.g. useless ``.to(device)``).
 
 AddressParser
 -------------

pyproject.toml

@@ -1,4 +1,6 @@
 [tool.black]
+target-version = ['py38', 'py39', 'py310', 'py311']
+
 line-length = 120
 skip-string-normalization = true
 required-version = "23.3.0"

styling_requirements.txt

@@ -1,3 +1,4 @@
 black==23.3.0
 pylint==2.16.2
 pylint-django[with_django]
+pre-commit