global: integrate pytest-invenio 1.4 and fixes

.travis.yml

@@ -20,8 +20,12 @@ language: python
 matrix:
   fast_finish: true
   allow_failures:
-    - env: REQUIREMENTS=devel EXTRAS=all,elasticsearch7 ES_URL=$ES7_DOWNLOAD_URL
+    - python: 3.6
+      env: REQUIREMENTS=devel EXTRAS=elasticsearch7 ES_URL=$ES7_DOWNLOAD_URL
+    - python: 3.7
+      env: REQUIREMENTS=devel EXTRAS=elasticsearch7 ES_URL=$ES7_DOWNLOAD_URL
     - python: 3.8
+      env: REQUIREMENTS=devel EXTRAS=elasticsearch7 ES_URL=$ES7_DOWNLOAD_URL
 
 cache:
   - pip

docs/api.rst

@@ -141,12 +141,11 @@ Views
 Permissions
 -----------
 
-Permissions are handled by the following decorator, which is used in the
-record's REST resources:
+Permissions are handled by the decorator
+:py:meth:`invenio_records_rest.views.need_record_permission`, which is used in
+the record's REST resources.
 
-.. autofunction:: invenio_records_rest.views.need_record_permission
-
-each resource method is checking the permissions through this decorator, with
+Each resource method is checking the permissions through this decorator, with
 ``read_permission_factory``, ``create_permission_factory``,
 ``update_permission_factory`` and ``delete_permission_factory`` as argument,
 where the implementation of each can be configured for each REST

invenio_records_rest/__init__.py

@@ -134,7 +134,8 @@
 is a small function which makes a record unique under a given schema
 (i.e. DOI, ORCID), so this identifier and any other field needed will be added
 to the record metadata. A
-`fetcher <http://invenio-pidstore.readthedocs.io/en/latest/usage.html#fetchers>`_
+`fetcher
+<http://invenio-pidstore.readthedocs.io/en/latest/usage.html#fetchers>`_
 on the other hand, is a small function that retrieves records minted
 following a certain persistent identifier schema such as DOI.
 
@@ -212,19 +213,23 @@
 ++++++++++++
 By exposing the Elasticsearch API by default, we can use the advanced
 features it provides, such as the `aggregations framework
-<https://www.elastic.co/guide/en/elasticsearch/reference/5.6/search-aggregations.html>`_.
+<https://www.elastic.co/guide/en/elasticsearch/reference/5.6/
+search-aggregations.html>`_.
 
 These features include:
 
-- `bucketing <https://www.elastic.co/guide/en/elasticsearch/reference/5.6/search-aggregations-bucket.html>`_
-  "used to group the documents by a certain criterion".
-- `metric <https://www.elastic.co/guide/en/elasticsearch/reference/5.6/search-aggregations-metrics.html>`_
-  "used to create different types of metrics from values extracted from the documents
-  being aggregated".
-- `matrix <https://www.elastic.co/guide/en/elasticsearch/reference/5.6/search-aggregations-matrix.html>`_
-  "used to produce a matrix result based on the values extracted from the requested document fields".
-- `pipeline <https://www.elastic.co/guide/en/elasticsearch/reference/5.6/search-aggregations-pipeline.html>`_
-  "used to aggregate the output of other aggregations and their associated metrics".
+- `bucketing <https://www.elastic.co/guide/en/elasticsearch/reference/5.6/
+  search-aggregations-bucket.html>`_ "used to group the documents
+  by a certain criterion".
+- `metric <https://www.elastic.co/guide/en/elasticsearch/reference/5.6/
+  search-aggregations-metrics.html>`_ "used to create different types of
+  metrics from values extracted from the documents being aggregated".
+- `matrix <https://www.elastic.co/guide/en/elasticsearch/reference/5.6/
+  search-aggregations-matrix.html>`_ "used to produce a matrix result based
+  on the values extracted from the requested document fields".
+- `pipeline <https://www.elastic.co/guide/en/elasticsearch/reference/5.6/
+  search-aggregations-pipeline.html>`_ "used to aggregate the output of other
+  aggregations and their associated metrics".
 
 
 Filters
@@ -236,7 +241,7 @@
 To set the aggregations and filters you want you can modify the
 `RECORDS_REST_FACETS`.
 
->>> from .facets import terms_filter
+>>> from invenio_records_rest.facets import terms_filter
 >>> app.config['RECORDS_REST_FACETS'] = {
 ...     'index_name': {
 ...         'aggs': {
@@ -261,9 +266,12 @@
 Sorting is based by default on a relevance score, but this can
 be configured as well. The following ways are possible:
 
-- `Sorting by field values <https://www.elastic.co/guide/en/elasticsearch/guide/current/_sorting.html#_sorting_by_field_values>`_
-- `Multilevel sorting <https://www.elastic.co/guide/en/elasticsearch/guide/current/_sorting.html#_multilevel_sorting>`_
-- `Sorting on multilevel fields <https://www.elastic.co/guide/en/elasticsearch/guide/current/_sorting.html#_sorting_on_multivalue_fields>`_
+- `Sorting by field values <https://www.elastic.co/guide/en/elasticsearch/
+  guide/current/_sorting.html#_sorting_by_field_values>`_
+- `Multilevel sorting <https://www.elastic.co/guide/en/elasticsearch/
+  guide/current/_sorting.html#_multilevel_sorting>`_
+- `Sorting on multilevel fields <https://www.elastic.co/guide/en/elasticsearch/
+  guide/current/_sorting.html#_sorting_on_multivalue_fields>`_
 
 This can be configured through the `RECORDS_REST_SORT_OPTIONS`:
 
@@ -300,8 +308,9 @@
 
 The search syntax for the various queries is powered by the query parser used.
 Internally the query parser is referred to as the search factory.
-This defaults to the `Q() <https://www.elastic.co/guide/en/elasticsearch/reference/2.4/query-dsl-query-string-query.html>`_ parser
-from elasticsearch_dsl.
+This defaults to the `Q() <https://www.elastic.co/guide/en/elasticsearch/
+reference/2.4/query-dsl-query-string-query.html>`_ parser from
+elasticsearch_dsl.
 
 ``/records/?q=``
 
@@ -368,8 +377,8 @@
   Creating a custom search factory can enable full control over Elasticsearch's
   Search API. Some of the features available include rank evaluation, post
   filters, highlighting, index boosting among others. For the full list see
-  the full Elasticsearch documentation for the `Search API
-  <https://www.elastic.co/guide/en/elasticsearch/reference/current/search.html>`_.
+  the full Elasticsearch documentation for the `Search API <https://
+  www.elastic.co/guide/en/elasticsearch/reference/current/search.html>`_.
 
 - As the default search behaviour can be adjusted, Invenio-Records-REST
   also offers the possibility to define custom search exceptions.
@@ -398,7 +407,8 @@
   fetcher, add the ``invenio_pidstore.fetchers`` entrypoint in the
   ``setup.py`` of a module and point it to the fetcher function. For more
   information on fetchers, see the corresponding paragraph in
-  `Invenio-PIDStore <https://invenio-pidstore.readthedocs.io/en/latest/api.html#module-invenio_pidstore.fetchers>`_.
+  `Invenio-PIDStore <https://invenio-pidstore.readthedocs.io/en/latest/
+  api.html#module-invenio_pidstore.fetchers>`_.
 
   The fetchers by default will query the database, but this can be redirected
   to Elasticsearch according to the needs of the application. More
@@ -508,7 +518,7 @@
 >>> from invenio_records_rest.serializers.response import record_responsify
 >>> csl_v1 = JSONSerializer(SimpleRecordSchema, replace_refs=True)
 >>> citeproc_v1 = CiteprocSerializer(csl_v1)
->>> citeproc_v1_response = record_responsify(citeproc_v1, 'text/x-bibliography')
+>>> citeproc_v1_resp = record_responsify(citeproc_v1, 'text/x-bibliography')
 
 and register it in our ``RECORDS_REST_ENDPOINTS``
 for ``Accept:text/x-bibliography``:
@@ -519,7 +529,7 @@
 ...          'application/json':('invenio_records_rest.serializers'
 ...                              ':json_v1_response'),
 ...          'text/x-bibliography':('invenio_records_rest.serializers'
-...                                 ':citeproc_v1_response'),
+...                                 ':citeproc_v1_resp'),
 ...       }
 ...    )
 ... )
@@ -555,8 +565,10 @@
 For some examples on the implementation of these serializers you can see
 the ones present in different invenio instances such as:
 
-- `CDS-Videos <https://github.com/CERNDocumentServer/cds-videos/tree/cdslabs_qa/cds/modules/records/serializers>`_
-- `Zenodo <https://github.com/zenodo/zenodo/tree/master/zenodo/modules/records/serializers>`_
+- `CDS-Videos <https://github.com/CERNDocumentServer/cds-videos/tree/master
+  /cds/modules/records/serializers>`_
+- `Zenodo <https://github.com/zenodo/zenodo/tree/master/zenodo/modules/
+  records/serializers>`_
 
 
 Tombstones and redirection
@@ -649,8 +661,10 @@
 more examples you can directly see the ones used in production in the
 Invenio instances:
 
-- `CDS's permission factories <https://github.com/CERNDocumentServer/cds-videos/blob/cdslabs_qa/cds/modules/records/permissions.py>`_
-- `Zenodo's permission factories <https://github.com/zenodo/zenodo/blob/master/zenodo/modules/records/permissions.py>`_
+- `CDS's permission factories <https://github.com/CERNDocumentServer/
+  cds-videos/blob/master/cds/modules/records/permissions.py>`_
+- `Zenodo's permission factories <https://github.com/zenodo/zenodo/blob/
+  master/zenodo/modules/records/permissions.py>`_
 
 Factories
 ~~~~~~~~~

invenio_records_rest/config.py

@@ -22,6 +22,7 @@ def _(x):
     """Identity function for string extraction."""
     return x
 
+
 RECORDS_REST_ENDPOINTS = dict(
     recid=dict(
         pid_type='recid',

invenio_records_rest/serializers/citeproc.py

@@ -26,7 +26,7 @@
 try:
     from citeproc_styles import get_style_filepath
     from citeproc_styles.errors import StyleNotFoundError
-except:
+except Exception:
     import warnings
     warnings.warn('citeproc_styles not found. '
                   'Please install to enable Citeproc Serialization.')
@@ -110,8 +110,8 @@ def _get_source(self, data):
 
     def _clean_result(self, text):
         """Remove double spaces, punctuation and escapes apostrophes."""
-        text = re.sub('\s\s+', ' ', text)
-        text = re.sub('\.\.+', '.', text)
+        text = re.sub(r'\s\s+', ' ', text)
+        text = re.sub(r'\.\.+', '.', text)
         text = text.replace("'", "\\'")
         return text
 

pytest.ini

@@ -7,8 +7,5 @@
 # under the terms of the MIT License; see LICENSE file for more details.
 
 [pytest]
-pep8ignore =
-    __init__.py E501
-    docs/conf.py ALL
-addopts = --pep8 --doctest-glob="*.rst" --doctest-modules --cov=invenio_records_rest --cov-report=term-missing
+addopts = --isort --pydocstyle --pycodestyle --doctest-glob="*.rst" --doctest-modules --cov=invenio_records_rest --cov-report=term-missing
 testpaths = docs tests invenio_records_rest

run-tests.sh

@@ -7,8 +7,9 @@
 # Invenio is free software; you can redistribute it and/or modify it
 # under the terms of the MIT License; see LICENSE file for more details.
 
-pydocstyle invenio_records_rest tests docs && \
-isort invenio_records_rest tests --check-only --diff && \
-check-manifest --ignore ".travis-*" && \
-sphinx-build -qnNW docs docs/_build/html && \
-py.test
+python -m check_manifest --ignore ".travis-*" && \
+python -m sphinx.cmd.build -qnNW docs docs/_build/html && \
+python -m pytest && \
+python -m sphinx.cmd.build -qnNW -b doctest docs docs/_build/doctest
+tests_exit_code=$?
+exit "$tests_exit_code"

setup.cfg

@@ -20,6 +20,9 @@ universal = 1
 [pydocstyle]
 add_ignore = D401
 
+[pycodestyle]
+exclude = invenio_records_rest/__init__.py,docs/conf.py
+
 [compile_catalog]
 directory = invenio_records_rest/translations/
 

setup.py

@@ -17,17 +17,11 @@
 history = open('CHANGES.rst').read()
 
 tests_require = [
-    'check-manifest>=0.25',
-    'coverage>=4.0',
     'Flask-Login>=0.3.2',
     'invenio-db[all]>=1.0.2',
     'invenio-indexer>=1.0.0',
-    'isort>=4.3.1',
-    'pydocstyle>=1.0.0',
-    'pytest-cov>=1.8.0',
-    'pytest-pep8>=1.0.6',
-    'pytest>=4.0.0,<5.0.0',
     'invenio-config>=1.0.2',
+    'pytest-invenio>=1.4.0',
 ]
 
 invenio_search_version = '1.2.0'
@@ -53,7 +47,7 @@
         'datacite>=1.0.1',
     ],
     'docs': [
-        'Sphinx>=1.6.7,<3',
+        'Sphinx>=3',
     ],
     'dublincore': [
         'dcxml>=0.1.0',
@@ -72,16 +66,15 @@
     extras_require['all'].extend(reqs)
 
 setup_requires = [
-    'Babel>=1.3',
-    'pytest-runner>=2.6.2'
+    'Babel>=2.8',
 ]
 
 install_requires = [
     'arrow>=0.12.1',
     'attrs>=17.4.0',
     'bleach>=2.1.3',
     'ftfy>=4.4.3',
-    'invenio-base>=1.2.2',
+    'invenio-base>=1.2.3',
     'invenio-pidstore>=1.2.0',
     'invenio-records>=1.0.0',
     'invenio-rest>=1.2.2',

tests/test_marshmallow_loader.py

@@ -52,12 +52,12 @@ class _TestSchemaNested(Schema):
 
 
 class _TestMetadataSchema(Schema):
-        """Test schema."""
+    """Test schema."""
 
-        title = fields.Str()
-        stars = fields.Integer()
-        year = fields.Integer()
-        control_number = PersistentIdentifier()
+    title = fields.Str()
+    stars = fields.Integer()
+    year = fields.Integer()
+    control_number = PersistentIdentifier()
 
 
 def test_marshmallow_load(app, db, es, test_data, search_url, search_class):

tests/test_views_item_delete.py

@@ -68,5 +68,5 @@ def raise_error():
             assert res.status_code == 500
 
     with app.test_client() as client:
-            res = client.get(record_url(pid))
-            assert res.status_code == 200
+        res = client.get(record_url(pid))
+        assert res.status_code == 200