Clean up top level directory of repository

[ltalirz]

fix #3730

[codecov]

Codecov Report

Merging #3738 into develop will decrease coverage by 0.00%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop    #3738      +/-   ##
===========================================
- Coverage    78.01%   78.00%   -0.01%     
===========================================
  Files          457      457              
  Lines        33830    33830              
===========================================
- Hits         26391    26390       -1     
- Misses        7439     7440       +1     

Flag	Coverage Δ
#django	70.05% <ø> (ø)
#sqlalchemy	70.87% <ø> (-0.01%)	⬇️

Impacted Files	Coverage Δ
aiida/transports/plugins/local.py	80.20% <0.00%> (-0.26%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f40f0d2...1c369a3. Read the comment docs.

[ltalirz]

@sphuber despite the current issues with CI, I believe the tests actually pass (see 2nd-to-last commit) and the coverage should not be decreasing.

[sphuber]

The tests are failing because it is still trying to install the bin/runaiida script during package setup:

error: [Errno 2] No such file or directory: '/home/runner/work/aiida-core/aiida-core/bin/runaiida'

You should remove the scripts key from the setup.json as well. Then we should really test that in a completely new environment the #!/usr/bin/env runaiida shebang still works exactly the same

[ltalirz]

The tests are failing because it is still trying to install the bin/runaiida script during package setup:

Ah... ;-) Thanks, fixed!

Then we should really test that in a completely new environment the #!/usr/bin/env runaiida shebang still works exactly the same

I checked it on my side and it worked. Please check as well.

[sphuber]

@ltalirz I also verified that the new runaiida still works. The pre-commit is now failing. These are the lines that you addressed in another PR, that we were wondering how they came there. I guess you must have mixed those files from this PR into the other one. Anyway, should be an easy fix. Once that is done I will merge this. Thanks!

[ltalirz]

These are the lines that you addressed in another PR, that we were wondering how they came there.

Indeed, this is what I introduced here because the tests suddenly started failing.
I don't see these changes anymore in the final squashed commit of yours in #3875

Very strange... how is it possible that the tests did pass for you while they did not for me?

Anyhow, I will re-introduce the changes here.

[ltalirz]

Just to follow up on this, here are links to recent github actions runs:

• passed (despite the # pylint:disable comments missing)
• failed

Looking at the python environment, the only differences are:

7c7
< Werkzeug-1.0.0
---
> Werkzeug-1.0.1
42c42
< django-2.2.11
---
> django-2.2.12
84c84
< nbformat-5.0.4
---
> nbformat-5.0.5

I don't think those have anything to do with it (also, if the reason for this would be updates of python dependencies, it would be unclear why the pre-commit hooks worked, then broke, then worked again, then broke again).

[sphuber]

The tests are failing for SqlAlchemy. They seem to be hanging at tests/backends/aiida_sqlalchemy/test_generic.py. I have already restarted them once but they timed out at the same point. I am not sure what is going on

[ltalirz]

I can reproduce the issue locally using

$ AIIDA_TEST_BACKEND=sqlalchemy pytest tests/backends/aiida_sqlalchemy

Actually, the tests in test_generic.py run fine for me:

 $ AIIDA_TEST_BACKEND=sqlalchemy pytest tests/backends/aiida_sqlalchemy/test_generic.py
============================================================================================================ test session starts =============================================================================================================
platform darwin -- Python 3.7.1, pytest-5.3.4, py-1.8.0, pluggy-0.13.1
rootdir: /Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq, inifile: pytest.ini
plugins: timeout-1.3.4, datadir-1.3.1, cov-2.8.1
collected 4 items

tests/backends/aiida_sqlalchemy/test_generic.py ....                                                                                                                                                                                   [100%]

============================================================================================================= 4 passed in 8.17s ==============================================================================================================

For me it's hanging at tests/backends/aiida_sqlalchemy/test_migrations.py

$ AIIDA_TEST_BACKEND=sqlalchemy pytest tests/backends/aiida_sqlalchemy
============================================================================================================ test session starts =============================================================================================================
platform darwin -- Python 3.7.1, pytest-5.3.4, py-1.8.0, pluggy-0.13.1
rootdir: /Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq, inifile: pytest.ini
plugins: timeout-1.3.4, datadir-1.3.1, cov-2.8.1
collected 39 items

tests/backends/aiida_sqlalchemy/test_generic.py ....                                                                                                                                                                                   [ 10%]
tests/backends/aiida_sqlalchemy/test_migrations.py ^C

[ltalirz]

It actually hangs here:

aiida-core/tests/backends/aiida_sqlalchemy/test_migrations.py

Line 67 in 6ce6cd4

self.migrate_db_down(self.migrate_from)

Is it a problem if a user is already stored in the DB at this point?
How do the migration tests work?

Perhaps @giovannipizzi could also comment

P.S. Fast way to reproduce:

AIIDA_TEST_BACKEND=sqlalchemy pytest tests/backends/aiida_sqlalchemy/test_migrations.py::TestMigrationEngine::test_revision_numbers

[giovannipizzi]

I might be fixing this in #3650

[EDIT: for reference, the problem that I am fixing is that the downward migration test was not resetting the migration to the most recent one, and the next one to run, if any, would fail; however it would not hang for me. I suggest to merge #3650 first and see if this solves the issue already]

[ltalirz]

Yeah, I introduced this fixture because it was necessary.

There are two options for this PR - either we fix the underlying issue or we keep the "pytest tests" separate from the rest (it's not strictly about cleaning up the top-level of the directory).

Note, however, that we will likely run into this issue again soon, namely when people start to use pytest fixtures in the test suite more actively.

[sphuber]

I am just playing around with this now and it indeed seems a problem in the two different ways be currently setup and clean the database. By adding the fixture, we ensure that a user instance is created for the pytest tests, however, this interferes with the setup of the AiidaTestCase tests and particularly with the migrations. Given that already the moving is really a bit outside of the scope of this PR, I would vote to remove it for now and continue in a separate issue.

[ltalirz]

Fine with me - can you take care of it?

[sphuber]

Fine with me - can you take care of it?

done

[ltalirz]

Looks good to me - you'll need to approve yourself ;-)

[sphuber]

Wij van WC-eend adviseren WC-eend 👍