Import error read endpoints and schema

[OmairK]

Closes #8130

Make sure to mark the boxes below before creating PR: [x]

• Description above provides context of the change
• Unit tests coverage for changes (not needed for documentation changes)
• Target Github ISSUE in description if exists
• Commits follow "How to write a good git commit message"
• Relevant documentation is updated including usage instructions.
• I will engage committers as explained in Contribution Workflow Example.

---

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.

[mik-laj]

😻

[mik-laj]

We are very close to the end. The main problem is static checking. As long as the change is not green (excluding quarantine tests), I cannot merge it.
Here is a guide about static checks: https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst
Can you do a rebase also? We have a merge conflicts.

[mik-laj]

tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorEndpoint::test_response_200
FAILED tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorsEndpoint::test_get_import_errors
FAILED tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorsEndpointPagination::test_limit_and_offset_0__api_v1_importErrors_limit_1
FAILED tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorsEndpointPagination::test_limit_and_offset_1__api_v1_importErrors_limit_110
FAILED tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorsEndpointPagination::test_limit_and_offset_2__api_v1_importErrors_offset_1
FAILED tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorsEndpointPagination::test_limit_and_offset_3__api_v1_importErrors_offset_3
FAILED tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorsEndpointPagination::test_limit_and_offset_4__api_v1_importErrors_offset_3_limit_3

Some API-related tests were unsuccessful. Could you look at it?

[OmairK]

tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorEndpoint::test_response_200
FAILED tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorsEndpoint::test_get_import_errors
FAILED tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorsEndpointPagination::test_limit_and_offset_0__api_v1_importErrors_limit_1
FAILED tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorsEndpointPagination::test_limit_and_offset_1__api_v1_importErrors_limit_110
FAILED tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorsEndpointPagination::test_limit_and_offset_2__api_v1_importErrors_offset_1
FAILED tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorsEndpointPagination::test_limit_and_offset_3__api_v1_importErrors_offset_3
FAILED tests/api_connexion/endpoints/test_import_error_endpoint.py::TestGetImportErrorsEndpointPagination::test_limit_and_offset_4__api_v1_importErrors_offset_3_limit_3

Some API-related tests were unsuccessful. Could you look at it?

That's weird, these tests are running fine in my breeze environment(both python3.6 python 3.7)

[mik-laj]

@OmairK Okay. I looked at what the problem was. The problem is that clear_db_import_errors function doesn't reset the sequence. Each newly added element has a new identifier. However, if the object was created in another test, then the numbering continues in your test. If you run one test several times, only the first run will be correct, but all others will fail. I have prepared a patch that normalizes identifiers in responses.
https://github.com/apache/airflow/pull/9217/commits
This should help. What do you think about this?

[OmairK]

@OmairK Okay. I looked at what the problem was. The problem is that clear_db_import_errors function doesn't reset the sequence. Each newly added element has a new identifier. However, if the object was created in another test, then the numbering continues in your test. If you run one test several times, only the first run will be correct, but all others will fail. I have prepared a patch that normalizes identifiers in responses.
https://github.com/apache/airflow/pull/9217/commits
This should help. What do you think about this?

Ah it was the sequence all along, I get it thank you so much for the help. A couple of quick questions

1. I was running these tests locally by ./breeze test-target tests/api_connexion/ these tests should have failed there too unless the sequence was automatically resetting after every test, which doesn't make sense. Is there some major difference between how breeze and github action CI handles the db state between tests ?
2. If yes, how do you reenact your airflow github actions locally ? (This would help me a lot in the future)

P.S.: +1 for the commit message, it perfectly sums up the situation :)

[mik-laj]

1. I will reproduce this problem locally. I launched Breeze with Postgres and Python 3.8 and have run the following command:

pytest tests/api_connexion/endpoints/test_import_error_endpoint.py

I suspect you use SQLite, which sometimes behaves differently

[mik-laj]

Caa you do a rebase to the latest master?

Is this still WIP? There is still a label in the PR title.

[OmairK]

Is this still WIP? There is still a label in the PR title.

No, I had 2 PRs to be merged at the same time so I made this PR WIP just to keep track.