schema tests

[A-Harby]

Description

Add tests for the schema checking server error, response code, content type, response schema, and response headers against the proxy schema itself.

Changes

Add a tests.py file, requirements.txt, and workflow.yml.

Related Issues

#51

Checklist

• Tests included
• Build pass
• Documentation
• Code format and docstring

[Omarabdul3ziz]

Good job ya @A-Harby
I have some comments, as far as I understood schemathesis reads the openapi spec and try to make each call to the server.

• we have more live proxy servers than the ones running on *.grid.tf and i don't see a need to test them all actually tests will fail cause it reads the specs from the pushed code which may contains some changes not deployed yet on the running server, so i guess it will be enough to test on push/pull-request for the server running from the code itself. what do you think?
• also, i see there is already an action from schemathsis themselves https://github.com/schemathesis/action, did you consider using it?

[A-Harby]

• we have more live proxy servers than the ones running on *.grid.tf and i don't see a need to test them all actually tests will fail cause it reads the specs from the pushed code which may contains some changes not deployed yet on the running server, so i guess it will be enough to test on push/pull-request for the server running from the code itself. what do you think?

I added the trigger on the PR on the main branch and just left the manual trigger for the test run on every network (to make sure everything is working fine).

• also, i see there is already an action from schemathsis themselves https://github.com/schemathesis/action, did you consider using it?

That does need a subscription, and running the coding does pretty much the same.

[Omarabdul3ziz]

correct me if I got it wrong, schematest checks the openapi spec file and do all the possible requests to the target servers.
it is frequently that we update these specs like adding new filters or changing the response schemes.
the code will read the specs from the pushed code - which will contain the changes - but will try to make the request to the running server - which doesn't have the changes yet - so it will fail.

isn't it enough to run the test once on the push/pr? if we do need to run it on all the servers can we get the specs for each server running? maybe from https://gridproxy.dev.grid.tf/swagger/doc.json instead of reading it from the codebase

[A-Harby]

isn't it enough to run the test once on the push/pr? if we do need to run it on all the servers can we get the specs for each server running? maybe from https://gridproxy.dev.grid.tf/swagger/doc.json instead of reading it from the codebase

I did that in 1811cac, the PR should be ready for review.

[Omarabdul3ziz]

code looks good to me, but the test fails due to some needed updates in the schema. we need to take care of this before merge this pr.
i will try to resolve them or you @A-Harby are very welcome to help :D

[sameh-farouk]

@Omarabdul3ziz Any schema or API fixes are outside the scope of this PR, so we can merge it (since it's working as expected).

We should create new issues/PRs for the discovered issues.