Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test for the new caching configuration options (#810) #199

Merged
merged 7 commits into from
Feb 5, 2022
Merged

Conversation

byko3y
Copy link
Contributor

@byko3y byko3y commented Jan 17, 2022

No description provided.

Copy link
Contributor

@krizhanovsky krizhanovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least these tests are missed:

  1. Missing functional tests for cache tempesta#810 (comment) max-age and all similar directives for requests and responses. Need to test that they work as expected, i.e. cache entries are retrieved or not depending on the directives
  2. testing point 3 from Automatic Platform Optimization tempesta#1516 (there are link to error output when Tempesta config is empty. #530)
  3. Test the header Cache-Control: public, no-cache="Set-Cookie", must-revalidate, max-age=120 and make sure that Set-Cookie isn't stored in the cache. Test other headers in no-cache. Test list of headers in no-cache (test case 3 from Cache behavior tuning tempesta#530)

cache/test_cache_control.py Outdated Show resolved Hide resolved
cache/test_cache_control.py Outdated Show resolved Hide resolved
cache/test_cache_control.py Outdated Show resolved Hide resolved
Ported to "framework".
Extended the test suit for additional Cache-Control directives.
Added PURGE-GET tests for header removal.
@byko3y
Copy link
Contributor Author

byko3y commented Jan 31, 2022

Did a complete rework of the test_cache_control suite to use framework.tester module instead of the legacy functional.FunctionalTest.
Extended the test suite to test additional directives, implemented the #1516 PURGE tests

Copy link
Contributor

@krizhanovsky krizhanovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Besides the comments, I also didn't find no-transform for either requests or responses and no-cache for requests.

Also I didn't do too deep review since I didn't touch the tests for a while, so I appreciate if @Dmitry-Gouriev also review the PR.

framework/tester.py Outdated Show resolved Hide resolved
framework/tester.py Show resolved Hide resolved
Copy link
Contributor

@Dmitry-Gouriev Dmitry-Gouriev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!
However it seems it is not finished yet.

cache/test_cache_control.py Outdated Show resolved Hide resolved
cache/test_cache_control.py Show resolved Hide resolved
cache/test_cache_control.py Outdated Show resolved Hide resolved
cache/test_cache_control.py Outdated Show resolved Hide resolved
cache/test_cache_control.py Show resolved Hide resolved
framework/tester.py Outdated Show resolved Hide resolved
Also removed duplicates of wait_all_connections() and fixed incorrect
dmesg string in the start_tempesta().
@byko3y
Copy link
Contributor Author

byko3y commented Feb 2, 2022

Updated the pull request. Also, there was a small bug in tester.start_all_servers(), where dmesg.wait_for_msg('[tempesta fw] modules are started', 1, True) was waiting for a string logged only on special debug builds of Tempesta FW, so wait_for_msg() was timing out on every test, thus making the tests run much slower.

Copy link
Contributor

@krizhanovsky krizhanovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for me, but there are comments from @Dmitry-Gouriev . Once the comments are resolved, the PR can be merged.

Copy link
Contributor

@Dmitry-Gouriev Dmitry-Gouriev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

* Fixed wrong check for second response status.
* Larger windows for cache lifetimes.
@Dmitry-Gouriev
Copy link
Contributor

Dmitry-Gouriev commented Feb 4, 2022

Looks fine in general.
I have some questions which are not critics.

  • Probably, windows could be even more large, taking in mind network depays.
  • ResponseCached3 seems to be non-informative name.

Also I wonder how you treat an Authorization: header, however this seems to be in accordance with never HTTP spec RFC 7234 3.2. Besides ResponsePublicCached2. Although your remark

Interestingly enough, RFC 7234 does not forbid serving cached response for
subsequent requests with "Authorization" header.

I can not inderstand why this response may be cached.

@byko3y
Copy link
Contributor Author

byko3y commented Feb 4, 2022

RFC 7234 only forbids serving responses from the first "Authorization" request to subsequent requests, it does not tell what to do when subsequent requests turn out to have "Authorization" header while the first request had no such header.
However, speaking about application logic, the server might need those subsequent "Authorization" requests to authorize some new clients while Tempesta FW by default serves them from cache, and thus clients cannot be authorized.

Talking about windows -- I'd be glad to make those larger, but that would make the tests run even longer. And we already have one test running for 3 minutes.

@Dmitry-Gouriev
Copy link
Contributor

Dmitry-Gouriev commented Feb 4, 2022

However, speaking about application logic, the server might need those subsequent "Authorization" requests to authorize some new clients while Tempesta FW by default serves them from cache, and thus clients cannot be authorized.

I'll simple repeat my response from private discussion.

When authorization is required, and Authorization: header is not present, the server normaly responds with 401 and such response may not be cached. If the server responds with 2xx this means that the particular resource is for public access, so can be responded from a cache with Authorization: header and without it. Analysing this case you have convinced me that everything goes OK and my question was unnecessary.

The only doubtful case remaining is when access rules change at server side and cached resource becomes for resticted access. I hope, cache cleaning or proxy restart can help in this rare situation.

@krizhanovsky
Copy link
Contributor

I found a mention about the case, when people need to cache responses for requests with Authorization header. It seems from the tests involving Authorization header in a request we are compliant with RFC 7234 3.2.

Regarding the network delays I think we're good with 1-2 second delays since the test suite already runs for a very long time and we either run it in a local host or in LAN, usually even inside a single physical server, so latency is quite small in our environments.

@byko3y byko3y merged commit 6c22c94 into master Feb 5, 2022
@krizhanovsky krizhanovsky deleted the pk-810 branch June 23, 2022 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants