Skip to content
This repository has been archived by the owner on Jan 10, 2023. It is now read-only.

darion: Use http.ServeContent for serving log data #102

Merged
merged 1 commit into from
May 14, 2018

Conversation

cHYzZQo
Copy link
Contributor

@cHYzZQo cHYzZQo commented May 7, 2018

This allows us to support range serving ranges.

I'm not sure I fully understood the way the virtual file mapping is to supposed to work, so while I believe the behavior will be the same as the old code, that part might need some extra scrutiny.

@sargun
Copy link
Contributor

sargun commented May 7, 2018

Hrm. Gotta figure out how to fix this.

@cHYzZQo cHYzZQo force-pushed the range-requests branch 2 times, most recently from 98b9db4 to f800a25 Compare May 8, 2018 00:00
@coveralls
Copy link

coveralls commented May 8, 2018

Pull Request Test Coverage Report for Build 799

  • 11 of 12 (91.67%) changed or added relevant lines in 1 file are covered.
  • 592 unchanged lines in 10 files lost coverage.
  • Overall coverage increased (+0.5%) to 26.129%

Changes Missing Coverage Covered Lines Changed/Added Lines %
darion/api/handlers.go 11 12 91.67%
Files with Coverage Reduction New Missed Lines %
api/netflix/titus/titus_agent_api.pb.go 8 4.64%
executor/runtime/types/types.go 11 49.09%
config/config.go 16 92.21%
executor/runtime/docker/docker_linux.go 20 0.0%
vpc/limits.go 23 0.0%
executor/metatron/metatron.go 31 0.0%
executor/runner/runner.go 41 47.98%
api/netflix/titus/agent.pb.go 86 18.68%
api/netflix/titus/titus_base.pb.go 103 4.73%
executor/runtime/docker/docker.go 253 2.46%
Totals Coverage Status
Change from base Build 784: 0.5%
Covered Lines: 2366
Relevant Lines: 9055

💛 - Coveralls

@codecov
Copy link

codecov bot commented May 8, 2018

Codecov Report

Merging #102 into master will decrease coverage by 10.67%.
The diff coverage is 83.33%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master     #102       +/-   ##
===========================================
- Coverage   34.97%   24.29%   -10.68%     
===========================================
  Files          65       64        -1     
  Lines        7302     7157      -145     
===========================================
- Hits         2554     1739      -815     
- Misses       4439     5219      +780     
+ Partials      309      199      -110
Impacted Files Coverage Δ
darion/api/handlers.go 65.51% <83.33%> (+4.73%) ⬆️
executor/mock/jobrunner.go 0% <0%> (-74.79%) ⬇️
vpc/bpfloader/bpfloader_linux.go 0% <0%> (-64.11%) ⬇️
executor/runtime/docker/docker_linux.go 0% <0%> (-50%) ⬇️
executor/runtime/docker/docker.go 2.39% <0%> (-46.66%) ⬇️
executor/runtime/types/types.go 47.61% <0%> (-36.7%) ⬇️
executor/drivers/titusdriver.go 50% <0%> (-25%) ⬇️
uploader/uploader.go 26.38% <0%> (-23.62%) ⬇️
executor/runner/runner.go 41.64% <0%> (-18.36%) ⬇️
filesystems/xattr/degrading_linux.go 75% <0%> (-14.29%) ⬇️
... and 12 more

@andrew-leung
Copy link
Contributor

FYI, this is for TITUS-1922 (which I realize doesn't help anyone external to Netflix).

@andrew-leung
Copy link
Contributor

@cHYzZQo, looks like the CI tests are now passing. Ready for review?

@sargun
Copy link
Contributor

sargun commented May 9, 2018

@cHYzZQo Just curious -- do you need this both on stdio and normal log files, or just on normal log files?

@cHYzZQo
Copy link
Contributor Author

cHYzZQo commented May 9, 2018

@sargun yes ready for review. Both stdio and normal log files would be good.

@sargun
Copy link
Contributor

sargun commented May 9, 2018

Okay, let me mull this over and I'll look at it EOD.

sargun
sargun previously requested changes May 14, 2018
Copy link
Contributor

@sargun sargun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Part of my almost wants to say not supported on the virtual log files. Is that a case you really need to support?

}

resp, e := client.Do(req)
if e != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -168,7 +215,7 @@ func verifyHelper(resp *http.Response, t *testing.T) string {
t.Fatal("Could not read response body: ", err)
}
dataStr := string(data)
if resp.StatusCode != 200 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When will we receive not 200? When it's range request not satisfiable?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that's the case, we should probably explicitly check for that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a range is requested http specs say the server should respond with a 206 "Partial Content".

In general though, any code between 200 and 399 is considered a success. And success checks should allow for anything in that range. For example 304s would be pretty common. But I can change the test to only allow 200 or 206.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW the code for unsatisfiable range requests is 416.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want add a specific test checking for the 416 case, and for the 206 case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, added. Though at that point we are testing the http lib not our code.

@cHYzZQo cHYzZQo force-pushed the range-requests branch 2 times, most recently from e644a86 to c3d2cba Compare May 14, 2018 16:10
This allows us to support range serving ranges
@cHYzZQo cHYzZQo dismissed sargun’s stale review May 14, 2018 16:33

Unfortunatly i think we need to support virtual files for stdout and err. Made the other suggested changes

@sargun sargun merged commit 11b8afb into Netflix:master May 14, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants