New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix serving files that clash with directories (#6222) #6231

Merged
merged 1 commit into from Jul 25, 2017

Conversation

Projects
None yet
7 participants
@MattSturgeon
Contributor

MattSturgeon commented Jul 18, 2017

Fix #6222.

To avoid having to re-implement the entire set_filename method, I instead extended search_index_file and temporarily mutate res.filename. Kinda hacky, so I tried to leave plenty of inline comments. (Also first time using ruby, so beware!)

For reference, here's the base class search_index_file implementation.

Feel free to checkout the commit message too.

@pathawks pathawks requested a review from jekyll/stability Jul 18, 2017

@pathawks

This comment has been minimized.

Show comment
Hide comment
@pathawks
Member

pathawks commented Jul 18, 2017

@pathawks

This comment has been minimized.

Show comment
Hide comment
@pathawks

pathawks Jul 18, 2017

Member

Thank you @MattSturgeon for learning Ruby just to fix this! 🍻

Member

pathawks commented Jul 18, 2017

Thank you @MattSturgeon for learning Ruby just to fix this! 🍻

@MattSturgeon

This comment has been minimized.

Show comment
Hide comment
@MattSturgeon

MattSturgeon Jul 19, 2017

Contributor

No problem, feel free to recommend any changes if required.

Just noticed the first mistake, I linked to the wrong super method in the commit message! Should be this one instead, so I'll amend the commit and we can wait for the CI builds to start from scratch!

Contributor

MattSturgeon commented Jul 19, 2017

No problem, feel free to recommend any changes if required.

Just noticed the first mistake, I linked to the wrong super method in the commit message! Should be this one instead, so I'll amend the commit and we can wait for the CI builds to start from scratch!

@ashmaroli

Hi, some minor changes:

  • == and === check for equality
  • = assigns a value to a variable
if File.extname == ".foo"
  puts "Foo Ahoy!"
  baz = "foobar"
end
Show outdated Hide outdated lib/jekyll/commands/serve/servlet.rb
@MattSturgeon

This comment has been minimized.

Show comment
Hide comment
@MattSturgeon

MattSturgeon Jul 19, 2017

Contributor

I've tried to make the code more readable, following @ashmaroli's feedback.

In 20a333f I replaced a conditional return with an unless branch and removed two out of three cases of assignment within conditions. I also re-write most of the comments.

It's a fixup commit, so intended to be rebased/autosquashed into the original commit, just thought I'd keep it separate for now in case anyone wanted to compare it with the original PR (?w=1 is your friend here).

Contributor

MattSturgeon commented Jul 19, 2017

I've tried to make the code more readable, following @ashmaroli's feedback.

In 20a333f I replaced a conditional return with an unless branch and removed two out of three cases of assignment within conditions. I also re-write most of the comments.

It's a fixup commit, so intended to be rebased/autosquashed into the original commit, just thought I'd keep it separate for now in case anyone wanted to compare it with the original PR (?w=1 is your friend here).

@ashmaroli

This comment has been minimized.

Show comment
Hide comment
@ashmaroli

ashmaroli Jul 19, 2017

Member

FYI, Currently, Jekyll's merge bot will squash all commits irrespective of the --fixup flag

Member

ashmaroli commented Jul 19, 2017

FYI, Currently, Jekyll's merge bot will squash all commits irrespective of the --fixup flag

@oe

oe approved these changes Jul 19, 2017

very nice! +1 for the extra documentation

Show outdated Hide outdated lib/jekyll/commands/serve/servlet.rb
@MattSturgeon

This comment has been minimized.

Show comment
Hide comment
@MattSturgeon

MattSturgeon Jul 19, 2017

Contributor

Ok, good to know. I think this is probably ready now, I've pushed up a slightly re-worded commit with all changes including switching to index based splitting:

i = res.filename.rindex "\/"
basename = res.filename[i..-1]
res.filename = res.filename[0, i]

And simply appending basename back onto filename instead of copying a backup.

From what I could tell, trailing slashes are unlikely to ever be an issue with WEBrick so I shifted to simply splitting on the index of the final / char without messing around with arrays or loops.

Contributor

MattSturgeon commented Jul 19, 2017

Ok, good to know. I think this is probably ready now, I've pushed up a slightly re-worded commit with all changes including switching to index based splitting:

i = res.filename.rindex "\/"
basename = res.filename[i..-1]
res.filename = res.filename[0, i]

And simply appending basename back onto filename instead of copying a backup.

From what I could tell, trailing slashes are unlikely to ever be an issue with WEBrick so I shifted to simply splitting on the index of the final / char without messing around with arrays or loops.

@ashmaroli

This comment has been minimized.

Show comment
Hide comment
@ashmaroli

ashmaroli Jul 19, 2017

Member

What we need now are tests.. 😈

Member

ashmaroli commented Jul 19, 2017

What we need now are tests.. 😈

@MattSturgeon

This comment has been minimized.

Show comment
Hide comment
@MattSturgeon

MattSturgeon Jul 19, 2017

Contributor

Where are the existing tests for the serve command?

Contributor

MattSturgeon commented Jul 19, 2017

Where are the existing tests for the serve command?

@ashmaroli

This comment has been minimized.

Show comment
Hide comment
@MattSturgeon

This comment has been minimized.

Show comment
Hide comment
@MattSturgeon

MattSturgeon Jul 19, 2017

Contributor

How did I miss that? I did look, I promise!

Contributor

MattSturgeon commented Jul 19, 2017

How did I miss that? I did look, I promise!

@MattSturgeon

This comment has been minimized.

Show comment
Hide comment
@MattSturgeon

MattSturgeon Jul 19, 2017

Contributor

I'm probably missing something, but I'm not sure there's any testable changes here. Short of running the server, and making a mini client within the test suit to assert that HTTP requests return expected results.

The tests for command serve seem to be mostly sanity checking configuration options to make sure config files and command line switches have taken affect.

I'd need to run the command asynchronously, then fire off a HTTP request to a few URLs that I know are either directories, files or both and assert that the result matches the local file I wanted.

Is this within scope of the tests? It sounds more complex than the entire feature to me. It doesn't seem to be something that was added for #3452

What kind of things should I be testing for this change?

Contributor

MattSturgeon commented Jul 19, 2017

I'm probably missing something, but I'm not sure there's any testable changes here. Short of running the server, and making a mini client within the test suit to assert that HTTP requests return expected results.

The tests for command serve seem to be mostly sanity checking configuration options to make sure config files and command line switches have taken affect.

I'd need to run the command asynchronously, then fire off a HTTP request to a few URLs that I know are either directories, files or both and assert that the result matches the local file I wanted.

Is this within scope of the tests? It sounds more complex than the entire feature to me. It doesn't seem to be something that was added for #3452

What kind of things should I be testing for this change?

@ashmaroli

needs some more edits..

Show outdated Hide outdated lib/jekyll/commands/serve/servlet.rb
Show outdated Hide outdated lib/jekyll/commands/serve/servlet.rb
Show outdated Hide outdated lib/jekyll/commands/serve/servlet.rb
@ashmaroli

This comment has been minimized.

Show comment
Hide comment
@ashmaroli

ashmaroli Jul 19, 2017

Member

What kind of things should I be testing for this change?

Mainly, if the original issue you reported has been fixed and stays fixed..

Member

ashmaroli commented Jul 19, 2017

What kind of things should I be testing for this change?

Mainly, if the original issue you reported has been fixed and stays fixed..

@MattSturgeon

This comment has been minimized.

Show comment
Hide comment
@MattSturgeon

MattSturgeon Jul 19, 2017

Contributor

What kind of things should I be testing for this change?

Mainly, if the original issue you reported has been fixed and stays fixed..

IMO that's not practical to test. Unless you have some test suite that can compare files served over HTTP to files on the filesystem?

Contributor

MattSturgeon commented Jul 19, 2017

What kind of things should I be testing for this change?

Mainly, if the original issue you reported has been fixed and stays fixed..

IMO that's not practical to test. Unless you have some test suite that can compare files served over HTTP to files on the filesystem?

@envygeeks

This comment has been minimized.

Show comment
Hide comment
@envygeeks

envygeeks Jul 20, 2017

Contributor

This is a lot of complicated code for such a simple problem.

Contributor

envygeeks commented Jul 20, 2017

This is a lot of complicated code for such a simple problem.

@MattSturgeon

This comment has been minimized.

Show comment
Hide comment
@MattSturgeon

MattSturgeon Jul 20, 2017

Contributor

Well the problem isn't as simple as it sounds, but I wouldn't say the solution was complex or even large (10 lines).

set_filename passes us a directory path (res.filename) and it expects us to return a file from within that directory to use as an index file (usually index.html).

We want to abuse this slightly and return a file from the parent directory of the one we have, so we have to modify res.filename and look for files in the modified directory path.

If we fail, we undo our modification so that we don't break WEBrick's Fancy Indexing.

Are you saying that it isn't worth this amount of complexity? Also, are you happy with the modification to your /file.* > /file/index.html > /file.html comment?

Contributor

MattSturgeon commented Jul 20, 2017

Well the problem isn't as simple as it sounds, but I wouldn't say the solution was complex or even large (10 lines).

set_filename passes us a directory path (res.filename) and it expects us to return a file from within that directory to use as an index file (usually index.html).

We want to abuse this slightly and return a file from the parent directory of the one we have, so we have to modify res.filename and look for files in the modified directory path.

If we fail, we undo our modification so that we don't break WEBrick's Fancy Indexing.

Are you saying that it isn't worth this amount of complexity? Also, are you happy with the modification to your /file.* > /file/index.html > /file.html comment?

@envygeeks

This comment has been minimized.

Show comment
Hide comment
@envygeeks

envygeeks Jul 20, 2017

Contributor

No, I'm not happy with the comment change, because it's arbitrary. I'm happy you learned Ruby and welcome to the Ruby community! This can be fixed by changing search file to super(req, res, ".html") || super || super(req, res, "#{basename}.html") without the extra 9 lines of code.

Sorry it took me so long to respond.

Contributor

envygeeks commented Jul 20, 2017

No, I'm not happy with the comment change, because it's arbitrary. I'm happy you learned Ruby and welcome to the Ruby community! This can be fixed by changing search file to super(req, res, ".html") || super || super(req, res, "#{basename}.html") without the extra 9 lines of code.

Sorry it took me so long to respond.

@MattSturgeon

This comment has been minimized.

Show comment
Hide comment
@MattSturgeon

MattSturgeon Jul 20, 2017

Contributor

Ah, much simpler. Probably should be ordered super || super(req, res, ".html") || super(req, res, "#{basename}.html") to prioritise the default.

As soon as I noticed search_index_file was used when searching a directory, I focused on that and didn't notice that you could just use ".html" in search_file and it would concatenate into a sane filename.

I've added your changes as 1dc2bc7.

Contributor

MattSturgeon commented Jul 20, 2017

Ah, much simpler. Probably should be ordered super || super(req, res, ".html") || super(req, res, "#{basename}.html") to prioritise the default.

As soon as I noticed search_index_file was used when searching a directory, I focused on that and didn't notice that you could just use ".html" in search_file and it would concatenate into a sane filename.

I've added your changes as 1dc2bc7.

@envygeeks

This comment has been minimized.

Show comment
Hide comment
@envygeeks

envygeeks Jul 20, 2017

Contributor

No, it should be in the order I put it in, as that was my intention, and it was my intention with this source to begin with. You should also rebase, squash, and only have commits in your name, it is bad form to commit in another's name, it should instead be credited in the extended message (if you wish,) as committing in my name forces me to have to say "that commit is not mine, it's not signed" in the future, should a problem arise, and it also alters the blame in a confusing way on Github. Sorry to be weird.

Thanks!

Contributor

envygeeks commented Jul 20, 2017

No, it should be in the order I put it in, as that was my intention, and it was my intention with this source to begin with. You should also rebase, squash, and only have commits in your name, it is bad form to commit in another's name, it should instead be credited in the extended message (if you wish,) as committing in my name forces me to have to say "that commit is not mine, it's not signed" in the future, should a problem arise, and it also alters the blame in a confusing way on Github. Sorry to be weird.

Thanks!

@envygeeks

This comment has been minimized.

Show comment
Hide comment
@envygeeks

envygeeks Jul 20, 2017

Contributor

I guess what I am saying is, I do wish to take blame if something goes wrong (and I'll happily help fix it,) but the commit should not be in my name, you should instead opt to link to my comment in the extended commit, so that people know that the source was originally my idea, and it's my fault if something is wrong.

Contributor

envygeeks commented Jul 20, 2017

I guess what I am saying is, I do wish to take blame if something goes wrong (and I'll happily help fix it,) but the commit should not be in my name, you should instead opt to link to my comment in the extended commit, so that people know that the source was originally my idea, and it's my fault if something is wrong.

@MattSturgeon

This comment has been minimized.

Show comment
Hide comment
@MattSturgeon

MattSturgeon Jul 20, 2017

Contributor

I didn't commit in your name, I listed you as the author of the changes. That's what --author is for, committing other's changes. It still list's me as the committer, so I still get the blame for any issues with the commit.

commit 1dc2bc74580060d4f00dd4a45e02a70574a56672
tree 4207e3ea2b7b52733fb6d3e3a85d5cc00d6b6522
parent 56546a28fda95098c44b31090050db0f415be574
author Jordon Bedwell <jordon@********> 1500420691 +0100
committer Matt Sturgeon <matt@********> 1500521221 +0100

    Fix serving files that clash with directories

Why did you want it to check ".html" before the default check of basename? Surely basename -> ".html" -> "basename.html" or basename -> "basename.html" -> ".html" makes more sense?


Either way since it is your change now, it is probably easiest if you make a separate PR.

Contributor

MattSturgeon commented Jul 20, 2017

I didn't commit in your name, I listed you as the author of the changes. That's what --author is for, committing other's changes. It still list's me as the committer, so I still get the blame for any issues with the commit.

commit 1dc2bc74580060d4f00dd4a45e02a70574a56672
tree 4207e3ea2b7b52733fb6d3e3a85d5cc00d6b6522
parent 56546a28fda95098c44b31090050db0f415be574
author Jordon Bedwell <jordon@********> 1500420691 +0100
committer Matt Sturgeon <matt@********> 1500521221 +0100

    Fix serving files that clash with directories

Why did you want it to check ".html" before the default check of basename? Surely basename -> ".html" -> "basename.html" or basename -> "basename.html" -> ".html" makes more sense?


Either way since it is your change now, it is probably easiest if you make a separate PR.

@parkr

parkr approved these changes Jul 21, 2017

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jul 21, 2017

Member

Thank you, all! I'm guessing the comment directly above this line is fine?

Member

parkr commented Jul 21, 2017

Thank you, all! I'm guessing the comment directly above this line is fine?

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jul 21, 2017

Member

Additionally, a test would be nice.

Member

parkr commented Jul 21, 2017

Additionally, a test would be nice.

@pathawks

This comment has been minimized.

Show comment
Hide comment
@pathawks

pathawks Jul 21, 2017

Member

Additionally, a test would be nice.

Suggestions would be appreciated. See above comment:

I'm probably missing something, but I'm not sure there's any testable changes here. Short of running the server, and making a mini client within the test suit to assert that HTTP requests return expected results.
...
What kind of things should I be testing for this change?

Member

pathawks commented Jul 21, 2017

Additionally, a test would be nice.

Suggestions would be appreciated. See above comment:

I'm probably missing something, but I'm not sure there's any testable changes here. Short of running the server, and making a mini client within the test suit to assert that HTTP requests return expected results.
...
What kind of things should I be testing for this change?

@envygeeks

This comment has been minimized.

Show comment
Hide comment
@envygeeks

envygeeks Jul 21, 2017

Contributor

This source was never really tested to begin with, I don't know if I would force that constraint on the new author, I'll take responsibility outside of this pull request if necessary, it shouldn't really fall on him.

Contributor

envygeeks commented Jul 21, 2017

This source was never really tested to begin with, I don't know if I would force that constraint on the new author, I'll take responsibility outside of this pull request if necessary, it shouldn't really fall on him.

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jul 25, 2017

Member

Ok! I'd love to get that module tested so we don't break this behaviour in the future, but it's not a super high priority. Thanks, all!

@jekyllbot: merge +minor

Member

parkr commented Jul 25, 2017

Ok! I'd love to get that module tested so we don't break this behaviour in the future, but it's not a super high priority. Thanks, all!

@jekyllbot: merge +minor

@jekyllbot jekyllbot merged commit ec84bec into jekyll:master Jul 25, 2017

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

jekyllbot added a commit that referenced this pull request Jul 25, 2017

@envygeeks

This comment has been minimized.

Show comment
Hide comment
@envygeeks

envygeeks Jul 26, 2017

Contributor

This commit is not what my code originally was, what it intended, and it still has my name stamped all over it as the author even though the person who used my name falsely refused to fix the problem, it is merged now and my name is associated with something I didn't write, please fix this problem.

Contributor

envygeeks commented Jul 26, 2017

This commit is not what my code originally was, what it intended, and it still has my name stamped all over it as the author even though the person who used my name falsely refused to fix the problem, it is merged now and my name is associated with something I didn't write, please fix this problem.

@MattSturgeon MattSturgeon deleted the MattSturgeon:fix-6222 branch Jul 26, 2017

@MattSturgeon

This comment has been minimized.

Show comment
Hide comment
@MattSturgeon

MattSturgeon Jul 26, 2017

Contributor

@envygeeks your name is not on the merged commit (ec84bec)

image

Contributor

MattSturgeon commented Jul 26, 2017

@envygeeks your name is not on the merged commit (ec84bec)

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment