Fix HEAD method for robots.txt

[wxiaoguang]

Fix #30601

~$ curl --head localhost:3000/robots.txt
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=0, private, must-revalidate
Content-Length: 5
Content-Type: text/plain; charset=utf-8
Last-Modified: Wed, 19 Jul 2023 04:56:12 GMT
X-Gitea-Debug: RUN_MODE=dev
Date: Fri, 19 Apr 2024 12:59:44 GMT

[silverwind]

HEAD requests by definition should not have a body, e.g. no Content-Length or Content-Length: 0 only. As per MDN:

Warning: A response to a HEAD method should not have a body. If it has one anyway, that body must be ignored: any representation headers that might describe the erroneous body are instead assumed to describe the response which a similar GET request would have received.

[wxiaoguang]

HEAD requests by definition must not have a body, e.g. no Content-Length or Content-Length: 0 only.

Are you sure? Would you like to learn it first before putting a change request?

https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD

[silverwind]

Ah, Content-Length is fine, but it does not send any actual content, right?

dismiss

[silverwind]

@wxiaoguang why are you dismissing without answering my question? That's rude.

[wxiaoguang]

Ah, Content-Length is fine, but it does not send any actual content, right?

It calls Golang's standard HTTP library, Golang already handles it correctly.

[wxiaoguang]

@wxiaoguang why are you dismissing without answering my question? That's rude.

I think the question has been answered and the concern has been addressed. All are standard.

[silverwind]

Ok, I think we should enable HEAD methods on all these GET methods defined in Routes.

[wxiaoguang]

Ok, I think we should enable HEAD methods on all these GET methods defined in Routes.

Actually no. Because most dynamic route handlers do not support HEAD.

Golang standard library only handles "file serving".

[silverwind]

@wxiaoguang why are you dismissing without answering my question? That's rude.

I think the question has been answered and the concern has been addressed. All are standard.

Blocks should be dismissed by their author only, unless the author is unresponsive, then it's fine when someone else dismisses it.

[silverwind]

Ok, I think we should enable HEAD methods on all these GET methods defined in Routes.

Actually no. Because most dynamic route handlers do not support HEAD.

Golang standard library only handles "file serving".

Yeah, it would be needed to be coded there. But I see we can maybe enable it for /metrics.

[wxiaoguang]

@wxiaoguang why are you dismissing without answering my question? That's rude.

I think the question has been answered and the concern has been addressed. All are standard.

Blocks should be dismissed by their author only, unless the author is unresponsive, then it's fine when someone else dismisses it.

TBH I was a little unhappy about that change request, I also felt a little rude of it too. Because, it seems that you didn't really look into it (your referred document also said the "content-length" could exist). Hopefully you could also understand my feeling.

Yeah, it would be needed to be coded there. But I see we can maybe enable it for /metrics.

If and only if the handlers really support HEAD method.

[silverwind]

My conclusion is that it's a waste of server resources to send Content-Length in HEAD because you are basically building a response body and then immediately throwing it away.

So a proper HEAD implementation would be a custom implementation and not that wasteful thing that golang does.