Attachments: Add extension support, allow all types for releases

[silverwind]

• Add support for file extensions, matching the accept attribute of <input type="file">
• Add support for type wildcard mime types, e.g. image/*
• Create repository.release.ALLOWED_TYPES setting (default unrestricted)
• Change default for attachment.ALLOWED_TYPES to a list of extensions
• Split out POST /attachments into two endpoints for issue/pr and releases to prevent circumvention of allowed types check

Fixes: #10172
Fixes: #7266
Fixes: #12460
Ref: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/file#Unique_file_type_specifiers

[silverwind]

Will do a few tweaks soon like using path over filepath because browsers generally only send Unix-style paths. I guess this may not fully resolve #8229 but at least users will not hit those mime-type problems any more in the default configuration.

[codecov-commenter]

Codecov Report

Merging #12465 into master will increase coverage by 0.03%.
The diff coverage is 72.97%.

@@            Coverage Diff             @@
##           master   #12465      +/-   ##
==========================================
+ Coverage   43.80%   43.83%   +0.03%     
==========================================
  Files         630      631       +1     
  Lines       69821    69892      +71     
==========================================
+ Hits        30585    30640      +55     
- Misses      34300    34305       +5     
- Partials     4936     4947      +11     

Impacted Files	Coverage Δ
modules/setting/repository.go	57.69% <ø> (ø)
modules/setting/setting.go	46.44% <ø> (ø)
routers/api/v1/repo/release_attachment.go	0.00% <0.00%> (ø)
routers/repo/editor.go	25.04% <0.00%> (+0.04%)	⬆️
routers/repo/attachment.go	49.43% <50.00%> (-0.57%)	⬇️
modules/upload/filetype.go	82.14% <95.83%> (+17.85%)	⬆️
routers/repo/release.go	26.52% <100.00%> (+0.32%)	⬆️
modules/indexer/stats/queue.go	52.94% <0.00%> (-11.77%)	⬇️
modules/util/sanitize.go	37.50% <0.00%> (-3.68%)	⬇️
modules/lfs/content_store.go	48.33% <0.00%> (-1.67%)	⬇️
... and 51 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update faa676c...503bebb. Read the comment docs.

[zeripath]

I think we have to mark this as breaking because server managers will have to do something to prevent uploads

[zeripath]

I think although this is breaking this is probably still the correct thing to do.

[silverwind]

The new default can't really "break" any existing uploads but it's certainly worth mentioning it in the release notes that we now allow all files to be uploaded by default.

[lunny]

I agree with @zeripath, For a security consideration, it's a break change.

[CirnoT]

While this is desired for content uploaded as releases, it is definitely not expected behavior when uploading content embedded in a comment.

We should distinguish them and keep comment one to sane text files + images while changing release one to empty by default. Improvements from this PR should of course be kept for both.

[silverwind]

it is definitely not expected behavior when uploading content embedded in a comment

What exactly is the issue with allowing all files? Security concerns? A user can already upload malicious files by just renaming their extension to .png and execution of that could be achieved through other means if such vulnerabilities exist.

These mime/extension filters does not really help much if any to prevent such attacks and golang's magic byte-based (I assume its based on that) detection can be fooled too. A much better alternative would be to mount the directory for uploads using noexec mount flag and restricting the gitea process through capabilities as much as possible.

sane text files + images

How would you match them using the new filter options? text/* may not be supported in all browsers as the spec only specifies audio,video,image wildcard (but we do support them all on the backend).

[CirnoT]

It would not be perfect of course, nothing ever will.

How would you match them using the new filter options?

Same as GitHub? .gif,.jpeg,.jpg,.png,.docx,.gz,.log,.pdf,.pptx,.txt,.xlsx,.zip

What exactly is the issue with allowing all files? Security concerns?

I suppose the main point would be that no .exe files and such are directly downloadable when uploaded by any user, thus helping site not being marked as hosting malicious content or using it directly to serve payloads. Of course malicious user can still create repository but such cases are easier to track and remove than random attachment on some comment.

[silverwind]

Seems resonable to change the default to the issue attachment filter to the extensions listed here.

[silverwind]

Default value .docx,.gif,.gz,.jpeg,.jpg,.log,.pdf,.png,.pptx,.txt,.xlsx,.zip added for issue/pr attachments and server-sent error string added to i18n.

[silverwind]

@zeripath: maybe you want to re-review. pr has substantially changed since your review.

[CirnoT]

Linter failure

[silverwind]

Lint fixed.

[silverwind]

Encryption removed, now using separate endpoints for issue and release attachment POSTs.

Known issue: Can not remove files from a release when editing it. Issue seems already present on master.

[silverwind]

Rebased, squashed and update OP. I'm not sure yet how hard the fix for the release edits will be, it'll likely requre quite a bit of fiddling with the quirky dropzone code.

[lafriks]

@silverwind ping

[silverwind]

Will have another look shortly. I opened #12963 for issue with editing so to not distract this PR further.

[silverwind]

Ready for review again.

[lunny]

@lafriks Need your review again.