Added 2 fuzzers

[AdamKorcz]

This PR adds 2 fuzzers that target markdown.RenderRaw and markup.PostProcess respectively. These fuzzers are implemented by way of the go-fuzz fuzzing engine.

Fuzzing is a way of testing programs whereby pseudo-random data is passed to a target with the goal of finding bugs and vulnerabilities.

I have worked on setting continuous fuzzing up for these two fuzzers which will allow them to run for longer and find harder-to-find bugs as well. I have set up the integration files for gitea on OSS-fuzz (google/oss-fuzz#4775), and upon merging the 2 fuzzers here, they would be able to run continuously through OSS-fuzz. OSS-fuzz is a free service for open source projects, and if/when bugs are found, maintainers get notified with an email containing a link to a detailed bug report with stacktrace and reproducer test-case. While it is a free service it is offered with an implied expectation that bugs are fixed, so that the resources spent on fuzzing gitea go towards resolving bugs in the codebase.

All that is needed to setup continuous fuzzing would be at least one maintainer email address.

For some examples of previous bugs found from fuzzing Golang projects I recommend checking out the trophy list on the go-fuzz repository: https://github.com/dvyukov/go-fuzz#trophies

Signed-off-by: AdamKorcz adam@adalogics.com

[AdamKorcz]

OSS-fuzz integration set up here: google/oss-fuzz#4775

[codecov-io]

Codecov Report

Merging #13818 (ee3b406) into master (1cb1fb8) will increase coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #13818      +/-   ##
==========================================
+ Coverage   42.21%   42.23%   +0.01%     
==========================================
  Files         710      710              
  Lines       77232    77232              
==========================================
+ Hits        32607    32616       +9     
+ Misses      39255    39252       -3     
+ Partials     5370     5364       -6     

Impacted Files	Coverage Δ
modules/util/timer.go	42.85% <0.00%> (-42.86%)	⬇️
modules/indexer/stats/queue.go	52.94% <0.00%> (-23.53%)	⬇️
modules/indexer/stats/db.go	43.47% <0.00%> (-8.70%)	⬇️
modules/charset/charset.go	68.53% <0.00%> (-4.50%)	⬇️
services/mailer/mail.go	60.21% <0.00%> (-1.08%)	⬇️
services/pull/pull.go	40.68% <0.00%> (-0.50%)	⬇️
modules/notification/mail/mail.go	39.08% <0.00%> (ø)
models/error.go	39.31% <0.00%> (+0.32%)	⬆️
models/notification.go	67.24% <0.00%> (+0.98%)	⬆️
services/mirror/mirror.go	16.89% <0.00%> (+1.10%)	⬆️
... and 5 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0c5fca2...ee3b406. Read the comment docs.

[lafriks]

Email for receiving reports could be admin@gitea.io

[silverwind]

Can we not introduce another top-level directory? I'd suggest making a tools/fuzz.go or tools/fuzz/fuzz.go for this.

[AdamKorcz]

@silverwind Not a problem from a fuzzing-perspective. Just let me know if that is what we go for.

@lafriks Duly noted!

[silverwind]

I'd say start with tools/fuzz.go if you expect this to stay a single file, otherwise make a directory.

[AdamKorcz]

@silverwind Done!

[silverwind]

Is it possible to make this runnable locally? e.g. through a make target like make fuzz.

[AdamKorcz]

Is it possible to make this runnable locally? e.g. through a make target like make fuzz.

Sure is. This would install some dependencies locally. I would prefer to set this up in a separate PR as it can be a quick fix or a longer task depending on the existing build system which I have not perused yet. Let me know if you are fine with that or would prefer it in this PR. Either way I am happy to do it.

[lunny]

Could this be a feature of Gitea to check repositories hosted on Gitea instance?

[AdamKorcz]

@lunny I am afraid I don't understand what you mean by "check"?

[silverwind]

Let me know if you are fine with that or would prefer it in this PR

I guess it's fine to do separately if you promise to follow up on it 😉

[AdamKorcz]

I guess it's fine to do separately if you promise to follow up on it 😉

I will be happy to. I would like to see this integrated into the build system.

[silverwind]

Oh, I think we need to add tools to GO_DIRS in Makefile so it gets linted and stuff.

[6543]

@AdamKorcz

diff --git a/Makefile b/Makefile
index 0e33047aa..e21cf20f8 100644
--- a/Makefile
+++ b/Makefile
@@ -110,7 +110,7 @@ TAGS ?=
 TAGS_SPLIT := $(subst $(COMMA), ,$(TAGS))
 TAGS_EVIDENCE := $(MAKE_EVIDENCE_DIR)/tags
 
-GO_DIRS := cmd integrations models modules routers build services vendor
+GO_DIRS := cmd integrations models modules routers build services vendor tools
 GO_SOURCES := $(wildcard *.go)

[6543]

@AdamKorcz ping ?

[AdamKorcz]

Still here. I will have a look at the diff this evening

[AdamKorcz]

@6543 Sorry for the delay. I ran the patch with git apply but I get the error "error: corrupt patch at line 12".

[zeripath]

@6543 Sorry for the delay. I ran the patch with git apply but I get the error "error: corrupt patch at line 12".

AFAIU it just needs an extra empty newline

[AdamKorcz]

AFAIU it just needs an extra empty newline

Thanks for the suggestion. That just gave "... at line 13".

Nonetheless, I have added tools to the Makefile manually. Let me now if any further changes are needed from my side.

[6543]

@AdamKorcz pleace follow up with a make target to fuzz localy :)