🧪 Fix fuzzing test failures

[chrisjsewell]

From https://github.com/google/oss-fuzz/tree/master/projects/markdown-it-py,
fixes issues 55363 and 55367

[codecov]

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.01 🎉

Comparison is base (34876b1) 96.05% compared to head (8f1aaf8) 96.07%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #254      +/-   ##
==========================================
+ Coverage   96.05%   96.07%   +0.01%     
==========================================
  Files          62       62              
  Lines        3223     3235      +12     
==========================================
+ Hits         3096     3108      +12     
  Misses        127      127              

Flag	Coverage Δ
pytests	96.07% <100.00%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
markdown_it/rules_block/blockquote.py	100.00% <100.00%> (ø)
markdown_it/rules_block/fence.py	100.00% <100.00%> (ø)
markdown_it/rules_block/hr.py	100.00% <100.00%> (ø)
markdown_it/rules_block/list.py	98.88% <100.00%> (+0.01%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[DavidKorczynski]

Thanks @chrisjsewell !

Would you be interested in moving the fuzzers from https://github.com/google/oss-fuzz/tree/master/projects/markdown-it-py upstream to here? Would be happy to create a set up that is easy for you to manage and also e.g. extend.

In the event you're curious, you can find a Fuzz Introspector report for markdown-it-py in this table: https://oss-fuzz-introspector.storage.googleapis.com/index.html where the current report (march 14th) links to: https://storage.googleapis.com/oss-fuzz-introspector/markdown-it-py/inspector-report/20230314/fuzz_report.html Here, there is yet another link to the "Project coverage" (the yellow button in top left corner) which shows code coverage achieved of the existing fuzzers: https://storage.googleapis.com/oss-fuzz-coverage/markdown-it-py/reports/20230314/linux/index.html -- 80% atm

[chrisjsewell]

Hey @DavidKorczynski 👋🏼

Would you be interested in moving the fuzzers from https://github.com/google/oss-fuzz/tree/master/projects/markdown-it-py upstream to here? Would be happy to create a set up that is easy for you to manage and also e.g. extend.

Yep that would be great thanks 🙏.
As it mentions in the docstring, I had a look, but didn't see any easy way to integrate it with pytest.
How long does the fuzzing take to run? I tried locally but it seemed to be running for a very long time. Obviously I don't want to have to wait ages every time I want to check the tests. In that case I would have them separate to the current unit tests.

[DavidKorczynski]

The fuzzers are meant to be run for a long period of time since the longer they run the more code they explore. There is currently a start-up overhead due to the use of atheris.instrument_all which can take a while (~45 sec?)

There are probably a few different ways to integrate -- what do you think of the following?

import sys
import atheris
from markdown_it import MarkdownIt

@pytest.mark.parametrize(
    "data",
    [
        b"\x42\65\xff.... (fuzzertest 1 case, check for regressions)",
        b"fuzzertest 2 case, check for regressions"
    ]
)
def TestOneInput(data):
  fdp = atheris.FuzzedDataProvider(data)
  md = MarkdownIt()
  raw_markdown = fdp.ConsumeUnicodeNoSurrogates(sys.maxsize)
  md.parse(raw_markdown)
  md.render(raw_markdown)


def main():
  atheris.instrument_all()
  atheris.Setup(sys.argv, TestOneInput)
  atheris.Fuzz()


if __name__ == "__main__":
  main()

Using the above solution you can just paste the byte arrays in from the OSS-Fuzz reports as pytest inputs to TestOneInput, which can be used as regression testing when run under pytest and the running time for the fuzzers when run under pytest will be low (i.e. like any other test) since atheris.instrument_all is not called. The fuzzers, however, when run under pytest will only exhibit the input specified -- OSS-Fuzz will continue to run main for each fuzzer and thus be able to pass arbitrary input to TestOneInput.