Add exception handlers for stoll in jit/frontend/schema_type_parser.cpp #94295

m4drat · 2023-02-07T10:19:31Z

Hi!

I've been fuzzing different pytorch modules, and found a few crashes.

Specifically, I'm talking about schema_type_parser.cpp and irparser.cpp. Inside these files, different standard conversion functions are used (such as stoll, stoi, stod, stoull). However, default std exceptions, such as std::out_of_range, std::invalid_argument, are not handled.

Some of the crash-files:

crash-493db74c3426e79b2bf0ffa75bb924503cb9acdc.zip - crash source: schema_type_parser.cpp:272
crash-67bb5d34ca48235687cc056e2cdeb2476b8f4aa5.zip - crash source: schema_type_parser.cpp:240
crash-0157bca5c41bffe112aa01f3b0f2099ca4bcc62f.zip - crash source: schema_type_parser.cpp:179
crash-430da923e56adb9569362efa7fa779921371b710.zip - crash source: schema_type_parser.cpp:196

The provided patch adds exception handlers for std::invalid_argument and std::out_of_range, to rethrow these exceptions with ErrorReport.

How to reproduce

To reproduce the crash, use provided docker: Dockerfile
Build the container: docker build -t oss-sydr-fuzz-pytorch-reproduce .
Copy crash file to the current directory
Run the container: docker run --privileged --network host -v `pwd`:/homedir --rm -it oss-sydr-fuzz-pytorch-reproduce /bin/bash
And execute the binary: /irparser_fuzz /homedir/crash-67bb5d34ca48235687cc056e2cdeb2476b8f4aa5

After execution completes you will see this error message:

terminate called after throwing an instance of 'std::out_of_range'
  what():  stoll

And this stacktrace:

==9626== ERROR: libFuzzer: deadly signal
    #0 0x5b4cf1 in __sanitizer_print_stack_trace /llvm-project/compiler-rt/lib/asan/asan_stack.cpp:87:3
    #1 0x529627 in fuzzer::PrintStackTrace() /llvm-project/compiler-rt/lib/fuzzer/FuzzerUtil.cpp:210:5
    #2 0x50f833 in fuzzer::Fuzzer::CrashCallback() /llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:233:3
    #3 0x7ffff7c3741f  (/lib/x86_64-linux-gnu/libpthread.so.0+0x1441f)
    #4 0x7ffff7a5700a in raise (/lib/x86_64-linux-gnu/libc.so.6+0x4300a)
    #5 0x7ffff7a36858 in abort (/lib/x86_64-linux-gnu/libc.so.6+0x22858)
    #6 0x7ffff7e74910  (/lib/x86_64-linux-gnu/libstdc++.so.6+0x9e910)
    #7 0x7ffff7e8038b  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xaa38b)
    #8 0x7ffff7e803f6 in std::terminate() (/lib/x86_64-linux-gnu/libstdc++.so.6+0xaa3f6)
    #9 0x7ffff7e806a8 in __cxa_throw (/lib/x86_64-linux-gnu/libstdc++.so.6+0xaa6a8)
    #10 0x7ffff7e7737d in std::__throw_out_of_range(char const*) (/lib/x86_64-linux-gnu/libstdc++.so.6+0xa137d)
    #11 0xbd0579 in long long __gnu_cxx::__stoa<long long, long long, char, int>(long long (*)(char const*, char**, int), char const*, char const*, unsigned long*, int) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/ext/string_conversions.h:86:2
    #12 0xc10f9c in std::__cxx11::stoll(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long*, int) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/basic_string.h:6572:12
    #13 0xc10f9c in torch::jit::SchemaTypeParser::parseRefinedTensor()::$_2::operator()() const::'lambda'()::operator()() const /pytorch_fuzz/torch/csrc/jit/frontend/schema_type_parser.cpp:240:25
    #14 0xc10f9c in void c10::function_ref<void ()>::callback_fn<torch::jit::SchemaTypeParser::parseRefinedTensor()::$_2::operator()() const::'lambda'()>(long) /pytorch_fuzz/c10/util/FunctionRef.h:43:12
    #15 0xbfbb27 in torch::jit::SchemaTypeParser::parseList(int, int, int, c10::function_ref<void ()>) /pytorch_fuzz/torch/csrc/jit/frontend/schema_type_parser.cpp:424:7
    #16 0xc0ef24 in torch::jit::SchemaTypeParser::parseRefinedTensor()::$_2::operator()() const /pytorch_fuzz/torch/csrc/jit/frontend/schema_type_parser.cpp:236:9
    #17 0xc0ef24 in void c10::function_ref<void ()>::callback_fn<torch::jit::SchemaTypeParser::parseRefinedTensor()::$_2>(long) /pytorch_fuzz/c10/util/FunctionRef.h:43:12
    #18 0xbfbb27 in torch::jit::SchemaTypeParser::parseList(int, int, int, c10::function_ref<void ()>) /pytorch_fuzz/torch/csrc/jit/frontend/schema_type_parser.cpp:424:7
    #19 0xbff590 in torch::jit::SchemaTypeParser::parseRefinedTensor() /pytorch_fuzz/torch/csrc/jit/frontend/schema_type_parser.cpp:209:3
    #20 0xc02992 in torch::jit::SchemaTypeParser::parseType() /pytorch_fuzz/torch/csrc/jit/frontend/schema_type_parser.cpp:362:13
    #21 0x9445642 in torch::jit::IRParser::parseVarWithType(bool) /pytorch_fuzz/torch/csrc/jit/ir/irparser.cpp:111:35
    #22 0x944ff4c in torch::jit::IRParser::parseOperatorOutputs(std::vector<torch::jit::VarWithType, std::allocator<torch::jit::VarWithType> >*)::$_0::operator()() const /pytorch_fuzz/torch/csrc/jit/ir/irparser.cpp:138:21
    #23 0x944ff4c in void std::__invoke_impl<void, torch::jit::IRParser::parseOperatorOutputs(std::vector<torch::jit::VarWithType, std::allocator<torch::jit::VarWithType> >*)::$_0&>(std::__invoke_other, torch::jit::IRParser::parseOperatorOutputs(std::vector<torch::jit::VarWithType, std::allocator<torch::jit::VarWithType> >*)::$_0&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60:14
    #24 0x94463a7 in torch::jit::IRParser::parseList(int, int, int, std::function<void ()> const&) /pytorch_fuzz/torch/csrc/jit/ir/irparser.cpp:498:7
    #25 0x94460a5 in torch::jit::IRParser::parseOperatorOutputs(std::vector<torch::jit::VarWithType, std::allocator<torch::jit::VarWithType> >*) /pytorch_fuzz/torch/csrc/jit/ir/irparser.cpp:137:3
    #26 0x944c1ce in torch::jit::IRParser::parseOperator(torch::jit::Block*) /pytorch_fuzz/torch/csrc/jit/ir/irparser.cpp:384:3
    #27 0x944bf56 in torch::jit::IRParser::parseOperatorsList(torch::jit::Block*) /pytorch_fuzz/torch/csrc/jit/ir/irparser.cpp:362:5
    #28 0x9444f5f in torch::jit::IRParser::parse() /pytorch_fuzz/torch/csrc/jit/ir/irparser.cpp:482:3
    #29 0x94448df in torch::jit::parseIR(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, torch::jit::Graph*, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, torch::jit::Value*, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, torch::jit::Value*> > >&) /pytorch_fuzz/torch/csrc/jit/ir/irparser.cpp:94:5
    #30 0x944526e in torch::jit::parseIR(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, torch::jit::Graph*) /pytorch_fuzz/torch/csrc/jit/ir/irparser.cpp:99:3
    #31 0x5e3ebd in LLVMFuzzerTestOneInput /irparser_fuzz.cc:43:5
    #32 0x510d61 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
    #33 0x4fac7c in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6
    #34 0x5009cb in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9
    #35 0x529f62 in main /llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #36 0x7ffff7a38082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082)
    #37 0x4f559d in _start (/irparser_fuzz+0x4f559d)

Following these steps with the remaining crashes will give you almost the same results.

pytorch-bot · 2023-02-07T10:19:33Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/94295

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

ROCm jobs fail to access AMD apt repo

✅ No Failures

As of commit e2349e6:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

davidberard98 · 2023-02-09T01:08:14Z

@pytorchbot rebase

pytorchmergebot · 2023-02-09T01:10:11Z

@pytorchbot successfully started a rebase job. Check the current status here

pytorchmergebot · 2023-02-09T01:10:18Z

Successfully rebased type-parser-stoll-exception-handlers onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout type-parser-stoll-exception-handlers && git pull --rebase)

davidberard98

thanks for the PR! tentatively it looks good, but please wait for CI to finish to make sure nothing fails (I rebased for you with the @ pytorchbot command, tests are still running)

davidberard98 · 2023-02-09T22:02:24Z

@pytorchbot merge

pytorchmergebot · 2023-02-09T22:06:01Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2023-02-10T04:04:37Z

The merge job was canceled. If you believe this is a mistake,then you can re trigger it through pytorch-bot.

davidberard98 · 2023-02-10T04:09:50Z

@pytorchbot merge

pytorchmergebot · 2023-02-10T04:11:55Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

cyyever · 2023-02-10T06:14:07Z

@m4drat I am also planing add fuzz testing to pytorch. Is it difficult for you to share the fuzzing code with Pytorch test CI?

SweetVishnya · 2023-02-10T10:54:12Z

@m4drat I am also planing add fuzz testing to pytorch. Is it difficult for you to share the fuzzing code with Pytorch test CI?

You can find all the fuzzing code here https://github.com/ispras/oss-sydr-fuzz/tree/master/projects/pytorch

pytorch-bot bot added the release notes: jit release notes category label Feb 7, 2023

pytorchbot added the open source label Feb 7, 2023

soulitzer requested a review from davidberard98 February 8, 2023 20:12

soulitzer added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Feb 8, 2023

Add exception handlers for stoll, stoi, stod in jit

e2349e6

pytorchmergebot force-pushed the type-parser-stoll-exception-handlers branch from 4de6d2a to e2349e6 Compare February 9, 2023 01:10

davidberard98 approved these changes Feb 9, 2023

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Feb 9, 2023

pytorchmergebot added the Merged label Feb 10, 2023

pytorchmergebot closed this in a1d210d Feb 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add exception handlers for stoll in jit/frontend/schema_type_parser.cpp #94295

Add exception handlers for stoll in jit/frontend/schema_type_parser.cpp #94295

m4drat commented Feb 7, 2023

pytorch-bot bot commented Feb 7, 2023 •

edited

Loading

davidberard98 commented Feb 9, 2023

pytorchmergebot commented Feb 9, 2023

pytorchmergebot commented Feb 9, 2023

davidberard98 left a comment

davidberard98 commented Feb 9, 2023

pytorchmergebot commented Feb 9, 2023

pytorchmergebot commented Feb 10, 2023

davidberard98 commented Feb 10, 2023

pytorchmergebot commented Feb 10, 2023

cyyever commented Feb 10, 2023 •

edited

Loading

SweetVishnya commented Feb 10, 2023

Add exception handlers for stoll in jit/frontend/schema_type_parser.cpp #94295

Add exception handlers for stoll in jit/frontend/schema_type_parser.cpp #94295

Conversation

m4drat commented Feb 7, 2023

How to reproduce

pytorch-bot bot commented Feb 7, 2023 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/94295

❗ 1 Active SEVs

✅ No Failures

davidberard98 commented Feb 9, 2023

pytorchmergebot commented Feb 9, 2023

pytorchmergebot commented Feb 9, 2023

davidberard98 left a comment

Choose a reason for hiding this comment

davidberard98 commented Feb 9, 2023

pytorchmergebot commented Feb 9, 2023

Merge started

pytorchmergebot commented Feb 10, 2023

davidberard98 commented Feb 10, 2023

pytorchmergebot commented Feb 10, 2023

Merge started

cyyever commented Feb 10, 2023 • edited Loading

SweetVishnya commented Feb 10, 2023

pytorch-bot bot commented Feb 7, 2023 •

edited

Loading

cyyever commented Feb 10, 2023 •

edited

Loading