Save reports can crash on too long name

[marusak]

save-reports crashed with this error

CRITICAL:faf:Action failed unexpectedly: DataError: (DataError) value too long for type character varying(8192)
 'INSERT INTO symbols (name, nice_name, normalized_path) VALUES (%(name)s, %(nice_name)s, %(normalized_path)s) RETURNING symbols.id' {'normalized_path': u'libfreeorionparse.so', 'nice_name': None, 'name': u'bool boost::spirit::qi::action<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::action<boost::spirit::qi::reference<boost::spirit::qi::rule<boost::spirit::lex::lexertl::iterator<boost::spirit::lex::lexertl::functor<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, boost::spirit::lex::lexertl::detail::data, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, mpl_::bool_<true>, mpl_::bool_<true> > >, boost::spirit::qi::in_state_skipper<boost::spirit::lex::detail::lexer_def_<boost::spirit::lex::lexer<boost::spirit::lex::lexertl::actor_lexer<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::spirit::lex::lexertl::functor<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, boost::spirit::lex::lexertl::detail::data, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, mpl_::bool_<true>, mpl_::bool_<true> > > > >, char const*>, ValueRef::ValueRefBase<double>* (), boost::spirit::locals<ValueRef::ValueRefBase<double>*, ValueRef::ValueRefBase<double>*, ValueRef::OpType, std::vector<ValueRef::ValueRefBase<double>*, std::allocator<ValueRef::ValueRefBase<double>*> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::spirit::unused_type> const>, boost::phoenix::actor<boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::assign, boost::proto::argsns_::list2<boost::proto::exprns_::expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<boost::spirit::local_variable<0> >, 0l>, boost::phoenix::actor<boost::spirit::argument<0> > >, 2l> > >, boost::fusion::cons<boost::spirit::qi::kleene<boost::spirit::qi::action<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::alternative<boost::fusion::cons<boost::spirit::qi::action<boost::spirit::qi::literal_char<boost::spirit::char_encoding::standard, true, false>, boost::phoenix::actor<boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::assign, boost::proto::argsns_::list2<boost::proto::exprns_::expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<boost::spirit::local_variable<2> >, 0l>, boost::phoenix::actor<boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<ValueRef::OpType>, 0l> > >, 2l> > >, boost::fusion::cons<boost::spirit::qi::action<boost::spirit::qi::literal_char<boost::spirit::char_encoding::standard, true, false>, boost::phoenix::actor<boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::assign, boost::proto::argsns_::list2<boost::proto::exprns_::expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<boost::spirit::local_variable<2> >, 0l>, boost::phoenix::actor<boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<ValueRef::OpType>, 0l> > >, 2l> > >, boost::fusion::nil_> > >, boost::fusion::cons<boost::spirit::qi::action<boost::spirit::qi::reference<boost::spirit::qi::rule<boost::spirit::lex::lexertl::iterator<boost::spirit::lex::lexertl::functor<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, boost::spirit::lex::lexertl::detail::data, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, mpl_::bool_<true>, mpl_::bool_<true> > >, boost::spirit::qi::in_state_skipper<boost::spirit::lex::detail::lexer_def_<boost::spirit::lex::lexer<boost::spirit::lex::lexertl::actor_lexer<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::spirit::lex::lexertl::functor<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, boost::spirit::lex::lexertl::detail::data, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, mpl_::bool_<true>, mpl_::bool_<true> > > > >, char const*>, ValueRef::ValueRefBase<double>* (), boost::spirit::locals<ValueRef::ValueRefBase<double>*, ValueRef::ValueRefBase<double>*, ValueRef::OpType, std::vector<ValueRef::ValueRefBase<double>*, std::allocator<ValueRef::ValueRefBase<double>*> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::spirit::unused_type> const>, boost::phoenix::actor<boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::assign, boost::proto::argsns_::list2<boost::proto::exprns_::expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<boost::spirit::local_variable<1> >, 0l>, boost::phoenix::actor<boost::proto::exprns_::basic_expr<boost::phoenix::tag::new_, boost::proto::argsns_::list4<boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<boost::phoenix::detail::target<ValueRef::Operation<double> > >, 0l>, boost::phoenix::actor<boost::spirit::local_variable<2> >, boost::phoenix::actor<boost::spirit::local_variable<0> >, boost::phoenix::actor<boost::spirit::argument<0> > >, 4l> > >, 2l> > >, boost::fusion::nil_> > >, boost::phoenix::actor<boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::assign, boost::proto::argsns_::list2<boost::proto::exprns_::expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<boost::spirit::local_variable<0> >, 0l>, boost::phoenix::actor<boost::spirit::local_variable<1> > >, 2l> > > >, boost::fusion::nil_> > >, boost::phoenix::actor<boost::proto::exprns_::basic_expr<boost::proto::tagns_::tag::assign, boost::proto::argsns_::list2<boost::proto::exprns_::expr<boost::proto::tagns_::tag::terminal, boost::proto::argsns_::term<boost::spirit::attribute<0> >, 0l>, boost::phoenix::actor<boost::spirit::local_variable<0> > >, 2l> > >::parse<boost::spirit::lex::lexertl::iterator<boost::spirit::lex::lexertl::functor<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, boost::spirit::lex::lexertl::detail::data, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, mpl_::bool_<true>, mpl_::bool_<true> > >, boost::spirit::context<boost::fusion::cons<ValueRef::ValueRefBase<double>*&, boost::fusion::nil_>, boost::fusion::vector<ValueRef::ValueRefBase<double>*, ValueRef::ValueRefBase<double>*, ValueRef::OpType, std::vector<ValueRef::ValueRefBase<double>*, std::allocator<ValueRef::ValueRefBase<double>*> > > >, boost::spirit::qi::state_switcher_context<boost::spirit::lex::reference<boost::spirit::lex::detail::lexer_def_<boost::spirit::lex::lexer<boost::spirit::lex::lexertl::actor_lexer<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::spirit::lex::lexertl::functor<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, boost::spirit::lex::lexertl::detail::data, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, mpl_::bool_<true>, mpl_::bool_<true> > > > > const, boost::spirit::unused_type>, char const* const>, boost::spirit::unused_type const>(boost::spirit::lex::lexertl::iterator<boost::spirit::lex::lexertl::functor<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, boost::spirit::lex::lexertl::detail::data, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, mpl_::bool_<true>, mpl_::bool_<true> > >&, boost::spirit::lex::lexertl::iterator<boost::spirit::lex::lexertl::functor<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, boost::spirit::lex::lexertl::detail::data, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, mpl_::bool_<true>, mpl_::bool_<true> > > const&, boost::spirit::context<boost::fusion::cons<ValueRef::ValueRefBase<double>*&, boost::fusion::nil_>, boost::fusion::vector<ValueRef::ValueRefBase<double>*, ValueRef::ValueRefBase<double>*, ValueRef::OpType, std::vector<ValueRef::ValueRefBase<double>*, std::allocator<ValueRef::ValueRefBase<double>*> > > >&, boost::spirit::qi::state_switcher_context<boost::spirit::lex::reference<boost::spirit::lex::detail::lexer_def_<boost::spirit::lex::lexer<boost::spirit::lex::lexertl::actor_lexer<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::spirit::lex::lexertl::functor<boost::spirit::lex::lexertl::position_token<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::mpl::vector<bool, int, double, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, mpl_::bool_<true>, unsigned long>, boost::spirit::lex::lexertl::detail::data, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, mpl_::bool_<true>, mpl_::bool_<true> > > > > const, boost::spirit::unused_type>, char const* const> const&, boost::spirit::unused_type const&) const'}

[marusak]

0f7ab13c-aadb-4a47-8247-d1c2091023d7.txt

[gayathrirajendar]

Hello @marusak ,

I have tried to reproduce this issue. But I am facing few problems in it. I think it is already resolved or I understood it somewhere differently.

I uploaded the mentioned report to /var/spool/faf/reports/incoming/ and then tried running sudo -E -u faf faf save-reports to save the report.

The above did not throw any errors to me. I was able to see the report in the /var/spool/faf/reports/saved/ folder. So I was confused.

Then as you helped that day to check entries in DB, I checked symbols table.

I tried select max(char_length(name)) from Symbols; which returned me 14670. But in the issue, it looks like it is throwing error for names that are having more than 8192 characters.

So, I checked the table definition using \d+ symbols and it returned me character varying(32768) for the name column which is 4 times of 8192.

Please tell me where to find the table definitions in the code. I see related things in faf/src/pyfaf/storage/migrations but facing difficulty about how it will work and how I should change it.

Thanks

[marusak]

Thank you for the update.

The table definition is in src/pyfaf/storage/symbol.py.
You are right, name size is set to 32768 characters. It was done in commit f7eaf2b . For some strange reason our db is set to 8192 (even though the limit before was 2048).

I think there should be some migration, but now it does not matter anymore.

Can you please edit the file and change the name to more than 32768 characters and then see what happens?

Thank you

[gayathrirajendar]

Thanks for link and sorry for the delay.

When I tried changing the function_name to be too long (Is that correct?), I see an error while saving the report.

I went through the code for some understanding and found that

faf/src/pyfaf/ureport.py

Line 136 in 79234b2

def validate_ureport2(ureport):

is where probably the error goes uncatched. I will try to raise exceptions and confirm the same.

How will the fix be for this issue? As it is limited to have only 32768 characters, anything above that causing the issue is expected. I think, this bug is related to handling it gracefully and reporting it to the user. Is it?

[marusak]

When I tried changing the function_name to be too long (Is that correct?), I see an error while saving the > report.

Yes, especially when you run save-reports command it fails. It should you inform that there is such report and move it to dereffered.

btw. I just remembered something. How you see the error from save-reports? Because if you look at https://github.com/abrt/faf/blob/master/src/pyfaf/problemtypes/core.py#L89 it should be safe. As we found out, in code there is set limit to 32768 but at our server we have 8192. And this check does not count on that.

[gayathrirajendar]

For record, I used the below ureport.
0f7ab13c-aadb-4a47-8247-d1c2091023d7.txt

Also, here is the error that I got.

I will check further about your comment.

Thanks

[marusak]

Okay, I am closing this issue.
It is fine. Problem of this issue was different limit in source code than in the DB. This should not happen.

Thanks for looking into this.