New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add mechanism to provide context to exceptions #1138
Conversation
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
aae6ccb
to
5a893aa
Compare
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
This is a nice idea. I’d suggest to make the setter take something like
Typedef std::string(*ContextMessageFunc)(void* atrg);
Setter::Setter(ContextMessageFunc func, void* arg);
Making the string when going inside the block is way too expensive. But pushing a function pointer and a pointer that can refer to stack allocated extra state information, e.g. the Expr(* costs nothing. Note that a lambda with capture, when its lifetime is not obvious and its only use inlined, will be heap allocated and thus expensive. So a C function pointer and a void8* argument are the best choice here since these can be on all the time with no measurable overhead.
Another idea is to also print these in the SIGSEGV handler, along with the status line from process/TraceContext.h.
|
Can we cache the string in |
0888541
to
f0f4866
Compare
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
@oerling Orri, thank you for the suggestion. I modified the code accordingly. Would you take another look? |
f0f4866
to
33142a5
Compare
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
33142a5
to
243fe2b
Compare
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
243fe2b
to
02c4de9
Compare
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
02c4de9
to
bee7569
Compare
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Summary: Add helper function to print query plan with runtime statistics. The existing PlanNode::toString(true, true) method allows to print query plan with details about each plan node, e.g. ``` ->hash join[FULL c0=u_c0] ->values[3456 rows in 2 vectors] ->project[expressions: (u_c0:INTEGER, ROW["c0"]), (u_c1:INTEGER, ROW["c1"]), ] ->values[123 rows in 1 vectors] ``` The new printPlanWithStats(PlanNode, TaskStats) function allows to print the query plan with runtime statistics about each plan node, e.g. ``` ->hash join[FULL c0=u_c0] Output: 20626 rows (809384 bytes), Cpu time: 2318889ns, Blocked wall time: 425000ns, Peak memory: 2097152 bytes HashBuild: Output: 0 rows (0 bytes), Cpu time: 347279ns, Blocked wall time: 0ns, Peak memory: 1048576 bytes HashProbe: Output: 20626 rows (809384 bytes), Cpu time: 1971610ns, Blocked wall time: 425000ns, Peak memory: 1048576 bytes ->values[3456 rows in 2 vectors] Input: 0 rows (0 bytes), Output: 3456 rows (36928 bytes), Cpu time: 5268ns, Blocked wall time: 0ns, Peak memory: 0 bytes ->project[expressions: (u_c0:INTEGER, ROW["c0"]), (u_c1:INTEGER, ROW["c1"]), ] Output: 123 rows (1408 bytes), Cpu time: 117442ns, Blocked wall time: 0ns, Peak memory: 0 bytes ->values[123 rows in 1 vectors] Input: 0 rows (0 bytes), Output: 123 rows (1408 bytes), Cpu time: 17204ns, Blocked wall time: 0ns, Peak memory: 0 bytes ``` Pull Request resolved: facebookincubator#1144 Reviewed By: pedroerp Differential Revision: D34722848 Pulled By: mbasmanova fbshipit-source-id: 55d9b2910389add678f56b7dfdeecb1f7cedd874
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
bee7569
to
23cb509
Compare
@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Summary: Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff add the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr. Differential Revision: D35557075 fbshipit-source-id: 1df3ce6f5d146124d691058c5002c0d32921059d
) Summary: Pull Request resolved: facebookincubator#1399 Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr. In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxRuntimeError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types. Differential Revision: D35557075 fbshipit-source-id: d8cb39209a1684304db642f63a151baa1badf5ae
) Summary: Pull Request resolved: facebookincubator#1399 Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr. In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxRuntimeError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types. Differential Revision: D35557075 fbshipit-source-id: c2ad807723c9079ded0b2e950c8e8ba6c5566d21
) Summary: Pull Request resolved: facebookincubator#1399 Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr. In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxRuntimeError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types. Differential Revision: D35557075 fbshipit-source-id: 8e2822380579accfc10a3597435729d266144302
) Summary: Pull Request resolved: facebookincubator#1399 Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr. In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxUserError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types. Reviewed By: Yuhta Differential Revision: D35557075 fbshipit-source-id: f559f5207faaecebd077489041c581d74fac31af
) Summary: Pull Request resolved: facebookincubator#1399 Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr. In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxRuntimeError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types. Differential Revision: D35557075 fbshipit-source-id: f2fcf2c465ca7e868ff1e0d81b2b4cf4f4a6d4f3
) Summary: Pull Request resolved: facebookincubator#1399 Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr. In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxRuntimeError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types. Differential Revision: D35557075 fbshipit-source-id: 3f7ef06da6093149b33bd40424e03a4639c40da9
Summary: Pull Request resolved: #1399 Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in #1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr. In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxUserError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types. Reviewed By: Yuhta Differential Revision: D35557075 fbshipit-source-id: 2307da2e2a837093e449d11c15147d6d7485283c
) Summary: Pull Request resolved: facebookincubator#1399 Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr. In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxUserError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types. Reviewed By: Yuhta Differential Revision: D35557075 fbshipit-source-id: 2307da2e2a837093e449d11c15147d6d7485283c
Summary: Errors happening during expression evaluation are hard to troubleshoot because it is not clear which sub-expression triggered the error. This PR adds a mechanism to specify context for exceptions via a thread_local variable and uses it to specify the expression being evaluated and add it to the exception. Here are some examples of the new "context" information being added to the exceptions: (c0 + c1) % 0 -> Context: mod(cast(plus(c0, c1)), literal) c0 + (c1 % 0) -> Context: mod(cast(c1), literal) Pull Request resolved: facebookincubator#1138 Reviewed By: laithsakka, pedroerp Differential Revision: D34704626 Pulled By: mbasmanova fbshipit-source-id: b504b776ea43dca7f32d278611a8601ae5e0fd01
) Summary: Pull Request resolved: facebookincubator#1399 Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr. In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxUserError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types. Reviewed By: Yuhta Differential Revision: D35557075 fbshipit-source-id: 2307da2e2a837093e449d11c15147d6d7485283c
Making the string at prepare time or first use would work in this case but would take a lot of memory. Consider strings of outer expressions contain those of inner ones, at least to a point, even if we print … past a certain depth. So making the string on first need is far better, specially if we have tens of thousands of Expr instances like in some very wide projections.
Another question of semantics is whether we just consider the innermost of all the nested contexts on the stack. Probably all, I’d say.
|
…or#1138) Make bufferSize of IteratorOptions configurable
Errors happening during expression evaluation are hard to troubleshoot because
it is not clear which sub-expression triggered the error. This PR adds a
mechanism to specify context for exceptions via a thread_local variable and
uses it to specify the expression being evaluated and add it to the exception.
Here are some examples of the new "context" information being added to the
exceptions:
(c0 + c1) % 0 -> Context: mod(cast(plus(c0, c1)), literal)
c0 + (c1 % 0) -> Context: mod(cast(c1), literal)