Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mechanism to provide context to exceptions #1138

Closed

Conversation

mbasmanova
Copy link
Contributor

@mbasmanova mbasmanova commented Mar 8, 2022

Errors happening during expression evaluation are hard to troubleshoot because
it is not clear which sub-expression triggered the error. This PR adds a
mechanism to specify context for exceptions via a thread_local variable and
uses it to specify the expression being evaluated and add it to the exception.

Here are some examples of the new "context" information being added to the
exceptions:

(c0 + c1) % 0 -> Context: mod(cast(plus(c0, c1)), literal)
c0 + (c1 % 0) -> Context: mod(cast(c1), literal)

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 8, 2022
@facebook-github-bot
Copy link
Contributor

@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@mbasmanova mbasmanova requested a review from oerling March 8, 2022 01:24
@mbasmanova mbasmanova changed the title [WIP] Add mechanism to provide context to exceptions Add mechanism to provide context to exceptions Mar 8, 2022
@facebook-github-bot
Copy link
Contributor

@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@oerling
Copy link
Contributor

oerling commented Mar 8, 2022 via email

@Yuhta
Copy link
Contributor

Yuhta commented Mar 8, 2022

This is a nice idea. I’d suggest to make the setter take something like Typedef std::string(ContextMessageFunc)(void atrg); Setter::Setter(ContextMessageFunc func, void* arg); Making the string when going inside the block is way too expensive. But pushing a function pointer and a pointer that can refer to stack allocated extra state information, e.g. the Expr(* costs nothing. Note that a lambda with capture, when its lifetime is not obvious and its only use inlined, will be heap allocated and thus expensive. So a C function pointer and a void8* argument are the best choice here since these can be on all the time with no measurable overhead. Another idea is to also print these in the SIGSEGV handler, along with the status line from process/TraceContext.h.

Can we cache the string in Expr to avoid recomputing for every batch?

@mbasmanova mbasmanova force-pushed the exception-context branch 2 times, most recently from 0888541 to f0f4866 Compare March 9, 2022 16:28
@facebook-github-bot
Copy link
Contributor

@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@mbasmanova
Copy link
Contributor Author

@oerling Orri, thank you for the suggestion. I modified the code accordingly. Would you take another look?

@facebook-github-bot
Copy link
Contributor

@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Summary:
Add helper function to print query plan with runtime statistics.

The existing PlanNode::toString(true, true) method allows to print query plan with details about each plan node, e.g.

```
->hash join[FULL c0=u_c0]
  ->values[3456 rows in 2 vectors]
  ->project[expressions: (u_c0:INTEGER, ROW["c0"]), (u_c1:INTEGER, ROW["c1"]), ]
    ->values[123 rows in 1 vectors]
```

The new printPlanWithStats(PlanNode, TaskStats) function allows to print the query plan with runtime statistics about each plan node, e.g.

```
->hash join[FULL c0=u_c0]
  Output: 20626 rows (809384 bytes), Cpu time: 2318889ns, Blocked wall time: 425000ns, Peak memory: 2097152 bytes
  HashBuild: Output: 0 rows (0 bytes), Cpu time: 347279ns, Blocked wall time: 0ns, Peak memory: 1048576 bytes
  HashProbe: Output: 20626 rows (809384 bytes), Cpu time: 1971610ns, Blocked wall time: 425000ns, Peak memory: 1048576 bytes
  ->values[3456 rows in 2 vectors]
    Input: 0 rows (0 bytes), Output: 3456 rows (36928 bytes), Cpu time: 5268ns, Blocked wall time: 0ns, Peak memory: 0 bytes
  ->project[expressions: (u_c0:INTEGER, ROW["c0"]), (u_c1:INTEGER, ROW["c1"]), ]
    Output: 123 rows (1408 bytes), Cpu time: 117442ns, Blocked wall time: 0ns, Peak memory: 0 bytes
    ->values[123 rows in 1 vectors]
      Input: 0 rows (0 bytes), Output: 123 rows (1408 bytes), Cpu time: 17204ns, Blocked wall time: 0ns, Peak memory: 0 bytes
```

Pull Request resolved: facebookincubator#1144

Reviewed By: pedroerp

Differential Revision: D34722848

Pulled By: mbasmanova

fbshipit-source-id: 55d9b2910389add678f56b7dfdeecb1f7cedd874
@facebook-github-bot
Copy link
Contributor

@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

kagamiori added a commit to kagamiori/velox that referenced this pull request Apr 11, 2022
Summary: Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff add the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr.

Differential Revision: D35557075

fbshipit-source-id: 1df3ce6f5d146124d691058c5002c0d32921059d
kagamiori added a commit to kagamiori/velox that referenced this pull request Apr 13, 2022
)

Summary:
Pull Request resolved: facebookincubator#1399

Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr.

In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxRuntimeError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types.

Differential Revision: D35557075

fbshipit-source-id: d8cb39209a1684304db642f63a151baa1badf5ae
kagamiori added a commit to kagamiori/velox that referenced this pull request Apr 15, 2022
)

Summary:
Pull Request resolved: facebookincubator#1399

Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr.

In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxRuntimeError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types.

Differential Revision: D35557075

fbshipit-source-id: c2ad807723c9079ded0b2e950c8e8ba6c5566d21
kagamiori added a commit to kagamiori/velox that referenced this pull request Apr 20, 2022
)

Summary:
Pull Request resolved: facebookincubator#1399

Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr.

In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxRuntimeError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types.

Differential Revision: D35557075

fbshipit-source-id: 8e2822380579accfc10a3597435729d266144302
kagamiori added a commit to kagamiori/velox that referenced this pull request Apr 26, 2022
)

Summary:
Pull Request resolved: facebookincubator#1399

Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr.

In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxUserError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types.

Reviewed By: Yuhta

Differential Revision: D35557075

fbshipit-source-id: f559f5207faaecebd077489041c581d74fac31af
kagamiori added a commit to kagamiori/velox that referenced this pull request Apr 26, 2022
)

Summary:
Pull Request resolved: facebookincubator#1399

Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr.

In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxRuntimeError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types.

Differential Revision: D35557075

fbshipit-source-id: f2fcf2c465ca7e868ff1e0d81b2b4cf4f4a6d4f3
kagamiori added a commit to kagamiori/velox that referenced this pull request Apr 26, 2022
)

Summary:
Pull Request resolved: facebookincubator#1399

Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr.

In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxRuntimeError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types.

Differential Revision: D35557075

fbshipit-source-id: 3f7ef06da6093149b33bd40424e03a4639c40da9
facebook-github-bot pushed a commit that referenced this pull request Apr 27, 2022
Summary:
Pull Request resolved: #1399

Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in #1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr.

In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxUserError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types.

Reviewed By: Yuhta

Differential Revision: D35557075

fbshipit-source-id: 2307da2e2a837093e449d11c15147d6d7485283c
Arty-Maly pushed a commit to Arty-Maly/velox that referenced this pull request May 13, 2022
)

Summary:
Pull Request resolved: facebookincubator#1399

Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr.

In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxUserError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types.

Reviewed By: Yuhta

Differential Revision: D35557075

fbshipit-source-id: 2307da2e2a837093e449d11c15147d6d7485283c
shiyu-bytedance pushed a commit to shiyu-bytedance/velox-1 that referenced this pull request Aug 18, 2022
Summary:
Errors happening during expression evaluation are hard to troubleshoot because
it is not clear which sub-expression triggered the error. This PR adds a
mechanism to specify context for exceptions via a thread_local variable and
uses it to specify the expression being evaluated and add it to the exception.

Here are some examples of the new "context" information being added to the
exceptions:

(c0 + c1) % 0 -> Context: mod(cast(plus(c0, c1)), literal)
c0 + (c1 % 0) -> Context: mod(cast(c1), literal)

Pull Request resolved: facebookincubator#1138

Reviewed By: laithsakka, pedroerp

Differential Revision: D34704626

Pulled By: mbasmanova

fbshipit-source-id: b504b776ea43dca7f32d278611a8601ae5e0fd01
shiyu-bytedance pushed a commit to shiyu-bytedance/velox-1 that referenced this pull request Aug 18, 2022
)

Summary:
Pull Request resolved: facebookincubator#1399

Context information about the expression being evaluated was added to error messages thrown in Expr::eval() in facebookincubator#1138. This diff adds the context infomration to errors messages thrown in evalSpecialForm() and evalSpecialFormSimplified() of ConstantExpr, FieldReference, SwitchExpr, ConjunctExpr, LambdaExpr, and CastExpr.

In addition, this diff changes the type of exceptions thrown in CastExpr from std::invalid_argument to velox::VeloxUserError for the consistency of error message format. So we also update the unit tests in Velox customer code that check the affected exception types.

Reviewed By: Yuhta

Differential Revision: D35557075

fbshipit-source-id: 2307da2e2a837093e449d11c15147d6d7485283c
@oerling
Copy link
Contributor

oerling commented Oct 11, 2022 via email

marin-ma pushed a commit to marin-ma/velox-oap that referenced this pull request Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants