Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom workload: Chaining responses of queries #136

Closed
15astro opened this issue May 7, 2020 · 2 comments
Closed

Custom workload: Chaining responses of queries #136

15astro opened this issue May 7, 2020 · 2 comments

Comments

@15astro
Copy link

15astro commented May 7, 2020

What is the ideal scenario for using this feature?
In a specific use-case specific workload is chaining responses from one statement to another is possible with nosqlbench?

For example:
response1 = "select * from table1 where a='a' and b='b'"
computed_value = response1.columnC+response1.columnD*100
"update table1 where a='a' and b='b' set d={computed_value}"

Is your feature request related to a problem? Please describe.
No
Would you like to help implementing or testing it?
Yes, testing.
Additional context

@jshook
Copy link
Contributor

jshook commented May 7, 2020

There is some support for this now, but it is not exposed in an easy to use format.
There are two threading models provided in nb, and the one that emulates "request per thread" as in many classic applications does treat all operations within the stride as linearized by default, even if they have no internal dependencies. The async mode treats all operations as independent. These are "all or none", which is also how many applications are built. It's realistic for many but not all scenarios in terms of emulating applications. Yet, there are some challenges with it that we want to improve:

  1. Specifying dependence between operations should be easy and explicit.
  2. Using async dispatch and linearized ops together in the same workload should be possible.

There is another implied requirement in your request which is to be able to take existing state from previous operations as input for dependent operations. We can already do this when using the request-per-thread model and the default operation sequencing that is available. However, it is not easy to express. One part of it requires you to use per-thread state variables and custom binding functions which are non-trivial. The other part requires you to understand how combine these functions with sequences so that they work well together.

If you need to be able to do basic chained operations, we may add an example that shows this.
However, we are actively tackling the challenge of how to make this type of workload easier to express overall. The next step will be to document the ideas in a design sketch and share a link to that here.

@jshook jshook added NEEDS REVIEW All issues are being reviewed to update the plan for NB and removed NEEDS REVIEW All issues are being reviewed to update the plan for NB labels Feb 9, 2022
@jshook
Copy link
Contributor

jshook commented Jul 19, 2024

#946 is where the discussion for this is happening now. This will be closed in lieu of that, and other more focused issues will be introduced to capture work from that.

@jshook jshook closed this as not planned Won't fix, can't repro, duplicate, stale Jul 19, 2024
@jshook jshook removed the NEEDS REVIEW All issues are being reviewed to update the plan for NB label Jul 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants