Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Max results #637

Closed
seanmcl opened this issue Apr 16, 2018 · 6 comments
Closed

Max results #637

seanmcl opened this issue Apr 16, 2018 · 6 comments

Comments

@seanmcl
Copy link
Contributor

seanmcl commented Apr 16, 2018

One feature that would be great would be to be able to stop souffle once a certain number of results were obtained for each output relation. For example, in our use case we typically have a single output relation, and only care about, say, the first 10 results, when hundreds of thousands are ultimately generated. I'd love a way to tell Souffle to stop so I could respond to our user requests faster.

@gkastrinis
Copy link

For my perspective, this sounds like an interesting feature to have :D
Another, similar feature to ask is a "max time" instead of max results.

@b-scholz
Copy link
Member

This would be a very nice feature. However, there is a problem with relation-dependencies. The only way to make it work is that relations that are not used in other relations, could have this feature. I could envisage a qualifier in the language declaration such as,

.decl A(x:number) sizelimit 10

For dependent relations, this is a very difficult feature to implement. Say, if we have two relations A and B, and B depends on the totality of A - especially in case of negation. The only way to implement this effectively is to switch the underlying evaluation from semi-naive evaluation to a stream-oriented evaluation. Lyndon is going to work on this as soon as he will commence his studies at USYD. Note that a stream-oriented approach for evaluation reduces the latency of results for relations; however, the throughput of evaluation might not be so perfect.

How do you feel about this?

@seanmcl
Copy link
Contributor Author

seanmcl commented Apr 19, 2018

Yes, I certainly see the problem, especially with respect to negation. All the relations that we want this for are not used in other relations, so that restriction is fine by me! The stream-based evaluator sounds neat, but if we could get some hack together to just stop at a certain number in the meantime that would be ideal.

@b-scholz
Copy link
Member

I will discuss with Martin.

@seanmcl
Copy link
Contributor Author

seanmcl commented Apr 19, 2018

Thinking about this more, what if I have

.decl a(x: number)
.decl b(x: number) sizeLimit 10 
.output b

a(0).
a(X+1) :- X < 10000000, a(X).
b(X) :- a(X).

This is the kind of situation we'd use the feature for. Is there an easy, hacky way to get what I'm hoping for here without full streaming support?

@azreika
Copy link
Member

azreika commented May 24, 2019

For an example like that, you could probably create an intermediate relation that uses the $ functor to index each tuple, then only grab the first 10 indexed.

.decl a(x: number)
a(1).
a(X*2) :- X < 1000, a(X).

// .decl b(x:number)
// .capacity b(10)

.decl b_indexed(x:number, y: number)
b_indexed(x,$) :- a(x).

.decl b(x:number)
b(x) :- b_indexed(x,y), y < min y : { b(_,y) } + 10.
.output b

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants