New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DHT queries #43

Closed
sim590 opened this Issue Feb 18, 2016 · 4 comments

Comments

Projects
None yet
2 participants
@sim590
Contributor

sim590 commented Feb 18, 2016

Traffic usage has not been concern as of yet, but with data persistence functionality, it is a major issue that has to be addressed. In order to do so, here is the idea of filtering values by field values and field themselves. The following table represents a Dht::Value:

Id ValueType Data OwnerPk RecipientHash UserType Signature EncryptedData
... ... ... ... ... ... ... ...
... ... ... ... ... ... ... ...
... ... ... ... ... ... ... ...

Operations Dht::get and Dht::put could optionally ask for a parameter specifying filters over rows and columns of the above table. However, this feature depends entirely on the completion of #42, since two separate queries over a same hash would have to be distinguished since they would not be necessarely satisfied in the same way.

@sim590 sim590 added this to the Data persistence milestone Feb 18, 2016

@sim590

This comment has been minimized.

Show comment
Hide comment
@sim590

sim590 May 2, 2016

Contributor

I have been working on this. Here's what I have done so far. Packing, unpacking and remote handling of queries is done. What's missing is the part where queries are actually locally processed and sent to nodes. Handling of the response, i.e. associating the query's response to the callback and filter to apply, is also missing. The following is summarizing an abstract model of how to handle this.

conception

NOTE

  • A Query distinguishes one Get structure instance from another. The same is true for LocalListener structure;

  • Upon receiving an answer for a query, you only call callbacks for which the associated query is satisfied by the query to which the response is targeted;

  • Request structure doesn't actually have a field std::shared_ptr q. The query shall only be captured into the Request::done_cb lambda.

  • In order to track which SearchNode has already been sent a given query, we keep a pointer of query associated to a network engine request using a map:

    std::map<std::shared_ptr<Query>, std::shared_ptr<Request>>

    That's why I use std::shared_ptr<Query>.

@aberaud: Do you have any thoughts? In the meantime, I'll be implementing this.

Contributor

sim590 commented May 2, 2016

I have been working on this. Here's what I have done so far. Packing, unpacking and remote handling of queries is done. What's missing is the part where queries are actually locally processed and sent to nodes. Handling of the response, i.e. associating the query's response to the callback and filter to apply, is also missing. The following is summarizing an abstract model of how to handle this.

conception

NOTE

  • A Query distinguishes one Get structure instance from another. The same is true for LocalListener structure;

  • Upon receiving an answer for a query, you only call callbacks for which the associated query is satisfied by the query to which the response is targeted;

  • Request structure doesn't actually have a field std::shared_ptr q. The query shall only be captured into the Request::done_cb lambda.

  • In order to track which SearchNode has already been sent a given query, we keep a pointer of query associated to a network engine request using a map:

    std::map<std::shared_ptr<Query>, std::shared_ptr<Request>>

    That's why I use std::shared_ptr<Query>.

@aberaud: Do you have any thoughts? In the meantime, I'll be implementing this.

@sim590

This comment has been minimized.

Show comment
Hide comment
@sim590

sim590 May 18, 2016

Contributor

The code has reached a mature state. I'll be opening a PR shortly.

Contributor

sim590 commented May 18, 2016

The code has reached a mature state. I'll be opening a PR shortly.

@sim590 sim590 referenced this issue May 19, 2016

Closed

[DHT] Queries: remote values filtering #70

2 of 2 tasks complete
@sim590

This comment has been minimized.

Show comment
Hide comment
@sim590

sim590 May 31, 2016

Contributor

#70 has been merged, but now there are some enhancements to add to it. As mentionned in 52b4f5a, the SELECT feature is unusable in many cases. I'm working on enhancing queries by splitting Queries into two seperate independant structures which are Select and Where. Typical get call from the API will let the user pass a Where and there'll be an additionnal call Dht::query that will accept a Query as parameter. It's 'get' callback will have a diffrent signature to solve the issue stated here.

Contributor

sim590 commented May 31, 2016

#70 has been merged, but now there are some enhancements to add to it. As mentionned in 52b4f5a, the SELECT feature is unusable in many cases. I'm working on enhancing queries by splitting Queries into two seperate independant structures which are Select and Where. Typical get call from the API will let the user pass a Where and there'll be an additionnal call Dht::query that will accept a Query as parameter. It's 'get' callback will have a diffrent signature to solve the issue stated here.

@kaldoran

This comment has been minimized.

Show comment
Hide comment
@kaldoran

kaldoran Jul 12, 2016

Collaborator

I think this issue is resolved since the merge of PR #79 into master 😎

Collaborator

kaldoran commented Jul 12, 2016

I think this issue is resolved since the merge of PR #79 into master 😎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment