New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Basic search #1187
Basic search #1187
Conversation
src/server/search_family.cc
Outdated
|
||
vector<vector<SerializedDocument>> docs(shard_set->size()); | ||
cntx->transaction->ScheduleSingleHop([&](Transaction* t, EngineShard* shard) { | ||
OpSearch(t->GetOpArgs(shard), &docs[shard->shard_id()], filter); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can have some contention here when pushing to the vectors... We have this pattern also in other places in the code as well, but this one is a bit worse because you're pushing to the vector in a loop during the search :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mean that we push in a loop without a resize ahead? We don't have any estimation for the number of docs 🤷🏻 I don't think that's as issue for now (MVP is brute force search, no performance goals)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, sorry that I've been unclear. We're pushing to docs[0]
and docs[1]
from different threads, but since they're allocated continuously inside docs
they're probably on the same cache line. And since each push_back
changes the vector's size member, you're going to get contention between the threads.
I agree it's not the end of the world, just wanted to point it out :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I see, you mean cache padding. Agree, we need a general solution for this for all other places as well:
- a
CachePadded<T>
wrapper - and possibly use an inlined vector for small thread counts
Transaction currently does all of this with its shard data, but its not re-usable.
0db2186
to
0e1aca1
Compare
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
0e1aca1
to
e5f44f0
Compare
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
class ConnectionContext; | ||
|
||
class SearchFamily { | ||
static void FtCreate(CmdArgList args, ConnectionContext* cntx); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Usually there's no need for static-private class members, we can just declare them in an anonymous namespace in the .cc file. Looks like we've already done this in bitops_family
:
dragonfly/src/server/bitops_family.h
Lines 20 to 28 in 79da3e6
class BitOpsFamily { | |
public: | |
/// @brief Register the function that would be called to operate on user commands. | |
/// @param registry The location to which the handling functions would be registered. | |
/// | |
/// We are assuming that this would have a valid registry to work on (i.e this do not point to | |
/// null!). | |
static void Register(CommandRegistry* registry); | |
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We do it very inconsistently, they're present mostly on the older files. I like when you have a quick overview over the command handlers from the header. Anyways I can remove them
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
100% your call, just a suggestion. As always, feel free to overrule :)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very rapid progress, good job! :)
class ConnectionContext; | ||
|
||
class SearchFamily { | ||
static void FtCreate(CmdArgList args, ConnectionContext* cntx); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
100% your call, just a suggestion. As always, feel free to overrule :)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Fixed comments |
Basic search (ft.create & ft.search)
Implements brute force search
Basic redis search interface: