Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Native Steem Search #1599

Closed
wants to merge 2 commits into from
Closed

Added Native Steem Search #1599

wants to merge 2 commits into from

Conversation

KylePiira
Copy link

Added the ability to query the steem blockchain directly through the free AskSteem API. Allows for making more advanced queries like by author, tag, and others listed here that are currently not supported by the existing Google CSE integration.

@sneak
Copy link
Contributor

sneak commented Aug 4, 2017

I appreciate the work you've done here, but we have some blockchain indexing services that I think will be a much better fit for something like this.

We also have a general prohibition on loading 3p resources (despite the cdnjs cloudflare link in this file - it should not have been there in the first place) so calls out to an external API are not something for which we are going to poke a hole in our CSP.

I'm thinking that sbds is the place we want to implement search. Do you hack python? Would you be interested in putting together the serverside component there?

@KylePiira
Copy link
Author

Hi @sneak thank you for your consideration, I'm not sure what you mean by "hack python" but I am very well versed in the Python programming language. What did you have in mind specifically for the server-side component?

@sneak
Copy link
Contributor

sneak commented Aug 4, 2017

I'm thinking a jsonrpc call in sbds (https://github.com/steemit/sbds) that does full text search.

You could do a simple mysql full text search over all of the post/edit rows, and fetch permalinks, then fetch complete edit history for each permalink that hits and apply it locally, and then filter the list of permalink/content pairs with the search terms.

@sneak
Copy link
Contributor

sneak commented Aug 4, 2017

https://github.com/steemit/sbds/blob/master/sbds/server/serve.py#L117

https://github.com/steemit/sbds/blob/master/sbds/server/methods.py#L64

If you add a method here that supports full text search (just using the db - the actual text content of the blockchain isn't that big yet) using the db of sbds, then we would happily merge in a condenser PR that queries it. This is the point of sbds and other indexers.

@sneak
Copy link
Contributor

sneak commented Aug 4, 2017

(the algorithm i described above would not be perfect, but would be an excellent first pass at getting search running without involving 3p services.)

@sneak
Copy link
Contributor

sneak commented Aug 4, 2017

@john-g-g and @roadscape might have better ideas about how to do the requisite post-history-stacking required for an accurate fulltext search call.

@KylePiira KylePiira closed this Aug 4, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants