New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Features suggestions #49
Comments
Regarding Searchkit, there is the same functionality in golang by using bleve, which can use boltdb. All Content in the CMS can then be searchable, with full facet searching. |
I was thinking to bridge elasticsearch with a docker container through a docker-compose file for larger dataset to index and query; bleve won't work on a dataset with 10k+ records. interesting projects using bleve:
Actually, I am giving a try to qor-admin and ponzu, as i d like to create an easy and quick web admin for a project like limo and maybe re-use this boilerplate for another project called dim; both are using bleve as a full-text search engine. |
I think your mistaken about how large a dataset bleve can handle. i have used bleve on huge datasets. It support alias and so can be run on clusters of servers. i have done 3 projects with bleve and its been solid as a rock. We were indexing 4 Petabytes through it a year ago with alias shards. Its very easy to make full text searching & faceted searching with bleve on any content. Also the developer behind bleve (Marty) is awesome and very approachable. |
Hi @roscopecoltran - Thank you for your feedback and suggestions - it's great to see other people enjoying Ponzu and sharing ideas about how to make it better. Regarding your suggestions:
I would be interested in making the CMS admin search feature more complete, moving beyond the current word/phrase exact match implementation. Is the enhancement you're suggesting for search inside the CMS as an admin, or outside as consumer (via some new I think @gedw99's suggestion to consider bleve is probably the right move, since the project is already entirely Go, and I'd like to keep it as such.
I'm planning to write more documentation on the
Would you clarify what the intended use-case is here? I'm not familiar with this project, but it seems like it would be replacing the UI already available at
Interesting project, but I think adding another dependency to simply manage HTML views may be a bit much. The first step to changing any UI code would be to attempt a port of the |
@gedw99 - first, nice profile picture. PF is the best. I've played guitar for the majority of my life and David Gilmour is just incredible. If you have suggestions as to how we could implement Bleve into our search system (or replace it), I would be grateful to hear them. Currently there is only a search handler inside the admin UI to do exact word/phrase matching on the JSON documents in Bolt. The purpose is solely to help admins find content they have added. I think that Bleve could support a really nice API endpoint for external search requests by outside users.. but I haven't used it and would need to make sure we can prevent indexing of the Bolt buckets that hold sensitive information, and the ones which are hidden as per a Ponzu developers implementation of the Another consideration would be how Bleve would work with other DB backends. There has been some talk about providing support for other common databases like Postgres & MySQL. Although adding swappable DB backends is probably fairly distant, its good to think about it now when possibly adding more DB-related features. |
Am on mobile..
It's maybe 50 lines of code.
Each content interface needs to only implement a bleve mapper. It can
handle numeric and time ranges too.
Then on search there is a standard bleve interface.
You probably want a global search that can display a GUI with a few etc for
each content item.
You can use aliases to ensure it does not clash with other boltdb buckets.
Also have a look at storm boltdb in github.
Might help everything else
That would do it.
I can write more when not on mobile with some links !!
…On Fri, 27 Jan 2017, 18:25 Steve, ***@***.***> wrote:
@gedw99 <https://github.com/gedw99> - first, nice profile picture. PF is
the best. I've played guitar for the majority of my life and David Gilmour
is just incredible.
If you have suggestions as to how we could implement Bleve into our search
system (or replace it), I would be grateful to hear them. Currently there
is only a search handler inside the admin UI to do exact word/phrase
matching on the JSON documents in Bolt. The purpose is solely to help
admins find content they have added.
I think that Bleve could support a really nice API endpoint for external
search requests by outside users.. but I haven't used it and would need to
make sure we can prevent indexing of the Bolt buckets that hold sensitive
information, and the ones which are hidden as per a Ponzu developers
implementation of the item.Hideable interface which prevents content
types from being exposed through the content API.
Another consideration would be how Bleve would work with other DB
backends. There has been some talk about providing support for other common
databases like Postgres & MySQL. Although adding swappable DB backends is
probably fairly distant, its good to think about it now when possibly
adding more DB-related features.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#49 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ATuCwgB_A1kbctcWOf2F35e7FYAUf5JXks5rWhpngaJpZM4LvEDa>
.
|
I wil now open a new issue just for the bleve stuff over at #51 . This issue has too many things in it. |
@gedw99 I looked at bleve, 1/2 year ago and asked the author about the index scalability of bleve, he did not told me such number (much more 200k). In one word, that would be awesome :-) I know that couchbase is doing some great things, for eg they did a couple of cool things for distributed deep learning like elastic-thoughts. I will take a couple of hours to read the article and make some tests. I saw the following interesting project: Cheers, |
Checked the new issue I created.
Should help slot.
…On Sat, 28 Jan 2017, 15:15 roscopecoltran, ***@***.***> wrote:
@gedw99 <https://github.com/gedw99> I looked at bleve, 1/2 year ago and
asked the author about the index scalability of bleve, he did not told me
such number (much more 200k).
In one word, that would be awesome :-) I know that *couchbase* is doing
some great things, for eg they did a couple of cool things for distributed
deep learning like elastic-thoughts
<https://github.com/tleyden/elastic-thought>.
I will take a couple of hours to read the article and make some tests.
I saw the following interesting project:
- https://github.com/couchbase/cbft
Cheers,
Richard
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#49 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ATuCwnzrrsqGrN0O_ushKgDtfsMaSlRGks5rWz9agaJpZM4LvEDa>
.
|
I need to familiarize myself with Bleve and check out storm. Will report back on #51. Thank you! |
I am curious myself because the design of Ponzi looks pretty awesome. It's
really similar to something I started to build, but now I am getting on
board with adding some of my ideas to ponzu. I will raise them as feature
discussion / issues if that's ok. See what you think...
The technical aspects are pretty easy thanks to the work and maturity that
Marty has put into bleve
…On Sat, 28 Jan 2017, 21:54 Steve, ***@***.***> wrote:
I need to familiarize myself with Bleve and check out storm. Will report
back on #51 <#51>. Thank you!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#49 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ATuCwjfDRGNanl9vT5oLDWjH2CgO8Z7tks5rW50PgaJpZM4LvEDa>
.
|
Thank you, I would be very happy to have contributions and get more people involved. I think the design already lends itself well to the approach you mentioned where Content items implement a mapper interface. If you look at the 'system/item' package, you'll see a number of other interfaces that are exported and that Item implements by default (most as no-ops, which the intention of them being overrriden). Do you think we could extend Item to implement a default mapping interface so the Ponzu developer wouldn't need to do this themselves in their content file? |
yes i saw that way the interface works. I really liked it. Very extendable design and innovative.
I answered on the proper page so we keep it all together:
#51
…On Sat, 28 Jan 2017, 22:26 Steve, ***@***.***> wrote:
Thank you, I would be very happy to have contributions and get more people
involved. I think the design already lends itself well to the approach you
mentioned where Content items implement a mapper interface.
If you look at the 'system/item' package, you'll see a number of other
interfaces that are exported and that Item implements by default (most as
no-ops, which the intention of them being overrriden).
Do you think we could extend Item to implement a default mapping interface
so the Ponzu developer wouldn't need to do this themselves in their content
file?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#49 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ATuCwvxHtcHzBNX2qPx78bE5tbqrfidhks5rW6RhgaJpZM4LvEDa>
.
|
@roscopecoltran - since it seems like the Bleve / full text search is the most interesting to us on this thread, I am going to close out this issue and we can carry out the conversation on #51. If you would like to discuss the other ideas you suggested, please read my responses and we can chat in Gopher Slack on the ponzu channel (join here: https://invite.slack.golangbridge.org/) Thanks! |
Hi,
Hope you are all well !
I am testing ponzu cms those days and made myself a couple of remarks about what could be awesome to have embedded in it.
It would be awesome to have:
What is the current road-map for ponzu ? What are your goals with this framework ?
Cheers,
Richard
The text was updated successfully, but these errors were encountered: