Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

redis support #1

Closed
inukshuk opened this issue Feb 1, 2012 · 3 comments
Closed

redis support #1

inukshuk opened this issue Feb 1, 2012 · 3 comments

Comments

@inukshuk
Copy link
Owner

inukshuk commented Feb 1, 2012

Currently, the ML model relies on a feature key-value store that is implemented using kyoto-cabinet or (as a fallback) a regular Ruby hash. Because the feature dictionary is fairly large (probably in excess of 5MB) it is not ideal to use a Ruby hash as this consumes a lot of memory of the Ruby process; the kyoto-cabinet uses a file-based approach which performs very well, however, some users have reported difficulties installing kyoto-cabinet. Redis seems like a good alternative and it should be fairly easy to add as third KV store option.

@dshorthouse
Copy link

+1 for Redis

inukshuk added a commit that referenced this issue Mar 1, 2012
inukshuk added a commit that referenced this issue Mar 1, 2012
@inukshuk
Copy link
Owner Author

inukshuk commented Mar 1, 2012

Fixed in master. To get it to work you need to adjust Anystyle::Parser::Dictionary.instance.options:

:mode => :redis
:path => unix socket

Or, if you want to use TCP:

:mode => :redis
:host => host
:port => port

The only snag at the moment is that you need to delete the :path option if you want to set a host/port combination, because the redis gem seems to give precedence to path. If we changed path to cabinet then the Kyoto-Cabinet option would not interfere.

inukshuk added a commit that referenced this issue Mar 1, 2012
@inukshuk
Copy link
Owner Author

inukshuk commented Mar 1, 2012

Alright, switching to redis should be more convenient now. Simply change mode to :redis and set host or path option. The port is set to the default redis port. Further convenience Dictionary#path returns the active path (file, unix socket or host/port, or 'hash' depending on the current mode).

@dshorthouse, let me know if you need anything else. Once we're good to go, we'll release this as 0.1.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants