Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

c++11, Config::show_progress option, icu::StringPiece ns #3

Merged
merged 1 commit into from
Dec 11, 2012
Merged

c++11, Config::show_progress option, icu::StringPiece ns #3

merged 1 commit into from
Dec 11, 2012

Conversation

graehl
Copy link
Contributor

@graehl graehl commented Dec 11, 2012

joint_sort.hh: minor c++11 bugfix - change implicit return type to non-const for
--std=c++11 compilation

config.{hh,cc}: add show_progress option (default true, per status quo) - when
we want to see warnings but not progress bars

search_trie.cc, model.cc: use show_progress config

binary_format.cc: warn about all trie types being slow when loading from ARPA

StringPiece.hh: add U_NAMESPACE_QUALIFIER macro for icu::StringPiece if -DHAVE_ICU

joint_sort.hh: minor c++11 bugfix - change implicit return type to non-const for
--std=c++11 compilation

config.{hh,cc}: add show_progress option (default true, per status quo) - when
we want to see warnings but not progress bars

search_trie.cc, model.cc: use show_progress config

binary_format.cc: warn about *all* trie types being slow when loading from ARPA

StringPiece.hh: add U_NAMESPACE_QUALIFIER macro for icu::StringPiece if -DHAVE_ICU
@@ -60,7 +60,7 @@ template <class KeyIter, class ValueIter> class JointProxy {
JointProxy(const KeyIter &key_iter, const ValueIter &value_iter) : inner_(key_iter, value_iter) {}
JointProxy(const JointProxy<KeyIter, ValueIter> &other) : inner_(other.inner_) {}

operator const value_type() const {
operator value_type() const {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for that. Maybe this fixes clang compilation with C++11.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah. i'm using gcc in production but i check clang warnings now and then.
it probably was clang.

thanks for the lm :)

unrelated, i'm curious about two things:

  1. if there's a version 6 binary kenlm format, how hard would it be to
    write a converter (perhaps using a temp file)?
  2. do you plan on templating State so that the max ngram size can be
    increased without cost? for now we're pretty standardized on 5gram but
    there are a few domains where 7gram is worthwhile ...

On Tue, Dec 11, 2012 at 2:59 PM, Kenneth Heafield
notifications@github.comwrote:

In util/joint_sort.hh:

@@ -60,7 +60,7 @@ template <class KeyIter, class ValueIter> class JointProxy {
JointProxy(const KeyIter &key_iter, const ValueIter &value_iter) : inner_(key_iter, value_iter) {}
JointProxy(const JointProxy<KeyIter, ValueIter> &other) : inner_(other.inner_) {}

  • operator const value_type() const {
  • operator value_type() const {

Thanks for that. Maybe this fixes clang compilation with C++11.


Reply to this email directly or view it on GitHubhttps://github.com//pull/3/files#r2386343.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably best done over direct e-mail. Anything based on a trie can be recovered exactly so a converter could be written. I'm not ready to guarantee that probing models will be convertible because hashing doesn't have an easy way to get the original n-grams out. As to templates for State, I'm a little worried that the decoder would then resort to dynamic memory allocation which is arguably worse. Moses already dynamically allocates and cdec has its weird string stuff, so this wouldn't be an issue there. Would probably break some existing code though. But I agree it would be good to get rid of the compile-time weirdness. Maybe not a template, just the option to use a fixed-size struct or manually managed variable size.

kpu added a commit that referenced this pull request Dec 11, 2012
c++11, Config::show_progress option, icu::StringPiece ns

Lots of spaces deleted at the ends of lines.
@kpu kpu merged commit a836f19 into kpu:master Dec 11, 2012
@cffan cffan mentioned this pull request Sep 26, 2014
frankier pushed a commit to frankier/kenlm that referenced this pull request Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants