Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hfst-sfstpl2fst fails with SFSTs with variable definition #369

Open
santhoshtr opened this Issue Oct 15, 2017 · 4 comments

Comments

Projects
None yet
3 participants
@santhoshtr
Copy link

santhoshtr commented Oct 15, 2017

This is my test fst in SFST formalism. Verbatim copy of example from SFST manual , variables introduction.

$Nsg$ = house | foot | mouse
$Npl$ = house<>:s | f o:e o:e t | {mouse}:{mice}
$Nsg$ <N>:<> <sg>:<> | $Npl$ <N>:<> <pl>:<>

Trying to compile:

practice$ fst-compiler-utf8  variables.fst variables.o                     
variables.fst: 4

As you can see it compiles perfectly.

practice$ fst-compiler-utf8 -v
fst-compiler version 1.4.7a

Version is 1.4.7a. Note that OS is Ubuntu 17.04.

Now using hfst-sfstpl2fst

practice$ hfst-sfstpl2fst -i variables.fst -o variables.o
undefined variable $Nsg$

Exception: HfstException in file: SfstCompiler.cc on line: 282

practice$ hfst-sfstpl2fst -V
hfst-sfstpl2fst 0.1 (hfst 3.13.0)
Copyright (C) 2017 University of Helsinki,
License GPLv3: GNU GPL version 3 <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
practice$ 

Error is "Exception: HfstException in file: SfstCompiler.cc on line: 282". This happens with any sfst with variable definitions.

@santhoshtr

This comment has been minimized.

Copy link
Author

santhoshtr commented Oct 15, 2017

This issue happens only when I compile and use hfst from the released tags. WIth hfst in ubuntu 17.04, all works fine with no issues. That is hfst 3.9.0 though.

@santhoshtr

This comment has been minimized.

Copy link
Author

santhoshtr commented Oct 15, 2017

I bisected till 5bcb93d
All works till there, may be a few more commits.
+cc @eaxelson
I think this is a serious regression for SFST support.

@eaxelson

This comment has been minimized.

Copy link
Member

eaxelson commented Dec 4, 2017

Now using strings instead of char pointers in variable maps (commit 8251357), does it work? This is a quick fix, the whole variable handling and unordered containers should be rewritten...

@eaxelson eaxelson self-assigned this Dec 4, 2017

@santhoshtr

This comment has been minimized.

Copy link
Author

santhoshtr commented Dec 4, 2017

Not really. I compiled the master version with the above commit and tried compiling my SFST files. Some are passing, but many failing. Here is an example failure:

ninfl (master)$ hfst-sfstpl2fst -f openfst-tropical -v -i accusative.fst -o accusative.a
accusative.fst: 5
  ../symbols.fst: 54
accusative.fst: 16terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_M_construct null not valid
Aborted (core dumped)

(the file is at https://github.com/smc/mlmorph/tree/master/ninfl )

Also compiling lexicon is tooo slow compared to sfst-compile-utf8 for https://github.com/smc/mlmorph/tree/master/lexicon

While we are fixing this, can I request some automatic tests in hfst for SFST PL with some medium sophisticated code?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.