move lemmatize_uk.so to /usr/share/manticore/modules/

[githubmanticore]

Currently columnar and secondary .so files are in /usr/share/manticore/modules/:

[root@f9e0664dbe8d ~]# rpm -ql manticore-columnar-lib   
/usr/share/manticore   
/usr/share/manticore/modules   
/usr/share/manticore/modules/lib_manticore_columnar.so   
/usr/share/manticore/modules/lib_manticore_secondary.so   

, but UK lemmatizer's lemmatize_uk.so is not:

[root@f9e0664dbe8d ~]# rpm -ql manticore-lemmatizer-uk   
/etc/ld.so.conf.d   
/etc/ld.so.conf.d/manticore_uk_lemmatizer.conf   
/usr/local   
/usr/local/manticore   
/usr/local/manticore/lib   
/usr/local/manticore/lib/lemmatize_uk.so   

Does it make sense to put it to the same dir?

Also, does:

common {   
    plugin_dir = /usr/local/manticore/lib/   
}   

make sense in the default config?

[githubmanticore]

➤ Sergey Nikolaev commented:

As discussed on dev call of Aug 18 2022 it makes sense to put the UA lemmatizer to /usr/share/manticore/modules/.

[githubmanticore]

➤ Sergey Nikolaev commented:

The temporary w/a for those who get this error:

MySQL [(none)]> create table tbl (f text) charset_table='non_cjk,U+0406->U+0456,U+0456,U+0407->U+0457,U+0457,U+0490->U+0491,U+0491' morphology='lemmatize_uk_all'; 
ERROR 1064 (42000): error adding index 'tbl': dlopen() failed: /usr/local/lib/manticore/lemmatize_uk.so: cannot open shared object file: No such file or directory 

is:

[root@20cff15ad6a1 ~]# mkdir /usr/local/lib/manticore/; ln -s /usr/local/manticore/lib/lemmatize_uk.so /usr/local/lib/manticore/lemmatize_uk.so 

After that the error should be gone:

MySQL [(none)]> create table tbl (f text) charset_table='non_cjk,U+0406->U+0456,U+0456,U+0407->U+0457,U+0457,U+0490->U+0491,U+0491' morphology='lemmatize_uk_all'; 
Query OK, 0 rows affected (0.23 sec) 

[githubmanticore]

➤ Stan commented:

should be fixed at https://gitlab.com/manticoresearch/lemmatizer-uk/-/commit/11f6e0d1591edd94c2f5c4cd7900e1758685d609

[githubmanticore]

➤ Sergey Nikolaev commented:

Please close if it's done

[githubmanticore]

➤ Stan commented:

added new packages for rhel9, bullseye, jammy, bookworm at https://gitlab.com/manticoresearch/lemmatizer-uk/-/commit/ab943c2d2ebd86329c3ef7b69081d0100a265a80

[githubmanticore]

➤ Stan commented:

all packages x86_64 got uploaded into package release repo

[sanikolaev]

As discussed in Slack let's look for the library not in plugin_dir, but in the modules dir the same way we do for the MCL libs:

mysql> create table t(f text) morphology='lemmatize_uk';
ERROR 1064 (42000): error adding table 't': dlopen() failed: /usr/local/lib/manticore/lemmatize_uk.so: cannot open shared object file: No such file or directory

[githubmanticore]

➤ Stan commented:

should be fixed at https://gitlab.com/manticoresearch/lemmatizer-uk/-/commit/6e7563d658744cdc59430f684d9796f44fc3afae

as discussed the package destination now is the plugin_dir and by default it is the /usr/local/lib/manticore/

The idea being tested is to put the package into /usr/share/manticore/modules/ but lemmatizer is internally works as a tokenizer filter and it uses plugin path to spawn on index creation. And adding a feature to load plugin from the module path needs additional development.

[sanikolaev]

I've also removed plugin_dir from the default config in https://gitlab.com/manticoresearch/dev/-/commit/c0924d152966d32b67a41757d49f6cad80cea3b1