pilerimport failed to import: error=table 'piler1' absent

[larsen0815]

I have installed piler on Ubuntu 24.04 via DEB file. Now I want to import existing mails but get an error:

pilerimport -e /var/piler/tmp/Folder_1/mailarchiver_-2147445508_b203d88e-c6f8-45ca-a30c-549de8099a1f.eml
failed to import: /var/piler/tmp/Folder_1/mailarchiver_-2147445508_b203d88e-c6f8-45ca-a30c-549de8099a1f.eml

From maillog

pilerimport[1406]: ERROR: 50000000692f4f87211ca57c00f102ed27f9 failed to store index data for id=26, errno=1064, append ret=0

From /var/log/manticore/query.log

error=table 'piler1' absent

# grep -A2 piler1 /etc/piler/manticore.conf
index piler1
{
    type = rt
    path = /var/piler/manticore/piler1
    rt_mem_limit = 512M
    stored_fields =

# ll /var/piler/manticore/piler1*
ls: cannot access '/var/piler/manticore/piler1*': No such file or directory

# mysql -P9306 Manticore -e "show tables"
+--------+------+
| Table  | Type |
+--------+------+
| audit1 | rt   |
+--------+------+

# pilerconf -q rtindex
rtindex=1

# ll /var/piler/manticore/*.spidx
-rw-r--r-- 1 piler piler 156 2025-12-02 15:40 /var/piler/manticore/dailydelta1.spidx
-rw-r--r-- 1 piler piler 156 2025-12-02 15:40 /var/piler/manticore/delta1.spidx
-rw-r--r-- 1 piler piler 156 2025-12-02 15:40 /var/piler/manticore/main1.spidx
-rw-r--r-- 1 piler piler 156 2025-12-02 15:40 /var/piler/manticore/main2.spidx
-rw-r--r-- 1 piler piler 156 2025-12-02 15:40 /var/piler/manticore/main3.spidx
-rw-r--r-- 1 piler piler 156 2025-12-02 15:40 /var/piler/manticore/main4.spidx
-rw-r--r-- 1 piler piler 127 2025-12-02 15:40 /var/piler/manticore/note1.spidx
-rw-r--r-- 1 piler piler 127 2025-12-02 15:40 /var/piler/manticore/tag1.spidx

In "/var/piler/manticore" there are no files for:
audit1
piler1

It seems like there should be more tables that somehow didn't get generated. What part of the installation should have done this and how can I fix it?

[jsuto]

Run /etc/piler/manticore.conf (it's an executable file) to see the actual config. The check the RT constant settings in the file. I suspect that you have define('RT', 0) which result the older main-delta indexing scheme. If so, then set it to 1, verify /etc/piler/manticore.conf out. If it looks correct, then stop manticore, remove all files in /var/piler/manticore dir, then restart it.

[larsen0815]

RT is 1, but as I had quite some problems with the installation, I think it's possible that at some point, it was 0. Didn't change it manually in that file, though.

# grep --no-ignore-case RT /etc/piler/manticore.conf
define('RT', 1);
<?php if(RT == 0) { ?>
<?php if(RT) { ?>

Did this but the files were not created:

# service manticore stop
# rm -rf /var/piler/manticore/*
# service manticore start
# ll /var/piler/manticore/
total 0

[jsuto]

Show me the output of /etc/piler/manticore.conf

[larsen0815]

manticore.conf.txt

[jsuto]

Not good. It's the legacy main-delta indexing scheme, it requires you to manually create these indexes. Instead you need to double check manticore.conf, verify that the real time index settings are enabled, and then manticore creates the rt indexes automatically. So you need to tweak manticore.conf until it prints the expected configuration, whatever it takes.

[larsen0815]

Well, this is the original file from the DEB from the project's GitHub page. I used piler_1.4.8-noble-2e863b0_amd64.deb

/usr/libexec/piler/postinstall.sh was run after installation and copied manticore.conf.dist to manticore.conf while setting the MySQL variables.

[larsen0815]

I found the culprit: Manticore was using the distribution's config file:

# grep ExecStart /usr/lib/systemd/system/manticore.service
# Run ExecStart with User=manticore / Group=manticore
ExecStart=/usr/bin/searchd --config /etc/manticoresearch/manticore.conf $_ADDITIONAL_SEARCHD_PARAMS

As a solution, use systemctl edit manticore to override it:

[Service]
User=piler
Group=piler
ExecStart=
ExecStart=/usr/bin/searchd --config /etc/piler/manticore.conf
ExecStop=
ExecStop=/usr/bin/searchd --config /etc/piler/manticore.conf --stopwait

After a restart, Manticore could be used by Piler.

However, the status of Manticore daemon is shown as "activating (start)". Is this normal?
Maybe has to do with manticoresoftware/manticoresearch#3646