Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash if columnar attribute using in ranker #1519

Closed
ousharovsky opened this issue Oct 23, 2023 · 6 comments
Closed

Crash if columnar attribute using in ranker #1519

ousharovsky opened this issue Oct 23, 2023 · 6 comments
Assignees
@tomatolog
Labels
bug rel::upcoming Upcoming release

Comments

@ousharovsky
Copy link

ousharovsky commented Oct 23, 2023 •
edited by sanikolaev

Oracle linux 8, Manticore 6.2.12

mysql> desc cmp_ru_0;
+--------------------------+------------+---------------------+
| Field                    | Type       | Properties          |
+--------------------------+------------+---------------------+
| id                       | bigint     | columnar fast_fetch |
| nameinrus                | text       | indexed stored      |
| entid                    | text       | indexed stored      |
| fullnameinrus            | text       | indexed stored      |
| actualaddress            | text       | indexed stored      |
| registeredaddress        | text       | indexed stored      |
| chief                    | text       | indexed stored      |
| regcodes                 | text       | indexed stored      |
| emails                   | text       | indexed stored      |
| sites                    | text       | indexed stored      |
| phones                   | text       | indexed stored      |
| chiefhist                | text       | indexed stored      |
| nameineng                | text       | indexed stored      |
| nameinrushist            | text       | indexed stored      |
| fullnameineng            | text       | indexed stored      |
| shnamesinrus             | text       | indexed stored      |
| shnamesinrushist         | text       | indexed stored      |
| shfullnamesinrus         | text       | indexed stored      |
| shfullnamesinrushist     | text       | indexed stored      |
| addrhist                 | text       | indexed stored      |
| innhist                  | text       | indexed stored      |
| ogrnhist                 | text       | indexed stored      |
| chiefeng                 | text       | indexed stored      |
| chiefhisteng             | text       | indexed stored      |
| registeredaddresseng     | text       | indexed stored      |
| actualaddresseng         | text       | indexed stored      |
| shnamesineng             | text       | indexed stored      |
| shfullnamesineng         | text       | indexed stored      |
| shnamesinenghist         | text       | indexed stored      |
| shfullnamesinenghist     | text       | indexed stored      |
| shortnameinrushist       | text       | indexed stored      |
| entitytype               | uint       | columnar fast_fetch |
| okvedcode                | uint       | columnar fast_fetch |
| isacting                 | uint       | columnar fast_fetch |
| isbranch                 | uint       | columnar fast_fetch |
| regioncode               | uint       | columnar fast_fetch |
| branchcount              | uint       | columnar fast_fetch |
| subsidiarycount          | uint       | columnar fast_fetch |
| popularity               | uint       | columnar fast_fetch |
| cmpsize                  | bigint     | columnar fast_fetch |
| latitude                 | float      |                     |
| longitude                | float      |                     |
| guid                     | string     |                     |
| okvedname                | string     |                     |
| okvednameeng             | string     |                     |
| regionname               | string     |                     |
| regionnameeng            | string     |                     |
| actingshareholders       | string     |                     |
| notactingshareholders    | string     |                     |
| allnames                 | string     |                     |
| nameinrus_len            | tokencount |                     |
| entid_len                | tokencount |                     |
| fullnameinrus_len        | tokencount |                     |
| actualaddress_len        | tokencount |                     |
| registeredaddress_len    | tokencount |                     |
| chief_len                | tokencount |                     |
| regcodes_len             | tokencount |                     |
| emails_len               | tokencount |                     |
| sites_len                | tokencount |                     |
| phones_len               | tokencount |                     |
| chiefhist_len            | tokencount |                     |
| nameineng_len            | tokencount |                     |
| nameinrushist_len        | tokencount |                     |
| fullnameineng_len        | tokencount |                     |
| shnamesinrus_len         | tokencount |                     |
| shnamesinrushist_len     | tokencount |                     |
| shfullnamesinrus_len     | tokencount |                     |
| shfullnamesinrushist_len | tokencount |                     |
| addrhist_len             | tokencount |                     |
| innhist_len              | tokencount |                     |
| ogrnhist_len             | tokencount |                     |
| chiefeng_len             | tokencount |                     |
| chiefhisteng_len         | tokencount |                     |
| registeredaddresseng_len | tokencount |                     |
| actualaddresseng_len     | tokencount |                     |
| shnamesineng_len         | tokencount |                     |
| shfullnamesineng_len     | tokencount |                     |
| shnamesinenghist_len     | tokencount |                     |
| shfullnamesinenghist_len | tokencount |                     |
| shortnameinrushist_len   | tokencount |                     |
+--------------------------+------------+---------------------+
80 rows in set (0,00 sec)

mysql> SELECT id, popularity  FROM cmp_ru_0 WHERE MATCH('(@* газпром ) ') LIMIT 0,10;
+---------+------------+
| id      | popularity |
+---------+------------+
|   82278 |        197 |
| 1658104 |         82 |
|   49329 |        250 |
|   50666 |        126 |
| 6218821 |        589 |
| 1137346 |         48 |
| 1286299 |         41 |
| 1143499 |         21 |
|  901306 |         25 |
| 5871124 |          0 |
+---------+------------+
10 rows in set (0,03 sec)


mysql> SELECT id  FROM cmp_ru_0 WHERE MATCH('(@* газпром ) ') LIMIT 0,10 OPTION ranker=expr('sum(popularity)');
ERROR 2013 (HY000): Lost connection to MySQL server during query

------ FATAL: CRASH DUMP -------
[Mon Oct 23 06:49:37.008 2023] [37763]

--- crashed SphinxQL request dump ---
SELECT id  FROM cmp_ru_0 WHERE MATCH('(@* газпром ) ') LIMIT 0,10 OPTION ranker=expr('sum(popularity)')
--- request dump end ---
--- local index:cmp_ru_0
Manticore 6.2.12 dc5144d35@230822 (columnar 2.2.4 5aec342@230822) (secondary 2.2.4 5aec342@230822)
Handling signal 11
-------------- backtrace begins here ---------------
Program compiled with Clang 15.0.7
Configured with flags: Configured with these definitions: -DDISTR_BUILD=rhel8 -DUSE_SYSLOG=1 -DWITH_GALERA=1 -DWITH_RE2=1 -DWITH_RE2_FORCE_STATIC=1 -DWITH_STEMMER=1 -DWITH_STEMMER_FORCE_STATIC=1 -DWITH_NLJSON=1 -DWITH_UNIALGO=1 -DWITH_ICU=1 -DWITH_ICU_FORCE_STATIC=1 -DWITH_SSL=1 -DWITH_ZLIB=1 -DWITH_ZSTD=1 -DDL_ZSTD=1 -DZSTD_LIB=libzstd.so.1 -DWITH_CURL=1 -DDL_CURL=1 -DCURL_LIB=libcurl.so.4 -DWITH_ODBC=1 -DDL_ODBC=1 -DODBC_LIB=libodbc.so.2 -DWITH_EXPAT=1 -DDL_EXPAT=1 -DEXPAT_LIB=libexpat.so.1 -DWITH_ICONV=1 -DWITH_MYSQL=1 -DDL_MYSQL=1 -DMYSQL_LIB=libmariadb.so.3 -DWITH_POSTGRESQL=1 -DDL_POSTGRESQL=1 -DPOSTGRESQL_LIB=libpq.so.5 -DLOCALDATADIR=/var/lib/manticore -DFULL_SHARE_DIR=/usr/share/manticore
Built on Linux x86_64 (rhel8) (cross-compiled)
Stack bottom = 0x7f9390087e20, thread stack size = 0x20000
Trying manual backtrace:
Something wrong with thread stack, manual backtrace may be incorrect (fp=0x1)
Wrong stack limit or frame pointer, manual backtrace failed (fp=0x1, stack=0x7f9390090000, stacksize=0x20000)
Trying system backtrace:
begin of system symbols:
/usr/bin/searchd(_Z12sphBacktraceib+0x22a)[0x5654c096d94a]
/usr/bin/searchd(_ZN11CrashLogger11HandleCrashEi+0x355)[0x5654c07ebd55]
/lib64/libpthread.so.0(+0x12d40)[0x7f93a1f47d40]
/usr/bin/searchd(_ZNK21Expr_GetColumnarInt_c7IntEvalERK9CSphMatch+0x6)[0x5654c16e3a36]
/usr/bin/searchd(_ZNK10Expr_Sum_TILb0ELb0EE7IntEvalERK9CSphMatch+0x1f7)[0x5654c157c3e7]
/usr/bin/searchd(_ZN19RankerState_Expr_fnILb0ELb0EE8FinalizeERK9CSphMatch+0x34)[0x5654c1578aa4]
/usr/bin/searchd(_ZN17ExtRanker_State_TI19RankerState_Expr_fnILb0ELb0EELb1EE10GetMatchesEv+0x4c6)[0x5654c15777c6]
/usr/bin/searchd(_ZNK13CSphIndex_VLN13MatchExtendedILb0ELb0ELb0EEEvR16CSphQueryContextRK9CSphQueryRK11VecTraits_TIP15ISphMatchSorterEP10ISphRankerii+0x85)[0x5654c0936f85]

`
@sanikolaev
Copy link
Collaborator

@PavelShilin89 pls prepare an MRE.

@tomatolog
Copy link
Contributor

I reproduced the case and preparing the fix

@tomatolog tomatolog assigned tomatolog and unassigned PavelShilin89 Oct 25, 2023
@tomatolog
Copy link
Contributor

the crash just fixed at c849394
now using columnar attribute at the expr ranker should work fine

@PavelShilin89
Copy link
Contributor

MRE

I acknowledge playback of the crash.

Steps to reproduce:

  1. Run the command
mysql -h0 -P9306 -e "CREATE TABLE a(id, p uint, abc text) engine='columnar';
DESC a;
INSERT INTO a(id,p, abc) VALUES (0,177,'def');
INSERT INTO a(id,p, abc) VALUES (0,17,'def');
INSERT INTO a(id,p, abc) VALUES (0,1,'def');
SELECT id  FROM a WHERE MATCH('def') LIMIT 0,10 OPTION ranker=expr('sum(p)');"

Log:

-------------- backtrace ends here ---------------
Please, create a bug report in our bug tracker (https://github.com/manticoresoftware/manticore/issues)
and attach there:
a) searchd log, b) searchd binary, c) searchd symbols.
Look into the chapter 'Reporting bugs' in the manual
(https://manual.manticoresearch.com/Reporting_bugs)
Dump with GDB via watchdog
--- active threads ---
thd 0 (work_4), proto mysql, state query, command select
--- Totally 2 threads, and 1 client-working threads ---
------- CRASH DUMP END -------
[Wed Oct 25 15:26:02.127 2023] [1700977] watchdog: main process 46616 killed dirtily with signal 5, will be restarted
[Wed Oct 25 15:26:02.130 2023] [1700977] watchdog: main process 46643 forked ok
[Wed Oct 25 15:26:02.134 2023] [1701349] starting daemon version '6.2.12 dc5144d35@230822 (columnar 2.2.4 5aec342@230822) (secondary 2.2.4 5aec342@230822)' ...
[Wed Oct 25 15:26:02.135 2023] [1701349] listening on 127.0.0.1:9312 for sphinx and http(s)
[Wed Oct 25 15:26:02.136 2023] [1701349] listening on 127.0.0.1:9306 for mysql
[Wed Oct 25 15:26:02.136 2023] [1701349] listening on 127.0.0.1:9308 for sphinx and http(s)
[Wed Oct 25 15:26:02.158 2023] [1701349] WARNING: failed to get MAC address, using random number 1445749771
[Wed Oct 25 15:26:02.158 2023] [1701357] binlog: replaying log /opt/homebrew/var/manticore/binlog/binlog.001
[Wed Oct 25 15:26:02.159 2023] [1701354] binlog: table a: recovered from tid 0 to tid 3
[Wed Oct 25 15:26:02.159 2023] [1701354] binlog: replay stats: 3 commits; 0 updates, 0 reconfigure; 0 pq-add; 0 pq-delete; 0 pq-add-delete, 1 tables
[Wed Oct 25 15:26:02.159 2023] [1701354] binlog: finished replaying /opt/homebrew/var/manticore/binlog/binlog.001; 0.0 MB in 0.000 sec
[Wed Oct 25 15:26:02.159 2023] [1701354] binlog: finished replaying total 1 in 0.001 sec
[Wed Oct 25 15:26:02.160 2023] [1701355] prereading 1 tables
[Wed Oct 25 15:26:02.161 2023] [1701355] preread 1 tables in 0.001 sec
[Wed Oct 25 15:26:02.161 2023] [1701349] accepting connections
[Wed Oct 25 15:26:02.179 2023] [1701353] WARNING: [BUDDY] invalid output, should be 'Buddy ver, started address:port', got 'dyld[46647]: Library not loaded: /opt/homebrew/opt/oniguruma/lib/libonig.5.dylib
  Referenced from: <4268FDBA-7F20-386B-A542-1FFE8EF92199> /opt/homebrew/Cellar/manticore-executor/0.7.6-230804-8f5cfa5/bin/manticore-executor
  Reason: tried: '/opt/homebrew/opt/oniguruma/lib/libonig.5.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/oniguruma/lib/libonig.5.dylib' (no such file), '/opt/homebrew/opt/oniguruma/lib/libonig.5.dylib' (no such file)
'
[Wed Oct 25 15:26:02.180 2023] [1701353] [BUDDY] restarting
[Wed Oct 25 15:26:02.198 2023] [1701357] WARNING: [BUDDY] invalid output, should be 'Buddy ver, started address:port', got 'dyld[46653]: Library not loaded: /opt/homebrew/opt/oniguruma/lib/libonig.5.dylib

@PavelShilin89
Copy link
Contributor

@tomatolog I checked the launch of the test on the docker image with the hash of your comment, the bug is reproduced, the problem is relevant.

My actions:

  1. Run the command
docker run -it -e EXTRA=1 --name manticore -p 9306:9306 -p 9308:9308 manticoresearch/manticore:dev-6.2.13-c849394
  1. Run the command
mysql -h0 -P9306 -e "CREATE TABLE a(id, p uint, abc text) engine='columnar';
DESC a;
INSERT INTO a(id,p, abc) VALUES (0,177,'def');
INSERT INTO a(id,p, abc) VALUES (0,17,'def');
INSERT INTO a(id,p, abc) VALUES (0,1,'def');
SELECT id  FROM a WHERE MATCH('def') LIMIT 0,10 OPTION ranker=expr('sum(p)');"
+-------+--------+---------------------+
| Field | Type   | Properties          |
+-------+--------+---------------------+
| id    | bigint | columnar fast_fetch |
| abc   | text   | indexed stored      |
| p     | uint   | columnar fast_fetch |
+-------+--------+---------------------+
ERROR 2013 (HY000) at line 6: Lost connection to MySQL server during query

@tomatolog tomatolog reopened this Oct 25, 2023
tomatolog added a commit that referenced this issue Oct 25, 2023
@tomatolog
fixed crash in expr ranker on using columnar attribute in RT index; a…
dbca00d 
…dded regression to test 430; fixed Github #1519
@tomatolog
Copy link
Contributor

seem the crash to plain index was fixed but the same crash to RT index got missed. Now at dbca00d I fixed the crash at RT index and the MRE passes well.

@sanikolaev sanikolaev added rel::upcoming Upcoming release and removed est::TO_ESTIMATE labels Oct 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tomatolog tomatolog

Labels
bug rel::upcoming Upcoming release
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@tomatolog @ousharovsky @sanikolaev @PavelShilin89