Bug - Elasticsearch - IndexQueueDataProcess error

[lis975]

Environment

• Server OS: Linux Rocky
• Browser: Chrome, Safari
• Znuny version: 6.5.6

Expected behavior

Elasticsearch plugin will keep the index up to date.

Actual behavior

for every new ticket crated in system error occurs:

ERROR: OTRS-otrs.Console.pl-Maint::Search::ES::IndexQueueDataProcess-70 Perl: 5.26.3 OS: linux Time: Mon Mar 18 09:26:20 2024

Message: Query failed for engine: Kernel::System::Search::Engine::ES. Message: [Request] ** [[https://xxxxxxxxxxxxxxxxx:9200]-[400]https://xxxxxxxxxxxxxxxxx:9200]-[400] Bad Request, called from sub Search::Elasticsearch::Transport::perform_request at /opt/otrs/Kernel/System/Search/Engine/ES.pm line 388. With vars: {'request' => {'qs' => {'pipeline' => 'attachment_nested'},'path' => 'ticket/_update_by_query','body' => {'query' => {'terms' => {'TicketID' => [9999891]}}},'method' => 'POST','ignore' => [],'mime_type' => 'application/json','serialize' => 'std'},'status_code' => 400,'body' => {'version_conflicts' => 0,'throttled_until_millis' => 0,'throttled_millis' => 0,'retries' => {'search' => 0,'bulk' => 0},'timed_out' => bless( do{(my $o = 0)}, 'JSON::PP::Boolean' ),'deleted' => 0,'took' => 8,'requests_per_second' => '-1','updated' => 0,'batches' => 1,'noops' => 0,'failures' => [{'index' => 'ticket','status' => 400,'id' => '9999891','cause' => {'type' => 'script_exception','reason' => 'runtime error','caused_by' => {'type' => 'class_cast_exception','reason' => 'Cannot cast java.lang.Integer to java.lang.String'},'position' => {'start' => 167,'end' => 248,'offset' => 194},'script_stack' => ['java.base/java.lang.Class.cast(Class.java:4067)','ArticleID = Articles.get(i).ArticleID;
ArrayList ',' ^---- HERE'],'script' => ' ...','lang' => 'painless'}}],'total' => 1}}

Traceback (3035849):
Module: Kernel::System::Search::Engine::ES::QueryExecute Line: 204
Module: Kernel::System::Search::Object::Engine::ES::Ticket::ObjectIndexGeneric Line: 1507
Module: Kernel::System::Search::Object::Engine::ES::Ticket::ObjectIndexUpdate Line: 1024
Module: Kernel::System::Search::ObjectIndexUpdate Line: 564
Module: Kernel::System::Search::Object::Base::ObjectIndexQueueHandle Line: 1203
Module: Kernel::System::Console::Command::Maint::Search::ES::IndexQueueDataProcess::Run Line: 97
Module: (eval) Line: 461
Module: Kernel::System::Console::BaseCommand::Execute Line: 455
Module: (eval) Line: 143
Module: Kernel::System::Daemon::DaemonModules::SchedulerTaskWorker::Cron::Run Line: 122
Module: Kernel::System::Daemon::DaemonModules::SchedulerTaskWorker::Run Line: 236
Module: (eval) Line: 332
Module: main::Start Line: 332
Module: /opt/otrs/bin/otrs.Daemon.pl Line: 153

How to reproduce

Steps to reproduce the behavior:

1. just create index ticket and article
2. wait for new ticket to be created by email or WS
3. look in the apache error_log or in the znuny system log

Additional information

My system is very large. Ticket index is about 1,5TB. Article: 61,6GB, ticket_history: 41,8GB.
ES configuration persists of 3 nodes, ES version is 8.12

Screenshots

[lis975]

my additional observation is: select count(*) from ticket returns above 7000000 records, but index ticket contains above 21000000 documents (approx 3x more than tickets in database). Maybe there is a problem in selecting tickets for indexing? The rest of indexes have almost the same count of objects (article, ticket_history).