[BUG] metadata extraction regex for s2 l2 is missing

[nleconte-csgroup]

Environment:

• Delivery tag: production-common 1.12.1-rc1
• Platform: IVV Orange Cloud

Test:

• Name: TST_PRO_LIST_S2L2
• Traceability (requirements):
  • RS-REQ-SRD-129

Current Behavior:
The metadata extraction is failing for the Sentinel 2 Level product due to wrong regex :
https://github.com/COPRS/production-common/blob/29e50e259b79b02537a6530882a6162e245b149e/processing-common/metadata/content/stream-parameters.properties#L52
app.metadata-extraction.worker.product-categories.s2-products.pattern-config=^(S2)(A|B|_)_([A-Z0-9]{4})_((MSI)_(L0_|L1A|L1B|L1C)_(GR|DS|TL|TC)|PRD_HKTM__)_\\w{0,4}_?(\\d{8}T\\d{6})(.*)$

It is missing L2A.

Expected Behavior:
The metadata extraction worker shall handle the L2A products.

Steps To Reproduce:

1. Produce Sentinel 2 Level 2 Datastrip/Tile/TCI
2. Check logs of metadata extraction

Test execution artefacts (i.e. logs, screenshots…)
{"header":{"type":"LOG","timestamp":"2023-04-05T12:46:37.599119Z","level":"ERROR","line":250,"file":"LogAccessor.java","thread":"KafkaConsumerDestination{consumerDestinationName='metadata-catalog-part1.metadata-filter', partitions=6, dlqName='error-warning'}.container-0-C-1"},"message":{"content":"org.springframework.messaging.MessageHandlingException: error occurred in message handler [org.springframework.cloud.stream.function.FunctionConfiguration$FunctionToDestinationBinder$1@666f6c1e]; nested exception is java.lang.RuntimeException: esa.s1pdgs.cpoc.common.errors.processing.MetadataFilePathException: Description extraction failed: File S2B_OPER_MSI_L2A_DS_REFS_20230405T124601_S20230325T203106_N04.00 does not match the configuration file pattern ^(S2)(A|B|_)_([A-Z0-9]{4})_((MSI)_(L0_|L1A|L1B|L1C)_(GR|DS|TL|TC)|PRD_HKTM__)_\\w{0,4}_?(\\d{8}T\\d{6})(.*)$, failedMessage=GenericMessage [payload=byte[581], headers={deliveryAttempt=3, kafka_timestampType=CREATE_TIME, kafka_receivedTopic=metadata-catalog-part1.metadata-filter, target-protocol=kafka, b3=f619e3264f40044f-851e7fa270db8076-0, nativeHeaders={b3=[f619e3264f40044f-851e7fa270db8076-0]}, skip-input-type-conversion=false, kafka_offset=657159, scst_nativeHeadersPresent=true, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@6543de62, kafka_receivedPartitionId=0, contentType=application/json, kafka_receivedTimestamp=1680698787532, kafka_groupId=metadata-catalog-part1}]\n\tat org.springframework.integration.support.utils.IntegrationUtils.wrapInHandlingExceptionIfNecessary(IntegrationUtils.java:191)\n\tat org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:65)\n\tat org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115)\n\tat org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133)\n\tat org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)\n\tat org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:72)\n\tat org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:317)\n\tat org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:272)\n\tat org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187)\n\tat org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166)\n\tat org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47)\n\tat org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109)\n\tat org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:216)\n\tat org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.sendMessageIfAny(KafkaMessageDrivenChannelAdapter.java:397)\n\tat org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.access$300(KafkaMessageDrivenChannelAdapter.java:83)\n\tat org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:454)\n\tat org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:428)\n\tat org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.lambda$onMessage$0(RetryingMessageListenerAdapter.java:125)\n\tat org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:329)\n\tat org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:255)\n\tat org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.onMessage(RetryingMessageListenerAdapter.java:119)\n\tat org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.onMessage(RetryingMessageListenerAdapter.java:42)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:2629)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeOnMessage(KafkaMessageListenerContainer.java:2609)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeRecordListener(KafkaMessageListenerContainer.java:2536)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeWithRecords(KafkaMessageListenerContainer.java:2427)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:2305)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:1979)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeIfHaveRecords(KafkaMessageListenerContainer.java:1364)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:1355)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:1247)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\nCaused by: java.lang.RuntimeException: esa.s1pdgs.cpoc.common.errors.processing.MetadataFilePathException: Description extraction failed: File S2B_OPER_MSI_L2A_DS_REFS_20230405T124601_S20230325T203106_N04.00 does not match the configuration file pattern ^(S2)(A|B|_)_([A-Z0-9]{4})_((MSI)_(L0_|L1A|L1B|L1C)_(GR|DS|TL|TC)|PRD_HKTM__)_\\w{0,4}_?(\\d{8}T\\d{6})(.*)$\n\tat esa.s1pdgs.cpoc.metadata.extraction.service.ExtractionService.apply(ExtractionService.java:108)\n\tat esa.s1pdgs.cpoc.metadata.extraction.service.ExtractionService.apply(ExtractionService.java:57)\n\tat org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.invokeFunctionAndEnrichResultIfNecessary(SimpleFunctionRegistry.java:897)\n\tat org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.invokeFunction(SimpleFunctionRegistry.java:853)\n\tat org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.doApply(SimpleFunctionRegistry.java:708)\n\tat org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.apply(SimpleFunctionRegistry.java:551)\n\tat org.springframework.cloud.stream.function.PartitionAwareFunctionWrapper.apply(PartitionAwareFunctionWrapper.java:84)\n\tat org.springframework.cloud.stream.function.FunctionConfiguration$FunctionWrapper.apply(FunctionConfiguration.java:754)\n\tat org.springframework.cloud.stream.function.FunctionConfiguration$FunctionToDestinationBinder$1.handleMessageInternal(FunctionConfiguration.java:586)\n\tat org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:56)\n\t... 32 more\nCaused by: esa.s1pdgs.cpoc.common.errors.processing.MetadataFilePathException: Description extraction failed: File S2B_OPER_MSI_L2A_DS_REFS_20230405T124601_S20230325T203106_N04.00 does not match the configuration file pattern ^(S2)(A|B|_)_([A-Z0-9]{4})_((MSI)_(L0_|L1A|L1B|L1C)_(GR|DS|TL|TC)|PRD_HKTM__)_\\w{0,4}_?(\\d{8}T\\d{6})(.*)$\n\tat esa.s1pdgs.cpoc.metadata.extraction.service.extraction.files.FileDescriptorBuilder.buildS2FileDescriptor(FileDescriptorBuilder.java:281)\n\tat esa.s1pdgs.cpoc.metadata.extraction.service.extraction.S2ProductMetadataExtractor.extract(S2ProductMetadataExtractor.java:64)\n\tat esa.s1pdgs.cpoc.metadata.extraction.service.ExtractionService.handleMessage(ExtractionService.java:136)\n\tat esa.s1pdgs.cpoc.metadata.extraction.service.ExtractionService.apply(ExtractionService.java:105)\n\t... 41 more\n"},"custom":{"logger_string":"org.springframework.integration.handler.LoggingHandler"}}

Whenever possible, first analysis of the root cause
Regex is wrong and the L2A part is missing.

Bug Generic Definition of Ready (DoR)

• The affect version in which the bug has been found is mentioned
• The context and environment of the bug is detailed
• The description of the bug is clear and unambiguous
• The procedure (steps) to reproduce the bug is clearly detailed
• The failed tests is linked to the bug : failed result % expected result
• The tested User Story / features is linked to the bug
• Logs are attached if available
• A data set attached if available
• Category label is link to the bug

Bug Generic Definition of Done (DoD)

• the modification implemented (the solution to fix the bug) is described in the bug.
• Unit tests & Continuous integration performed - Test results available - Structural Test coverage reported by SONAR
• Code committed in GIT with right tag or Analysis/Trade Off documentation up-to-date in reference-system-documentation repository
• Code is compliant with coding rules (SONAR Report as evidence)
• Acceptance criteria of the related User story are checked and Passed

[nleconte-csgroup]

The workaround is pending in the rs-config : https://github.com/COPRS/rs-config/pull/125

[nleconte-csgroup]

The workaround was deployed and products are well ingested in the MDC.

[LAQU156]

IVV_CCB_2023_w15 : Moved into "Accepted Werum", Priority minor (workaround). This update have to be done into Werum default conf

[LAQU156]

Werum_CCB_2023_w15 : Moved into "Sprint Backlog", seems to be easy to fix

[w-jka]

Added fix to default configuration. Will be included in next delivery.

[w-fsi]

This issue was tackled in V1.13.1 and thus will be contained in V2.
(refer to https://github.com/COPRS/production-common/releases/tag/1.13.1-rc1)

[LAQU156]

To_validate_CCB_2023_w24 : To be tested by @nleconte-csgroup

[nleconte-csgroup]

Fix is well delivered since 1.13.1 : COPRS/production-common@571091b

[LAQU156]

IVV_CCB_2023_w28 : Validated, closed (V1.13.1)