Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OPS] SL1_RBT short duration product generate many error on production chain #1091

Open
12 tasks
suberti-ads opened this issue Jul 12, 2024 · 1 comment
Open
12 tasks
Labels
bug Something isn't working CCB Issue for CCB ops Ticket from ADS operation team

Comments

@suberti-ads
Copy link

suberti-ads commented Jul 12, 2024

Environment:

  • Delivery tag:
  • Platform: OPS Orange Cloud
  • Configuration:
    rs-addon-s3-sy2-stc: 1.15.0
    rs-addon-s3-sl1-nrt: 1.15.0
    rs-addon-s3-pug-nrt: 1.15.0

Traceability:

Current Behavior:
Some SY2 products failed to be ingested by metatda-catalog.

Metadata extraction failed: java.lang.NumberFormatException: For input string: "nan" at java.base/jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2054) at java.base/jdk.internal.math.FloatingDecimal.parseDouble(FloatingDecimal.java:110) at java.base/java.lang.Double.parseDouble(Double.java:543) at esa.s1pdgs.cpoc.metadata.extraction.service.extraction.files.ExtractMetadata.transformFromOpengis(ExtractMetadata.java:889) at esa.s1pdgs.cpoc.metadata.extraction.service.extraction.files.ExtractMetadata.processS3Coordinates(ExtractMetadata.java:861) at esa.s1pdgs.cpoc.metadata.extraction.service.extraction.files.ExtractMetadata.processProductXFDUFile(ExtractMetadata.java:585) at esa.s1pdgs.cpoc.metadata.extraction.service.extraction.files.MetadataBuilder.buildS3LevelProductFileMetadata(MetadataBuilder.java:243) at esa.s1pdgs.cpoc.metadata.extraction.service.extraction.S3LevelProductMetadataExtractor.extract(S3LevelProductMetadataExtractor.java:33) at esa.s1pdgs.cpoc.metadata.extraction.service.ExtractionService.handleMessage(ExtractionService.java:136) at esa.s1pdgs.cpoc.metadata.extraction.service.ExtractionService.apply(ExtractionService.java:105) at esa.s1pdgs.cpoc.metadata.extraction.service.ExtractionService.apply(ExtractionService.java:57) at org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.invokeFunctionAndEnrichResultIfNecessary(SimpleFunctionRegistry.java:897) at org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.invokeFunction(SimpleFunctionRegistry.java:853) at org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.doApply(SimpleFunctionRegistry.java:708) at org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.apply(SimpleFunctionRegistry.java:551) at org.springframework.cloud.stream.function.PartitionAwareFunctionWrapper.apply(PartitionAwareFunctionWrapper.java:84) at org.springframework.cloud.stream.function.FunctionConfiguration$FunctionWrapper.apply(FunctionConfiguration.java:754) at org.springframework.cloud.stream.function.FunctionConfiguration$FunctionToDestinationBinder$1.handleMessageInternal(FunctionConfiguration.java:586) at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:56) at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115) at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133) at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106) at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:72) at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:317) at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:272) at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187) at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166) at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47) at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109) at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:216) at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.sendMessageIfAny(KafkaMessageDrivenChannelAdapter.java:397) at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.access$300(KafkaMessageDrivenChannelAdapter.java:83) at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:454) at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:428) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:2629) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeOnMessage(KafkaMessageListenerContainer.java:2609) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeRecordListener(KafkaMessageListenerContainer.java:2536) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeWithRecords(KafkaMessageListenerContainer.java:2427) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:2305) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:1979) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeIfHaveRecords(KafkaMessageListenerContainer.java:1364) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:1355) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:1247) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.lang.Thread.run(Thread.java:829)

It appears the sensing duration for these product is very short (less than one second).

Expected Behavior:
Product should not be generated in case of too short duration

Steps To Reproduce:
Sample processing

Test execution artefacts (i.e. logs, screenshots…)
see bellow

Whenever possible, first analysis of the root cause
It seems issue was due to SY2 rs-addon should not generate these products.
Log Preparation worker
On execution generate :
S3A_SY_1_MISR___20240703T142014_20240703T142014_20240703T183453_0000_114_153______LN3_D_ST_002.SEN3
S3A_SY_2_SYN____20240703T142014_20240703T142014_20240703T183504_0000_114_153______LN3_D_ST_002.SEN3
S3A_SY_2_VGP____20240703T142014_20240703T142014_20240703T183506_0000_114_153______LN3_D_ST_002.SEN3
S3A_SY_2_VGK____20240703T142014_20240703T142014_20240703T183509_0000_114_153______LN3_D_ST_002.SEN3
All these product have a short duration.
hereafeter job associated:
appjob151730.txt

Hereafter log from preparation worker
Explore-logs-2024-07-12 13_54_52.txt

Input for SY2-STC job come from PUG-NRT which have also issue due to low duration procut.
All these products seems to be generated by SL1-RBT Products.
Input product: S3A_SL_1_RBT____20240703T142014_20240703T142514_20240703T171153_0299_114_153______LN3_O_NR_002.SEN3

Hereafter job for preparation-worker for Sl1-NRT
appjob151442.txt

Hereafter preparation log for job generation:
Explore-logs-2024-07-12 14_53_53.txt

Bug Generic Definition of Ready (DoR)

  • The affect version in which the bug has been found is mentioned
  • The context and environment of the bug is detailed
  • The description of the bug is clear and unambiguous
  • The procedure (steps) to reproduce the bug is clearly detailed
  • The tested User Story / features is linked to the bug if available
  • Logs are attached if available
  • A data set attached if available

Bug Generic Definition of Done (DoD)

  • the modification implemented (the solution to fix the bug) is described in the bug.
  • Unit tests & Continuous integration performed - Test results available - Structural Test coverage reported by SONAR
  • Code committed in GIT with right tag or Analysis/Trade Off documentation up-to-date in reference-system-documentation repository
  • Code is compliant with coding rules (SONAR Report as evidence)
  • Acceptance criteria of the related User story are checked and Passed
@suberti-ads suberti-ads added bug Something isn't working CCB Issue for CCB ops Ticket from ADS operation team labels Jul 12, 2024
@suberti-ads
Copy link
Author

This issue seem to be the root cause for #1090

@suberti-ads suberti-ads changed the title [OPS] Sl1_RBT short duration product generate many error on production chain [OPS] SL1_RBT short duration product generate many error on production chain Jul 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working CCB Issue for CCB ops Ticket from ADS operation team
Projects
None yet
Development

No branches or pull requests

1 participant