Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] [S3] Around 50% of NTC "SR_1_SRA_A_" product have got a size of 2970 seconds (instead of 3029 seconds). #1053

Open
5 of 12 tasks
pcuq-ads opened this issue Jul 27, 2023 · 4 comments
Open
5 of 12 tasks

[BUG] [S3] Around 50% of NTC "SR_1_SRA_A_" product have got a size of 2970 seconds (instead of 3029 seconds). #1053

pcuq-ads opened this issue Jul 27, 2023 · 4 comments
Assignees
@w-fsi @w-jka
Labels
bug Something isn't working CCB Issue for CCB ops Ticket from ADS operation team priority:major Set the priority to major because the production is heavily impacted S3 Relative to Sentinel-3 RS Addons to_be_fixed_phase1 Issue to be fixed for RS phase 1

Comments

@pcuq-ads
Copy link

pcuq-ads commented Jul 27, 2023 •
edited
Loading

Environment:

  • Platform: OPS Orange Cloud
  • Configuration: OPS
  • Infrastructure v1.5.0
  • RS add-on S3-SR1-NRT 1.13.1-rc1
  • RS add-on S3-SR1-NTC 1.13.1-rc1

Traceability:

  • Test(s):
  • Requirements:

Current Behavior:
Around 50% of NTC "SR_1_SRA_A_" product have got a size of 2970 seconds instead of 3029 seconds.
During test 24 hours, we check that all NTC product with type "SR_1_SRA_A_" have got a length of 3029 seconds (sometimes 3026 or 3027).

Expected Behavior:
Reference System shall generate same kind of products than the one from the production services.

Steps To Reproduce:
Check product name from Test 24 hours.

Test execution artefacts (i.e. logs, screenshots…)
Here is the production from RS for NTC "SR_1_SRA_A_"

 S3B_SR_1_SRA_A__20230428T014224_20230428T023254_20230630T202422_3029_079_003______LN3_O_NT_002.SEN3.zip
S3B_SR_1_SRA_A__20230428T023254_20230428T032224_20230630T195950_2970_079_003______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230427T225744_20230427T234814_20230630T225604_3029_098_144______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230428T022210_20230428T031240_20230630T215009_3029_098_146______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230428T072842_20230428T081912_20230630T212440_3029_098_149______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230427T220651_20230427T225744_20230630T193235_3053_098_143______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230428T013013_20230428T021943_20230630T190353_2970_098_145______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230427T234814_20230428T003744_20230630T185834_2970_098_144______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230428T054703_20230428T063733_20230630T182513_3029_098_148______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230428T045514_20230428T054444_20230630T173710_2970_098_147______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230428T081912_20230428T090842_20230630T173322_2970_098_149______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230428T040444_20230428T045514_20230630T163531_3029_098_147______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230427T225721_20230427T225744_20230630T160806_0023_098_144______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230428T031240_20230428T040210_20230630T153038_2970_098_146______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230428T063733_20230428T072703_20230630T145018_2970_098_148______LN3_O_NT_002.SEN3.zip
S3A_SR_1_SRA_A__20230428T003943_20230428T013013_20230630T135404_3029_098_145______LN3_O_NT_002.SEN3.zip
S3B_SR_1_SRA_A__20230428T064919_20230428T073949_20230630T183147_3029_079_006______LN3_O_NT_002.SEN3.zip
S3B_SR_1_SRA_A__20230428T055755_20230428T064725_20230630T183108_2970_079_005______LN3_O_NT_002.SEN3.zip
S3B_SR_1_SRA_A__20230428T032458_20230428T041528_20230630T181234_3029_079_004______LN3_O_NT_002.SEN3.zip
S3B_SR_1_SRA_A__20230428T050725_20230428T055755_20230630T173821_3029_079_005______LN3_O_NT_002.SEN3.zip
S3B_SR_1_SRA_A__20230427T230850_20230427T235820_20230630T164816_2970_079_001______LN3_O_NT_002.SEN3.zip
S3B_SR_1_SRA_A__20230428T073949_20230428T082919_20230630T155658_2970_079_006______LN3_O_NT_002.SEN3.zip
S3B_SR_1_SRA_A__20230428T041528_20230428T050458_20230630T144957_2970_079_004______LN3_O_NT_002.SEN3.zip
S3B_SR_1_SRA_A__20230427T221820_20230427T230850_20230630T140705_3029_079_001______LN3_O_NT_002.SEN3.zip

and here the corresponding production from PS :

S3A_SR_1_SRA_A__20230427T093615_20230427T102644_20230523T123409_3029_098_136______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T102644_20230427T111714_20230523T123314_3029_098_136______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T111714_20230427T120743_20230523T141859_3029_098_137______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T120743_20230427T125813_20230523T141848_3029_098_137______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T125813_20230427T134843_20230523T155929_3029_098_138______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T134843_20230427T143912_20230523T155612_3029_098_138______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T143912_20230427T152942_20230523T173712_3029_098_139______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T152942_20230427T162012_20230523T174107_3029_098_139______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T162012_20230427T171041_20230523T192851_3029_098_140______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T171041_20230427T180110_20230523T192811_3029_098_140______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T180112_20230427T185140_20230523T205703_3028_098_141______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T185141_20230427T194210_20230523T205836_3029_098_141______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T194210_20230427T203240_20230523T224645_3029_098_142______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T203240_20230427T212309_20230523T224305_3029_098_142______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T212309_20230427T221339_20230524T002656_3029_098_143______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T221339_20230427T230409_20230524T002806_3029_098_143______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T230409_20230427T235438_20230524T021215_3029_098_144______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230427T235438_20230428T004508_20230524T021122_3029_098_144______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230428T004508_20230428T013537_20230524T033847_3029_098_145______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230428T013537_20230428T022607_20230524T034239_3029_098_145______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230428T022607_20230428T031637_20230524T052309_3029_098_146______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230428T031637_20230428T040706_20230524T052144_3029_098_146______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230428T040706_20230428T045736_20230524T070948_3029_098_147______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230428T045736_20230428T054805_20230524T070749_3029_098_147______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230428T054806_20230428T063835_20230524T084838_3029_098_148______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230428T063835_20230428T072905_20230524T085159_3029_098_148______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230428T072905_20230428T081934_20230524T102802_3029_098_149______PS1_O_NT_004.SEN3.zip
S3A_SR_1_SRA_A__20230428T081934_20230428T091004_20230524T103616_3029_098_149______PS1_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T094737_20230427T103807_20230521T143042_3029_078_378______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T103806_20230427T112836_20230521T140822_3029_078_379______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T112836_20230427T121906_20230521T155020_3029_078_379______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T121905_20230427T130935_20230521T154321_3029_078_380______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T130935_20230427T140005_20230521T172142_3029_078_380______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T140005_20230427T145035_20230521T172312_3029_078_381______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T145035_20230427T154105_20230521T190620_3029_078_381______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T154104_20230427T163134_20230521T190323_3029_078_382______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T163134_20230427T172204_20230521T205150_3029_078_382______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T172203_20230427T181233_20230521T204701_3029_078_383______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T181233_20230427T190303_20230521T223002_3029_078_383______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T190303_20230427T195333_20230521T222530_3029_078_384______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T195333_20230427T204400_20230522T000932_3026_078_384______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T204404_20230427T213432_20230522T001009_3027_078_385______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T213432_20230427T222502_20230522T014436_3029_078_385______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T222501_20230427T231531_20230522T014815_3029_079_001______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230427T231531_20230428T000601_20230522T033227_3029_079_001______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230428T000600_20230428T005630_20230522T033036_3029_079_002______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230428T005630_20230428T014700_20230522T051329_3029_079_002______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230428T032759_20230428T041829_20230522T065557_3029_079_004______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230428T041829_20230428T050859_20230522T084148_3029_079_004______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230428T050858_20230428T055928_20230522T085156_3029_079_005______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230428T055928_20230428T064958_20230522T102233_3029_079_005______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230428T064957_20230428T074027_20230522T101535_3029_079_006______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230428T074027_20230428T083057_20230522T120546_3029_079_006______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230428T083056_20230428T092126_20230522T120659_3029_079_007______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230428T014659_20230428T023729_20230522T051209_3029_079_003______PS2_O_NT_004.SEN3.zip
S3B_SR_1_SRA_A__20230428T023729_20230428T032759_20230522T070603_3029_079_003______PS2_O_NT_004.SEN3.zip

Whenever possible, first analysis of the root cause
Hypothesis :

  1. The difference is due to the baseline collection : 002 vs 004
  2. The difference is due to some parameter configuration. For example these parameters from PUG-NRT
    image.png

Bug Generic Definition of Ready (DoR)

  • The affect version in which the bug has been found is mentioned
  • The context and environment of the bug is detailed
  • The description of the bug is clear and unambiguous
  • The procedure (steps) to reproduce the bug is clearly detailed
  • The tested User Story / features is linked to the bug if available
  • Logs are attached if available
  • A data set attached if available

Bug Generic Definition of Done (DoD)

  • the modification implemented (the solution to fix the bug) is described in the bug.
  • Unit tests & Continuous integration performed - Test results available - Structural Test coverage reported by SONAR
  • Code committed in GIT with right tag or Analysis/Trade Off documentation up-to-date in reference-system-documentation repository
  • Code is compliant with coding rules (SONAR Report as evidence)
  • Acceptance criteria of the related User story are checked and Passed
@pcuq-ads pcuq-ads added bug Something isn't working CCB Issue for CCB ops Ticket from ADS operation team priority:major Set the priority to major because the production is heavily impacted S3 Relative to Sentinel-3 RS Addons to_be_fixed_phase1 Issue to be fixed for RS phase 1 labels Jul 27, 2023
@w-jka
Copy link

w-jka commented Jul 31, 2023

@pcuq-ads
The time intervals of the SR1 production are the same as the ones of the PUG production of the L0 SR products. Therefor the difference is inherited by the step before.

My assumption is, that we are running into a follow up issue on #1055. Based on the given information it could be, that the input products were produced before the PDU timeout was increased to 6 hours, therefore the last (half-)granule of the SR_0_SRA__G was missing. This would explain why every second PDU is missing the same amount of time.

Based on this assumption and the current configuration, this issue should not be present, if the production would be started from the granules.

Can you confirm my analysis?

@LAQU156
Copy link

LAQU156 commented Aug 2, 2023

System_CCB_2023_w31 : Moved into "Under Analysis", Waiting for the platform tests with 1.14.0 version

@LAQU156
Copy link

LAQU156 commented Aug 9, 2023 •
edited
Loading

System_CCB_2023_w32 : Validation ongoing with #1055 on last production

@LAQU156
Copy link

LAQU156 commented Aug 16, 2023

System_CCB_2023_w33 : Validation to be done due to #1055 closure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@w-fsi w-fsi

@w-jka w-jka

Labels
bug Something isn't working CCB Issue for CCB ops Ticket from ADS operation team priority:major Set the priority to major because the production is heavily impacted S3 Relative to Sentinel-3 RS Addons to_be_fixed_phase1 Issue to be fixed for RS phase 1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@pcuq-ads @w-fsi @w-jka @LAQU156