Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat[server]: offline products order handling #918

Merged
merged 10 commits into from
Apr 10, 2024
Merged

feat[server]: offline products order handling #918

merged 10 commits into from
Apr 10, 2024

Conversation

dalpasso
Copy link
Collaborator

@dalpasso dalpasso commented Nov 13, 2023 •
edited by sbrunato

When downloading an OFFLINE product in server mode:

  • return HTTP Status 202 with new download link containing order status check information as query parameters
  • don't retry to download the product again

Closes #871

Also fixes missing dataset parameter after BuildSearchResult search-by-id in server mode

@dalpasso dalpasso linked an issue Nov 13, 2023 that may be closed by this pull request
staging status in server mode download #871
Closed
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 13, 2023 •
edited

Test Results

    4 files  ±0      4 suites  ±0   5m 27s ⏱️ ±0s
  505 tests +1    502 ✅ +1   3 💤 ±0  0 ❌ ±0 
2 020 runs  +4  1 932 ✅ +2  88 💤 +2  0 ❌ ±0 

Results for commit 7dc9cee. ± Comparison against base commit 4286d13.

This pull request removes 1 and adds 2 tests. Note that renamed tests count towards both. 
tests.units.test_download_plugins.TestDownloadPluginHttp ‑ test_plugins_download_http_order_download_cds

tests.units.test_download_plugins.TestDownloadPluginHttp ‑ test_plugins_download_http_order_download_cop_ads
tests.units.test_http_server.RequestTestCase ‑ test_download_offline_item_from_catalog

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 13, 2023 •
edited

Code Coverage (Ubuntu)

File Coverage
All files 82% ✅

Minimum allowed coverage is 70%

Generated by 🐒 cobertura-action against a35b21c

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 16, 2023 •
edited

Code Coverage (Windows)

File Coverage
All files 76% ✅

Minimum allowed coverage is 70%

Generated by 🐒 cobertura-action against a35b21c

sbrunato
sbrunato requested changes Nov 23, 2023
View reviewed changes
eodag/rest/utils.py Outdated Show resolved Hide resolved
@dalpasso dalpasso force-pushed the 871-staging-status-in-server-mode-download branch from 3d25558 to a35b21c Compare November 23, 2023 16:49
@sbrunato sbrunato marked this pull request as draft March 20, 2024 15:39
@sbrunato sbrunato force-pushed the 871-staging-status-in-server-mode-download branch from a35b21c to 4142cd1 Compare April 9, 2024 21:43
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 9, 2024 •
edited

badge

Code Coverage (Ubuntu)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     11       0  100.00%
cli.py                                         300      47  84.33%   61, 647-686, 788-839, 843
config.py                                      318      27  91.51%   80-82, 91, 99, 103-105, 178, 190, 391-393, 457-460, 507-508, 517-518, 597, 666-671, 673
crunch.py                                        6       6  0.00%    18-24
api/__init__.py                                  1       0  100.00%
api/core.py                                    727      76  89.55%   87-96, 369, 582, 626-629, 667, 772, 776-781, 807, 877, 954, 1068-1073, 1159-1171, 1211, 1213, 1217, 1238-1240, 1244-1255, 1268-1274, 1364-1367, 1396-1416, 1464, 1470-1473, 1482, 1848, 1881-1887, 2152, 2156-2159, 2173-2175, 2210
api/search_result.py                            44       6  86.36%   33-35, 70, 79, 86, 100
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          44       5  88.64%   27-29, 79, 129
api/product/_product.py                        211      32  84.83%   53-60, 64-66, 170-177, 261-262, 352, 388, 449, 463-466, 479, 503-506, 549-555
api/product/metadata_mapping.py                649      82  87.37%   67-69, 130-132, 233, 265-266, 318-330, 332, 343, 349-361, 402-405, 442, 463-466, 489, 497-498, 571-572, 596-597, 603-606, 621-622, 771, 817, 934-939, 1070, 1084-1104, 1124, 1129, 1239, 1261, 1275, 1288-1307, 1346, 1398, 1436-1440, 1459
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              1       0  100.00%
plugins/base.py                                 23       3  86.96%   25, 48, 55
plugins/manager.py                             127      14  88.98%   49-51, 95-100, 146, 185, 205, 231, 270-271
plugins/apis/__init__.py                         1       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           99      15  84.85%   47-55, 156-158, 205-206, 232-234
plugins/apis/usgs.py                           172      36  79.07%   59-64, 128, 201, 235, 270-272, 277, 303-304, 309, 339-346, 357-362, 384-390, 392-398, 421
plugins/authentication/__init__.py               7       1  85.71%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                17       1  94.12%   27
plugins/authentication/keycloak.py              88      17  80.68%   32-34, 159-160, 190-212, 238-243
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       103      64  37.86%   39-41, 138-150, 154-172, 180-222, 228-237, 246-286, 291-299, 304-305
plugins/authentication/qsauth.py                36       2  94.44%   32, 83
plugins/authentication/sas_auth.py              49       2  95.92%   32, 76
plugins/authentication/token.py                 81      13  83.95%   35-37, 86, 99, 101, 114-117, 165-168
plugins/crunch/__init__.py                       1       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      10  80.00%   32-34, 51-52, 71, 80-83, 85, 92-95
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      17  75.00%   28-30, 33, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     1       0  100.00%
plugins/download/aws.py                        491     165  66.40%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 458-459, 465-469, 502, 537, 541, 548, 578-586, 590, 628-636, 643-645, 686-760, 778-839, 850-855, 871-884, 913, 928-930, 933, 943-951, 959-972, 982-1001, 1008-1020, 1061, 1087, 1132-1134, 1354
plugins/download/base.py                       261      57  78.16%   58-64, 145, 180, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 689, 711, 719
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       458      94  79.48%   82-88, 122, 196-208, 211, 292-297, 321, 323, 387, 414-416, 426, 432, 434, 458, 497-501, 558, 644-700, 718, 751-760, 786-787, 814, 838, 846-851, 873-874, 881, 942-948, 1003-1004, 1010, 1020, 1086, 1090, 1104-1120
plugins/download/s3rest.py                     117      27  76.92%   55-58, 124, 165, 199, 229-236, 239-241, 245, 258-264, 272-273, 276-280, 303, 324-327
plugins/search/__init__.py                       1       0  100.00%
plugins/search/base.py                         127       9  92.91%   49-54, 108, 112, 275, 295, 378
plugins/search/build_search_result.py          179      22  87.71%   62, 97, 142-143, 149, 160, 290-293, 327, 384-396, 458, 461, 471, 488, 516, 518
plugins/search/creodias_s3.py                   54       3  94.44%   55, 73, 107
plugins/search/csw.py                          107      83  22.43%   43-45, 57-58, 62-63, 74-122, 128-141, 149-181, 199-240
plugins/search/data_request_search.py          198      65  67.17%   52, 89-92, 108, 120, 124-125, 139, 144, 149, 156, 169-172, 226-227, 231, 241-247, 252, 280-283, 291-302, 319, 321, 328-329, 331-332, 350-354, 387, 394, 405, 418, 424-436, 441
plugins/search/qssearch.py                     561      65  88.41%   89, 361-367, 375-376, 483-489, 544-547, 620-621, 662, 680, 695, 748, 769, 772-773, 782, 793, 802, 825, 885-890, 894-895, 923, 994, 1013-1030, 1069, 1145-1149, 1215-1216, 1237, 1262, 1268, 1299-1300, 1310-1316, 1359, 1373, 1393, 1483
plugins/search/static_stac_search.py            47       3  93.62%   39-40, 82
rest/__init__.py                                 1       0  100.00%
rest/core.py                                   199      15  92.46%   71-77, 261, 586, 588, 591-593, 665, 672-676
rest/server.py                                 299      54  81.94%   80-81, 107, 130-131, 242-244, 260, 300-301, 313-329, 415-420, 451, 612-619, 651, 694-695, 783-785, 802-807, 837, 839, 843-844, 848-849
rest/stac.py                                   436      95  78.21%   59-61, 229-231, 273, 286-295, 314-320, 365, 402-404, 427, 462-463, 549-594, 639, 659-660, 840, 905-907, 1126, 1136-1148, 1161-1183, 1197-1242, 1401-1402
rest/types/__init__.py                           1       0  100.00%
rest/types/eodag_search.py                     185       9  95.14%   52-55, 231-235, 288, 291, 359
rest/types/stac_queryables.py                   30       2  93.33%   28, 114
rest/types/stac_search.py                      122      10  91.80%   48-51, 167, 182-184, 192, 196
rest/utils/__init__.py                         116      14  87.93%   53, 108-109, 128-130, 180, 190-204, 236
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           31       3  90.32%   78, 90, 92
types/__init__.py                               76       6  92.11%   53, 87, 152, 172, 177, 185
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              496      43  91.33%   83, 88, 109-111, 189-190, 199-226, 229, 243, 325-329, 405-409, 430-432, 514, 519, 529, 567-568, 964-967, 975-976, 1017-1018, 1098, 1182, 1200, 1375
utils/constraints.py                           123      42  65.85%   31, 84-93, 134, 139, 143, 154, 177-178, 189-197, 206, 220-236, 245-256
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/stac_reader.py                            91      28  69.23%   55-56, 63-86, 93-95, 99, 141, 155-158
TOTAL                                         8697    1532  82.38%

Diff against develop

Filename                                 Stmts    Miss  Cover
-------------------------------------  -------  ------  --------
plugins/authentication/base.py               0      +1  -5.26%
plugins/download/http.py                   +15      +1  +0.47%
plugins/search/build_search_result.py       +1       0  +0.07%
plugins/search/qssearch.py                  +1      +1  -0.16%
rest/core.py                               +30      +5  -1.62%
rest/server.py                              +1       0  +0.06%
utils/__init__.py                           +1       0  +0.02%
TOTAL                                      +49      +8  +100.00%

Results for commit: 7dc9cee

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 9, 2024 •
edited

badge

Code Coverage (Windows)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     11       0  100.00%
cli.py                                         300      47  84.33%   61, 647-686, 788-839, 843
config.py                                      318      28  91.19%   80-82, 91, 99, 103-105, 178, 190, 391-393, 457-460, 507-508, 517-518, 597, 631, 666-671, 673
crunch.py                                        6       6  0.00%    18-24
api/__init__.py                                  1       0  100.00%
api/core.py                                    727      76  89.55%   87-96, 369, 582, 626-629, 667, 772, 776-781, 807, 877, 954, 1068-1073, 1159-1171, 1211, 1213, 1217, 1238-1240, 1244-1255, 1268-1274, 1364-1367, 1396-1416, 1464, 1470-1473, 1482, 1848, 1881-1887, 2152, 2156-2159, 2173-2175, 2210
api/search_result.py                            44       6  86.36%   33-35, 70, 79, 86, 100
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          44       5  88.64%   27-29, 79, 129
api/product/_product.py                        211      32  84.83%   53-60, 64-66, 170-177, 261-262, 352, 388, 449, 463-466, 479, 503-506, 549-555
api/product/metadata_mapping.py                649      83  87.21%   67-69, 130-132, 233, 265-266, 318-330, 332, 343, 349-361, 402-405, 442, 463-466, 489, 497-498, 571-572, 596-597, 603-606, 621-622, 771, 817, 934-939, 1070, 1084-1104, 1124, 1129, 1239, 1261, 1275, 1288-1307, 1346, 1398, 1421, 1436-1440, 1459
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              1       0  100.00%
plugins/base.py                                 23       4  82.61%   25, 48, 55, 68
plugins/manager.py                             127      14  88.98%   49-51, 95-100, 146, 185, 205, 231, 270-271
plugins/apis/__init__.py                         1       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           99      15  84.85%   47-55, 156-158, 205-206, 232-234
plugins/apis/usgs.py                           172      36  79.07%   59-64, 128, 201, 235, 270-272, 277, 303-304, 309, 339-346, 357-362, 384-390, 392-398, 421
plugins/authentication/__init__.py               7       1  85.71%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                17       1  94.12%   27
plugins/authentication/keycloak.py              88      17  80.68%   32-34, 159-160, 190-212, 238-243
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       103      64  37.86%   39-41, 138-150, 154-172, 180-222, 228-237, 246-286, 291-299, 304-305
plugins/authentication/qsauth.py                36       2  94.44%   32, 83
plugins/authentication/sas_auth.py              49       2  95.92%   32, 76
plugins/authentication/token.py                 81      13  83.95%   35-37, 86, 99, 101, 114-117, 165-168
plugins/crunch/__init__.py                       1       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      35  30.00%   32-34, 48-53, 69-114
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      17  75.00%   28-30, 33, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     1       0  100.00%
plugins/download/aws.py                        491     165  66.40%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 458-459, 465-469, 502, 537, 541, 548, 578-586, 590, 628-636, 643-645, 686-760, 778-839, 850-855, 871-884, 913, 928-930, 933, 943-951, 959-972, 982-1001, 1008-1020, 1061, 1087, 1132-1134, 1354
plugins/download/base.py                       261      59  77.39%   58-64, 145, 180, 250-252, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 689, 711, 719
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       458      94  79.48%   82-88, 122, 196-208, 211, 292-297, 321, 323, 387, 414-416, 426, 432, 434, 458, 497-501, 558, 644-700, 718, 751-760, 786-787, 814, 838, 846-851, 873-874, 881, 942-948, 1003-1004, 1010, 1020, 1086, 1090, 1104-1120
plugins/download/s3rest.py                     117      27  76.92%   55-58, 124, 165, 199, 229-236, 239-241, 245, 258-264, 272-273, 276-280, 303, 324-327
plugins/search/__init__.py                       1       0  100.00%
plugins/search/base.py                         127       9  92.91%   49-54, 108, 112, 275, 295, 378
plugins/search/build_search_result.py          179      29  83.80%   62, 97, 142-143, 149, 160, 290-293, 327, 384-396, 458, 461, 471, 488, 508-523
plugins/search/creodias_s3.py                   54       3  94.44%   55, 73, 107
plugins/search/csw.py                          107      83  22.43%   43-45, 57-58, 62-63, 74-122, 128-141, 149-181, 199-240
plugins/search/data_request_search.py          198      65  67.17%   52, 89-92, 108, 120, 124-125, 139, 144, 149, 156, 169-172, 226-227, 231, 241-247, 252, 280-283, 291-302, 319, 321, 328-329, 331-332, 350-354, 387, 394, 405, 418, 424-436, 441
plugins/search/qssearch.py                     561      89  84.14%   89, 361-367, 375-376, 483-489, 544-547, 620-621, 662, 680, 695, 748, 769, 772-773, 782, 793, 802, 825, 885-890, 894-895, 923, 994, 1013-1030, 1069, 1145-1149, 1215-1216, 1237, 1262, 1268, 1299-1300, 1310-1316, 1359, 1373, 1393, 1443-1512
plugins/search/static_stac_search.py            47       3  93.62%   39-40, 82
rest/__init__.py                                 1       0  100.00%
rest/core.py                                   199      88  55.78%   71-77, 158, 160, 162, 168-169, 186-194, 200-206, 253-302, 313-340, 490-517, 535, 585-624, 665, 672-676
rest/server.py                                 299     299  0.00%    18-862
rest/stac.py                                   436     154  64.68%   59-61, 214, 229-231, 273, 286-295, 314-320, 365, 402-404, 427, 462-463, 549-594, 639, 647-648, 652-660, 782, 840, 905-907, 924-926, 934-936, 949-951, 965-982, 992-1013, 1023-1045, 1053-1070, 1093-1116, 1126, 1136-1148, 1161-1183, 1197-1242, 1395-1421
rest/types/__init__.py                           1       0  100.00%
rest/types/eodag_search.py                     185      18  90.27%   52-55, 231-235, 268-270, 288, 291, 297, 301, 359, 371-374
rest/types/stac_queryables.py                   30       6  80.00%   28, 53-58, 114
rest/types/stac_search.py                      122      12  90.16%   48-51, 167, 182-184, 192, 196, 240, 243
rest/utils/__init__.py                         116      31  73.28%   53, 79-85, 105, 108-109, 128-130, 147, 173-181, 188-209, 236
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           31       5  83.87%   73-74, 78, 90, 92
types/__init__.py                               76      11  85.53%   53, 87, 125, 152, 162-164, 172, 177, 185, 195
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              496      44  91.13%   83, 88, 109-111, 189-190, 199-226, 229, 243, 325-329, 405-409, 430-432, 514, 519, 529, 567-568, 964-967, 975-976, 1017-1018, 1098, 1181-1182, 1200, 1375
utils/constraints.py                           123      42  65.85%   31, 84-93, 134, 139, 143, 154, 177-178, 189-197, 206, 220-236, 245-256
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/stac_reader.py                            91      28  69.23%   55-56, 63-86, 93-95, 99, 141, 155-158
TOTAL                                         8697    2010  76.89%

Diff against develop

Filename                                 Stmts    Miss  Cover
-------------------------------------  -------  ------  --------
plugins/download/http.py                   +15      +1  +0.47%
plugins/search/build_search_result.py       +1       0  +0.09%
plugins/search/qssearch.py                  +1      +1  -0.15%
rest/core.py                               +30     +27  -8.13%
rest/server.py                              +1      +1  +100.00%
utils/__init__.py                           +1       0  +0.02%
TOTAL                                      +49     +30  -0.21%

Results for commit: 7dc9cee

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@sbrunato sbrunato marked this pull request as ready for review April 10, 2024 10:23
@sbrunato sbrunato force-pushed the 871-staging-status-in-server-mode-download branch from 386f28d to 7dc9cee Compare April 10, 2024 11:54
@sbrunato sbrunato changed the title 871 staging status in server mode download feat[server]: offline products order handling Apr 10, 2024
@sbrunato sbrunato merged commit dbfca9b into develop Apr 10, 2024
10 checks passed
@sbrunato sbrunato deleted the 871-staging-status-in-server-mode-download branch April 10, 2024 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sbrunato sbrunato Awaiting requested review from sbrunato

Assignees

@dalpasso dalpasso

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

staging status in server mode download
2 participants
@dalpasso @sbrunato