Amazonia-1 pipeline initial implementation #33
Conversation
|
@fredliporace Apologies for the delay in reviewing this. I'll try to check over things today. |
There was a problem hiding this comment.
@fredliporace I deployed this pipeline to our dev environment. The collection was created successfully
curl --location --request GET 'https://dev.asdi-catalog.org/collections/AMAZONIA1-WFI'
I'll monitor things to verify that the SNS topic subscription is working correctly and then merge this.
| @@ -0,0 +1,3 @@ | |||
| aws-lambda-powertools | |||
| jmespath | |||
| stactools-amazonia-1 >= 0.1.1 | |||
There was a problem hiding this comment.
@fredliporace Can you also include s3fs in the requirements as it is necessary when fsspec infers the s3 protocol when using use_fsspec.
There was a problem hiding this comment.
Done in 638779e
| xml_key = xml_key_from_quicklook_key(png_key) | ||
| href = f"s3://{bucket}/{xml_key}" | ||
| print(href) | ||
| stac = create_item(asset_href=href) |
There was a problem hiding this comment.
@fredliporace I also created another new issue over on the stactools package regarding the hardcoded bucket prefix of cbers-pds.
| from aws_asdi_pipelines.cognito.utils import get_token | ||
|
|
||
|
|
||
| def xml_key_from_quicklook_key(key: str) -> str: |
There was a problem hiding this comment.
@fredliporace It looks like this function might need to be extended for some different cases of inferring the xml key. As an example looking at s3://amazonia-pds/AMAZONIA1/WFI/033/018/AMAZONIA_1_WFI_20230621_033_018_L4/.
The xml key is actually AMAZONIA_1_WFI_20230621_033_018_L4_LEFT_BAND2.xml rather than the key AMAZONIA_1_WFI_20230621_033_018_L4_BAND2.xml which the function produces.
There was a problem hiding this comment.
@sharkinsspatial see #34, working on that.
There was a problem hiding this comment.
@sharkinsspatial #34 is fixed by bdfe86c but my testing in actual aws environment is limited, let me know if any additional work is required.
What I am changing
How I did it
How you can test it