Skip to content

Conversation

kevinlacaille
Copy link
Contributor

@kevinlacaille kevinlacaille commented Mar 1, 2023

Related Issue(s):

Closes #505, #506, #507

Proposed Changes:

For inclusion in changelog (if applicable):

  1. Add planet data asset-activate, asset-wait, asset-download CLI commands to help with activating and downloading assets.

Not intended for changelog:

Diff of User Interface

Old behavior:

N/A

New behavior:

asset-activate

planet data asset-activate PSScene 20221003_002705_38_2461 basic_udm2

asset-wait

planet data asset-wait PSScene 20221003_002705_38_2461 basic_udm2
00:00 - order my asset - state: active
{'_links': {'_self': 'https://api.planet.com/data/v1/assets/eyJpIjogIjIwMjIxMDAzXzAwMjcwNV8zOF8yNDYxIiwgImMiOiAiUFNTY2VuZSIsICJ0IjogImJhc2ljX3VkbTIiLCAiY3QiOiAiaXRlbS10eXBlIn0', 'activate': 'https://api.planet.com/data/v1/assets/eyJpIjogIjIwMjIxMDAzXzAwMjcwNV8zOF8yNDYxIiwgImMiOiAiUFNTY2VuZSIsICJ0IjogImJhc2ljX3VkbTIiLCAiY3QiOiAiaXRlbS10eXBlIn0/activate', 'type': 'https://api.planet.com/data/v1/asset-types/basic_udm2'}, '_permissions': ['download'], 'expires_at': '2023-03-01T23:25:19.908175', 'location': 'https://api.planet.com/data/v1/download?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzUxMiJ9.eyJzdWIiOiI0c0NHUW1rcXk4REg1RUpnYS1mV2RGRHp6VlJiT2VtdzFEVjU1OFpPQzE3ZE41THJlcmtxVzJ4c2J0T0t3bEREaWttenAzeHVPZ0w3OU1YNkQ1ZUxaZz09IiwiZXhwIjoxNjc3NzEzMTE5LCJ0b2tlbl90eXBlIjoidHlwZWQtaXRlbSIsIml0ZW1fdHlwZV9pZCI6IlBTU2NlbmUiLCJpdGVtX2lkIjoiMjAyMjEwMDNfMDAyNzA1XzM4XzI0NjEiLCJhc3NldF90eXBlIjoiYmFzaWNfdWRtMiJ9.i4ynH7e8xz-Sv6bUtYETKkujEczooMWD3p65Qf3am2OHTv6amnirBjSySsSTfG1gPcHz0smpNK_QvXD6108COA', 'md5_digest': '3a9f7dd1ce500f699d0a96afdd0e3aa2', 'status': 'active', 'type': 'basic_udm2'}

asset-download

planet data asset-download PSScene 20221003_002705_38_2461 basic_udm2 --overwrite --checksum                 
/Users/kevin.lacaille/code/repos/planet/planet-client-python/20221003_002705_38_2461_1A_udm2.tif: 100%|████████████████████████████| 3.16k/3.16k [00:00<00:00, 26.5MB/s]

Putting it all together

planet data asset-activate PSScene 20221003_002705_38_2461 basic_udm2 && \
planet data asset-wait PSScene 20221003_002705_38_2461 basic_udm2 && \
planet data asset-download PSScene 20221003_002705_38_2461 basic_udm2 --overwrite
00:00 - order my asset - state: active
{'_links': {'_self': 'https://api.planet.com/data/v1/assets/eyJpIjogIjIwMjIxMDAzXzAwMjcwNV8zOF8yNDYxIiwgImMiOiAiUFNTY2VuZSIsICJ0IjogImJhc2ljX3VkbTIiLCAiY3QiOiAiaXRlbS10eXBlIn0', 'activate': 'https://api.planet.com/data/v1/assets/eyJpIjogIjIwMjIxMDAzXzAwMjcwNV8zOF8yNDYxIiwgImMiOiAiUFNTY2VuZSIsICJ0IjogImJhc2ljX3VkbTIiLCAiY3QiOiAiaXRlbS10eXBlIn0/activate', 'type': 'https://api.planet.com/data/v1/asset-types/basic_udm2'}, '_permissions': ['download'], 'expires_at': '2023-03-02T19:30:48.942718', 'location': 'https://api.planet.com/data/v1/download?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzUxMiJ9.eyJzdWIiOiJQYWtGNHZuUEs3WXRmSFNGUklHY2I3YTNXT3piaTlaam4zWUpZMmxnd0x5cVlFMVBRSHU5QXNCcjR5Q3FxSjBNbl9yN3VwVEFQYUI1ZzhYNUJmcDhmUT09IiwiZXhwIjoxNjc3Nzg1NDQ4LCJ0b2tlbl90eXBlIjoidHlwZWQtaXRlbSIsIml0ZW1fdHlwZV9pZCI6IlBTU2NlbmUiLCJpdGVtX2lkIjoiMjAyMjEwMDNfMDAyNzA1XzM4XzI0NjEiLCJhc3NldF90eXBlIjoiYmFzaWNfdWRtMiJ9.Dd0opDjW3bBS6qLLZoNiJkfBsO2n5Xz9pM5apEUz_K6viDPFexhJiy6bMbaySbby8W0YvuATdb1uYXS2FkweDg', 'md5_digest': '3a9f7dd1ce500f699d0a96afdd0e3aa2', 'status': 'active', 'type': 'basic_udm2'}
/Users/kevin.lacaille/code/repos/planet/planet-client-python/20221003_002705_38_2461_1A_udm2.tif: 100%|██████████████████████████████████| 3.16k/3.16k [00:00<00:00, 32.0MB/s]

PR Checklist:

  • This PR is as small and focused as possible
  • If this PR includes proposed changes for inclusion in the changelog, the title of this PR summarizes those changes and is ready for inclusion in the Changelog.
  • I have updated docstrings for function changes and docs in the 'docs' folder for user interface / behavior changes
  • This PR does not break any examples or I have updated them

(Optional) @mentions for Notifications:

@kevinlacaille kevinlacaille added CLI/SDK Interface Update the CLI and SDK to the finalized interface data api labels Mar 1, 2023
@kevinlacaille kevinlacaille self-assigned this Mar 1, 2023
@kevinlacaille kevinlacaille marked this pull request as ready for review March 1, 2023 22:41
@kevinlacaille kevinlacaille changed the title Data asset download 507 Implement CLI commands: planet data asset-get, download, activate, wait Mar 1, 2023
Copy link
Contributor

@jreiberkyle jreiberkyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EDIT:
Ok, wait, as I am looking at the usage in the PR desc I'm seeing an issue. If you save asset.json, then use planet data asset-download asset.json, you're not going to be successful because asset.json has not been updated since the asset was activated, and so it won't have the download urls. OK, let's go back to how the CLI-data docs specify the UI (that is, design-docs/CLI-Data.md), aka, input for the CLI commands are the item-type, item-id, and asset-type. This matches the orders UI as well.

INITIAL:
Ok so I tested this and found it to be a delight to use. The comments I have in the review are mostly fyis and style I think, I can't remember any necessary changes.

The one thing that DOES need to change is design-docs/CLI-Data.md since the UI in this PR differs. Ideally, we'd be working from that doc and have discussed the different UI before you got to work here, but I'm not sure I called that out in the ticket for one, and for two, I am on board with this change. So, it should be a quick change but it's one we should get in there before moving forward. I'll approve when it's in!

@jreiberkyle
Copy link
Contributor

Ok, wait, as I am looking at the usage in the PR desc I'm seeing an issue. If you save asset.json, then use planet data asset-download asset.json, you're not going to be successful because asset.json has not been updated since the asset was activated, and so it won't have the download urls. OK, let's go back to how the CLI-data docs specify the UI, that is, input for the CLI commands are the item-type, item-id, and asset-type. This matches the orders UI as well.

kevinlacaille and others added 3 commits March 1, 2023 19:40
Co-authored-by: Jennifer Reiber Kyle <jreiberkyle@users.noreply.github.com>
Co-authored-by: Jennifer Reiber Kyle <jreiberkyle@users.noreply.github.com>
@kevinlacaille
Copy link
Contributor Author

kevinlacaille commented Mar 2, 2023

@jreiberkyle alternatively, perhaps a better way of doing it would be:
planet data asset-get PSScene 20221003_002705_38_2461 basic_udm2 | planet data asset-download -

Or, maybe we stick to the CLI docs and then something like this works, too:
planet data asset-download PSScene 20221003_002705_38_2461 basic_udm2

I would just have to add the get_asset call to the download function.

async with data_client(ctx) as cl:
    asset = await cl.get_asset(item_type, item_id, asset_type_id)
    path = await cl.download_asset(asset=asset,
                                    filename=filename,
                                    directory=Path(directory),
                                    overwrite=overwrite,
                                    progress_bar=not quiet)
    if checksum:
        cl.validate_checksum(asset, path)

I think the later looks a little more elegant, let's go with it.

@kevinlacaille
Copy link
Contributor Author

@jreiberkyle Here’s what I did since you last reviewed it:

  • Removed asset as an input to asset-activate, asset-wait, asset-download, and replaced it with item_id, item_type, and asset_type
    • This got get_asset() involved in all three functions
  • Changed the tests for all three CLI functions to accommodate the new parameters
    • Created get_asset() ’s mocked response in a pytest fixture, removed a lot of clutter
  • Commented out asset-get from the CLI and tests
  • Added data asset commands to the docs

Copy link
Contributor

@jreiberkyle jreiberkyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just the item-type type needs to change to str (one entry) instead of comma-separated string (multiple entries)

For example, this is confusing to me:

planet data asset-activate PSScene,SkySatScene 20221003_002705_38_2461 basic_udm2
Error: {"general": [{"message": "Could not find the requested item."}], "field": {}}

For 1 how am I ordering the same ID/asset for two different satellites and for 2 the error message isn't telling me anything about how I specified two item types but only one will be used, and which one was used, which would help me understand the source of this error (maybe... I can't say I'm in love with the API error either).

Copy link
Contributor

@jreiberkyle jreiberkyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so close, so close! just three changes requested, two can be as simple as accepting suggestions, the other is an argument name change.

kevinlacaille and others added 3 commits March 3, 2023 19:48
Co-authored-by: Jennifer Reiber Kyle <jreiberkyle@users.noreply.github.com>
Co-authored-by: Jennifer Reiber Kyle <jreiberkyle@users.noreply.github.com>
@jreiberkyle jreiberkyle self-requested a review March 6, 2023 16:38
Copy link
Contributor

@jreiberkyle jreiberkyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@kevinlacaille kevinlacaille merged commit 150f438 into main Mar 6, 2023
@kevinlacaille kevinlacaille deleted the data-asset-download-507 branch March 6, 2023 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLI/SDK Interface Update the CLI and SDK to the finalized interface data api
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement CLI command: planet data asset-activate
2 participants