Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"parameter-filters" not working for "aws ssm get-parameters-by-path" ? #2850

Closed
jamietsao opened this issue Sep 22, 2017 · 10 comments
Closed
Assignees
Labels
guidance Question that needs advice or information.

Comments

@jamietsao
Copy link

jamietsao commented Sep 22, 2017

Unless I'm mistaken, the documentation for get-parameters-by-path says you can filter the parameter results via parameter-filters: http://docs.aws.amazon.com/cli/latest/reference/ssm/get-parameters-by-path.html

I can't seem to get this to work:

AWS_DEFAULT_REGION=us-west-2 aws ssm get-parameters-by-path --parameter-filters Key=tag:xyz  --path /my/path

This returns ALL parameters in /my/path instead of only those tagged with xyz.

describe-parameters has the same parameter-filters and it works:

AWS_DEFAULT_REGION=us-west-2 aws ssm describe-parameters --parameter-filters Key=tag:xyz

This will correctly return only parameters tagged with xyz.

Am I missing something?

@joguSD joguSD self-assigned this Sep 25, 2017
@joguSD joguSD added api-question closing-soon This issue will automatically close in 4 days unless further comments are made. labels Sep 25, 2017
@joguSD
Copy link
Contributor

joguSD commented Sep 25, 2017

Could you post debug logs by adding --debug?
It would seem that this should be possible and as long as the cli is properly sending the request this might actually be an issue with the service.

@jamietsao
Copy link
Author

jamietsao commented Sep 25, 2017

I've created two test parameters under /test:

➜  ~ AWS_DEFAULT_REGION=us-west-2 aws ssm get-parameters-by-path  --path /test
{
    "Parameters": [
        {
            "Type": "String",
            "Name": "/test/foo",
            "Value": "bar"
        },
        {
            "Type": "String",
            "Name": "/test/hello",
            "Value": "world"
        }
    ]
}

I added a baz tag to one of them:

➜  ~ AWS_DEFAULT_REGION=us-west-2 aws ssm describe-parameters --parameter-filters Key=tag:baz
{
    "Parameters": [
        {
            "LastModifiedUser": "arn:aws:iam::<redacted>:user/<redacted>",
            "LastModifiedDate": 1506377527.11,
            "Type": "String",
            "Name": "/test/foo"
        }
    ]
}

But when I try to filter by baz tag using get-parameters-by-path, it still returns both:

➜  ~ AWS_DEFAULT_REGION=us-west-2 aws ssm get-parameters-by-path --parameter-filters Key=tag:baz --path /test --debug
2017-09-25 15:16:02,231 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.11.150 Python/2.7.10 Darwin/16.7.0 botocore/1.7.8
2017-09-25 15:16:02,231 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['ssm', 'get-parameters-by-path', '--parameter-filters', 'Key=tag:baz', '--path', '/test', '--debug']
2017-09-25 15:16:02,231 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_scalar_parsers at 0x109a9d578>
2017-09-25 15:16:02,231 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x1097b9d70>
2017-09-25 15:16:02,233 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.11.150/libexec/lib/python2.7/site-packages/botocore/data/ssm/2014-11-06/service-2.json
2017-09-25 15:16:02,275 - MainThread - botocore.hooks - DEBUG - Event service-data-loaded.ssm: calling handler <function register_retries_for_service at 0x109375758>
2017-09-25 15:16:02,275 - MainThread - botocore.handlers - DEBUG - Registering retry handlers for service: ssm
2017-09-25 15:16:02,279 - MainThread - botocore.hooks - DEBUG - Event building-command-table.ssm: calling handler <function add_waiters at 0x109aa6668>
2017-09-25 15:16:02,284 - MainThread - awscli.clidriver - DEBUG - OrderedDict([(u'path', <awscli.arguments.CLIArgument object at 0x10a2fb190>), (u'recursive', <awscli.arguments.BooleanArgument object at 0x10a2fb1d0>), (u'no-recursive', <awscli.arguments.BooleanArgument object at 0x10a2fb210>), (u'parameter-filters', <awscli.arguments.ListArgument object at 0x10a2fb250>), (u'with-decryption', <awscli.arguments.BooleanArgument object at 0x10a2f0dd0>), (u'no-with-decryption', <awscli.arguments.BooleanArgument object at 0x10a2fb290>), (u'max-results', <awscli.arguments.CLIArgument object at 0x10a2fb2d0>), (u'next-token', <awscli.arguments.CLIArgument object at 0x10a2fb310>)])
2017-09-25 15:16:02,284 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ssm.get-parameters-by-path: calling handler <function add_streaming_output_arg at 0x109a9d7d0>
2017-09-25 15:16:02,284 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ssm.get-parameters-by-path: calling handler <function add_cli_input_json at 0x1097c2b90>
2017-09-25 15:16:02,284 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ssm.get-parameters-by-path: calling handler <function unify_paging_params at 0x109a16d70>
2017-09-25 15:16:02,288 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.11.150/libexec/lib/python2.7/site-packages/botocore/data/ssm/2014-11-06/paginators-1.json
2017-09-25 15:16:02,288 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.ssm.get-parameters-by-path: calling handler <function add_generate_skeleton at 0x109a0e1b8>
2017-09-25 15:16:02,288 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.ssm.get-parameters-by-path: calling handler <bound method CliInputJSONArgument.override_required_args of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x10a2fb350>>
2017-09-25 15:16:02,288 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.ssm.get-parameters-by-path: calling handler <bound method GenerateCliSkeletonArgument.override_required_args of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x10a2d6150>>
2017-09-25 15:16:02,291 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.path: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,291 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.ssm.get-parameters-by-path: calling handler <awscli.argprocess.ParamShorthandParser object at 0x109aa83d0>
2017-09-25 15:16:02,291 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'/test' for parameter "path": u'/test'
2017-09-25 15:16:02,291 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.recursive: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,291 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.parameter-filters: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,291 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.ssm.get-parameters-by-path: calling handler <awscli.argprocess.ParamShorthandParser object at 0x109aa83d0>
2017-09-25 15:16:02,291 - MainThread - awscli.argprocess - DEBUG - Parsing param --parameter-filters as shorthand
2017-09-25 15:16:02,292 - MainThread - awscli.arguments - DEBUG - Unpacked value of [u'Key=tag:baz'] for parameter "parameter_filters": [{u'Key': u'tag:baz'}]
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.with-decryption: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.max-results: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.next-token: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.cli-input-json: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.ssm.get-parameters-by-path.generate-cli-skeleton: calling handler <function uri_param at 0x109655938>
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event calling-command.ssm.get-parameters-by-path: calling handler <bound method CliInputJSONArgument.add_to_call_parameters of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x10a2fb350>>
2017-09-25 15:16:02,292 - MainThread - botocore.hooks - DEBUG - Event calling-command.ssm.get-parameters-by-path: calling handler <bound method GenerateCliSkeletonArgument.generate_json_skeleton of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x10a2d6150>>
2017-09-25 15:16:02,292 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2017-09-25 15:16:02,292 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2017-09-25 15:16:02,293 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2017-09-25 15:16:02,293 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2017-09-25 15:16:02,293 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.11.150/libexec/lib/python2.7/site-packages/botocore/data/endpoints.json
2017-09-25 15:16:02,329 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.ssm: calling handler <function add_generate_presigned_url at 0x1093459b0>
2017-09-25 15:16:02,329 - MainThread - botocore.args - DEBUG - The s3 config key is not a dictionary type, ignoring its value of: None
2017-09-25 15:16:02,341 - MainThread - botocore.endpoint - DEBUG - Setting ssm timeout as (60, 60)
2017-09-25 15:16:02,342 - MainThread - botocore.client - DEBUG - Registering retry handlers for service: ssm
2017-09-25 15:16:02,342 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.ssm.GetParametersByPath: calling handler <function generate_idempotent_uuid at 0x1093751b8>
2017-09-25 15:16:02,343 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=GetParametersByPath) (verify_ssl=True) with params: {'body': '{"Path": "/test", "ParameterFilters": [{"Key": "tag:baz"}]}', 'url': u'https://ssm.us-west-2.amazonaws.com/', 'headers': {'User-Agent': 'aws-cli/1.11.150 Python/2.7.10 Darwin/16.7.0 botocore/1.7.8', 'Content-Type': u'application/x-amz-json-1.1', 'X-Amz-Target': u'AmazonSSM.GetParametersByPath'}, 'context': {'auth_type': None, 'client_region': 'us-west-2', 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x10a4b9d50>}, 'query_string': '', 'url_path': '/', 'method': u'POST'}
2017-09-25 15:16:02,343 - MainThread - botocore.hooks - DEBUG - Event request-created.ssm.GetParametersByPath: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x10a4b9cd0>>
2017-09-25 15:16:02,343 - MainThread - botocore.hooks - DEBUG - Event choose-signer.ssm.GetParametersByPath: calling handler <function set_operation_specific_signer at 0x1093750c8>
2017-09-25 15:16:02,343 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2017-09-25 15:16:02,343 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
POST
/

content-type:application/x-amz-json-1.1
host:ssm.us-west-2.amazonaws.com
x-amz-date:20170925T221602Z
x-amz-target:AmazonSSM.GetParametersByPath

content-type;host;x-amz-date;x-amz-target
472e8bd620027d5bcbfae2b28aae0c138963d1ce4201a45657ba3eef65d15667
2017-09-25 15:16:02,343 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20170925T221602Z
20170925/us-west-2/ssm/aws4_request
8634a86420a036bb701d34ff2f827b56fa7dd9792956370682ba5b4df729d4af
2017-09-25 15:16:02,343 - MainThread - botocore.auth - DEBUG - Signature:
9e491139a8fa51c3164aed3d5eee90d826f082c5495a8634854c7969382e1543
2017-09-25 15:16:02,345 - MainThread - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [POST]>
2017-09-25 15:16:02,345 - MainThread - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): ssm.us-west-2.amazonaws.com
2017-09-25 15:16:02,541 - MainThread - botocore.vendored.requests.packages.urllib3.connectionpool - DEBUG - "POST / HTTP/1.1" 200 122
2017-09-25 15:16:02,542 - MainThread - botocore.parsers - DEBUG - Response headers: {'x-amzn-requestid': '1dd11356-a23f-11e7-96b2-41ae96746da4', 'date': 'Mon, 25 Sep 2017 22:16:02 GMT', 'content-length': '122', 'content-type': 'application/x-amz-json-1.1'}
2017-09-25 15:16:02,542 - MainThread - botocore.parsers - DEBUG - Response body:
{"Parameters":[{"Name":"/test/foo","Type":"String","Value":"bar"},{"Name":"/test/hello","Type":"String","Value":"world"}]}
2017-09-25 15:16:02,543 - MainThread - botocore.hooks - DEBUG - Event needs-retry.ssm.GetParametersByPath: calling handler <botocore.retryhandler.RetryHandler object at 0x10a2d60d0>
2017-09-25 15:16:02,543 - MainThread - botocore.retryhandler - DEBUG - No retry needed.
2017-09-25 15:16:02,543 - MainThread - awscli.formatter - DEBUG - RequestId: 1dd11356-a23f-11e7-96b2-41ae96746da4
{
    "Parameters": [
        {
            "Type": "String",
            "Name": "/test/foo",
            "Value": "bar"
        },
        {
            "Type": "String",
            "Name": "/test/hello",
            "Value": "world"
        }
    ]
}

As you can see, the filtering works for describe-parameters, but not get-parameters-by-path even though the documentation for parameter-filters is identical for both commands.

@jamietsao
Copy link
Author

jamietsao commented Sep 28, 2017

@joguSD - any updates?

@kyleknap kyleknap added investigating This issue is being investigated and/or work is in progress to resolve the issue. and removed closing-soon This issue will automatically close in 4 days unless further comments are made. api-question labels Oct 2, 2017
@joguSD
Copy link
Contributor

joguSD commented Oct 2, 2017

As far as I can tell the CLI is passing along the correct arguments in the same manner in both cases so this is ultimately a difference in the service behavior between the two API calls. You should open up a support ticket to get a hold of the service team, I'll try pinging them as well. Closing out the issue here as there's no actionable item in this repo.

@joguSD joguSD closed this as completed Oct 2, 2017
@joguSD joguSD added move-to-forums and removed investigating This issue is being investigated and/or work is in progress to resolve the issue. labels Oct 2, 2017
@diehlaws diehlaws added guidance Question that needs advice or information. and removed move-to-forums labels Jan 4, 2019
@satterly
Copy link

satterly commented Nov 14, 2019

The documentation says it's not supported...

GetParametersByPath
Retrieve parameters in a specific hierarchy. For more information, see Working with Systems Manager Parameters in the AWS Systems Manager User Guide.
Note
This API action doesn't support filtering by tags.

https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParametersByPath.html

@jamietsao
Copy link
Author

jamietsao commented Nov 16, 2019

Yeah, that note wasn't there when I posted this over 2 years ago. Thanks.

@Obirah
Copy link

Obirah commented Mar 11, 2020

Just if someone stumbles across this issue like I did: I worked around this restriction by labeling the parameters.

@Michael1d
Copy link

Michael1d commented Apr 3, 2020

Genius @Obirah ! Spent hours trying to work out why the tag functionality didn't work. Found your comment - sorted! Cheers :)

@cosmok
Copy link

cosmok commented Aug 20, 2020

Another option is to use describe-parameters instead (when we need to filter by tags) and then pipe the result to get-parameter.

See:
https://www.trk7.com/blog/aws-ssm-parameter-store-iam-tag-support/

@jamietsao
Copy link
Author

jamietsao commented Nov 29, 2021

To close the loop on this, I ended up doing something similar to @cosmok's suggestion: A call to describe-parameters to filter by tags, then call get-parameters to retrieve the actual parameter values.

thoward-godaddy pushed a commit to thoward-godaddy/aws-cli that referenced this issue Feb 12, 2022
* get questions' default answers from toml

* make black happy

* add more docs

* rename question's attribute 'default_from_toml' to 'defaultFromToml' and rename 'valueof' to 'key' and add some docs

* Add preload_value

* Allow to pass toml file to interactive flow run()

* Update related classes to utilize proload value context object

* Update test

* Add missing docstring

* Remove samconfig change

* Rename extra_context to context because it is required field now

* Remove toml logics from this PR

* Update comment

Co-authored-by: Sam Liu <xinhol@amazon.com>
Co-authored-by: _sam <3804518+aahung@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
guidance Question that needs advice or information.
Projects
None yet
Development

No branches or pull requests

8 participants