Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow passing aws config from CLI; prep for podman-remote pipeline work #2962

Merged
merged 4 commits into from
Jul 6, 2022

Conversation

dustymabe
Copy link
Member

See individual commits:

commit 65610231ceed734bfc3db5722343ac33db31936a
Author: Dusty Mabe <dusty@dustymabe.com>
Date:   Tue Jul 5 18:56:10 2022 -0400

    cmd-buildfetch: add --aws-config-file option
    
    This will allow us to specify the AWS_CONFIG_FILE from the command
    line without having to set it in the environment.
    
    This required reworking the cosalib/s3 library into an S3() class
    so that we could call boto3.client('s3') after setting the
    AWS_CONFIG_FILE environment variable in cmd-buildfetch. Otherwise
    it would get called as soon as the library was imported and setting
    the environment variable would have no effect (too late).

commit ba4cd8f600b3940694cb486fcbfbbb7059b2849c
Author: Dusty Mabe <dusty@dustymabe.com>
Date:   Tue Jul 5 18:54:20 2022 -0400

    cmd-buildupload: add s3 --aws-config-file option
    
    This will allow us to specify the AWS_CONFIG_FILE from the command
    line without having to set it in the environment.

commit 23c6f1aa000581c98a81df814312b423c6a73c13
Author: Dusty Mabe <dusty@dustymabe.com>
Date:   Tue Jul 5 18:53:06 2022 -0400

    cosalib/aws: allow passing through --credentials-file to ore
    
    This will allow us to specify where the AWS_CONFIG_FILE is from
    the CLI without having to set it in the environment.

commit c4f8d50b0ef594d3f49e3d8f010c0412ced9ad7d
Author: Dusty Mabe <dusty@dustymabe.com>
Date:   Tue Jul 5 18:49:27 2022 -0400

    cmd-meta: support setting/getting fedora-coreos.parent-{commit,version}
    
    The command previously wouldn't allow us to get/set fedora-coreos.parent-commit
    and fedora-coreos.parent-version because it would try to traverse the
    `fedora-coreos.` path. Add an exception here similar to `coreos-assembler.`.
    
    Now we can do something like this in the pipeline:
    
    ```
    cosa meta \
        --set fedora-coreos.parent-commit=${parent_commit} \
        --set fedora-coreos.parent-version=${parent_version}

The command previously wouldn't allow us to get/set fedora-coreos.parent-commit
and fedora-coreos.parent-version because it would try to traverse the
`fedora-coreos.` path. Add an exception here similar to `coreos-assembler.`.

Now we can do something like this in the pipeline:

```
cosa meta \
    --set fedora-coreos.parent-commit=${parent_commit} \
    --set fedora-coreos.parent-version=${parent_version}
```
This will allow us to specify where the AWS_CONFIG_FILE is from
the CLI without having to set it in the environment.
This will allow us to specify the AWS_CONFIG_FILE from the command
line without having to set it in the environment.
@cgwalters
Copy link
Member

Looks like some legit CI failures in unit tests. Otherwise (somewhat superficial) LGTM.

This will allow us to specify the AWS_CONFIG_FILE from the command
line without having to set it in the environment.

This required reworking the cosalib/s3 library into an S3() class
so that we could call boto3.client('s3') after setting the
AWS_CONFIG_FILE environment variable in cmd-buildfetch. Otherwise
it would get called as soon as the library was imported and setting
the environment variable would have no effect (too late).
@dustymabe
Copy link
Member Author

Thanks @cgwalters - CI failures (python flake8) should be fixed now.

@cgwalters
Copy link
Member

One other note; I think some of the problems in this would have already been fixed by some progress on #2821 - I mean, right now we have to handle both Python libraries and Go libraries to access S3. If we got that down to only using the Go AWS libraries, things like this would have been easier/more avoidable.

@dustymabe dustymabe merged commit 9d55a20 into coreos:main Jul 6, 2022
@dustymabe dustymabe deleted the dusty-podman-remote-prep branch July 6, 2022 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants