aws s3 sync --exclude & --include not working

[segersniels]

I'm trying to sync a local directory to an S3 bucket with the following command:

aws s3 sync foo s3://foo/bar --region eu-west-1 --metadata-directive REPLACE --acl public-read --exclude "*" --include "*.js" --include "*.css" --content-encoding gzip --cache-control max-age=86400,public

This used to just work normally, only setting cache headers on specific objects. But recently it has started throwing the following error:

Unknown options: Gemfile,Gemfile.lock,Makefile,README.md,Web.js,android,app.json,assets,circle.yml,index.js,index.web.js ........

Basically trying to resolve the files found by the glob pattern * as command line arguments? Is this something on my end or is this a known issue?

Also worth noting I'm using this inside a Makefile in combination with aws-vault which is probably why this is acting all weird.

[jamesls]

Nothing's changed in that part of the code. I'm guessing it's related to the Makefile stuff as well. A quick way to check is to add --debug to the end of the command and look at the first few lines . It will show the exact commands the CLI received (after it's been processed by bash/whatever shell you use). Ideally you should see this:

 $ aws s3 sync foo s3://foo/bar --region eu-west-1 --metadata-directive REPLACE --acl public-read --exclude "*" --include "*.js" --include "*.css" --content-encoding gzip --cache-control max-age=86400,public --debug
2018-01-02 13:09:26,324 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.14.17 Python/2.7.14 Darwin/16.7.0 botocore/1.8.21
2018-01-02 13:09:26,325 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3', 'sync', 'foo', 's3://foo/bar', '--region', 'eu-west-1', '--metadata-directive', 'REPLACE', '--acl', 'public-read', '--exclude', '*', '--include', '*.js', '--include', '*.css', '--content-encoding', 'gzip', '--cache-control', 'max-age=86400,public', '--debug']

Specifically you want to make sure you see [..., '--include', '*.js', ...] in that list. If instead you see the list of .js files that suggests that your shell is expanding the wildcards before it makes it to the CLI process.

Let me know if you have any more questions.

[segersniels]

Sorry for my late answer, this is indeed the case. It entirely is an issue due to calling an aws-vault exec from within a subshell environment created by either a shell script or Makefile. It somehow forced the * wildcard to expand to command line arguments.

Moving the exec out of the GNU Make target itself and prefixing the target on the CLI instead fixed the issue.

[stealthycoin]

Glad that worked for you!