Environment variables in config files aren't being expanded

[juneb]

aws-encryption-sdk-cli/1.0.2.5 aws-encryption-sdk/1.3.2
Windows 10.0.14393.1884
Windows PowerShell 5.1.14393.1884

When a parameter value in a config file includes an environment variable (e.g. $home), the CLI does not expand the variable.

(py) PS C:\ps-test> cat .\conf\home.conf
--encrypt
--input .\TestCLI\Hello.txt
--output .\TestEnc\ChildDir
--metadata-output $home\Documents\metadata.txt   #<--- $home
@conf\cmk.conf


(py) PS C:\ps-test> aws-crypto `@conf\encrypt.conf
2017-11-16 18:27:37,588 - MainThread - aws_encryption_sdk_cli - WARNING - Overwriting existing target file because no ac
tion was specified otherwise: .\TestEnc\ChildDir\Hello.txt.encrypted
--- Logging error ---
Traceback (most recent call last):
  File "c:\py\lib\site-packages\aws_encryption_sdk_cli\__init__.py", line 266, in cli
    process_cli_request(stream_args, args)
  File "c:\py\lib\site-packages\aws_encryption_sdk_cli\__init__.py", line 209, in process_cli_request
    destination=_destination
  File "c:\py\lib\site-packages\aws_encryption_sdk_cli\internal\io_handling.py", line 331, in process_single_file
    destination=destination
  File "c:\py\lib\site-packages\aws_encryption_sdk_cli\internal\io_handling.py", line 261, in process_single_operation
    destination_writer=destination_writer
  File "c:\py\lib\site-packages\aws_encryption_sdk_cli\internal\io_handling.py", line 197, in _single_io_write
    with aws_encryption_sdk.stream(source=_source, **stream_args) as handler, self.metadata_writer as metadata:
  File "c:\py\lib\site-packages\aws_encryption_sdk_cli\internal\metadata.py", line 99, in __enter__
    self.open()
  File "c:\py\lib\site-packages\aws_encryption_sdk_cli\internal\metadata.py", line 94, in open
    self._output_stream = open(self.output_file, self._output_mode)
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\ps-test\\$home\\Documents\\metadata.txt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\junebl\AppData\Local\Programs\Python\Python36-32\Lib\logging\__init__.py", line 992, in emit
    msg = self.format(record)
  File "C:\Users\junebl\AppData\Local\Programs\Python\Python36-32\Lib\logging\__init__.py", line 838, in format
    return fmt.format(record)
  File "c:\py\lib\site-packages\aws_encryption_sdk_cli\internal\logging_utils.py", line 131, in format
    _record = self.__redact_record(record)
  File "c:\py\lib\site-packages\aws_encryption_sdk_cli\internal\logging_utils.py", line 117, in __redact_record
    _record = copy.deepcopy(record)
  File "c:\py\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "c:\py\lib\copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "c:\py\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "c:\py\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "c:\py\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "c:\py\lib\copy.py", line 220, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "c:\py\lib\copy.py", line 220, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "c:\py\lib\copy.py", line 169, in deepcopy
    rv = reductor(4)
TypeError: can't pickle traceback objects
Call stack:
  File "C:\Users\junebl\AppData\Local\Programs\Python\Python36-32\Lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\junebl\AppData\Local\Programs\Python\Python36-32\Lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\py\Scripts\aws-crypto.exe\__main__.py", line 9, in <module>
    sys.exit(cli())
  File "c:\py\lib\site-packages\aws_encryption_sdk_cli\__init__.py", line 273, in cli
    _LOGGER.exception(message)
Message: 'Encountered unexpected FileNotFoundError: increase verbosity to see details'
Arguments: ()
Encountered unexpected FileNotFoundError: increase verbosity to see details

[mattsb42-aws]

This is ultimately caused by our config file line parser not expanding environment variables (or user for that matter). Expanding works outside of the config file because the shell is silently doing that transformation before passing the path to us.

Solution

Expand environment variables and user shorthand in config parser.