Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add small fix and test to assert prepare_host hook is called with correct dirs #8585

Merged
merged 1 commit into from Jun 28, 2023

Conversation

whummer
Copy link
Member

@whummer whummer commented Jun 28, 2023

Add small fix and test to assert that prepare_host hooks are called with appropriate LocalStack directories layout. This is a follow-up from #8384 .

The previous PR #8384 adjusted the logic to start up LocalStack using the CLI when running inside a Docker container, for example to run localstack start in a Github Codespaces environment. This works well for the Community version, but unfortunately there was an oversight for the Pro version, where we're performing operations on the host via prepare_host hooks. In particular, the hook used to cache the key locally was failing with a PermissionError, as it was trying to access the wrong directory (using the container dirs layout, not the host dirs layout).

Traceback (most recent call last):
  File "/usr/local/python/3.10.4/lib/python3.10/site-packages/localstack_ext/bootstrap/licensing.py", line 64, in fetch_key
    C=json.loads(to_str(B.content))[_C];cache_key_locally(A,C)
  File "/usr/local/python/3.10.4/lib/python3.10/site-packages/localstack_ext/bootstrap/licensing.py", line 75, in cache_key_locally
    A=to_str(base64.b64encode(to_bytes(C)));E=get_key_cache();E.update({_E:int(B),_G:md5(api_key),_C:A});E.save()
  File "/usr/local/python/3.10.4/lib/python3.10/site-packages/localstack/utils/json.py", line 84, in save
    os.makedirs(os.path.dirname(self.path), exist_ok=True)
  File "/usr/local/python/3.10.4/lib/python3.10/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/local/python/3.10.4/lib/python3.10/os.py", line 225, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/var/lib/localstack'

This PR contains a small follow-up change in cmd_start(..) to ensure bootstrap.prepare_host(..) is called after we're performing the config.* adjustments introduced in #8384 . The change has been tested in Github Codespaces, which now allows to spin up a Pro container via the CLI if the API key is configured. A unit test is added to cover the functionality, with assertions about the "is in Docker" status and the accessibility of the dirs.cache directory.

Note: The PR also contains a few minor cleanups/refactorings (add type hints, f-strings, etc) along the way.

@whummer whummer added the semver: patch Non-breaking changes which can be included in patch releases label Jun 28, 2023
@@ -403,6 +404,9 @@ def cmd_start(docker: bool, host: bool, no_banner: bool, detached: bool) -> None
config.is_in_docker = False
config.dirs = config.init_directories()

# call hooks to prepare host (note that this call should stay below the config overrides above)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is effectively the main change in the PR - calling the hooks after we've adjusted the config values above.

)

# prepare config options
image_name = ls_service_details.get("image")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: removed unused variable (duplicate from line 295 above, and not used further below)

@@ -153,7 +153,7 @@ def _worker(*_args):

def is_command_available(cmd: str) -> bool:
try:
run("which %s" % cmd, print_error=False)
run(["which", cmd], print_error=False)
Copy link
Member Author

@whummer whummer Jun 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security: changing to the non-shell subprocess execution here, to prevent potential command injection.

@coveralls
Copy link

Coverage Status

coverage: 82.647% (-0.007%) from 82.654% when pulling a03fc9b on prepare-host-dirs into bee5f76 on master.

@github-actions
Copy link

LocalStack Community integration with Pro

       2 files         2 suites   1h 36m 25s ⏱️
2 166 tests 1 830 ✔️ 336 💤 0
2 167 runs  1 830 ✔️ 337 💤 0

Results for commit a03fc9b.

Copy link
Member

@thrau thrau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch. changes make sense to me!

@thrau thrau merged commit 3f88f59 into master Jun 28, 2023
32 of 35 checks passed
@thrau thrau deleted the prepare-host-dirs branch June 28, 2023 13:19
@dfangl dfangl added this to the 2.2 milestone Jul 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver: patch Non-breaking changes which can be included in patch releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants