Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] adding s3 local stack #1593

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

thetechnocrat-dev
Copy link
Contributor

I ran into some issues locally around S3 when trying to create questions or posts from the frontend connected to my backend running in dev-containers.

I attempted to add S3 LocalStack to fix the errors. And am still getting the error. I've searched for ENV variables, but can't seem to find what I'm missing. Any suggestions?

When starting the server I get this warning:

vscode ➜ /workspaces/researchhub-backend/src (master) $ ./manage.py runserver
Parameter validation failed:
Invalid bucket name "": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$"
Parameter validation failed:
Invalid bucket name "": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$" None None
Parameter validation failed:
Invalid bucket name "": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$"
Parameter validation failed:
Invalid bucket name "": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$" None None
Watching for file changes with StatReloader
Performing system checks...

When submitting a post request through the frontend I get this 500 error:

Internal Server Error: /api/researchhubpost/
Traceback (most recent call last):
  File "/home/vscode/.local/lib/python3.8/site-packages/asgiref/sync.py", line 534, in thread_handler
    raise exc_info[1]
  File "/home/vscode/.local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 43, in inner
    response = await get_response(request)
  File "/home/vscode/.local/lib/python3.8/site-packages/django/core/handlers/base.py", line 253, in _get_response_async
    response = await wrapped_callback(
  File "/home/vscode/.local/lib/python3.8/site-packages/asgiref/sync.py", line 479, in __call__
    ret: _R = await loop.run_in_executor(
  File "/home/vscode/.local/lib/python3.8/site-packages/asgiref/current_thread_executor.py", line 40, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/vscode/.local/lib/python3.8/site-packages/asgiref/sync.py", line 538, in thread_handler
    return func(*args, **kwargs)
  File "/home/vscode/.local/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 55, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/vscode/.local/lib/python3.8/site-packages/rest_framework/viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/vscode/.local/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/home/vscode/.local/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/vscode/.local/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/home/vscode/.local/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/workspaces/researchhub-backend/src/analytics/amplitude.py", line 145, in inner
    res = func(*args, **kwargs)
  File "/workspaces/researchhub-backend/src/researchhub_document/views/researchhub_post_views.py", line 75, in create
    return self.upsert_researchhub_posts(request)
  File "/workspaces/researchhub-backend/src/researchhub_document/views/researchhub_post_views.py", line 100, in upsert_researchhub_posts
    return self.create_researchhub_post(request)
  File "/workspaces/researchhub-backend/src/utils/siftscience.py", line 449, in inner
    res = func(*args, **kwargs)
  File "/workspaces/researchhub-backend/src/researchhub_document/views/researchhub_post_views.py", line 166, in create_researchhub_post
    rh_post.discussion_src.save(file_name, full_src_file)
  File "/home/vscode/.local/lib/python3.8/site-packages/django/db/models/fields/files.py", line 92, in save
    self.name = self.storage.save(name, content, max_length=self.field.max_length)
  File "/home/vscode/.local/lib/python3.8/site-packages/django/core/files/storage.py", line 55, in save
    name = self.get_available_name(name, max_length=max_length)
  File "/home/vscode/.local/lib/python3.8/site-packages/storages/backends/s3boto3.py", line 589, in get_available_name
    return super().get_available_name(name, max_length)
  File "/home/vscode/.local/lib/python3.8/site-packages/django/core/files/storage.py", line 95, in get_available_name
    while self.exists(name) or (max_length and len(name) > max_length):
  File "/home/vscode/.local/lib/python3.8/site-packages/storages/backends/s3boto3.py", line 461, in exists
    self.connection.meta.client.head_object(Bucket=self.bucket_name, Key=name)
  File "/home/vscode/.local/lib/python3.8/site-packages/storages/backends/s3boto3.py", line 342, in connection
    self._connections.connection = session.resource(
  File "/home/vscode/.local/lib/python3.8/site-packages/boto3/session.py", line 446, in resource
    client = self.client(
  File "/home/vscode/.local/lib/python3.8/site-packages/boto3/session.py", line 299, in client
    return self._session.create_client(
  File "/home/vscode/.local/lib/python3.8/site-packages/botocore/session.py", line 997, in create_client
    client = client_creator.create_client(
  File "/home/vscode/.local/lib/python3.8/site-packages/botocore/client.py", line 161, in create_client
    client_args = self._get_client_args(
  File "/home/vscode/.local/lib/python3.8/site-packages/botocore/client.py", line 520, in _get_client_args
    return args_creator.get_client_args(
  File "/home/vscode/.local/lib/python3.8/site-packages/botocore/args.py", line 137, in get_client_args
    endpoint = endpoint_creator.create_endpoint(
  File "/home/vscode/.local/lib/python3.8/site-packages/botocore/endpoint.py", line 402, in create_endpoint
    raise ValueError("Invalid endpoint: %s" % endpoint_url)
ValueError: Invalid endpoint: https://s3..amazonaws.com
HTTP POST /api/researchhubpost/? 500 [1.03, 127.0.0.1:39462]

@thetechnocrat-dev thetechnocrat-dev requested a review from a team as a code owner May 14, 2024 21:09
@thetechnocrat-dev thetechnocrat-dev marked this pull request as draft May 14, 2024 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant