Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CTFd Google Cloud Storage support for uploads #1953

Closed
arxenix opened this issue Jul 20, 2021 · 4 comments
Closed

CTFd Google Cloud Storage support for uploads #1953

arxenix opened this issue Jul 20, 2021 · 4 comments

Comments

@arxenix
Copy link

arxenix commented Jul 20, 2021

It would be nice if CTFd had built-in support for gcloud storage as an upload backend.

I managed to get cloud storage working under it's S3 interop compatibility mode, but had to make a minor change in order to sign keys differently (changed the boto3 signature version to s3v4)

--- CTFd/utils/uploads/uploaders.py     2021-07-19 23:36:03.755843407 +0000
+++ uploaders.py        2021-07-19 23:47:11.148425129 +0000
@@ -1,10 +1,10 @@
 import os
 import posixpath
 import string
-from shutil import copyfileobj
+from pathlib import PurePath
+from shutil import copyfileobj, rmtree

 import boto3
-from botocore.client import Config
 from flask import current_app, redirect, send_file
 from flask.helpers import safe_join
 from werkzeug.utils import secure_filename
@@ -65,7 +65,8 @@

     def delete(self, filename):
         if os.path.exists(os.path.join(self.base_path, filename)):
-            os.unlink(os.path.join(self.base_path, filename))
+            file_path = PurePath(filename).parts[0]
+            rmtree(os.path.join(self.base_path, file_path))
             return True
         return False

@@ -88,7 +89,6 @@
             aws_access_key_id=access_key,
             aws_secret_access_key=secret_key,
             endpoint_url=endpoint,
-            config=Config(signature_version='s3v4'),
         )
         return client
@ColdHeat
Copy link
Member

Does #1882 resolve this for you perhaps? I've been waiting to see if I run into this or if it affects someone else before merging it.

@arxenix
Copy link
Author

arxenix commented Jul 20, 2021

unfortunately not, it seems like i need to explicitly need to set the signature version to s3v4 in order for it to work. Just v4 makes gcloud storage return 'SignatureDoesNotMatch'

@Matir
Copy link

Matir commented Nov 25, 2021

This seems to be caused by the use of the ResponseContentDisposition parameter, which is included in boto3's signature, but not in GCPs, per https://cloud.google.com/storage/docs/access-control/signed-urls-v2. I can't find good documentation on AWS v2 signatures, but it does seem they're deprecated, so need to be moved to v4 anyway.

@ColdHeat
Copy link
Member

ColdHeat commented Dec 3, 2021

I believe this is closed by #1882

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants