Add `OBJECTSTORE_S3_SSE_C_KEY` environment variable to use S3 SSE-C encryption (server side encryption) #2098

jessebot · 2023-11-07T14:08:24Z

Thanks to everyone who maintains this repo!

I recently learned I could pass in env vars for using S3 as the primary object store, but I was missing the encryption key.

This PR would add an environment variable called OBJECTSTORE_S3_SSE_C_KEY to the docs and if set, it will be used to provide the sse_c_key argument to the $CONFIG array in s3.config.php. If the user doesn't provide OBJECTSTORE_S3_SSE_C_KEY, we don't set ss3_c_key, because it has no default.

It's mentioned in the docs here:

Nextcloud supports server side encryption, also known as SSE-C, with compatible S3 bucket provider. The encryption and decryption happens on the S3 bucket side with a key provided by the Nextcloud server.

The key can be specified with the sse_c_key parameter which needs to be provided as a base64 encoded string with a maximum length of 32 bytes. A random key could be generated using the the following command:

openssl rand 32 | base64

…FIG array Signed-off-by: JesseBot <jessebot@linux.com>

…README.md Signed-off-by: JesseBot <jessebot@linux.com>

joshtrichards · 2023-11-08T14:16:09Z

Hi @jessebot - Thanks for including the matching README update. :-)

This variable should get _FILE (Docker secrets) support too if we're going to do it. Feel up for adding it? Can be modeled after the others at the same spot in the code.

jessebot · 2023-11-08T15:14:18Z

README.md

@@ -180,6 +180,7 @@ To use an external S3 compatible object store as primary storage, set the follow
 - `OBJECTSTORE_S3_LEGACYAUTH` (default: `false`): Not required for AWS S3
 - `OBJECTSTORE_S3_OBJECT_PREFIX` (default: `urn:oid:`): Prefix to prepend to the fileid
 - `OBJECTSTORE_S3_AUTOCREATE` (default: `true`): Create the container if it does not exist
+- `OBJECTSTORE_S3_SSE_C_KEY`: Server side encryption key - must be a base64 encoded string with a maximum length of 32 bytes


Suggested change

- `OBJECTSTORE_S3_SSE_C_KEY`: Server side encryption key - must be a base64 encoded string with a maximum length of 32 bytes

- `OBJECTSTORE_S3_SSE_C_KEY`: _Optional_ Server side encryption key - must be a base64 encoded string with a maximum length of 32 bytes

We might want to include the word optional, so users know it is not required.

The existing wording in the README is (not set by default) - not _Optional_. Could you please add it to your PR?

jessebot · 2023-11-08T15:15:06Z

Hi @jessebot - Thanks for including the matching README update. :-)

This variable should get _FILE (Docker secrets) support too if we're going to do it. Feel up for adding it? Can be modeled after the others at the same spot in the code.

No problem :) I can take a look the docker secrets now

Signed-off-by: jessebot <jessebot@linux.com>

jessebot · 2023-11-08T16:03:44Z

@joshtrichards I've added the OBJECTSTORE_S3_SSE_C_KEY_FILE env var support as well. Let me know if I can help with anything else :)

jessebot · 2023-11-13T12:54:40Z

do these jobs normally pass? @joshtrichards They look like they're failing on not having the hostname set. I took a quick peak at the github action workflows for these and couldn't find where we set any sort of env vars. Happy to update them if you can point me in the right direction.

jessebot · 2023-12-15T08:25:26Z

please let me know if there's anything I can do to help more this forward.

pathob

Hi @jessebot , I'm not a maintainer here, but highly interested in this change, so I would like to help moving it forward. I've added two suggestions that might help get this merged. It would be great if you could make these two small adjustments. Thank you!

pathob · 2024-01-07T14:30:01Z

.config/s3.config.php

+  if (getenv('OBJECTSTORE_S3_SSE_C_KEY_FILE') && file_exists(getenv('OBJECTSTORE_S3_SSE_C_KEY_FILE'))) {
+    $CONFIG['objectstore']['arguments']['sse_c_key'] = trim(file_get_contents(getenv('OBJECTSTORE_S3_SSE_C_KEY_FILE'));
+  } elseif (getenv('OBJECTSTORE_S3_SSE_C_KEY')) {
+    $CONFIG['objectstore']['arguments']['sse_c_key'] = getenv('OBJECTSTORE_S3_SSE_C_KEY');
+  }
+


IMHO this block should go to the end of the file since OBJECTSTORE_S3_KEY_FILE and OBJECTSTORE_S3_SECRET_FILE have much more relevance than this.

pathob · 2024-01-07T14:31:08Z

README.md

@@ -180,6 +180,7 @@ To use an external S3 compatible object store as primary storage, set the follow
 - `OBJECTSTORE_S3_LEGACYAUTH` (default: `false`): Not required for AWS S3
 - `OBJECTSTORE_S3_OBJECT_PREFIX` (default: `urn:oid:`): Prefix to prepend to the fileid
 - `OBJECTSTORE_S3_AUTOCREATE` (default: `true`): Create the container if it does not exist
+- `OBJECTSTORE_S3_SSE_C_KEY`: Server side encryption key - must be a base64 encoded string with a maximum length of 32 bytes


The existing wording in the README is (not set by default) - not _Optional_. Could you please add it to your PR?

pathob · 2024-01-08T10:10:32Z

.config/s3.config.php

@@ -23,6 +23,12 @@
    )
  );

+  if (getenv('OBJECTSTORE_S3_SSE_C_KEY_FILE') && file_exists(getenv('OBJECTSTORE_S3_SSE_C_KEY_FILE'))) {
+    $CONFIG['objectstore']['arguments']['sse_c_key'] = trim(file_get_contents(getenv('OBJECTSTORE_S3_SSE_C_KEY_FILE'));


I've tried to run it... there is a third closing bracket ) missing at the end

J0WI · 2024-01-25T23:33:21Z

closing in favour of #2151

jessebot · 2024-04-30T20:13:18Z

Hi @jessebot , I'm not a maintainer here, but highly interested in this change, so I would like to help moving it forward. I've added two suggestions that might help get this merged. It would be great if you could make these two small adjustments. Thank you!

Sorry for missing this, but thank you for carrying the torch to the finish line! 🙏 Glad to see the feature implemented :)

jessebot added 2 commits November 7, 2023 14:50

Update s3.config.php - add sse_c_key to objectstore arguments in $CON…

7ea6eac

…FIG array Signed-off-by: JesseBot <jessebot@linux.com>

add OBJECTSTORE_S#_SSE_C_KEY to possible S3 environment variables in …

cffa34c

…README.md Signed-off-by: JesseBot <jessebot@linux.com>

joshtrichards added enhancement feature: auto config (environment variables) Auto configuring via environment variables feature: auto config (Docker secrets) labels Nov 8, 2023

jessebot commented Nov 8, 2023

View reviewed changes

also support getting S3 SSE C Key from KEY_FILE

c9f1ba1

Signed-off-by: jessebot <jessebot@linux.com>

jessebot force-pushed the patch-1 branch from 68a5af9 to c9f1ba1 Compare November 8, 2023 15:44

jessebot mentioned this pull request Nov 8, 2023

Add default S3 object storage config from nextcloud/docker nextcloud/helm#464

Open

3 tasks

pathob reviewed Jan 7, 2024

View reviewed changes

pathob suggested changes Jan 8, 2024

View reviewed changes

joshtrichards added the 3. to review label Jan 8, 2024

pathob mentioned this pull request Jan 25, 2024

Allow setting AWS S3 SSE-C key #2151

Merged

J0WI closed this Jan 25, 2024

jessebot deleted the patch-1 branch April 30, 2024 20:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `OBJECTSTORE_S3_SSE_C_KEY` environment variable to use S3 SSE-C encryption (server side encryption) #2098

Add `OBJECTSTORE_S3_SSE_C_KEY` environment variable to use S3 SSE-C encryption (server side encryption) #2098

jessebot commented Nov 7, 2023

joshtrichards commented Nov 8, 2023

jessebot Nov 8, 2023 •

edited

pathob Jan 7, 2024

jessebot commented Nov 8, 2023

jessebot commented Nov 8, 2023

jessebot commented Nov 13, 2023

jessebot commented Dec 15, 2023

pathob left a comment

pathob Jan 7, 2024

pathob Jan 7, 2024

pathob Jan 8, 2024

J0WI commented Jan 25, 2024

jessebot commented Apr 30, 2024

	- `OBJECTSTORE_S3_SSE_C_KEY`: Server side encryption key - must be a base64 encoded string with a maximum length of 32 bytes
	- `OBJECTSTORE_S3_SSE_C_KEY`: _Optional_ Server side encryption key - must be a base64 encoded string with a maximum length of 32 bytes

Add OBJECTSTORE_S3_SSE_C_KEY environment variable to use S3 SSE-C encryption (server side encryption) #2098

Add OBJECTSTORE_S3_SSE_C_KEY environment variable to use S3 SSE-C encryption (server side encryption) #2098

Conversation

jessebot commented Nov 7, 2023

joshtrichards commented Nov 8, 2023

jessebot Nov 8, 2023 • edited

Choose a reason for hiding this comment

pathob Jan 7, 2024

Choose a reason for hiding this comment

jessebot commented Nov 8, 2023

jessebot commented Nov 8, 2023

jessebot commented Nov 13, 2023

jessebot commented Dec 15, 2023

pathob left a comment

Choose a reason for hiding this comment

pathob Jan 7, 2024

Choose a reason for hiding this comment

pathob Jan 7, 2024

Choose a reason for hiding this comment

pathob Jan 8, 2024

Choose a reason for hiding this comment

J0WI commented Jan 25, 2024

jessebot commented Apr 30, 2024

Add `OBJECTSTORE_S3_SSE_C_KEY` environment variable to use S3 SSE-C encryption (server side encryption) #2098

Add `OBJECTSTORE_S3_SSE_C_KEY` environment variable to use S3 SSE-C encryption (server side encryption) #2098

jessebot Nov 8, 2023 •

edited