Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Encrypted blob store repository - take I #46170

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
1b648a7
POC
albertzaharovits Aug 20, 2019
e48c568
EncryptedRepository lifecycle
albertzaharovits Aug 20, 2019
61e4f2a
encryptionMetadataBlobPath
albertzaharovits Aug 20, 2019
5b7f5ea
Almost...
albertzaharovits Aug 21, 2019
23bcd23
Done???
albertzaharovits Aug 22, 2019
9e50384
Merge branch 'master' into encrypted-repo-poc
albertzaharovits Aug 22, 2019
f7ac3ed
Nit
albertzaharovits Aug 22, 2019
d892c2c
WORKS!
albertzaharovits Aug 23, 2019
5f8d77b
Merge branch 'master' into encrypted-repo-poc
albertzaharovits Aug 26, 2019
ad6f14a
Merge branch 'master' into encrypted-repo-poc
albertzaharovits Aug 26, 2019
f1a44de
Chunk size
albertzaharovits Aug 26, 2019
7b3eb4d
SunJCE mrrrr
albertzaharovits Aug 29, 2019
a54513c
Merge branch 'master' into encrypted-repo-poc
albertzaharovits Sep 1, 2019
43087e5
Always failIfExists for encryption metadata
albertzaharovits Sep 2, 2019
c160245
Parameterize for provider and chunk size
albertzaharovits Sep 2, 2019
85b1803
compile oversight
albertzaharovits Sep 2, 2019
7345c9b
License
albertzaharovits Sep 2, 2019
5fd4e61
Adjust sizes
albertzaharovits Sep 2, 2019
24378fc
Refactoring in a new plugin WIP
albertzaharovits Sep 3, 2019
c1649c8
Works!
albertzaharovits Sep 4, 2019
411f5da
Merge branch 'master' into encrypted-repo-poc
albertzaharovits Sep 12, 2019
ab8d6c6
Changes to move encrypted snapshots code to x-pack module
Sep 6, 2019
5e75538
Merge branch 'encrypted-repo-poc' of github.com:albertzaharovits/elas…
albertzaharovits Oct 3, 2019
de4aeb9
Merge branch 'master' into encrypted-repo-poc
albertzaharovits Oct 3, 2019
69fc7e5
FIPS libs
albertzaharovits Oct 7, 2019
b483b57
Straight GCM but with the bc-fips lib
albertzaharovits Oct 9, 2019
8345c3a
Merge branch 'master' into encrypted-repo-poc
albertzaharovits Oct 9, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ protected AzureBlobStore createBlobStore() {
}

@Override
protected ByteSizeValue chunkSize() {
public ByteSizeValue chunkSize() {
return chunkSize;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ protected GoogleCloudStorageBlobStore createBlobStore() {
}

@Override
protected ByteSizeValue chunkSize() {
public ByteSizeValue chunkSize() {
return chunkSize;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,7 @@ protected HdfsBlobStore createBlobStore() {
}

@Override
protected ByteSizeValue chunkSize() {
public ByteSizeValue chunkSize() {
return chunkSize;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ protected BlobStore getBlobStore() {
}

@Override
protected ByteSizeValue chunkSize() {
public ByteSizeValue chunkSize() {
return chunkSize;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,8 @@ public abstract class BlobStoreRepository extends AbstractLifecycleComponent imp

protected final ChecksumBlobStoreFormat<SnapshotInfo> snapshotFormat;

private final NamedXContentRegistry namedXContentRegistry;

private final boolean readOnly;

private final ChecksumBlobStoreFormat<BlobStoreIndexShardSnapshot> indexShardSnapshotFormat;
Expand All @@ -194,6 +196,10 @@ public abstract class BlobStoreRepository extends AbstractLifecycleComponent imp

private final BlobPath basePath;

protected BlobStoreRepository(BlobStoreRepository other) {
this(other.metadata, other.namedXContentRegistry, other.threadPool, other.basePath);
}

/**
* Constructs new BlobStoreRepository
* @param metadata The metadata for this repository including name and settings
Expand All @@ -211,6 +217,7 @@ protected BlobStoreRepository(
restoreRateLimiter = getRateLimiter(metadata.settings(), "max_restore_bytes_per_sec", new ByteSizeValue(40, ByteSizeUnit.MB));
readOnly = metadata.settings().getAsBoolean("readonly", false);
this.basePath = basePath;
this.namedXContentRegistry = namedXContentRegistry;

indexShardSnapshotFormat = new ChecksumBlobStoreFormat<>(SNAPSHOT_CODEC, SNAPSHOT_NAME_FORMAT,
BlobStoreIndexShardSnapshot::fromXContent, namedXContentRegistry, compress);
Expand Down Expand Up @@ -343,7 +350,7 @@ protected final boolean isCompress() {
*
* @return chunk size
*/
protected ByteSizeValue chunkSize() {
public ByteSizeValue chunkSize() {
return null;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ protected BlobStore createBlobStore() throws Exception {
}

@Override
protected ByteSizeValue chunkSize() {
public ByteSizeValue chunkSize() {
return chunkSize;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,6 @@
import java.time.Clock;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Optional;
Expand Down Expand Up @@ -313,7 +312,7 @@ public static Path resolveConfigFile(Environment env, String name) {
@Override
public Map<String, Repository.Factory> getRepositories(Environment env, NamedXContentRegistry namedXContentRegistry,
ThreadPool threadPool) {
return Collections.singletonMap("source", SourceOnlySnapshotRepository.newRepositoryFactory());
return Map.of("source", SourceOnlySnapshotRepository.newRepositoryFactory());
}

@Override
Expand Down
16 changes: 16 additions & 0 deletions x-pack/plugin/repository-encrypted/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
evaluationDependsOn(xpackModule('core'))

apply plugin: 'elasticsearch.esplugin'
esplugin {
name 'repository-encrypted'
description 'Elasticsearch Expanded Pack Plugin - client-side encrypted repositories.'
classname 'org.elasticsearch.repositories.encrypted.EncryptedRepositoryPlugin'
extendedPlugins = ['x-pack-core']
}

dependencies {
compile "org.bouncycastle:bc-fips:1.0.1"
compile "org.bouncycastle:bcpkix-fips:1.0.3"
}

integTest.enabled = false
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ed8dd3144761eaa33b9c56f5e2bef85f1b731d6f
12 changes: 12 additions & 0 deletions x-pack/plugin/repository-encrypted/licenses/bc-fips-LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Copyright (c) 2000 - 2019 The Legion of the Bouncy Castle Inc. (https://www.bouncycastle.org)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the
following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Empty file.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
33c47b105777c9dcc8a08188186bd35401366bd1
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Copyright (c) 2000 - 2019 The Legion of the Bouncy Castle Inc. (https://www.bouncycastle.org)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the
following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Empty file.
Loading