Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SOLR-15501: GCSBackupRepository operations without credentials - Branch 8 11 #2634

Closed
wants to merge 3 commits into from
Closed

SOLR-15501: GCSBackupRepository operations without credentials - Branch 8 11 #2634

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b385b71
SOLR-15501 made it possible to use GCSBackupRepository without creden…
jaceq Dec 20, 2021
e10f574
SOLR-15501 add Test for not setting GCS credentials
linuxswords Dec 21, 2021
49c96fc
SOLR-15501 update documentation in solr-ref-guid
linuxswords Dec 21, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
15 changes: 8 additions & 7 deletions solr/contrib/gcs-repository/src/java/org/apache/solr/gcs/GCSBackupRepository.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,14 +83,15 @@ protected Storage initStorage() {
return storage;

try {
if (credentialPath == null) {
throw new IllegalArgumentException(GCSConfigParser.missingCredentialErrorMsg());
if (credentialPath != null) {
log.info("Creating GCS client using credential at {}", credentialPath);
// 'GoogleCredentials.fromStream' closes the input stream, so we don't
GoogleCredentials credential = GoogleCredentials.fromStream(new FileInputStream(credentialPath));
storageOptionsBuilder.setCredentials(credential);
} else {
// nowarn compile time string concatenation
log.info(GCSConfigParser.missingCredentialMsg()); //nowarn
}

log.info("Creating GCS client using credential at {}", credentialPath);
// 'GoogleCredentials.fromStream' closes the input stream, so we don't
GoogleCredentials credential = GoogleCredentials.fromStream(new FileInputStream(credentialPath));
storageOptionsBuilder.setCredentials(credential);
storage = storageOptionsBuilder.build().getService();
} catch (IOException e) {
throw new IllegalStateException(e);
Expand Down
12 changes: 7 additions & 5 deletions solr/contrib/gcs-repository/src/java/org/apache/solr/gcs/GCSConfigParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@
* Parses configuration for {@link GCSBackupRepository} from NamedList and environment variables
*/
public class GCSConfigParser {
private static final String GCS_BUCKET_ENV_VAR_NAME = "GCS_BUCKET";
private static final String GCS_CREDENTIAL_ENV_VAR_NAME = "GCS_CREDENTIAL_PATH";
protected static final String GCS_BUCKET_ENV_VAR_NAME = "GCS_BUCKET";
protected static final String GCS_CREDENTIAL_ENV_VAR_NAME = "GCS_CREDENTIAL_PATH";

private static final String GCS_BUCKET_PARAM_NAME = "gcsBucket";
private static final String GCS_CREDENTIAL_PARAM_NAME = "gcsCredentialPath";
Expand Down Expand Up @@ -93,9 +93,11 @@ private String parseCredentialPath(NamedList<Object> repoConfig, Map<String, Str
return envVars.get(GCS_CREDENTIAL_ENV_VAR_NAME);
}

public static String missingCredentialErrorMsg() {
return "GCSBackupRepository requires a credential for GCS communication, but none was provided. Please specify a " +
"path to this GCS credential by adding a '" + GCS_CREDENTIAL_PARAM_NAME + "' property to the repository " +
public static String missingCredentialMsg() {
return "GCSBackupRepository credential path is missing. GCSBackupRepository will only work within GCP when role " +
"based access is configured for backup bucket." +
"If you'd like to use credentials, set path to this GCS credential by adding a '" +
GCS_CREDENTIAL_PARAM_NAME + "' property to the repository " +
"definition in your solrconfig, or by setting the path value in an env-var named '" +
GCS_CREDENTIAL_ENV_VAR_NAME + "'";
}
Expand Down
14 changes: 14 additions & 0 deletions solr/contrib/gcs-repository/src/test/org/apache/solr/gcs/GCSBackupRepositoryTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,4 +68,18 @@ protected BackupRepository getRepository() {
protected URI getBaseUri() throws URISyntaxException {
return new URI("tmp");
}

@Test
public void testInitStoreDoesNotFailWithMissingCredentials()
{
Map<String, String> config = new HashMap<>();
config.put(GCS_BUCKET_ENV_VAR_NAME, "TestBucketName");
// explicitely setting credential name to null; will work inside google-cloud project
config.put(GCS_CREDENTIAL_ENV_VAR_NAME, null);
config.put(BACKUP_LOCATION, "/testPath");

BackupRepository gcsBackupRepository = getRepository();

gcsBackupRepository.init(new NamedList<>(config));
}
}
2 changes: 1 addition & 1 deletion solr/solr-ref-guide/src/making-and-restoring-backups.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,7 @@ If both values are absent, the value `solrBackupsBucket` will be used as a defau
`gcsCredentialPath`::
A path on the local filesystem (accessible by Solr) to a https://cloud.google.com/iam/docs/creating-managing-service-account-keys[Google Cloud service account key] file.
If not specified, GCSBackupRepository will use the value of the `GCS_CREDENTIAL_PATH` environment variable.
If both values are absent, an error will be thrown as GCS requires credentials for most usage.
If both values are absent, no error will be thrown as running solr in google cloud will handle authentication/authorization internally.

`location`::
A valid "directory" path in the given GCS bucket to us for backup strage and retrieval.
Expand Down