Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(java): adding resource and reflection configurations for native image testing #809

Merged
merged 96 commits into from
May 31, 2022
Merged
Show file tree
Hide file tree
Changes from 75 commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
1f3feba
build: execute parallel integration tests
olavloite Apr 12, 2022
b8bc04a
fix: remove commented code
olavloite Apr 12, 2022
d4c2115
fix: only create relevant database
olavloite Apr 12, 2022
266f5ac
test: skip PG on emulator
olavloite Apr 12, 2022
6f8aff1
fix: skip table creation if dialect is not supported
olavloite Apr 12, 2022
ff95391
build: make sure db is created before tables
olavloite Apr 12, 2022
eae50a2
fix: ensure own instance on emulator
olavloite Apr 12, 2022
c976538
refactor: cleanup test setup
olavloite Apr 15, 2022
a8900ae
fix: use correct script file
olavloite Apr 15, 2022
596d59d
fix: metadata test cases
olavloite Apr 15, 2022
b1ded0c
fix: reduce fork count + remove error name
olavloite Apr 15, 2022
f0d821d
fix: type in sql file name
olavloite Apr 15, 2022
5822006
build: increase fork count to 8
olavloite Apr 15, 2022
ffb3e3f
fix: return database id as catalog name
olavloite Apr 20, 2022
a3eab4c
fix(java): testing native image support with parallel test execution
mpeddada1 Apr 21, 2022
7860362
initialize class at build time
mpeddada1 Apr 21, 2022
ebd05b9
add reflection config
mpeddada1 Apr 22, 2022
9f0921d
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 22, 2022
12740ba
add dependency
mpeddada1 Apr 22, 2022
d2f4099
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 22, 2022
b746473
address dependency problem
mpeddada1 Apr 22, 2022
3010940
register .sql files
mpeddada1 Apr 22, 2022
c8f51a4
add reflection configuration
mpeddada1 Apr 22, 2022
eb07ecb
Merge branch 'main' into troubleshoot-native-2
mpeddada1 Apr 22, 2022
f1d9aca
add reflection config for QueryOptions classes
mpeddada1 Apr 22, 2022
4c01edb
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 22, 2022
92f8045
fix compilation errors
mpeddada1 Apr 22, 2022
9b08905
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 22, 2022
c6e5fa7
run single test
mpeddada1 Apr 23, 2022
7721df9
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 23, 2022
7ce9ba3
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 23, 2022
2aaed1d
exclude from owlbot for testing
mpeddada1 Apr 23, 2022
7d00763
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 23, 2022
9ad18f9
exclude from owlbot
mpeddada1 Apr 23, 2022
cbbac93
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 23, 2022
5e16b56
add graalvm-native.cfg to owlbot exclusion
mpeddada1 Apr 23, 2022
d69d6b7
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 23, 2022
a16573e
add parameter back in
mpeddada1 Apr 23, 2022
a3370a2
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 23, 2022
17f46c9
add more reflection config
mpeddada1 Apr 23, 2022
ca9513e
modify owlbot config
mpeddada1 Apr 23, 2022
83ecbb8
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 23, 2022
94c4869
add resource config
mpeddada1 Apr 23, 2022
0d16095
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 23, 2022
1a01c1b
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 23, 2022
deeccef
fix param
mpeddada1 Apr 23, 2022
16a4a5b
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 23, 2022
efe210b
add resource and reflection config
mpeddada1 Apr 23, 2022
6b44c60
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 23, 2022
c6e50ad
register ConnectionImpl
mpeddada1 Apr 23, 2022
2a48809
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 23, 2022
833ed3a
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 23, 2022
12f8318
add debugging statements; register test classes for reflection
mpeddada1 Apr 24, 2022
16aa6cd
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 24, 2022
c565200
fix lint errors
mpeddada1 Apr 24, 2022
390ee30
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 24, 2022
b7f1647
add debugging print statements to QueryOptions
mpeddada1 Apr 24, 2022
9f26538
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 24, 2022
c766e9a
compilation error
mpeddada1 Apr 24, 2022
6d5a80b
include fixes for ITJdbcQueryOptionsTests and ITJdbcPreparedStatement…
mpeddada1 Apr 25, 2022
77d3db9
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 25, 2022
5268a1b
add graalvm sdk to ignoreUnusedDependency
mpeddada1 Apr 25, 2022
c03dbaa
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 25, 2022
9537a80
include only necessary changes
mpeddada1 Apr 26, 2022
58a0adc
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 26, 2022
17a9312
resolve dependency issue for graal-sdk
mpeddada1 Apr 26, 2022
4af00d3
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 26, 2022
0528c01
remove print statements
mpeddada1 Apr 26, 2022
90a25db
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Apr 26, 2022
207a66d
resolve dependendency issue
mpeddada1 Apr 27, 2022
5b77d0c
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 Apr 27, 2022
c7e0367
remove spaces in pom.xml
mpeddada1 Apr 27, 2022
7037d64
fix formatting in pom
mpeddada1 Apr 27, 2022
e17fb73
Merge branch 'main' of github.com:googleapis/java-spanner-jdbc into t…
mpeddada1 May 12, 2022
3ae43a7
initialize integrationTestEnv at build time
mpeddada1 May 12, 2022
971b520
Merge branch 'main' of github.com:googleapis/java-spanner-jdbc into t…
mpeddada1 May 25, 2022
b3c0751
include integration tests for native testing
mpeddada1 May 25, 2022
dfde319
use 22.1.0 for testing
mpeddada1 May 25, 2022
d8b42e3
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 25, 2022
f9d39ac
add configs from spanner
mpeddada1 May 25, 2022
6d0f8ca
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 May 25, 2022
f74ace0
add pg executor config
mpeddada1 May 25, 2022
6cc6ffa
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 25, 2022
83fe960
Merge branch 'main' of github.com:googleapis/java-spanner-jdbc into t…
mpeddada1 May 27, 2022
c72a337
bring in latest spanner
mpeddada1 May 27, 2022
7bdc7c8
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 May 27, 2022
d5a03bf
upgrade to 6.25.4
mpeddada1 May 27, 2022
cb480b6
remove sdk dependency
mpeddada1 May 27, 2022
d1bc3fa
temporary config
mpeddada1 May 27, 2022
4e1cdc4
verify with only one test
mpeddada1 May 27, 2022
3a35691
config for constructor
mpeddada1 May 27, 2022
bcce92c
add config
mpeddada1 May 27, 2022
b39f8b3
run all tests
mpeddada1 May 27, 2022
93592f7
Merge branch 'main' into troubleshoot-native-2
mpeddada1 May 31, 2022
4aed30e
remove reflection config
mpeddada1 May 31, 2022
c38c951
Merge branch 'troubleshoot-native-2' of github.com:googleapis/java-sp…
mpeddada1 May 31, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,15 @@
<artifactId>proto-google-cloud-spanner-v1</artifactId>
</dependency>

<!-- Native Image dependencies-->
<dependency>
<groupId>org.graalvm.nativeimage</groupId>
<artifactId>svm</artifactId>
<version>22.0.0.2</version>
<scope>provided</scope>
</dependency>
<!-- Native Image dependencies-->

<!-- Test dependencies -->
<dependency>
<groupId>com.google.cloud</groupId>
Expand Down Expand Up @@ -222,6 +231,14 @@
<configuration>
<exclude>com.google.cloud.spanner.IntegrationTest</exclude>
<reportNameSuffix>sponge_log</reportNameSuffix>
<systemPropertyVariables>
<spanner.testenv.config.class>
com.google.cloud.spanner.GceTestEnvConfig
</spanner.testenv.config.class>
<spanner.testenv.instance>
projects/gcloud-devel/instances/spanner-testing-east1
</spanner.testenv.instance>
</systemPropertyVariables>
</configuration>
</plugin>
<plugin>
Expand Down Expand Up @@ -311,6 +328,12 @@
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<configuration>
<ignoredUsedUndeclaredDependencies>
<ignoredUsedUndeclaredDependency>org.graalvm.sdk:graal-sdk</ignoredUsedUndeclaredDependency>
<ignoredUsedUndeclaredDependency>com.google.api.grpc:grpc-google-cloud-spanner-v1</ignoredUsedUndeclaredDependency>
<ignoredUsedUndeclaredDependency>com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1</ignoredUsedUndeclaredDependency>
<ignoredUsedUndeclaredDependency>com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1</ignoredUsedUndeclaredDependency>
</ignoredUsedUndeclaredDependencies>
<ignoredUnusedDeclaredDependencies>
<!-- TODO: Remove grpc-alts from ignored list once it has been removed from java-spanner -->
<ignoredDependency>io.grpc:grpc-alts</ignoredDependency>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
/*
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.google.cloud.spanner.jdbc.nativeimage;

import com.google.api.gax.nativeimage.NativeImageUtils;
import com.oracle.svm.core.annotate.AutomaticFeature;
import com.oracle.svm.core.configure.ResourcesRegistry;
import org.graalvm.nativeimage.ImageSingletons;
import org.graalvm.nativeimage.hosted.Feature;
import org.graalvm.nativeimage.impl.ConfigurationCondition;

/** Registers Spanner library classes for reflection. */
@AutomaticFeature
final class SpannerFeature implements Feature {

private static final String SPANNER_CLASS = "com.google.spanner.v1.SpannerGrpc";
private static final String SPANNER_TEST_CLASS = "com.google.cloud.spanner.GceTestEnvConfig";
private static final String MOCK_CLASS = "com.google.cloud.spanner.MockDatabaseAdminServiceImpl";
private static final String CLIENT_SIDE_IMPL_CLASS =
"com.google.cloud.spanner.connection.ClientSideStatementImpl";
private static final String CLIENT_SIDE_VALUE_CONVERTER =
"com.google.cloud.spanner.connection.ClientSideStatementValueConverters";
private static final String CONNECTION_IMPL =
"com.google.cloud.spanner.connection.ConnectionImpl";
private static final String CLIENT_SIDE_STATEMENTS =
"com.google.cloud.spanner.connection.ClientSideStatements";
private static final String CONNECTION_STATEMENT_EXECUTOR =
"com.google.cloud.spanner.connection.ConnectionStatementExecutor";
private static final String CLIENT_SIDE_STATEMENT_NO_PARAM_EXECUTOR =
"com.google.cloud.spanner.connection.ClientSideStatementNoParamExecutor";
private static final String CLIENT_SIDE_STATEMENT_SET_EXECUTOR =
"com.google.cloud.spanner.connection.ClientSideStatementSetExecutor";
private static final String ABSTRACT_STATEMENT_PARSER =
"com.google.cloud.spanner.connection.AbstractStatementParser";
private static final String STATEMENT_PARSER =
"com.google.cloud.spanner.connection.SpannerStatementParser";
private static final String POSTGRESQL_STATEMENT_PARSER =
"com.google.cloud.spanner.connection.PostgreSQLStatementParser";
private static final String DIALECT = "com.google.cloud.spanner.Dialect";
private static final String STATEMENT_RESULT =
"com.google.cloud.spanner.connection.StatementResult$ResultType";

@Override
public void beforeAnalysis(BeforeAnalysisAccess access) {
registerSpannerTestClasses(access);
if (access.findClassByName(CLIENT_SIDE_IMPL_CLASS) != null) {
NativeImageUtils.registerClassHierarchyForReflection(access, CLIENT_SIDE_IMPL_CLASS);
}
if (access.findClassByName(CLIENT_SIDE_STATEMENT_NO_PARAM_EXECUTOR) != null) {
NativeImageUtils.registerClassForReflection(access, CLIENT_SIDE_STATEMENT_NO_PARAM_EXECUTOR);
}
if (access.findClassByName(CLIENT_SIDE_STATEMENT_SET_EXECUTOR) != null) {
NativeImageUtils.registerClassForReflection(access, CLIENT_SIDE_STATEMENT_SET_EXECUTOR);
}
if (access.findClassByName(CLIENT_SIDE_VALUE_CONVERTER) != null) {
NativeImageUtils.registerClassHierarchyForReflection(access, CLIENT_SIDE_VALUE_CONVERTER);
}
if (access.findClassByName(CLIENT_SIDE_STATEMENTS) != null) {
NativeImageUtils.registerClassForReflection(access, CLIENT_SIDE_STATEMENTS);
}
if (access.findClassByName(CONNECTION_STATEMENT_EXECUTOR) != null) {
NativeImageUtils.registerClassForReflection(access, CONNECTION_STATEMENT_EXECUTOR);
}
if (access.findClassByName(CONNECTION_IMPL) != null) {
NativeImageUtils.registerClassForReflection(access, CONNECTION_IMPL);
}
if (access.findClassByName(ABSTRACT_STATEMENT_PARSER) != null) {
NativeImageUtils.registerClassHierarchyForReflection(access, ABSTRACT_STATEMENT_PARSER);
NativeImageUtils.registerClassForReflection(access, DIALECT);
}
if (access.findClassByName(STATEMENT_PARSER) != null) {
NativeImageUtils.registerConstructorsForReflection(access, STATEMENT_PARSER);
}
if (access.findClassByName(POSTGRESQL_STATEMENT_PARSER) != null) {
NativeImageUtils.registerConstructorsForReflection(access, POSTGRESQL_STATEMENT_PARSER);
}
if (access.findClassByName(STATEMENT_RESULT) != null) {
NativeImageUtils.registerClassForReflection(access, STATEMENT_RESULT);
}

Class<?> spannerClass = access.findClassByName(SPANNER_CLASS);
if (spannerClass != null) {
NativeImageUtils.registerClassHierarchyForReflection(
access, "com.google.spanner.admin.database.v1.Database");
NativeImageUtils.registerClassHierarchyForReflection(
access, "com.google.spanner.admin.instance.v1.Instance");
NativeImageUtils.registerClassForReflection(
access, "com.google.spanner.admin.database.v1.RestoreInfo");

// Resources
ResourcesRegistry resourcesRegistry = ImageSingletons.lookup(ResourcesRegistry.class);
resourcesRegistry.addResources(
ConfigurationCondition.alwaysTrue(),
"\\Qcom/google/cloud/spanner/connection/ClientSideStatements.json\\E");
resourcesRegistry.addResources(
ConfigurationCondition.alwaysTrue(),
"\\Qcom/google/cloud/spanner/connection/PG_ClientSideStatements.json\\E");
resourcesRegistry.addResources(
"\\Qcom/google/cloud/spanner/spi/v1/grpc-gcp-apiconfig.json\\E");
resourcesRegistry.addResources(
ConfigurationCondition.alwaysTrue(),
"\\Qcom/google/cloud/spanner/connection/ITSqlScriptTest_TestQueryOptions.sql\\E");
}
}

private void registerSpannerTestClasses(BeforeAnalysisAccess access) {
Class<?> spannerTestClass = access.findClassByName(SPANNER_TEST_CLASS);
if (spannerTestClass != null) {
NativeImageUtils.registerConstructorsForReflection(access, SPANNER_TEST_CLASS);
}
Class<?> mockClass = access.findClassByName(MOCK_CLASS);
if (mockClass != null) {
NativeImageUtils.registerClassForReflection(
access, "com.google.cloud.spanner.MockDatabaseAdminServiceImpl$MockBackup");
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Args = --initialize-at-build-time==com.google.cloud.spanner.IntegrationTestEnv,\
com.google.cloud.spanner.jdbc.it.JdbcIntegrationTestEnv,\
com.google.common.collect.RegularImmutableMap
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"resources":{
"includes":[
{"pattern":".*.sql"},
{"pattern":".*.json"},
{"pattern":".*.txt"}
]
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@
import com.google.cloud.spanner.testing.EmulatorSpannerHelper;
import com.google.common.base.Strings;
import com.google.common.io.BaseEncoding;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.StringReader;
import java.math.BigDecimal;
import java.sql.BatchUpdateException;
Expand All @@ -54,7 +54,6 @@
import java.util.Arrays;
import java.util.Calendar;
import java.util.List;
import java.util.Objects;
import java.util.Scanner;
import java.util.TimeZone;
import java.util.UUID;
Expand Down Expand Up @@ -1106,14 +1105,14 @@ private void assertDefaultParameterMetaData(ParameterMetaData pmd, int expectedP
}

private List<String> readValuesFromFile(String filename) {
File file = new File(Objects.requireNonNull(getClass().getResource(filename)).getFile());
StringBuilder builder = new StringBuilder();
try (Scanner scanner = new Scanner(file)) {
try (InputStream stream = ITJdbcPreparedStatementTest.class.getResourceAsStream(filename)) {
Scanner scanner = new Scanner(stream);
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
builder.append(line).append("\n");
}
} catch (FileNotFoundException e) {
} catch (IOException e) {
throw new RuntimeException(e);
}
String[] array = builder.toString().split(";");
Expand Down