Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Destination Oracle: use airbyte default column name + correct ci tests #5746

Merged
merged 3 commits into from
Aug 31, 2021
Merged

Destination Oracle: use airbyte default column name + correct ci tests #5746

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
36ae284
column name + ci tests
marcosmarxm Aug 30, 2021
b6f6c51
bump version
marcosmarxm Aug 30, 2021
a517519
Merge branch 'master' into marcosmarxm/oracle-dest-column-name
marcosmarxm Aug 30, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...esources/config/STANDARD_DESTINATION_DEFINITION/3986776d-2319-4de9-8af8-db14c0996e72.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@
"destinationDefinitionId": "3986776d-2319-4de9-8af8-db14c0996e72",
"name": "Oracle (Alpha)",
"dockerRepository": "airbyte/destination-oracle",
"dockerImageTag": "0.1.6",
"dockerImageTag": "0.1.7",
"documentationUrl": "https://docs.airbyte.io/integrations/destinations/oracle"
}
2 changes: 1 addition & 1 deletion airbyte-config/init/src/main/resources/seed/destination_definitions.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@
- destinationDefinitionId: 3986776d-2319-4de9-8af8-db14c0996e72
name: Oracle (Alpha)
dockerRepository: airbyte/destination-oracle
dockerImageTag: 0.1.6
dockerImageTag: 0.1.7
documentationUrl: https://docs.airbyte.io/integrations/destinations/oracle
- destinationDefinitionId: 9f760101-60ae-462f-9ee6-b7a9dafd454d
name: Kafka
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/destination-oracle/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@ COPY build/distributions/${APPLICATION}*.tar ${APPLICATION}.tar

RUN tar xf ${APPLICATION}.tar --strip-components=1

LABEL io.airbyte.version=0.1.6
LABEL io.airbyte.version=0.1.7
LABEL io.airbyte.name=airbyte/destination-oracle
6 changes: 3 additions & 3 deletions ...on-oracle/src/main/java/io/airbyte/integrations/destination/oracle/OracleDestination.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,9 @@ public class OracleDestination extends AbstractJdbcDestination implements Destin

public static final String DRIVER_CLASS = "oracle.jdbc.OracleDriver";

public static final String COLUMN_NAME_AB_ID = JavaBaseConstants.COLUMN_NAME_AB_ID.substring(1).toUpperCase();
public static final String COLUMN_NAME_DATA = JavaBaseConstants.COLUMN_NAME_DATA.substring(1).toUpperCase();
public static final String COLUMN_NAME_EMITTED_AT = JavaBaseConstants.COLUMN_NAME_EMITTED_AT.substring(1).toUpperCase();
public static final String COLUMN_NAME_AB_ID = "\"" + JavaBaseConstants.COLUMN_NAME_AB_ID.toUpperCase() + "\"";
public static final String COLUMN_NAME_DATA = "\"" + JavaBaseConstants.COLUMN_NAME_DATA.toUpperCase() + "\"";
public static final String COLUMN_NAME_EMITTED_AT = "\"" + JavaBaseConstants.COLUMN_NAME_EMITTED_AT.toUpperCase() + "\"";

public OracleDestination() {
super(DRIVER_CLASS, new OracleNameTransformer(), new OracleOperations("users"));
Expand Down
11 changes: 5 additions & 6 deletions ...st-integration/java/io/airbyte/integrations/destination/oracle/OracleIntegrationTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ protected JsonNode getConfig() {
protected List<JsonNode> retrieveRecords(TestDestinationEnv env, String streamName, String namespace, JsonNode streamSchema) throws Exception {
return retrieveRecordsFromTable(namingResolver.getRawTableName(streamName), namespace)
.stream()
.map(r -> Jsons.deserialize(r.get(OracleDestination.COLUMN_NAME_DATA).asText()))
.map(r -> Jsons.deserialize(r.get(OracleDestination.COLUMN_NAME_DATA.replace("\"", "")).asText()))
.collect(Collectors.toList());
}

Expand Down Expand Up @@ -111,10 +111,10 @@ protected List<String> resolveIdentifier(String identifier) {
}

private List<JsonNode> retrieveRecordsFromTable(String tableName, String schemaName) throws SQLException {
List<org.jooq.Record> result = getDatabase().query(ctx -> ctx
.fetch(String.format("SELECT * FROM %s.%s ORDER BY %s ASC", schemaName, tableName, OracleDestination.COLUMN_NAME_EMITTED_AT))
.stream()
.collect(Collectors.toList()));
List<org.jooq.Record> result = getDatabase()
.query(ctx -> ctx.fetch(String.format("SELECT * FROM %s.%s ORDER BY %s ASC", schemaName, tableName, OracleDestination.COLUMN_NAME_EMITTED_AT))
.stream()
.collect(Collectors.toList()));
return result
.stream()
.map(r -> r.formatJSON(JSON_FORMAT))
Expand Down Expand Up @@ -165,7 +165,6 @@ protected void tearDown(TestDestinationEnv testEnv) {

final Database database = getDatabase();
var tables = getAllTables(database);
tables.removeAll(allTables);
Comment on lines 167 to -168
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step is strange. We get all tables and remove all of them. The variable tables was empty at the end and we didn't remove data/tables in tearDown. The function getAllTables(database) get all tables where the user is the owner.

try {
for (String table : tables) {
database.query(ctx -> {
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/destinations/oracle.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ You should now have all the requirements needed to configure Oracle as a destina
## Changelog
| Version | Date | Pull Request | Subject |
| :--- | :--- | :--- | :--- |
| 0.1.7 | 2021-08-30 | [#5746](https://github.com/airbytehq/airbyte/pull/5746) | Use default column name for raw tables |
| 0.1.6 | 2021-08-23 | [#5542](https://github.com/airbytehq/airbyte/pull/5542) | Remove support for Oracle 11g to allow normalization |
| 0.1.5 | 2021-08-10 | [#5307](https://github.com/airbytehq/airbyte/pull/5307) | 🐛 Destination Oracle: Fix destination check for users without dba role |
| 0.1.4 | 2021-07-30 | [#5125](https://github.com/airbytehq/airbyte/pull/5125) | Enable `additionalPropertities` in spec.json |
Expand Down