Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #157: Create new SQL Source #224

Merged
merged 6 commits into from
Jun 5, 2024

Conversation

TNohaic
Copy link
Contributor

@TNohaic TNohaic commented Jun 4, 2024

Creation of new type of extension: SourceComputationExtension Creation of new SQL extension for SQL Sources

Creation of new type of extension: SourceComputationExtension
Creation of new SQL extension for SQL Sources
@TNohaic TNohaic linked an issue Jun 4, 2024 that may be closed by this pull request
Copy link
Member

@NassimBtk NassimBtk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @TNohaic! Minor updates here and there then we are good to go.

@Builder
@AllArgsConstructor
@NoArgsConstructor
public class SqlColumn {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SqlColumn needs to implement Serializable.
Add

private static final long serialVersionUID = 1L;

* @return A new instance of {@link SqlTable} with the same source, alias and columns.
*/
public SqlTable copy() {
return SqlTable.builder().source(source).alias(alias).columns(columns).build();
Copy link
Member

@NassimBtk NassimBtk Jun 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return SqlTable.builder().source(source).alias(alias).columns(new ArrayList<>(columns)).build(); 🤦‍♂️

	public SqlTable copy() {
		return SqlTable.builder().source(source).alias(alias).columns(columns.stream().map(SqlColumn::copy).collect(Collectors.toCollection(ArrayList::new))).build();
	}

stringBuilder.append(name);
stringBuilder.append(", type=");
stringBuilder.append(type);
stringBuilder.append(", column number=");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

	stringBuilder.append(", number=");

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's how I set it at first, but I have found that it was not explicit at all when I was lloking at the logs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a big deal, keep it as it is 😉.

@NoArgsConstructor
public class SqlTable implements Serializable {

private static final long serialVersionUID = 2338817623743864427L;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

private static final long serialVersionUID = 1L;

.forceSerialization(forceSerialization)
.computes(getComputes() != null ? new ArrayList<>(getComputes()) : null)
.executeForEachEntryOf(executeForEachEntryOf != null ? executeForEachEntryOf.copy() : null)
.tables(tables != null ? new ArrayList<>(tables) : null)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.tables(tables != null ? tables.stream().map(SqlTable::copy).collect(Collectors.toCollection(ArrayList::new)) : null)

for (final List<String> row : table) {
final List<String> rowValues = new ArrayList<>();
for (final SqlColumn sqlColumn : sqlColumns) {
final String separator = sqlColumn.getType().contains("VARCHAR") ? "'" : "";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting the value at the beginning is better String value = row.get(sqlColumn.getNumber() - 1);


@Override
public SourceTable processSource(Source source, String connectorId, TelemetryManager telemetryManager) {
final String hostname = telemetryManager.getHostConfiguration().getHostname();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

final String hostname = telemetryManager.getHostname();


SourceTable sourceTable = new SourceTable();

if (executeSqlQuery != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.executeQuery(sqlTables, query); shouldn't return null table please.

pom.xml Outdated
@@ -342,6 +343,11 @@
<artifactId>ssh</artifactId>
<version>1.0.01</version>
</dependency>
<dependency>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this dependency please , it doesn't exist.

<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>2.2.224</version>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the version it is managed in the main pom.

Correction after PR review
@@ -72,7 +74,7 @@ public class SqlTable implements Serializable {
* @return A new instance of {@link SqlTable} with the same source, alias and columns.
*/
public SqlTable copy() {
return SqlTable.builder().source(source).alias(alias).columns(columns).build();
return SqlTable.builder().source(source).alias(alias).columns(new ArrayList<>(columns)).build();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

	public SqlTable copy() {
		return SqlTable.builder().source(source).alias(alias).columns(columns.stream().map(SqlColumn::copy).collect(Collectors.toCollection(ArrayList::new))).build();
	}

More corrections after PR review
Add handling of null values
* Reverted untested changes.
NassimBtk
NassimBtk previously approved these changes Jun 5, 2024
@TNohaic TNohaic dismissed NassimBtk’s stale review June 5, 2024 16:42

The merge-base changed after approval.

@NassimBtk NassimBtk self-requested a review June 5, 2024 16:58
@NassimBtk NassimBtk merged commit cbfcb56 into main Jun 5, 2024
2 checks passed
@NassimBtk NassimBtk deleted the feature/issue-157-create-new-source-sqlquery branch June 5, 2024 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create new SQL Source
2 participants