Skip to content

[PECO-1080] Added version check that throws on low protocol version #181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 20, 2023
Merged

[PECO-1080] Added version check that throws on low protocol version #181

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 20 additions & 7 deletions lib/DBSQLSession.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@ import * as fs from 'fs';
import * as path from 'path';
import { stringify, NIL, parse } from 'uuid';
import fetch, { HeadersInit } from 'node-fetch';
import { Thrift } from 'thrift';
import {
TSessionHandle,
TStatus,
TOperationHandle,
TSparkDirectResults,
TSparkArrowTypes,
TSparkParameter,
TProtocolVersion,
} from '../thrift/TCLIService_types';
import HiveDriver from './hive/HiveDriver';
import { Int64 } from './hive/Types';
Expand Down Expand Up @@ -81,17 +83,28 @@ function getArrowOptions(): {
}

function getQueryParameters(
sessionHandle: TSessionHandle,
namedParameters?: Record<string, DBSQLParameter | DBSQLParameterValue>,
): Array<TSparkParameter> {
const result: Array<TSparkParameter> = [];

if (namedParameters !== undefined) {
for (const name of Object.keys(namedParameters)) {
const value = namedParameters[name];
const param = value instanceof DBSQLParameter ? value : new DBSQLParameter({ value });
const sparkParam = param.toSparkParameter();
sparkParam.name = name;
result.push(sparkParam);
if (
sessionHandle?.serverProtocolVersion &&
sessionHandle.serverProtocolVersion >= TProtocolVersion.SPARK_CLI_SERVICE_PROTOCOL_V8
Copy link
Contributor

@kravets-levko kravets-levko Sep 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, one more thing I juts realized - when we merged parameters PR, the current protocol version at that moment was SPARK_CLI_SERVICE_PROTOCOL_V6, and the feature worked. Maybe we can be less strict here?

Copy link
Contributor Author

@nithinkdb nithinkdb Sep 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, there were no server side checks for protocol, which is why the protocol didn't matter before. But Yunbo has said that we shouldn't enable parameterized queries on anything past V8, and we've actually prevented parameters from being passed on the server side unless the protocol version is V8.

) {
for (const name of Object.keys(namedParameters)) {
const value = namedParameters[name];
const param = value instanceof DBSQLParameter ? value : new DBSQLParameter({ value });
const sparkParam = param.toSparkParameter();
sparkParam.name = name;
result.push(sparkParam);
}
} else {
throw new Thrift.TProtocolException(
Thrift.TProtocolExceptionType.BAD_VERSION,
'Server version does not support parameterized queries',
);
}
}

Expand Down Expand Up @@ -164,7 +177,7 @@ export default class DBSQLSession implements IDBSQLSession {
...getDirectResultsOptions(options.maxRows),
...getArrowOptions(),
canDownloadResult: options.useCloudFetch ?? globalConfig.useCloudFetch,
parameters: getQueryParameters(options.namedParameters),
parameters: getQueryParameters(this.sessionHandle, options.namedParameters),
});
const response = await this.handleResponse(operationPromise);
const operation = this.createOperation(response);
Expand Down