Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Athena driver: Typescript, stream, unload, integration & e2e tests #4039

Merged
merged 97 commits into from
Feb 9, 2022

Conversation

cristipp
Copy link
Contributor

@cristipp cristipp commented Feb 4, 2022

Check List

  • Tests has been run in packages where changes made if available
  • Linter has been run for changed code
  • Tests for the changes have been added if not covered yet
  • Docs have been added / updated if required

Issue Reference this PR resolves

N/A

Description of Changes Made (if issue reference is not provided)

  • Ports AthenaDriver to idiomatic Typescript.
  • Adds batching (AthenaDriver.stream) support.
  • Adds export (AthenaDriver.unload) support.
  • Adds integration tests for simple/stream/unload modes, for AthenaDriver and BigQueryDriver.
    • Currently not integrated with github presubmit queue, need to sort out service accounts & secrets.
  • Adds e2e tests for simple mode, for AthenaDriver and BigQueryDriver.
  • Adds CubeStore support for importing tables with no headers, CREATE TABLE ... WITH(skip_header = 'true')

@codecov
Copy link

codecov bot commented Feb 4, 2022

Codecov Report

Merging #4039 (bd766e6) into master (75545e8) will increase coverage by 0.07%.
The diff coverage is 75.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4039      +/-   ##
==========================================
+ Coverage   57.98%   58.06%   +0.07%     
==========================================
  Files         127      127              
  Lines       10659    10684      +25     
  Branches     2517     2520       +3     
==========================================
+ Hits         6181     6204      +23     
- Misses       4199     4201       +2     
  Partials      279      279              
Flag Coverage Δ
cube-backend 58.06% <75.00%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
packages/cubejs-backend-shared/src/http-utils.ts 27.11% <33.33%> (+0.70%) ⬆️
packages/cubejs-backend-shared/src/helpers.ts 65.51% <92.85%> (+25.51%) ⬆️
packages/cubejs-server-core/src/core/server.ts 63.03% <0.00%> (+0.18%) ⬆️
...es/cubejs-schema-compiler/src/adapter/BaseQuery.js 63.78% <0.00%> (+0.33%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 75545e8...bd766e6. Read the comment docs.

@paveltiunov paveltiunov merged commit 108f42a into master Feb 9, 2022
@paveltiunov paveltiunov deleted the adbatchx branch February 9, 2022 21:14
@rpaik rpaik removed the pr:community Contribution from Cube.js community members. label May 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants