Skip to content

FEATURE: MSF-20653 Support Blob Storage input source#1784

Merged
1 commit merged intogooddata:developfrom
tqtu:MSF-20653-2
Jun 21, 2021
Merged

FEATURE: MSF-20653 Support Blob Storage input source#1784
1 commit merged intogooddata:developfrom
tqtu:MSF-20653-2

Conversation

@tqtu
Copy link
Copy Markdown
Contributor

@tqtu tqtu commented Jun 14, 2021

No description provided.

@ghost
Copy link
Copy Markdown

ghost commented Jun 14, 2021

Build succeeded (check pipeline).

@ghost
Copy link
Copy Markdown

ghost commented Jun 14, 2021

Build succeeded (check pipeline).

@ghost
Copy link
Copy Markdown

ghost commented Jun 14, 2021

Build succeeded (check pipeline).

raise("Data Source needs a client to Blob Storage to be able to get blob file but 'blobStorage_client' is empty.") unless options['blobStorage_client']

# rubocop:disable Naming/VariableName
if options['blobStorage_client']['connectionString'] || options['blobStorage_client']['container']
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both connectionString and container are mandatory

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, connectionString and container are mandatory

@ghost
Copy link
Copy Markdown

ghost commented Jun 14, 2021

Build succeeded (check pipeline).

Comment thread lib/gooddata/helpers/data_helper.rb Outdated
require 'csv'
require 'digest'
require 'open-uri'
require_relative '../cloud_resources/blobstorage/blobstorage_client'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should load blob storage api only when user defined blob storage input source

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, Moved require_relative into blobStorage

@phong-nguyen-duy
Copy link
Copy Markdown
Contributor

@tqtu please add integration test to cover the new input source type

@ghost
Copy link
Copy Markdown

ghost commented Jun 15, 2021

Build succeeded (check pipeline).

@ghost
Copy link
Copy Markdown

ghost commented Jun 16, 2021

Build succeeded (check pipeline).

GoodData.gd_logger.info("Realizing download from Blob Storage. Container #{@container}.")
connect
filename = "#{SecureRandom.urlsafe_base64(6)}_#{Time.now.to_i}.csv"
blob_name = @path ? "#{file}" : "#{@path}/#{file}"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should handle the case @path ends with slash (/)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The path added delete_suffix to remove slash

@ghost
Copy link
Copy Markdown

ghost commented Jun 18, 2021

Build succeeded (check pipeline).

@yenkins
Copy link
Copy Markdown

yenkins commented Jun 21, 2021

Sonar scan result

More detail, see in https://sonarqube-gate.intgdc.com/dashboard?id=gooddata-ruby-gate-PR1784

To scan for vulnerabilities in dependencies and run unit tests (to get coverage report in sonar) please comment your PR with 'extended check sonar'.

@ghost ghost removed the merge label Jun 21, 2021
@ghost ghost merged commit a960c93 into gooddata:develop Jun 21, 2021
@ghost
Copy link
Copy Markdown

ghost commented Jun 21, 2021

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants