Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented MysqlFileManager #3

Merged
merged 9 commits into from Feb 26, 2019

Conversation

Projects
None yet
3 participants
@lwj5
Copy link
Member

commented Feb 14, 2019

Added an implementation of filemanager using MySQL for recording and indexing and file system to store files.

lwj5 added some commits Feb 13, 2019

@lwj5 lwj5 added the enhancement label Feb 14, 2019

@lwj5 lwj5 self-assigned this Feb 14, 2019

@lwj5 lwj5 requested a review from sitfoxfly Feb 14, 2019

@coveralls

This comment has been minimized.

Copy link

commented Feb 14, 2019

Pull Request Test Coverage Report for Build 113

  • 262 of 403 (65.01%) changed or added relevant lines in 16 files are covered.
  • 2 unchanged lines in 2 files lost coverage.
  • Overall coverage decreased (-0.1%) to 74.749%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/main/java/ai/preferred/venom/fetcher/AsyncResponseConsumer.java 2 3 66.67%
src/main/java/ai/preferred/venom/job/PriorityQueueScheduler.java 1 2 50.0%
src/main/java/ai/preferred/venom/Crawler.java 2 4 50.0%
src/main/java/ai/preferred/venom/response/StorageResponse.java 9 11 81.82%
src/main/java/ai/preferred/venom/storage/FileManagerCallback.java 9 11 81.82%
src/main/java/ai/preferred/venom/request/StorageFetcherRequest.java 12 16 75.0%
src/main/java/ai/preferred/venom/fetcher/StorageFetcher.java 53 59 89.83%
src/main/java/ai/preferred/venom/storage/DummyFileManager.java 23 33 69.7%
src/main/java/ai/preferred/venom/storage/MysqlFileManager.java 87 200 43.5%
Files with Coverage Reduction New Missed Lines %
src/main/java/ai/preferred/venom/fetcher/AsyncResponseConsumer.java 1 83.33%
src/main/java/ai/preferred/venom/job/AbstractQueueScheduler.java 1 63.16%
Totals Coverage Status
Change from base Build 97: -0.1%
Covered Lines: 1042
Relevant Lines: 1394

💛 - Coveralls
@sitfoxfly

This comment has been minimized.

Copy link
Member

commented Feb 18, 2019

Only test phase requires mysql-connector-java dependency, right?

lwj5 added some commits Feb 22, 2019

@lwj5

This comment has been minimized.

Copy link
Member Author

commented Feb 22, 2019

No, it is needed to use MysqlFileManager, the user can override it if they require a different version.
For now we will consolidate and include the storage dependencies. If the project grows or if we require other implementations of FileManager, we will move to a multi-module maven project.

lwj5 added some commits Feb 22, 2019

@lwj5 lwj5 merged commit 0ad77a5 into master Feb 26, 2019

2 of 3 checks passed

coverage/coveralls Coverage decreased (-0.1%) to 74.749%
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@lwj5 lwj5 deleted the storage branch Feb 26, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.