Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[HBase] refactoring #614

Merged
merged 26 commits into from
Jun 30, 2022
Merged

[HBase] refactoring #614

merged 26 commits into from
Jun 30, 2022

Conversation

JulienPeloton
Copy link
Member

@JulienPeloton JulienPeloton commented Jun 17, 2022 •
edited
Loading

IMPORTANT: Please create an issue first before opening a Pull Request.
Linked to issue(s):

What changes were proposed in this pull request?

This PR does 2 things:

  1. code refactoring when writing data on HBase. Now one single routine for all.
  2. catalogs are saved on disk for later re-use

How was this patch tested?

Cluster & CI

@JulienPeloton JulienPeloton added this to the 2.2 milestone Jun 17, 2022
@codecov
Copy link

codecov bot commented Jun 28, 2022 •
edited
Loading

Codecov Report

Merging #614 (e175daf) into master (6bb1c02) will decrease coverage by 0%.
The diff coverage is 90%.

❗ Current head e175daf differs from pull request most recent head 4ba7a94. Consider uploading reports for the commit 4ba7a94 to get more accurate results

@@          Coverage Diff          @@
##           master   #614   +/-   ##
=====================================
- Coverage      94%    94%   -1%     
=====================================
  Files          19     19           
  Lines         921    913    -8     
=====================================
- Hits          869    859   -10     
- Misses         52     54    +2
Impacted Files Coverage Δ
fink_broker/parser.py 100% <ø> (ø)
fink_broker/hbaseUtils.py 93% <82%> (-4%) ⬇️
bin/access_science_db.py 100% <100%> (ø)
bin/daily_stats.py 100% <100%> (ø)
bin/index_sso_cand_archival.py 100% <100%> (ø)
bin/science_archival.py 100% <100%> (+2%) ⬆️
fink_broker/science.py 100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6bb1c02...4ba7a94. Read the comment docs.

@JulienPeloton
Add Gaia, GCVS, VSX (#620)
e0ee97e 
* Add Gaia, GCVS, VSX

* Fix missing module, and syntax for the constructor

* Fix PEP8
@JulienPeloton
Disable object table creation for now
70cc631
@JulienPeloton JulienPeloton mentioned this pull request Jun 29, 2022
Open
@sonarcloud
Copy link

sonarcloud bot commented Jun 30, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@JulienPeloton JulienPeloton merged commit 7a875f6 into master Jun 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
hbase refactoring
Projects
None yet
Milestone
2.2
Development

Successfully merging this pull request may close these issues.

[HBase] refactor index table script [HBase] store HBase catalogs on the repo
1 participant
@JulienPeloton