feat(db-engine): Add support for Apache Solr #12403

aadel · 2021-01-10T16:00:01Z

SUMMARY

Adding support for Apache Solr through SQLAlchemy DBAPI subject to features provided by and limitations of Solr Parallel SQL interface. An engine spec is included.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Example Charts:

TEST PLAN

Most of the testing is maintained on sqlalchemy-solr package since the current engine spec is skeletal.

ADDITIONAL INFORMATION

codecov-io · 2021-01-10T16:07:47Z

Codecov Report

Merging #12403 (b34c404) into master (76b06b2) will decrease coverage by 4.59%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master   #12403      +/-   ##
==========================================
- Coverage   66.94%   62.35%   -4.60%     
==========================================
  Files        1014     1015       +1     
  Lines       49516    49602      +86     
  Branches     5068     4839     -229     
==========================================
- Hits        33147    30927    -2220     
- Misses      16239    18477    +2238     
- Partials      130      198      +68

Flag	Coverage Δ
cypress	`?`
javascript	`60.73% <ø> (+0.03%)`	⬆️
python	`63.40% <100.00%> (-0.81%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset/db_engine_specs/solr.py	`100.00% <100.00%> (ø)`
superset-frontend/src/SqlLab/App.jsx	`0.00% <0.00%> (-100.00%)`	⬇️
superset-frontend/src/explore/App.jsx	`0.00% <0.00%> (-100.00%)`	⬇️
superset-frontend/src/dashboard/App.jsx	`0.00% <0.00%> (-100.00%)`	⬇️
superset-frontend/src/chart/ChartContainer.jsx	`0.00% <0.00%> (-100.00%)`	⬇️
superset-frontend/src/explore/reducers/index.js	`0.00% <0.00%> (-100.00%)`	⬇️
...et-frontend/src/dashboard/containers/Dashboard.jsx	`0.00% <0.00%> (-100.00%)`	⬇️
...et-frontend/src/filters/components/Select/types.ts	`0.00% <0.00%> (-100.00%)`	⬇️
...t-frontend/src/dashboard/containers/SliceAdder.jsx	`0.00% <0.00%> (-100.00%)`	⬇️
...t-frontend/src/explore/reducers/getInitialState.js	`0.00% <0.00%> (-100.00%)`	⬇️
... and 261 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 76b06b2...b34c404. Read the comment docs.

villebro

This is awesome @aadel, thanks so much for adding support for Solr! 🎉 LGTM, but requesting review from @srinify to check the docs changes.

aadel · 2021-01-10T19:55:44Z

Thanks @villebro! I'm also excited adding support for Solr! 💯

amitmiran137 · 2021-01-10T22:23:19Z

Is there a plan on the roadmap to support facets?
How would a SQL query would look like for that ?

aadel · 2021-01-11T08:05:24Z

@amitmiran137 Actually the concept of facets is associated with Lucene/Solr however, the SQL interface uses supports aggregations, predicates and sorting using standard syntax which give the same results for supported scenarios. Internally, the SQL interface in the default mode uses the JSON Facet API or StatsComponent for aggregations. In this mode, the aggregations logic is pushed down into the search engine and the aggregates are returned.

willbarrett · 2021-01-11T14:15:51Z

Nice! Added @dpgaspar as a reviewer as well.

dpgaspar

Awesome!

duynguyenhoang · 2021-01-12T06:11:46Z

superset/db_engine_specs/solr.py

+    """Engine spec for Apache Solr"""
+
+    engine = "solr"
+    engine_name = "Apach Solr"


nit: typo here. It should be "Apache"

Thanks for catching @duynguyenhoang ! This would be good to fix before merging.

hey @aadel this PR looks good minus this one typo! "Apach" -> "Apache"

Fix this and I'd be happy to approve!

villebro

Request to fix typo that was picked up by @duynguyenhoang

villebro · 2021-01-12T07:04:13Z

superset/db_engine_specs/solr.py

+    """Engine spec for Apache Solr"""
+
+    engine = "solr"
+    engine_name = "Apach Solr"


Thanks for catching @duynguyenhoang ! This would be good to fix before merging.

srinify · 2021-01-14T16:15:09Z

superset/db_engine_specs/solr.py

+    """Engine spec for Apache Solr"""
+
+    engine = "solr"
+    engine_name = "Apach Solr"


hey @aadel this PR looks good minus this one typo! "Apach" -> "Apache"

Fix this and I'd be happy to approve!

* [db engine] Add support for Apache Solr * Fixing typo

* release: bump to 1.0.0 and CHANGELOG * fix(explore): long metric name display (#12387) * fix(explore): long metric name display * add tooltip to control * chore: Show datasets when search input is empty (#12391) * chore: Fix typo “Rest” to “Reset” (#12392) * chore: upgrade eslint, babel, and prettier (#12393) * feat(explore): add tooltip to timepicker label (#12401) * chore: change Datasource to Dataset in Explore ui (#12402) * chore(explore):change dataset to datasource in ui * modal * Add space * Changing it back🤦🏾‍♀️ * Chargeback * fix: Refresh Interval Modal dropdown (#12406) * fix(native-filters): incorrect queriesData state (#12409) * refactor: from superset.utils.core break down date_parser (#12408) * Fixes control panel fields styling (#12236) (#12326) * feat: Resizable dataset and controls panels on Explore view (#12411) * Implement resizable panels on explore view * Optimize chart rendering while resizing * Make dataset column narrower Co-authored-by: Evan Rusackas <evan@preset.io> * fix(dashboard): artefacts shown while drag and dropping deck.gl charts (#12418) * [12181] Fix artifacts while drag and dropping deck.gl charts. * Run prettier * bump superset-ui packages for rolling window change (#12426) * chore: bump superset-ui deckgl plugin (#12466) * fix: do not show vertical scrollbar for charts in dashboard (#12478) * fix: do not show vertical scrollbar for charts in dashboard * Proper fix for #11419 Co-authored-by: Jesse Yang <jesse.yang@airbnb.com> * fix(dashboard): use datasource id from slice metadata (#12483) * fix(timepicker): make pyparsing thread safe (#12489) * fix: make pyparsing thread safe * remove parenthesis for decorator * fix (SQL Lab): disappearing results on tab switch (#12472) * fix (SQL Lab): disappearing results on tab switch * Remove state * Fix test * fix: import ZIP files that have been modified (#12425) * fix: import ZIP files that have been modified * Add unit test * update changelog with rc2 entries * fix: impose dataset ownership check on old API (#12491) * fix: impose dataset ownership check on old API * update UPDATING.md * partially protect the old MVC also * prevent metric and column add and update * ci: remove refs/tags from docker tags on a release (#12518) * ci: remove refs/tags from docker tags on a release * wider head * fix: lowercase all columns in examples (#12530) * fix(explore): time table control panel (#12532) * fix(explore): Add Time section back to FilterBox (#12537) * Fixing Pinot queries for time granularities: WEEKS/MONTHS/QUARTERS/YEARS (#12536) * fix: Select options overflowing Save chart modal on Explore view (#12522) * Fix select options overflowing modal * fix unit test Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com> * Fix list filters vertical alignment (#12497) * feat(db-engine): Add support for Apache Solr (#12403) * [db engine] Add support for Apache Solr * Fixing typo * chore: rename docker image in build_docker_image.sh, docker-compose.yml and helm values.yaml (#12337) * add rc3 changelog entries * fix: Popover closes on change of dropdowns values (#12410) * fix: Add MAX_SQL_ROW value to LIMIT_DROPDOWN (#12555) * fix(viz): missing groupby and broken adhoc metrics for boxplot (#12556) * fix: height on grid results (#12558) * fix: case expression should not have double quotes (#12562) * Fix 500 error when loading dashboards with slice having deleted dataset (#12535) * add rc4 changelog entries * Fixed typo on line 348 * Added files Co-authored-by: Daniel Gaspar <danielvazgaspar@gmail.com> Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com> Co-authored-by: Geido <60598000+geido@users.noreply.github.com> Co-authored-by: Junlin Chen <junlin@preset.io> Co-authored-by: Jesse Yang <jesse.yang@airbnb.com> Co-authored-by: Agata Stawarz <47450693+agatapst@users.noreply.github.com> Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com> Co-authored-by: Michael S. Molina <70410625+michael-s-molina@users.noreply.github.com> Co-authored-by: Kamil Gabryjelski <kamil.gabryjelski@gmail.com> Co-authored-by: Evan Rusackas <evan@preset.io> Co-authored-by: Kasia Kucharczyk <2536609+kkucharc@users.noreply.github.com> Co-authored-by: Phillip Kelley-Dotson <pkelleydotson@yahoo.com> Co-authored-by: Grace Guo <grace.guo@airbnb.com> Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com> Co-authored-by: Xiang Fu <fx19880617@gmail.com> Co-authored-by: Ahmed Adel <github@aadel.io> Co-authored-by: Amit Miran <47772523+amitmiran137@users.noreply.github.com> Co-authored-by: Hugh A. Miles II <hughmil3s@gmail.com> Co-authored-by: Shuyao Bi <shuyaob@andrew.cmu.edu> Co-authored-by: Lyndsi Kay Williams <lyndsikaywilliams@Lyndsis-MacBook-Pro.local>

[db engine] Add support for Apache Solr

43077e7

pull-request-size bot added the size/M label Jan 10, 2021

aadel changed the title ~~[db engine] Add support for Apache Solr~~ [feat] Add support for Apache Solr Jan 10, 2021

aadel changed the title ~~[feat] Add support for Apache Solr~~ feat(db-engine): Add support for Apache Solr Jan 10, 2021

villebro requested a review from srinify January 10, 2021 19:29

villebro approved these changes Jan 10, 2021

View reviewed changes

willbarrett requested a review from dpgaspar January 11, 2021 14:15

dpgaspar approved these changes Jan 11, 2021

View reviewed changes

duynguyenhoang reviewed Jan 12, 2021

View reviewed changes

villebro requested changes Jan 12, 2021

View reviewed changes

srinify requested changes Jan 14, 2021

View reviewed changes

Fixing typo

b34c404

villebro approved these changes Jan 15, 2021

View reviewed changes

villebro merged commit e6a76a5 into apache:master Jan 15, 2021

villebro added the v1.0 label Jan 15, 2021

villebro pushed a commit that referenced this pull request Jan 15, 2021

feat(db-engine): Add support for Apache Solr (#12403)

739ab14

* [db engine] Add support for Apache Solr * Fixing typo

anjackson mentioned this pull request Jan 29, 2021

Superset Solr setup does not support time-interval GROUP BY aggregations ukwa/docker-superset#1

Open

mistercrunch added 🍒 1.0.0 🍒 1.0.1 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.2.0 labels Mar 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(db-engine): Add support for Apache Solr #12403

feat(db-engine): Add support for Apache Solr #12403

aadel commented Jan 10, 2021

codecov-io commented Jan 10, 2021 •

edited

villebro left a comment

aadel commented Jan 10, 2021

amitmiran137 commented Jan 10, 2021

aadel commented Jan 11, 2021

willbarrett commented Jan 11, 2021

dpgaspar left a comment

duynguyenhoang Jan 12, 2021

villebro Jan 12, 2021

srinify Jan 14, 2021

villebro left a comment

villebro Jan 12, 2021

srinify Jan 14, 2021

feat(db-engine): Add support for Apache Solr #12403

feat(db-engine): Add support for Apache Solr #12403

Conversation

aadel commented Jan 10, 2021

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

ADDITIONAL INFORMATION

codecov-io commented Jan 10, 2021 • edited

Codecov Report

villebro left a comment

Choose a reason for hiding this comment

aadel commented Jan 10, 2021

amitmiran137 commented Jan 10, 2021

aadel commented Jan 11, 2021

willbarrett commented Jan 11, 2021

dpgaspar left a comment

Choose a reason for hiding this comment

duynguyenhoang Jan 12, 2021

Choose a reason for hiding this comment

villebro Jan 12, 2021

Choose a reason for hiding this comment

srinify Jan 14, 2021

Choose a reason for hiding this comment

villebro left a comment

Choose a reason for hiding this comment

villebro Jan 12, 2021

Choose a reason for hiding this comment

srinify Jan 14, 2021

Choose a reason for hiding this comment

codecov-io commented Jan 10, 2021 •

edited