Releases: metatron-app/metatron-discovery
2021.2
Discovery feature
Enhancement
Support the format for each numeric column (#3572)
Added detailed setting function for grid measurement value column (#3606)
Added saving function for changing grid width (#3613)
Add configuration and api for mobile support (#3632)
Adds subtotal functionality for grid charts. (#3637)
Bump TypeScript, Angular (#3638)
Combine Chart sub-axis function (#3639)
Custom color for each measure in DataGrid (#3641)
Display url when entering the Dashboard screen (#3643)
Support the aggregation type of thetaSketch in data ingestion (#3644)
Line chart, bar chart tooltip function improvement (#3671)
Change datasource in dashboard (#3687)
Set countd in aggregation type in widget (#3691)
filter Set maximum number of views (#3717)
Added number format setting for each measure in KPI (#3720)
Added color setting for each measure in KPI (#3721)
Korean language support for numerical abbreviations (#3729)
Bug
Cannot change aggregation function of user defined column used by multiple charts (#3582)
Local filter is changed to global filter in the copied widget (#3651)
PivotGrid Performance issue, after legacy code merge. (#3668)
Line Chart tooltip missing cross-shelf content (#3718)
Grid horizontal scroll not displaying normally (#3726)
The subtotal function does not work properly in the grid chart. (#3736)
If TimeFilter is registered as a global filter and is not displayed on the dashboard, it will not be applied. (#3741)
Fix grid error (#3744)
Chart widget and selection filter z-index error (#3745)
Druid features
You need to upgrade your Druid to 2021.2 version unless you did it.
The druid binary was build based on hadoop version 3.1.1.3.1.4.0-315.
Full features
#3755 Release druid-2021.2
#3754 isNull on bitset column throws exception
#3753 Fix test fails
#3752 More bitset functions
#3749 Invalid value matcher for count aggregator with field and predicate specified
#3747 Port regex filter using FST from apache Pinot
#3742 Support FST indexing on dictionary
#3735 Support building tuning config in parquet/orc resolver
#3734 Exception with table datasource in join element
#3731 Support direct evaluation of bitset index
#3730 Decouple dimension type from string type
#3723 BitmapIndexSelector.getBitmapIndex should return null if bitmap index is not provided
#3722 Support inverted index for bitset column
#3714 Support filter on bitset type
#3712 Add some functions dealing bitset
#3709 Support indexing as bitset type and querying on it
#3708 Introduce extractHints for aggregator factory
#3705 Translate explicit 'null' string for numeric column as null
#3701 Support LOAD for non-schema aware format
#3700 Transient rows for query is not cleaned-up timely manner
#3697 Channel closed even when it succeeded and returned to pool
#3693 Set proper context loader for schema resolver
#3692 Return QIE instead of RE when channel disconnected
#3689 Discard MergeIndexSorting which can make invalid result
#3688 Fix exception on search query not-using cache
#3685 Support ITEM operator conversion in SQL
#3684 StackOverflow from accessing array virtual column
#3681 Provide page key for buffered aggregator
#3680 Optimize case function with many params
#3679 Minor test cleanups
#3677 Revert #3284 and fix it properly
#3675 Minor code refactorings
#3631 Broken error message from StreamHandler
#3670 Support rollup/cube for SQL
#3669 Port some more sql functions from apache druid
#3667 CCE retrieving topN query result from cache
#3666 Parallel histogram building based on id optimization
#3665 No need to read HLL context for buffer aggregator
#3663 Refactoring and bug fix for IntIterators
#3662 Remove unnecessary buffer duplication for bitmap serde
#3661 Minor optimization for difference of two bitmaps
#3660 Consolidate murmur3 hashing util
#3659 Split window functions to separate library class
#3657 CCE on schema query
#3655 Support TIME_CEIL function for sql
#3653 Support sub query on aggregation query with window
#3650 Use buffer reference instead of slice of it for simple comparing expression
#3646 Implement simple FilterableTable for sys.segments
#3642 Still query blocks on indexing node
#3636 Fix CCE on bound filter in subquery
#3631 Remove Futures.getUnchecked for query processing
#3629 Add point gathering aggregator factory
#3626 Improve performance of expression parsing
#3625 Allow table construct for CTAS
#3624 Support creating table with explicit values
#3619 Include star-schema benchmark as test case
#3618 Add type detail for column information
#3617 Simplify type coercing
2021.1
Discovery feature
New Feature
- Introduce engine(druid) monitoring function
- You can monitor the druid engine status.
- You can receive notifications of the status of each node through the slack channel.
- Support real-time data source
- You can create a data source using Kafka topics.
- Snapshot as an SQL script
- New JDBC connection support for workbench
- Athena
- BigQuery
- Added security enhancement option
- Various options such as multiple access restrictions and timeout... have been added.
Bug Fixes
In addition, various bug fixes were included.
For details, please refer to the existing RC Release note.
Druid features
You need to upgrade your Druid to 2021.1 version unless you did it.
The druid binary was build based on hadoop version 3.1.1.3.1.4.0-315.
3.4.0-rc4
Discovery features
Bug
Snapshot result error (#2315)
Grid is shown as blank on multiple rename page (#2844)
Extract rule should yield nulls instead of empty strings (#2873)
Fix extract, countpattern behavior (#2879)
Bug on merge with null (#2886)
Bug on unnest test cases (#2896)
I.DS disappears while data swapping (#2907)
Enhancement
Engine Monitoring - add to show datasources information (#2681)
Support custom script for chart widget on dashboard (#2803)
Improve preview stage of progress on stream Datasource creation (#2857)
Provide detail popup view on overview widgets in engine monitoring (#2869)
Support multi-element unnest (#2871)
Different policies should be applied for reading JSON (#2893)
Druid features
You need to upgrade your Druid to 3.3.3 version unless you did it.
Use druid-metatron-3.3.3.tar.gz from this link
3.3.4
Discovery features
The source code for this version can be found in the 3.3.x branch.
Bug Fix
Fix error for setting up a Jupyter notebook in a new workspace (#2948)
Add type to result object converter on Postgresql (#2943)
Add option to show value as original format in original view (#2908)
Fix duplication error when creating a datasource with a registered disabled datasource (#2905)
Add 'VIEW' type for counting table on Postgresql (#2868)
Fix trivial ui-related errors (#2946)
Druid features
You need to upgrade your Druid to 3.3.3 version unless you did it.
Use druid-metatron-3.3.3.tar.gz from this link
3.4.0-rc3
Discovery features
❗️Notice
We've found a bug when initialize this version with H2 database.
If you faced this issue, you can checkout hotfix branch 3.4.0-rc3-hotfix. And if you have any queries about it, please feel free to create a github issue.
New feature
Snapshot as an SQL script (#1976)
Enhancement
Add query information tab in engine monitoring (#1417)
Add 'Reversed' option for general color template (#1442)
Convert JSON string column into Map or Array column type (#1493)
Improve performance for searching original data (#1605)
Snapshot as an SQL script (#1976)
Too much space in dataset swap pop-up (#2230)
Nulls and type-mismatched values in data snapshots (#2752)
Apply Liquibase for Schema Management (#2756)
Show more information for applied Column Dictionary and Code Table (#2758)
Fill condition automatically using histogram bar selection (#2760)
Add permission to lnb extension (#2764)
creating a csv dataset with quote character (#2835)
Bug
A long snapshot name throws an error in attempting snapshot table generation. (#1752)
Some rules doesn't apply when create a data snapshot (#2066)
Add seq number to new datasets named duplicated (#2156)
Snapshot pop-up is not shown (#2350)
Need to show an error when fetch failed from Hive (#2476)
'Color by' values are not kept when setting layer in map view chart. (#2746)
Checking the status of snapshot is repeated (#2753)
Preview is too slow (#2795)
Preview row count error when downloading original data in chart (#2804)
Exploration Catalog's CSS Bug (#2806)
Cannot create/set a notebook server if you do not have an existing notebook server. (#2824)
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.3.3.tar.gz from this link
Full features
Support fieldName for count aggregator (#2848)
Add FFT post processor (#2840)
Rollback propagating typeString(#2838)
Throw exception when partition values are given to non-partitioned table in hive path spec (#2837)
Add compression information in index viewer (#2832)
Support block compression on string metric (#2828)
Log row count for sql query (#2815)
Allow file for baseDir of local firehose (#2814)
Optimize trivial ordering on time column with stream query (#2810)
Cleanup hadoop dependencies (#2809)
Support hive3 for hive-udf extension (#2799)
Add some tests for stream query on multi-value dimension (#2797)
Fix NPE when input format is not given for query forwarding (#2793)
Support minCount, maxDistance for kmeans query (#2792)
Fix possible NPE in cache runner (#2787)
Use iterator instead of full cluster materialization for dbscan (#2786)
Performance improvement of dbscan post processor (#2766)
Add dbScan post processor (#2762)
Support dimension spec / grouping set for frequency query (#2744)
Meta query for splitting should respect context of original query (#2730)
Regard query cancel as query success (#2720)
3.3.3
Discovery features
The source code for this version can be found in the 3.3.x branch.
Bug Fix
Change select query type from select to select.stream (#1605)
Apply timestamp format to file type snapshots (#2066)
'Color by' values are not kept when setting layer in map view chart (#2746)
Improve Preview load speed (#2795)
Fix append parameter for ingestion (#2812)
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.3.3.tar.gz from the assets of this release page.
Full features
Support fieldName for count aggregator (#2848)
Add FFT post processor (#2840)
Rollback propagating typeString(#2838)
Throw exception when partition values are given to non-partitioned table in hive path spec (#2837)
Add compression information in index viewer (#2832)
Support block compression on string metric (#2828)
Log row count for sql query (#2815)
Allow file for baseDir of local firehose (#2814)
Optimize trivial ordering on time column with stream query (#2810)
Cleanup hadoop dependencies (#2809)
Support hive3 for hive-udf extension (#2799)
Add some tests for stream query on multi-value dimension (#2797)
Fix NPE when input format is not given for query forwarding (#2793)
Support minCount, maxDistance for kmeans query (#2792)
Fix possible NPE in cache runner (#2787)
Use iterator instead of full cluster materialization for dbscan (#2786)
Performance improvement of dbscan post processor (#2766)
Add dbScan post processor (#2762)
Support dimension spec / grouping set for frequency query (#2744)
Meta query for splitting should respect context of original query (#2730)
Regard query cancel as query success (#2720)
3.4.0-rc2
Discovery features
New feature
Support real-time data source (#2562)
Bug Fix
Each screen has different representation for null in data grid (#1750)
Union rule error in data preparation (#2063)
Cannot close preview result panel in join pop-up (#2293)
Download as CSV does not work (#2329)
Wrong function explanation (#2530)
Measure aliases cannot be changed. (#2533)
UI hang after delimiter error (#2610)
Maintain search result list view in datasource mgmt. after moving to detailed view (#2635)
postgresql sql syntax error (#2686)
Dashboard widget 'save table as file' screen error in dark theme (#2695)
If you change from a dark theme to a general theme and refresh the screen, it appears as a dark theme instead of a general theme (#2696)
Change Filter in Exploration list and bug fix. (#2699)
'Size by', 'Radius Range' values are not kept when setting layer in map view chart. (#2722)
'Custom Color Range' values are not kept when setting layer in map view chart. (#2734)
Enhancement
UI enhancements for engine (druid) monitoring (#2680)
When generate a new WDS, add seq# for distinction (#2705)
Refactoring
Unified processing for CSV file (#1815)
Use Better coding patterns and reuse code blocks (#2517)
Unify charset detecting code (#2640)
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.3.2.tar.gz from this link
Full features
Support search query on virtual columns (#2719)
Add option for automatic applying of #2702 (#2712)
Optimize finding most frequent item with count-min sketch (#2702)
Fix CCE in frequency post processor (#2700)
Expect list type instead of array for struct type (#2694)
Stream query with order by on metric is not working (#2690)
Sporadic timeout exception on query cancel (#2689)
Indexing speed is severely degraded (#2688)
Support trivial loading of orc/parquet format (#2687)
Add API for retrieving running queries (#2685)
Historical node is not shown in jmx query (#2684)
Support running query in shell (#2683)
Support parquet formatter (#2669)
3.3.2
Discovery features
The source code for this version can be found in the 3.3.x branch.
Bug Fix
Fix 'Custom Color range' does not reset when creating a new map view widget (#2745)
Fix null value representation in ORC files (#2752)
Checking the status of snapshot is repeated (#2753)
Use fetch size and set auto_commit=false for PostgreSQL (#2748)
Unify null representations in datapreparation (#1750)
Unify charset detecting code (#2640)
When generate a new dataset, add sequence for distinction (#2705)
Create dataset button is not clicked (#2723)
Contains function explanation was corrected (#2530)
Cannot close preview result (#2293)
Preview reload problem (#2610)
Big refactoring (Exception handling, dividing mega classes, etc.) (#2517)
Adding union rule shows field type as icon correctly (#2063)
Fix can add an aggregate rule without group (#2666)
Fix snapshot list loads infinitely (#2664)
Decide column count only by the first line (#1798)
Use type information from snapshot when ingestion (#2375)
Remove unused dataprep code (#1326)
Reformat by standard code style file by IDE (#2516)
A proper error message when move inconsecutive columns (#1771)
Troubleshooting Chrome compatibility issues with custom fields (#2750)
'Size by', 'Radius Range' values are not kept when setting layer in map view chart (#2722)
'Custom Color Range' values are not kept when setting layer in map view chart. (#2734)
Fix error for regex filtering mode in advance filter (#2727)
Add missing "union" case in switch-case
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.3.2.tar.gz from the assets of this release page.
Full features
Support search query on virtual columns (#2719)
Add option for automatic applying of #2702 (#2712)
Optimize finding most frequent item with count-min sketch (#2702)
Fix CCE in frequency post processor (#2700)
Expect list type instead of array for struct type (#2694)
Stream query with order by on metric is not working (#2690)
Sporadic timeout exception on query cancel (#2689)
Indexing speed is severely degraded (#2688)
Support trivial loading of orc/parquet format (#2687)
Add API for retrieving running queries (#2685)
Historical node is not shown in jmx query (#2684)
Support running query in shell (#2683)
Support parquet formatter (#2669)
3.4.0-rc1
Discovery features
New feature
Introduce engine(druid) monitoring function (#1373)
Add "recently used (dashboard)" section in metadata detail view (#2603)
Bug Fix
Unknown error when move inconsecutive columns (#1771)
Disable spark engine when not properly configured (#2584)
Contents in Column List of Metadata Management and Exploration Data Do Not Match (#2592)
Error for changing column order in table chart when changing column in shelf (#2632)
Some trivial css issues (#2633)
Missing view in Postgresql table list (#2672)
Enhancement
Change auto-detection of column count (#1798)
Use data snapshot column type when creating datasource with data snapshot (#2375)
Apply black design theme and change global UI function (#2388)
Pause and resume batch datasource ingestion (#2463)
Support to show calculated column in table charts (#2539)
Synchronize css before applying black theme (#2602)
Add link to move Dashboard and Workbench in "Top User" Section (#2604)
Improve filtering and sorting for metadata list view (#2605)
Keep sorting conditions used in the workspace (#2617)
Improves the Layers menu in the panel for dimension filters (#2625)
Improves the Layers menu in the panel for time filters (#2626)
Refactoring
dataprep code cleaning (#1326)
Reformat by the coding style file. Remove unused codes (#2516)
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.3.1.tar.gz from this link
Full features
Indexing speed is severely degraded (#2688)
Support trivial loading of orc/parquet format (#2687)
Add API for retrieving running queries (#2685)
Historical node is not shown in jmx query (#2684)
Support running query in shell (#2683)
Support parquet formatter (#2669)
Comma is not allowed in format of timestamp spec (#2656)
Keep ordering of pivot value columns as user specified (#2663)
Remove hive dependency from hdfs module (#2662)
Contents are not needed for cancel query (#2639)
Apply estimation on result of join for determining hashing it or not for next join (#2637)
Consolidate console shell with index viewer (#2636)
Return consistent timestamp for timeseries and group-by query with all granularity (#2630)
Use cached string for well-known datetime instance (#2624)
Remove Iterable from IndexedInts (#2623)
Simplify cache key generation (#2619)
Optimize filter on boolean column (#2618)
Support converting lucene filter to expression filter in realtime node (#2608)
Split shape functions into two modules (#2615)
Support boolean type column (#2613)
Keep ordering of postProcessing --> finalDecoration in local nodes (#2611)
Use fill if possible for numeric metrics (#2600)
Remove deprecated methods in stream query (#2599)
Check on immutable roaring bitmap is slow (#2597)
Possible invalid NULL for generic min aggregator (#2595)
Remove unnecessary instantiation of DateTime for timeseries query (#2594)
Use raw accessor for cardinality aggregator if possible (#2591)
Verify no duplicate datasource in supervisors (#2580)
Decouple segment partitioning from Kafka partitioning (#2526)
Add optional "earlyMessageRejectionPeriod" config (#2525)
Refactoring Appenderator Driver (#2524)
Support null in orc formatter (#2582)
Support running SQL file in druid shell (#2556)
Support type-less null (aka NULL) in expression (#2541)
Concurrency issue when using deprecated UDAF class in hive-udf extension (#2520)
Balancing is not working properly (#2515)
3.3.1
Discovery features
The source code for this version can be found in the 3.3.x branch.
Bug Fix
Missing view in Postgresql table list (#2672)
Fix uppercase table name in postgresql (#2629)
Fix error for displaying measure alias in original mode of table(#2533)
Add timer to cancel button in workbench(#2648)
Fix presto array column handling as string (#2653)
Fix ifcountd function error (#2616)
Uppercase table name of postgresql to create dataset (#2654)
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.3.1.tar.gz from the assets of this release page.
Full features
Indexing speed is severely degraded (#2688)
Support trivial loading of orc/parquet format (#2687)
Add API for retrieving running queries (#2685)
Historical node is not shown in jmx query (#2684)
Support running query in shell (#2683)
Support parquet formatter (#2669)
Comma is not allowed in format of timestamp spec (#2656)
Keep ordering of pivot value columns as user specified (#2663)
Remove hive dependency from hdfs module (#2662)
Contents are not needed for cancel query (#2639)
Apply estimation on result of join for determining hashing it or not for next join (#2637)
Consolidate console shell with index viewer (#2636)
Return consistent timestamp for timeseries and group-by query with all granularity (#2630)
Use cached string for well-known datetime instance (#2624)
Remove Iterable from IndexedInts (#2623)
Simplify cache key generation (#2619)
Optimize filter on boolean column (#2618)
Support converting lucene filter to expression filter in realtime node (#2608)
Split shape functions into two modules (#2615)
Support boolean type column (#2613)
Keep ordering of postProcessing --> finalDecoration in local nodes (#2611)
Use fill if possible for numeric metrics (#2600)
Remove deprecated methods in stream query (#2599)
Check on immutable roaring bitmap is slow (#2597)
Possible invalid NULL for generic min aggregator (#2595)
Remove unnecessary instantiation of DateTime for timeseries query (#2594)
Use raw accessor for cardinality aggregator if possible (#2591)
Verify no duplicate datasource in supervisors (#2580)
Decouple segment partitioning from Kafka partitioning (#2526)
Add optional "earlyMessageRejectionPeriod" config (#2525)
Refactoring Appenderator Driver (#2524)
Support null in orc formatter (#2582)
Support running SQL file in druid shell (#2556)
Support type-less null (aka NULL) in expression (#2541)
Concurrency issue when using deprecated UDAF class in hive-udf extension (#2520)
Balancing is not working properly (#2515)