Releases: metatron-app/metatron-discovery
3.3.0
Discovery features
- Introduce data exploration feature (beta)
- You can search and find the data you want to use.
- Add additional options when creating data sources.
- Schema modifications have been improved in the ingestion stage.
- The re-ingestion feature has been added.
- Improved metadata management
- You can do metadata management on all db connections supported by metatron.
- You can see sample data in metadata details view.
- Improved notebook functionality
- Change Notebook connection information from existing hostname and port to url
- Added the function to release notebook server in workspace
- Improved the item view in the workspace.
- Along with some design changes, let you know the (un)publishing status of the resources associated with the item.
- Compatible with the latest Java version(12)
- Support spark as data trasformation engine in data-preparation
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.3.0.tar.gz from the assets of this release page.
3.3.0-rc3
Discovery features
Enhancement
When you request a shared workspace, there are unnecessary query calls. (#2366)
Put column type information into snapshot entity (#2335)
Filter Popup Enhancements (#2322)
Generate metadata when data source creation is successful. (#2298)
Change to url mode when creating notebook server (#2240)
Initialize selection value when selecting notebook server in workspace (#2170)
Initialization function when changing value alias (#2086)
Bug Fix
When creating a data source, up to 20 table lists are displayed. (#2363)
Error adding or changing global variables in workbench (#2360)
Values in KPI charts are displayed incorrectly if user-defined columns exist (#2342)
KPI chart spelling mistake (#2334)
Map view chart can not be added to the dashboard. (#2331)
Change pop-up title when deleting data connection (#2323)
Can't configure datasource column as date/time if data set starts with null / empty values (#2294)
Bugs that do not reflect values when changing the last value of several fields (#2085)
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.2.6.tar.gz from this link
Full features
Remove replication throttle (#2389)
Prevent pre-matured cleanup in coordinator (#2383)
Nuke segments in batch for purge datasource (#2379)
Fix CCE in nvl on none-string tagged as string type (#2378)
Support lock listing API in overlord resource (#2377)
Support returning null for basic aggregators(min/max/var/stddev) (#2373)
Remove some unnecessary classes/methods (#2372)
Support implicit casting of boolean to number (#2357)
Support wasbs scheme for deep storage (#2353)
Expression evaluation on unknown type should return unknown type (#2351)
Fix compilation with Java 8 (#2347)
Support log extraction API in servers (#2338)
Handle numeric nulls in bulk sequence (#2337)
Minor log improvements (#2330)
Make resource files in extension directory loadable (#2327)
Some dependencies are missed for extensions in packaging (#2326)
Balancing is biased to a server (#2283)
Use buffered stream for zipping (#2282)
3.2.6
Discovery features
The source code for this version can be found in the 3.2.x branch.
Bug Fix
Fix setting domain cookie in external login page (#2325)
Values in KPI charts are displayed incorrectly if user-defined columns exist (#2342)
Enable logout in the external application (#2364)
When creating a data source, up to 20 table lists are displayed (#2363)
When you request a shared workspace, there are unnecessary query calls. (#2366)
Fix snapshot loading and add name title (#2385)
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.2.6.tar.gz from the assets of this release page.
Full features
Remove replication throttle (#2389)
Prevent pre-matured cleanup in coordinator (#2383)
Nuke segments in batch for purge datasource (#2379)
Fix CCE in nvl on none-string tagged as string type (#2378)
Support lock listing API in overlord resource (#2377)
Support returning null for basic aggregators(min/max/var/stddev) (#2373)
Remove some unnecessary classes/methods (#2372)
Support implicit casting of boolean to number (#2357)
Support wasbs scheme for deep storage (#2353)
Expression evaluation on unknown type should return unknown type (#2351)
Fix compilation with Java 8 (#2347)
Support log extraction API in servers (#2338)
Handle numeric nulls in bulk sequence (#2337)
Minor log improvements (#2330)
Make resource files in extension directory loadable (#2327)
Some dependencies are missed for extensions in packaging (#2326)
Balancing is biased to a server (#2283)
Use buffered stream for zipping (#2282)
3.3.0-rc2
Discovery features
Enhancement
Enhance displaying the resources associated with the item in the workspace (#2167)
Add re-ingestion setting with the file (#2239)
Disable roll-up mode in case of including geo type column (#2299)
Data exploration based on popularity, updated-date, catalog, tags using metadata (#2131)
Display and download metadata's sample data(#2172)
Improve Java version(11 or 12) compatibility (#1704)
Documentation for application properties (#2246)
Bug Fix
Back button toward dataflow list acts like browser back button (#2232)
'Series Percent' needs to be modified so that the sum of the bar charts is 100 percent. (#2184)
The user-defined field is not working in the prediction line (#2227)
In dataflows no other join type besides an inner join works (#2241)
Validation is disabled when a column dictionary of type timestamp is connected. (#2242)
If a permission schema is deleted, workspaces which are linked in the permission schema is not changed (#2214)
If the measurement is null, it must be able to be replaced with a negative value in the data source ingestion process. (#2120)
Unexpected retry occurs when a query fails. (#2272)
Fix error during processing DB connection in workbench (#2250)
A snapshot that contains join rules looks like empty. (#2216)
Treat null values and empty strings the same way. (#2243)
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.2.5.tar.gz from this link
Full features
Type of PostAggregator is not valid (#2281)
Support wasb scheme for deep storage (#2278)
Allow numeric null from text file (#2274)
Minor log improvement of QueryResource (#2270)
Add licenses to LICENSE (#2268)
Remove unnecessary wrapping in hash join on single key (#2265)
Keep collation information of join for further processing (#2262)
Exclude NaN from covariance query results (#2261)
Append task stats in task status (#2260)
Lessen gc pressure of coordinator with a lot of segments (#2258)
Trivialize timeout exception on scheduleNow() (#2257)
Support ordering pushdown for stream query (#2256)
Change license headers (#2252)
Make option for bush join optimizer (#2238)
Fix some regressions from #2133 (#2236)
Support heuristic join reordering in SQL (#2233)
Add prepare-metric.sh example for engine monitor (#2231)
Add option for dumping details of query planning in SQL (#2226)
Keep some silly legacy convention for pivot (#2207)
Amplify cost reduction of scan projection (#2020)
Simplify state machine of query generator in SQL (#2194)
Support windowing functions in SQL (#1282)
Support windowing in stream query (#2133)
Improve general type resolution (#2171)
3.2.5
Discovery features
The source code for this version can be found in the 3.2.x branch.
Bug Fix
Fix Exceptionhandler to include HttpMessageNotWritableException (#2321)
Fix the count of workbook dashboard in workspace (#2314)
Fix dataconnection un-publishing checkbox (#2311)
Add format placeholder in column dictionary detail (#2308)
Fix authentication script error when updating connection component (#2306)
Fix default dataconnection icon of extensions (#2303)
Remove browser scrolling when updating a dashboard (#2297)
Fix column dictionary's name duplicated check (#2296)
History back is not allowed on dataflow pages (#2232)
Fix error for including user defined field in prediction line (#2227)
If a permission schema deleted then set default permission schema (#2214)
Fix error for displaying percentage on bar chart series (#2184)
Change hibernate property(connection.release_mode) to after_transaction (#2250)
Add document for application properties guide (#2246)
Fix typos and add comment for extensions (#2248)
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.2.5.tar.gz from the assets of this release page.
Full features
Type of PostAggregator is not valid (#2281)
Support wasb scheme for deep storage (#2278)
Allow numeric null from text file (#2274)
Minor log improvement of QueryResource (#2270)
Add licenses to LICENSE (#2268)
Remove unnecessary wrapping in hash join on single key (#2265)
Keep collation information of join for further processing (#2262)
Exclude NaN from covariance query results (#2261)
Append task stats in task status (#2260)
Lessen gc pressure of coordinator with a lot of segments (#2258)
Trivialize timeout exception on scheduleNow() (#2257)
Support ordering pushdown for stream query (#2256)
Change license headers (#2252)
Make option for bush join optimizer (#2238)
Fix some regressions from #2133 (#2236)
Support heuristic join reordering in SQL (#2233)
Add prepare-metric.sh example for engine monitor (#2231)
Add option for dumping details of query planning in SQL (#2226)
Keep some silly legacy convention for pivot (#2207)
Amplify cost reduction of scan projection (#2020)
Simplify state machine of query generator in SQL (#2194)
Support windowing functions in SQL (#1282)
Support windowing in stream query (#2133)
Improve general type resolution (#2171)
3.3.0-rc1
Discovery features
The first release candidate version for 3.3.0.
Enhancement
Enable ioConfig on the realtime ingestion (#2158)
Enable explicit garbage collection in dataprep (#2144)
Users should be able to search for hidden value in filter widget (#2045)
Support changing column schema on ingestion stage (#1920)
User Delete Policy with workspace (#2115)
Simplify color by option on mapview layer option and add cluster view mode on point layer (#1443)
Enhance re-ingestion datasource with file (#2178)
Creating a metadata by database in data connection (#1539)
Bug Fix
Script error when moving to next page of datasource creation with file (#2163)
Infinite loop when the total count of list is zero (#2176)
Cannot register the StorageProperties bean (#2179)
limit is not support in oracle (#2177)
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.2.4.tar.gz from this link
Update easymock to 3.4 (#2162)
Support 한글 token in SQL parser (#2160)
Support router in single mode (#2155)
Fix tasks command at DruidShell (#2143)
Support more result formats and column header at SQL (#2141)
Support full param to supervisor api (#2140)
Support for router forwarding request to active coordinator/overlord (#2139)
Implment SQLMetadataStorageActionHandler (#2138)
Improve task retrieval APIs on Overlord (#2137)
Apply java8 (#2136)
Introduce SystemSchema tables (#2135)
Add Unified web console (#2134)
Hadoop ingestion in ReduceMerge mode does not progress from second index (#2151)
Do not keep index in memory for index viewer (#2121)
Minor log message improvements (#2124)
Remove classpath from logging in MiddleManager (#2123)
Regard U+00A0 as whitespace (#2122)
Fix NPE when dimension spec is not exists (#2117)
Result of hive udaf is not deterministic (#2116)
3.2.4
Discovery features
We've efforted to catch the bugs in this release. Hope you will experience more stability.
The source code for this version can be found in the 3.2.x branch.
Bug Fix
Enable ioConfig on the realtime ingestion (#2158)
Transfer workspace to user whose name contains dot(.) (#2152)
Fix script error when moving to next page of datasource creation with file (#2163)
Fix infinite loop when list is empty (#2176)
DataSource for webSocket is not existed (#2168)
Apply common jdbc module to remove bug when using oracle (#2177)
Fill all fields of DataConnection class while dataset creation (#2197)
Removing mysql dedicated string template in audit stats query. (#2205)
Fix custom field filter and filter widget errors (#2221)
Fix display workbench update user when initializing the page (#2228)
Show histogram correctly for oracle number datatype (#2224)
Fix csv total row counts (#2235)
Fix metadata change column error (#2234)
Represent selected cols when the col-name is enclosed in quotation marks (#2223)
Fix error message on header rule against an empty dataset (#2219)
Fix datasource and connection ui error (#2222)
Druid features
You need to upgrade your Druid according to the Discovery version.
Use druid-metatron-3.2.4.tar.gz from the assets of this release page.
We backported the unified console of Apache Druid.
To run the unified consle, you need to follow this guidance.
Full features
Update easymock to 3.4 (#2162)
Support 한글 token in SQL parser (#2160)
Support router in single mode (#2155)
Fix tasks command at DruidShell (#2143)
Support more result formats and column header at SQL (#2141)
Support full param to supervisor api (#2140)
Support for router forwarding request to active coordinator/overlord (#2139)
Implment SQLMetadataStorageActionHandler (#2138)
Improve task retrieval APIs on Overlord (#2137)
Apply java8 (#2136)
Introduce SystemSchema tables (#2135)
Add Unified web console (#2134)
Hadoop ingestion in ReduceMerge mode does not progress from second index (#2151)
Do not keep index in memory for index viewer (#2121)
Minor log message improvements (#2124)
Remove classpath from logging in MiddleManager (#2123)
Regard U+00A0 as whitespace (#2122)
Fix NPE when dimension spec is not exists (#2117)
Result of hive udaf is not deterministic (#2116)
3.2.3
Discovery Features
Enhancement
Maintain search result list view after moving to detailed view (#1062)
Support s3 storage (#1503)
API Documentation - Common / DataSource mgmt. and Search Query (#1814)
Enhanced presto connection valid check (#1890)
Hide geo-type column from the filtering list. (#1989)
Improve phrase and event in Connection info. view (#1990)
Some unnatural Korean translations (#1996)
Partition validation should always be visible in stagedb ingestion (#2023)
Need a submit button to time filter in dashboard (#2036)
Bug Fix
When joining the data source of the dashboard creation, detailed error history is displayed in the toast popup. (#1231)
Error displaying array type of column on presto connection (#1309)
The user must not be able to move the workbook to a workspace with watcher privileges. (#1850)
Script error when moving to previous page of workbench (#1889)
fix dimension alias script, css error. (#1904)
Chart selection filter error (#1907)
The number of data sources in the workspace UIs are different (#1921)
Improved combo box interaction. (#1925)
Not working "MultiLineString" geometry (#1965)
Some of the sorting functions in the grid do not work (#1966)
[IE] data source icon not visible in dataflow (#2009)
Error checking column similarity between join tables (#2060)
The BBox filter does not work properly when filtering around the corner of the map (#2061)
Druid Features
You need to upgrade your Druid according to the Discovery version.
Cannot find broker address in shell (#2105)
Fix ping-pong on load balancing (#2104)
Add some columns in requestLog parser (#2103)
Quote column name for summary post processor (#2101)
Throw exception early when join column is not exist in the alias (#2093)
Allow multiple hadoop version in distribution (#2039)
Support appending for index task (#2080)
Support type inference for expression function (#2073)
Use natural ordering for non-primitive types instead of exception in SQL (#2072)
Except hive operators from expressions function (#2017)
Register more expression functions to SQL (#2067)
Error checking column similarity between join tables (#2060)
The BBox filter does not work properly when filtering around the corner of the map (#2061)
Fix typo in SpatialOperations (#2062)
Support calling aggregation functions in SQL (#2059)
Add schema query test for geo indexed columns (#2048)
'no lucene index' is thrown when column name contains dot (#2037)
Fix ArrayStoreException when time column is also a dimension (#2035)
Minor refactoring for dictionary compression (#2034)
Covariance query returns empty result (#2027)
Fix CCE on covariance query (#2026)
Add option to view user provided segment file (#2025)
Allow named parameters of shape_buffer to be assigned with normal parameters (#2010)
Support explicit constant bounds for geo.boundary query (#2007)
More JTS-shape functions (#2005)
Support multi-line history in sql mode shell (#2003)
Support (internal) boolean type (#1999)
Support geospatial functions in SQL (#1998)
Fix CCE when rectangle is given as polygon type (#1994)
Follow expression convention for some SQL functions (#1984)
Fix some warnings building sql module (#1963)
Support registering UDFs in Druid (#1957)
Support calling hive UDAFs (#1955)
Add hive-udf-extensions (#1949)
Hadoop index task with REDUCE_MERGE does not need large sort.mb (#1945)
No need to use hash shard spec when numShards = 1 (#1938)
Exception thrown in processing row in index task should not be regarded as parsing fail (#1936)
Possible maximum number of rows exception when multiple-indexing in mapper (#1926)
Remove deprecated methods of guava (#1924)
Minor refactoring (#1919)
Return exact number of rows when using bulk transfer (#1918)
Fix sporadic fail of KMeansQueryRunnerTest (#1916)
Optimize timeseries query with smaller granularity than segment (#1915)
Upgrade jetty from 9.2.5 to 9.3.24 (#1913)
Support remote shutdown (#1908)
Reuse StringBuilder for time formatting functions (#1903)
Support decommission of historical node (#1898)
3.2.2
Discovery Features
Enhancement
#1816 Add progress bar for local file upload on ingestion stage
#1933 Mail template internationalization
#1940 Improved UI performance when column content size is long on ingestion stage
Bug Fix
#1391 SQL Syntax error when selecting 'Overwrite only incremental' in creating DB type datasource
#1517 When workbook is moved to shared workspace, need to check datasource permission
#1882 Can not draw prediction line
#1894 Error when hadoop is not configured
#1901 If two table has same column names filter cannot apply to both table
#1911 Error for loading linked datasource on the druid connection
#1912 Upstream dataset affects creation of hive type snapshot
#1922 Fix access denied when clientId is different
#1928 User-defined field can't put on the shelf
#1948 Incremental ingestion operation failure at presto
#1954 Unnecessary pop-ups when creating datasource via workbench
#1959 Errors that can not be entered into a workbook or workbench in I.E
#1968 #1977 CSS broken on IE 11
#1969 Custom color settings in map charts are initialized when changing other filter conditions.
#1978 "Ingest periodically" is not working properly
#1979 Support additional delimiters on stagingDB ingestion spec
Druid Features
From this release, we'll describe a compatible version of druid and attach the binary in this page.
There is NO druid upgrade for this patch version.
But if you are using an old version of the druid, we highly recommend use attached version.