Skip to content
de03025
Compare
Choose a tag to compare

bugfix

  • Fixed the bug that the nebula connection will be interrupted if the YIELD clause in LOOKUP contains aggregate function. #3245

download:

3ba41bd
Compare
Choose a tag to compare

feature

  • Support TOSS. #2525
  • Support ZONE. #2604
  • Support Geo Spatial. #2954 #2979 #3043
  • Support crypto in transportation. #2584
  • Support return query result as JSON format. #2824
  • Support to clone space meta. #2763
  • Support lookup index scan using IN expression as filter. #2906
  • Integration Breakpad. #2536
  • Support copying the local folder of metad to remote. #2532
  • Support DELETE TAG. #2520
  • Support concat function. #2540
  • Support SHOW META LEADER. #2542

enhancement

  • Optimize the computation of index scan limit pushdown. #2905 #2823 #2796
  • Optimize the calculation of sampling and limit pushdown at each step of the go statement. #2904 #2853 #2831
  • Better yield data format. #2555 #2572 #2779 #2895 #2944
  • Enable prefix bloom filter by default to improve performance. #2860
  • Support server to verify client version to make sure the connection reliability(client version start from v2.6.0). #2965
  • Optimize flow control when pulling the entire partition. #2557
  • SHOW JOBS only show space related. #2872
  • Grant job permission for all roles except GUEST. #2928
  • Improve memory watermark detection. #2885
  • Support to kill the slow queries of storage. #2534

bugfix

  • Fixed the bug that clean part rocksdb data when raftpart::reset. #2522
  • Fixed the bug which insert mismatched date time type. #2527
  • Fixed the bug that setting millisecond failed but microsecond worked. #2781
  • Fixed the meta service crash when inserting too much data in batches(millions lines). #2813
  • Fixed the crash that getting edge when no edge schema exist in space. #2571
  • Fixed the bug for that GO with WHERE clause expression eval when prop data type is fixed_string. #2762
  • Fixed the bug when find all path. #2773
  • Fixed the bug that users without roles have the permission to find all roles of SPACE. #2778
  • Fixed the bug of case expression. #2819
  • Fixed the infinite loop when use time(). #2820
  • Fixed the bug while task node shutdown, this job will display "running" forever. #2843
  • Fixed the bug insert statements may cause inconsistent attribute values between replicas in the case of multiple replicas. #2862
  • Fixed the bug that space not right when submit job after USE. #3010
  • Fixed the bug that getting attribute error of thrift struct when column is not nullable. #3011
  • Fixed the bug that graphd is always running even if meta is not ready. #3048
  • Fixed the bug that dangling edge will return a null vertex when use FIND PATH WITH PROP. #3008
  • Fixed the crash when YIELD DISTINCT map value. #3051
  • Fixed the bug that the service still starts with a wrong ip/host. #3057
  • Fixed the bug that alter the same property in one statement. #3036
  • Fixed the bug that the multi-step filtering on edge is invalid. #3144

download:

721ae51
Compare
Choose a tag to compare

New Features

  • YIELD clause supports DISTINCT  #2368
  •  Add lookup_concurrently option. LOOKUP supports traversing all partitions concurrently, which improves the performance significantly  #2417
  • LOOKUP clause supports DISTINCTLOOKUP ON supports returning srcid and dstid of an edge  #2415

Bugfix

  • Fix the issue that YIELD -1*1 may result in an overflow  #2411
  • Fix the issue that leader election may be unstable when RPC times out #2419
  • Fix UnaryExpression bug which causes storaged crash  #2412
  • Fix the issue LIMIT and OFFSET do not return the expected result #2428
  • Fix the issue that partial results may return after the leader change. #2465
  • Fix the issue that executing FETCH PROP ON *  may lead to graphd crash after altering schema. #2461

Changelog

53f56b6
Compare
Choose a tag to compare

New Features

  • FIND PATH support to find paths with or without regard to direction. And also support to exclude loops in paths. #2398
  • Add ARM64 support to Nebula Graph. #2392
  • Add auto_remove_invalid_space option to indicate whether to remove data from a deleted graph space when restarting the services. #2375 #196

Improvements

  • Improve the DELETE edges performance. #2404

Bugfix

  • Fix the issue that FETCH PROP ON timestamp properties returns an int64 value. #2389
  • Fix the issue that GO m TO n STEPS may lead to OOM errors. #2386
  • Fix the issue that FETCH PROP ON * might return the wrong results when the tag on the vertex has no properties. #2359
9154dde
Compare
Choose a tag to compare

Improvements

  • Remove the lock inside FunctionManager for better performance. #2273
  • Improve property-fetching performance by refactoring VertexHolder::getDefaultProp.#2249
  • Optimize GO statement performance by adopting unordered_map::reserve and vector::reserve, and removing storage query lock. #2268
  • Reduce the unnecessary creation of snapshots and optimize performance. #2287 #2318
  • Support configuring the prefix bloom filter for RocksDB. The prefix blooms can optimize QueryEdgeProps and QueryVertexProps performance, as well as reduce memory usage. #2274
  • Support configuring compaction_thread_limiter and rate_limiter options of RocksDB. #2251
  • Support obtaining RocksDB statistics through web service. #2262 #2243
  • Support more dynamically configurable options of RocksDB, such as max_open_files, memtable_huge_page_size. #2291
  • FETCH PROP ON supports fetching multiple tag properties on multiple vertexes. The previous command’s output can be passed to FETCH PROP ON as its input in the pipeline statement ("|"). FETCH PROP ON * supports getting the properties of multiple vertices. #2222 #117
  • Add a warning message for a partially successful query. #2290
  • Raft lease is always valid when replica_factor is set to 1. #2276

Bugfixes

  • Fix the leader election failure that may occur during the balancing process. #2232
  • Fix the issue that there may be more than one election request after the RAFT election failed. #2305
  • Fix the issue that the logging of the original service can't be recorded to the log file when the service was started multiple times.  #2278 #2289
  • Fix the issue that StorageClient scanVertex and scanEdge interfaces failed to return the results after enable_multi_versions  is set to true. #2312
  • Fix the issue that USE SPACE and FIND PATH statements cannot work together in a query. #2303
  • Fix the issue that snapshots may be triggered unexpectedly. #2316
  • Fix the issue that storaged may crash when a vertex with related edges and index is deleted. #2335
  • Fix the issue that after  ALTER TAG/EDGE ADD <props>, LOOKUP ON the newly added properties may make graphd crash. #2341
  • Fix the stale data problem when using LRUCache. #2267
  • Fix the bug that meta service cannot be stopped in some cases. #2240
  • Fix the issue that MetaClientTest.HeartbeatTest may be hung if the target port has already been used. #2296
  • Fix the bug when stopping metad  before the initialization of job manager, metad may crash. #2332
  • Fix the bug that multiple indexes can exist for the same property set. #1695

Changelog

  • Forbid range access for string-property indexes. For LOOKUP ON xxx WHERE xxx, if there are string properties in the WHERE clause, the exact-match index would be used to look up records. #2283 #2277
  • Forbid OR and XOR conditions in LOOKUP ON statement. #2283
  • The pipe (|) operation has deduped the input. #2292
  • The default value for rocksdb_disable_wal option has been changed to false. #2236
3132e39
Compare
Choose a tag to compare

New Features

  • Support uuid() in FIND PATH statement #2209.
  • Support disabling block cache of rocksdb by setting rocksdb_block_cache of storaged.conf to less than or equal to 0 #2177
  • Adopt folly::SingletonThreadLocal in meta client to avoid multi-threads mitigating lock contention on localCacheLock_, which improves the performance significantly. #2165
  • Storage supports different compression algorithms of rocksdb and setting different compressions for different levels. #2179

Bugfix

  • Fix the issue that compaction filter does not work when calls manual compaction #2184
  • Fix the issue that Leader sends Snapshot in some cases #2150
  • Fix the issue when UPSERT/UPDATE the new prop of existing data after adding a new prop of tag/edgetype, an error return #2186
  • Fix the issue that GO m ... n returns an error when m is set to 0 #2202
  • Fix the issue when combining GO n STEPS and | pipe statement,an incomplete result is returned in some cases #2203
06a5db4
Compare
Choose a tag to compare

Basic Features

  • Online DDL & DML. Support updating schemas and data without stopping or affecting your ongoing operations.
  • Graph traversal. GO statement supports forward/reverse and bidirectional graph traversal. GO minHops TO maxHops is supported to get variable hops relationships.
  • Aggregate. Support aggregation functions such as GROUP BY, ORDER BY, and LIMIT.
  • Composite query. Support composite clauses: UNION, UNION DISTINCT, INTERSECT, and MINUS.
  • PIPE statements. The result yielded from the previous statement could be piped to the next statement as input.
  • Use defined variables. Support user-defined variables to pass the result of a query to another.
  • Index. Both the single-property index and composite index are supported to make searches of related data more efficient. LOOKUP ON statement is to query on the index.

Advanced Features

  • Privilege Management. Support user authentication and role-based access control. Nebula Graph can easily integrate with third-party authentication systems. There are five built-in roles in Nebula Graph: GOD, ADMIN, DBA, USER, and GUEST. Each role has its corresponding privileges.
  • Support Reservoir Sampling, which will retrieve k elements randomly for the sampling of the supernode at the complexity of O(n).
  • Cluster snapshot. Support creating snapshots for the cluster as an online backup strategy.
  • TTL. Support TTL to expire items after a certain amount of time automatically.
  • Operation & Maintenance
    • Scale in/out. Support online scale in/out and load balance for storage
    • HOSTS clause to manage storage hosts
    • CONFIGS clause to manage configuration options
  • Job Manager & Scheduler. A tool for job managing and scheduling. Currently, COMPACT and FLUSH jobs are supported.
  • Graph Algorithms. Support finding the full path and the shortest path between vertices.
  • Provide OLAP interfaces to integrate with third-party graph analytics platforms.
  • Support multiple character sets and collations. The default CHARSET and COLLATE are utf8 and utf8_bin.

Clients

  • Java Client. Support source code building and downloading from the MVN repository, see Java Client for more details.
  • Python Client. Support source code building and installation with pip, see Python Client for more details.
  • Golang Client. Install the client with the command go get -u -v github.com/vesoft-inc/nebula-go,see Go Client for more details.

Nebula Graph Studio

A graphical user interface for working with Nebula Graph. Support querying, designing schema, data loading, and graph exploring. See Nebula Graph Studio for more details.

Tools

  • Data Import
    • Nebula-Importer is used to import data from the CSV file.
    • Spark Writer, a Spark-based distributed data import tool that converts data from multiple data resources into the Nebula Graph.
  • Data Export
    • Dump Tool. A single-machine off-line data dumping tool to dump or count data with specified conditions.
  • Monitoring
    • Nebula Stats Exporter (for Prometheus), is to collect database metrics and expose metrics to Prometheus. And Grafana has integrated for metrics Visualization and Alerting.
d7422b6
Compare
Choose a tag to compare
Pre-release

!! Note:RC4 is not compatible with the Data inserted before commitID 43453a0 (2020.02.06) due to the change of the underlying data structure.

New features

  • Support Index to make searches of related data more efficient. An index on a property/properties combination can be created with CREATE INDEX. DROP INDEX is to drop an index #1776. REBUILD INDEX is to reindex the data #1566.

  • Support LOOKUP ON to query on index #1705. See #1738 for the performance of Storage Engine when inserting data with an index.

  • Introduces account management and access control #1842, #1873. All users may perform only the operations permitted to them. Roles provided by Nebula Graph and their privileges please refer to #1929. Add --enable_authorize=true to nebula-graphd.conf and restart the services to enable authentication.

  • Support TTL to remove items after a certain amount of time automatically #1584#422, #1934.

  • Enhance DELETE VERTEX to support deleting a batch of vertices. And supports hash() and uuid() functions for VertexID #1317, #1759.

  • Introduces Job Manager, to manage the jobs that take a long time of Storage Engine. At present, it supports flush and compact operations. SUBMIT JOB is to submit a job, STOP JOB stops the running jobs,SHOW JOB shows the details info of a job, RECOVER JOB is to put back the failed job to the queue #1424.

  • Support BIDIRECT for GO query to traverse along with both ingoing and outgoing directions #1740, #1752.

  • Support Reservoir Sampling, set enable_reservoir_sampling to TRUE is to turn on the sampling. Max_edge_returned_per_vertex in nebula-storage.conf is to configure the number of returned edges #1746, #1915.

  • Support more character sets and collations. SHOW CHARSET and SHOW COLLATION statement show all available character sets and collations. It can be configured when creating the space. The default CHARSET is utf8, and the corresponding COLLATE is utf8_bin #1709.

OLAP Interface

Tools

  • Support Deploying Nebula Graph on Kubernetes with Helm #1473
  • Introduce Nebula Stats Exporter (for Prometheus) to collect database metrics and expose metrics to Prometheus. And Grafana has integrated for metrics Visualization and Alerting.

Change

  • RC4 is not compatible with the Data inserted before commitID 43453a0 (2020.02.06) due to the change of the underlying data structure.
  • DOC move to repo vesoft-inc/nebula-doc
  • For Nebula Python Client, is_async is not supported when creating ConnectionPool. Async client will be introduced later.
  • _src in REVERSELY and BIDIRECT statements represent the queried node instead of the src of the edge. While _dst refers to the other connected nodes. e.g. GO FROM <vid> OVER <edgetype> YIELD edgetype._src returns the vid. #1740
48569fa
Compare
Choose a tag to compare
Pre-release

Query engine

  • Support fetching all props of a given vertex #1486
  • Support DELETE EDGE to delete the given edge #1063
  • Add IF EXISTS to conditionally drop a tag/edgetype only if it exists. #1505
  • Add IF NOT EXISTS to conditionally create a space/tag/edge only if it does not exist #1379
  • Export graphd metrics #1451

Storage

  • Add scan edge/vertex interface to retrieve data from storage for OLAP, #1381
  • Support heartbeat_interval_secs option to config heartbeat interval between storage/graph and meta #1540
  • Pushdown filter to minimize data transfer and improve query performance #947
  • Support local conf mode, using local conf rather than config in meta server #1411
  • Add timeout for storage/meta clients, the default value is 60s and configured by meta_client_timeout_ms option #1399
  • Support creating a snapshot for the whole cluster #1199 #1372
  • Both support reading from leader/follow and support only read from the leader #1363
  • Add check step for each balance task during the balancing process. #1378

Build

  • Simplify the build process, support most of Linux Kernel 2.6.32+ system #1332
    Index
  • Support create an index, get a list of all indexes in the space and drop an index #1459 #1360

Tools

  • dump_tools, an off-line data dumping tool that can be used to dump or count data with specified conditions. #1479 #1554
  • Spark Writer adopts async client,add Hash and UUID support,support load data into the same schema from different data source #1405 #1512. It also supports configuring Spark partition #1412.

UI

  • Nebula Graph Studio is the graphical user interface for working with Nebula. Query, visualize nebula and import CSV data to Nebula. Visualize and explore graph data via the interface; an editor with syntax highlighting feature enables users to design queries fast and view query results in a structured manner; support data import via GUI. Here's the repo of this tool (including documentation and deployment files): https://github.com/vesoft-inc/nebula-web-docker
Compare
Choose a tag to compare
Pre-release

Query Engine

  • Support GROUP BY to group items that have the same values,often used with aggregate functions (COUNT, MAX, MIN, SUM, AVG, etc.) to group the result-set by one or more property field. #749
  • UPDATE CONFIGS supports expression #1273
  • Support changing log severity level and verbose level from console #1273
  • Support IF... RETURN... to return the result if a specified condition is true #1233 #1246
  • String properties could have unlimited length #1103
  • Support GO FROM ... REVERSELY to implement reverse query #1349
  • When insert vertices and edges, users can specify properties order instead of following the schema defined order #1219

Storage

  • Support SHOW PARTS to fetch the partition information of current space #1086
  • Support BALANCE STOP to stop load balance process #1238
  • Support triggering the compaction and flush manually #677 #1240
  • Change BlockBasedTable default block cache size from 4MB to 1024MB #1248
  • Add max_edge_returned_per_vertex gflags option to limit the return size of supper vertex #1221
  • Add cache for vertex #1294 #1268
  • Upgrade dependencies folly and fbthrift  #1161

Tools

  • Enhance CSV Importer, user can import CSV using docker image. See Nebula Importer for more details.

Client

Changed/Removed

  • Remove commands ADD/REMOVE HOSTS, the hosts could be discovered automatically #1172  

Coming soon

  • See RoadMap for the features coming soon