Wanna contribute the nebula jvm version [jdbc | orm | batch insert pure code style| visual |csv&neo4j with import&export ] project for scala and java

[mullerhai]

Hi all:
I want to contribute one nebula sidebar project named nebula-veggie or nebula-curling, this project may have these features like:[jdbc | orm | batch insert pure code style| graph data visual |neo4j record with import&export |csv with import&export] ,how do you think it will have some effect to the nebula ?

[jamieliu1023]

@mullerhai Great proposal! As far as I know, the Nebula team has their own design principles as to what the product should be like. Could you please share more of your design ideas about how you are going to implement these features? Will involve more dev people to discuss what can be done and how it can be implemented. @Nicole00

[wey-gu]

Awsome @mullerhai! Thank you!!!

For graph data visualization, if I understand it correctly, it will be:

• A frontend Application
• A backend application in scale based on your own JDBC/ORM lib in the same project

And regarding neo4j record with import&export |csv with import&export, it will be:
A scala application with interfaces to:

• Connecting neo4j for write/read
• CSV import/export

correct?

ICYMI, In Nebula Community, there are some existing projects:

• Nebula Studio:
  • Web GUI App with nodejs + go(HTTP gateway) backend to enable: Data Visualization(no in 2D only), data import(csv) by wiring nebula-importer
• Nebula Exchange:
  • support 10+ data sources including neo4j importing to the nebula cluster, while, it's spark based

Thanks!!!

[Nicole00]

Great ideas! @mullerhai Can you describe each feature in detail?

And for some features, I wanna know your intention to support this feature in java or scala：

1. what's the meaning of batch insert pure code style, we have several jdks to support the execute statement, and the execute statement can be batch insert ngqls. Besides, we have connector product which support batch insert/update/delete for csv(json, parquet and so on) data. So what's this feature's difference with other products?
2. For data importing and exporting, we have nebula importer and nebula exchange products, which support import neo4j and csv data into Nebula Graph. Are you going to support the import function with pure java (not depend on other frameworks )?
3. For jdbc，there‘s a java jdbc here https://github.com/vesoft-inc/nebula-jdbc , What's the difference between this and your jdbc? And maybe we don't need to implement it with both java and scala, since interfaces of java and scala can call each other.
4. I’m very interested in your orm feature, could you please post your design ideas or architecture here?

[mullerhai]

In my opinion, excellent graph database need support so many features. would you like to take them to the real world. Because of the features too many , I consider to use Chinese to describe them.

1. 支持 多进程 多线程 读取和写入 和读写分离
2. 支持pure code style 的批量创建节点和关系 [ batch insert pure code style ],不必写ngql ，直接调用insert or create node &relation 即可实现插入节点和关系
3. 支持远端批量插入
4. 支持批量的导入导出csv操作
5. 支持离线文件的导入导出操作
6. 支持jvm orm 双向映射
7. 支持jdbc 和grpc
8. 支持远端 json化节点和关系数据插入解析
9. 19.支持十亿级节点的创建和查询
10. 支持udf定制
11. 自研图论算法实现
12. 支持GNN神经网络向量化计算
13. 支持数据可视化
14. 支持可逆的与neo4j相互转换操作
15. 支持和spark Graphx 的集成
16. 支持和elasticSearch 上下游协作
17. 支持GQL Cypher
18. 支持读取 rdf jena protege
19. 支持图谱推理机 sparql
20. 支持 分布式部署
21. 性能稳定 速度够快
22. 支持 程序嵌入式embedded graph database调用和实现
23. 开源 免费 有偿技术支持 社区贡献
24. 添加索引 过滤查询不能性能剧烈下降
25. 在idea 和 zeppelin有相关的DataSource & visual &execute插件

[jamieliu1023]

@mullerhai Thanks for sharing your idea about "what is a great graph database"! Appreciate it. This is a great topic and I think it is worth a new thread for further discussion. :) I am trying to translate your ideas into English so that every community member who drops by can understand what we are talking about. @wey-gu and @Sophie-Xie may help clarify which of the following are already supported, which are on the near-future roadmap of the core team, and which are expected from the community.

1. Support read and write in multi-process and multi-thread; support read and write separation.
2. Support batch insert pure code style to insert or create node and relation, where nGQL is not mandatory
3. Support batch insert from remote
4. Support batch import & export via CSV
5. Support offline file import & export
6. Support jvm orm. There is a pull request pending review: graph ORM nebula-java#349 would appreciate if you are available for code review
7. Support jdbc and grpc, which I think are already available. Here is the jdbc repo: https://github.com/vesoft-inc/nebula-jdbc
8. Support parsing inserts of node and edge in json format from remote
9. Support billion level node creation and query
10. Support UDF. One of the Nebula Hackathon 2021 teams has already made it happen. We would encourage you to also participate in such activities and make what you need and want happen.
11. Support self developed algorithms. Good point. Will definitely develop more graph algorithms and we encourage the community to work together with the core team for public good.
12. Support GNN
13. Support graph data visualization. You may consider Nebula Graph Studio, Nebula Explorer, and Nebula Dashboard.
14. Support convertible operational integration with Neo4j. Hope I understand your point correctly.
15. Support integration with Spark GraphX, which is already available to my understanding.
16. Support integration with ElasticSearch
17. Support GQL and Cypher. GQL is soon to be released and we will definitely embrace the standards because it does good for the entire graph community. As to Cypher, I am sure you already know that Nebula Graph supports the openCypher.
18. Support read rdf jena protege
19. Support sparql
20. Support distributed deployment, which is supported by design. You know Nebula Graph is designed distributed.
21. Stable and fast
22. Support embedded graph database
23. Open source. Free to use. Paid tech support. Community contribution.
24. Support index-filtered query without obvious performance declining
25. idea and zeppelin plugins for DataSource & visual &execute. You may further explain what this is.

Back to your topic, we would appreciate if you can provide some design ideas for further discussion. Thanks!

[Nicole00]

That's great ideas, for so many features, we should categorize them.

1. ORM: this feature is on the way, and we are fortunate received a pr contributed by contributor, we need time to discuss the plan and the framework. If you are interested please join us and discuss it together. graph ORM nebula-java#349
2. Spring-Boot: ** We will open a separate repository, and if you are interested in this feature, you can submit the spring boot related code to this repo**.
3. Import & export: Now we have such tools to support these features.
4. Graph Computing: We have two graph computing product, which are integrated with Graphx and Plato.
5. JDBC: Read-write separation can be encapsulated in jdbc, but maybe it will be more flexible to implement this feature in business layer.
6. Json: Nebula client supports the json execute interface.
7. others maybe the DataBase's feature or the third party‘s feature, such as inference engine, neo4j conversion tool.