Quarter_Release_2017_Q4

DataFibers has more than 40+ changes tasks applied in this release as follows. Now, DataFibers is able seamless submit stream, batch connects and transformations.

• New Web admin UI released using ReactJs based AOR. More powerful and more beautiful.
• Improved Connect Import completeness
• Completed basic demo by using df to build real-time data pipeline for the dashboard in 10 min.
• Certified and tested Hive/JDBC/Stock API connectors
• Added topic data preview especially for data in DataFibers queue
• Added more task/job operation, such as pause and restart, in service and web ui
• Added feature to run batch SQL using Spark and Livy
• Added feature to stream back batch result to Queue
• Default value is provided in the UI for better user experience
• Confluent is upgraded to v3.3.2. Vertx is updated to v3.5.2. Flink is updated to v1.3.2
• Changed Flink job submission from java API to rest API to achieve non-blocking
• Couple of improvement on dfops command as well as sandbox update command

Quarter_Release_2017_Q3

This is DataFibers quarter release for Q3 in 2017. We have more than 20 issues closed.

Release Scope

• Support Flink Table API and SQL API
• Flink upgrade to v1.3
• Merge Topic and Schema together
• New UI Preview
• Adding logging to MongoDB
• New API Message handling is added for better format
• Fix issues in VM setup working enviroment

Monthly_Release_2016_DEC

This is DataFibers monthly release for Dec. 2016, which is the last release in 2016. Happy New Year, the community friends.

Release Scope

In this release, we complete 15+ core tasks covering things as follows.

1. Integrate REST API Document to the DF Application #73
2. Added landing welcome page for DF at localhost:8000
3. Added "processed history" view in UI and API endpoint
4. Added more information in the view "installed connects"
5. Added "avro file, hadoop file" connect in UI connect view
6. Added metadata mongo sink and schema in background of DF
7. Added prototyped metadata generation in generic file connect
8. Certified mongodb connect - sink
9. Update API document to add history API query #72
10. Redesign connect name in POPJ #58
11. Refactory schema processing logic #74
12. Add -- debug or -d to CLI for DF jar launch and default logging as INFO #70
13. Shell Scripts enhancement for start DF Env #59
14. Bug Fix - When there is no schema, the API returns exception to UI #62
15. Bug Fix - Null issues in certified connect - generic file connect #63
16. Bug Fix - Schema View has schema update issue #55

Other Activities

Started two regular team meetings
Assign the tasks for coming demo milestone/release.
@schubert Zhu @will, thanks for the contribution

Monthly_Release_2016_NOV

This is DataFibers monthly release for Nov. 2016. In this release, we complete 10+ tasks covering things as follows.

• Rebuilt feature to get Flink Job id from blocking streaming submission #36
• Created Flink Avro source and customized stream client #27
• Added Flink SQL transform avro to json #19
• Added Flink UDF transform #16
• Added data schema view #25 #6
• Bug fix on service and UI

For other activity

• Started two regular team meetings
• Assign the tasks for coming demo milestone/release.

@schubert Zhu @will, thanks for the contribution