What difference from spark and deltaStreamer? which more efficient？

[poiyyq]

as I know, deltaStreamer is a tool to operate hudi .

If it's just a tool, I just use spark syntax instant of deltaStreamer. right?

[bvaradar]

DeltaStreamer gives you ability to continuously ingest data from upstream sources such as kafka/DFS log files and other hoodie tables. It manages checkpoints as well. Yes, you can also use Spark DataSource for writes where your input batch is a Spark dataset.

[poiyyq]

I can use spark streaming or flink to consume kafka data , then write hoodie table with Spark DataSource, right?

[bvaradar]

Flink support is still in the works. Structured streaming support is already available. Please look out for 0.6.0 release sometime next week where we have added support for running async compaction in structured streaming.