What's the difference between Trino and spark?

[zcw5116]

The latest version 3.0 of spark is also very fast and stable. According to the official meeting materials, some scenes are faster than presto. According to the third-party test, Presto is faster than spark, but not much faster. In particular, the performance of the new version of spark has been improved significantly, and spark supports a lot of data sources, and the expansion of data sources is much simpler than Trino.

What's the difference between Trino and spark?

What is the future orientation of Trino?

I've been troubled by this problem for a long time,

[findinpath]

Here is a comparison between Presto/Trino & Spark

https://www.concurrencylabs.com/blog/starburst-enterprise-vs-aws-emr-sql-tpcds/

[hashhar]

Another big difference is the breadth of connectors Trino has (see the list https://trino.io/docs/current/connector.html).

Also Trino can push-down many operations to the underlying system to reduce data movement. For example if you use a JDBC source to create a data-frame in Spark it cannot push-down things like JOINs, aggregation functions etc. Spark pulls the entire raw data and then performs operations while in contrast Trino will pushdown whatever operations it can (without sacrificing correctness) and do the remaining itself.

Spark is more suited to long-lived batch processes (or Spark streaming if you are into it). Trino serves the ad-hoc and interactive use-cases much better than Spark and can also be used for the batch processes.

Also, Trino supports ANSI SQL instead of it's own bespoke dialect of SQL (Spark has SparkSQL) which means it's behaviour and functions are what you'd expect in other database systems and it's easier for new people to get started with.

[hashhar]

Feel free to join our Slack and continue the conversation there.

There are more people there who use both Spark + Trino or have been where you are making a decision to choose what to use. So they can share their inputs and findings too.