Datetime subtraction

[brookewenig]

I can't subtract two datetimes from each other. I can do this in pandas without issues. See this notebook: https://databricks-prod-cloudfront.cloud.databricks.com/public/793177bc53e528530b06c78a4fa0e086/0/12638605/100107/latest.html

Tony would like to show off this feature in his blog.

[rxin]

What's the return type of date subtraction and timestamp subtraction for Pandas?

In Spark there is a datediff function and perhaps we can use that to do date subtraction. It'd return the number of days.

There is also to_unix_timestamp that turns a timestamp into the number of seconds and we can use that to compute the diff (in secs) between two timestamp values.