Add README note about project status and recommended alternative

[nchammas]

spark-ec2 is no longer under active development. This has been the reality for about a year or so, but I have been asked by @shivaram to submit a PR making this official by adding a note to the README and redirecting people to Flintrock, a project I created as a sort of spiritual successor to spark-ec2.

I think @shivaram can provide a proper post-mortem on his end for why spark-ec2 development has effectively stalled. From my perspective, I believe the main reasons are:

• spark-ec2's dependence on custom AMIs makes it difficult for the project to keep up pace with Spark itself. This is a problem I tried unsuccessfully to tackle for spark-ec2 years ago and have discussed here as well.
• Major improvements to spark-ec2's user experience (launch speed, persistent configs) require a lot of review time, and the core people on this project (mainly @shivaram these days, myself and others to a lesser degree) don't have the bandwidth to keep up.

Flintrock addresses these key issues -- they were, in fact, the main motivation for the project -- and I'd be happy to help onboard users who decide to switch from spark-ec2 to Flintrock.

[shivaram]

+1 - Thanks @nchammas for the PR. I think the need for custom AMIs and tcode changes for every Spark release / EC2 instance release results in significant burden in terms of maintenance for spark-ec2. I think Flintrock's architecture avoids some of these issues and there are also more mature commercial tools like EMR Spark or Databricks for more advanced use cases.

I'll leave this PR open for a couple of days to see if there are any other comments.

[nchammas]

I should probably push additional commits to this branch to add PR and Issue GitHub templates so that when people file new issues or open new PRs they are told that this project is inactive and are redirected to the README.

[shivaram]

That would be great @nchammas -- Thanks

[shivaram]

LGTM. Thanks -- Merging this.

[nchammas]

@shivaram - Looks like GitHub recently added the ability to archive repositories. That will obviously work much better than the PR and Issue templates I added here.

Perhaps it's a good idea to make a final sweep through any open issues and PRs to leave comments where appropriate and then archive this repo.

[shivaram]

Thanks for the pointer. I will take a look at it.

[shivaram]

@nchammas Given that the oldest PRs are more than 6 months old I'm not sure its useful to go and ping them again. I think archiving just makes the PRs and issues read-only which should be fine ?

[nchammas]

@shivaram - That's fair. I was thinking of the more recent issues and PRs where the authors might still be expecting a response, but I suppose they will find out sooner or later when they check and see that the repo was archived.