Skip to content

v1.0.9 - Fix Spot VM Termination Issue

Choose a tag to compare

View all tags
@deeco deeco released this 20 Jun 10:54
· 14 commits to main since this release
v1.0.9
8261018

What's Changed

🐛 Bug Fixes

  • Fixed Spot VM termination: Updated cluster configuration to use On-Demand instances for driver
  • Improved cluster stability: Prevents driver loss due to Spot VM reclamation by AWS

📋 Technical Details

  • Added to AWS attributes in cluster configuration
  • Added for automatic availability zone selection
  • Maintains existing EBS volume configuration and Spark version 16.3

💡 Why This Fixes the Issue

Spot instances can be reclaimed by AWS at any time, causing the Spark driver to be lost. By using On-Demand instances for the driver, we ensure stable cluster operation and prevent unexpected failures.

This release should resolve the 'Spark Driver running on Spot VM was lost' error.

Assets 2
Loading