Skip to content

Conversation

@nabuskey
Copy link
Collaborator

@nabuskey nabuskey commented Jul 1, 2025

🔄 Pull Request

This makes tasks that should run in in FG interactive and be able to see outputs.
Right now, task results are not visible.

$ task start-spark
task: [start-spark] ./start_local_spark_history.sh
🚀 Starting Local Spark History Server for MCP Testing
=======================================================
🔍 Checking prerequisites...
🛑 Stopping any existing Spark History Server containers...
   No existing container found

📊 Available Test Applications:
===============================
📋 spark-110be3a8424d4a2789cb88134418217b (488K)
📋 spark-bcec39f6201b42b9925124595baad260 (108K)
📋 spark-cc4d115f011443d787f03a71a476a745 (656K)

📁 Event directories found:
   eventlog_v2_spark-110be3a8424d4a2789cb88134418217b
   eventlog_v2_spark-bcec39f6201b42b9925124595baad260
   eventlog_v2_spark-cc4d115f011443d787f03a71a476a745

📋 Configuration:
   Log Directory: spark.history.fs.logDirectory=/mnt/data/events
   Port: 18080
   Docker Image: apache/spark:3.5.5

🚀 Starting Spark History Server...
📍 Will be available at: http://localhost:18080
📍 Web UI: http://localhost:18080
📍 API: http://localhost:18080/api/v1/

⚠️  Keep this terminal open - Press Ctrl+C to stop the server
⚠️  It may take 30-60 seconds for the server to fully start

🐳 Starting Docker container...
Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties

Signed-off-by: Manabu McCloskey <manabu.mccloskey@gmail.com>
Copy link
Collaborator

@vara-bonthu vara-bonthu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@vara-bonthu vara-bonthu merged commit fb6dd8e into main Jul 1, 2025
5 checks passed
@nabuskey nabuskey deleted the int-tasks branch July 1, 2025 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants