Skip to content

Add Helion Kernel Challenge competition#105

Merged
msaroufim merged 5 commits intomainfrom
helion-competition
Mar 4, 2026
Merged

Add Helion Kernel Challenge competition#105
msaroufim merged 5 commits intomainfrom
helion-competition

Conversation

@msaroufim
Copy link
Copy Markdown
Member

@msaroufim msaroufim commented Feb 26, 2026

No description provided.

New competition inspired by Helion kernel ideas covering attention mechanisms,
sampling strategies, quantization, and sequence modeling operators from
production LLM architectures.

Problems:
- GQA: Causal Grouped Query Attention (Llama 3 style)
- MLA: Multi-Head Latent Attention decode (DeepSeek-V2/V3)
- KDA: Kimi Delta Attention (linear attention + delta rule)
- Causal Conv1d: Causal depthwise 1D convolution (Mamba)
- FP8 Quant: Per-token-group FP8 E4M3 quantization
- INT8 Quant: Per-token INT8 symmetric quantization
- Min-P: Adaptive probability threshold sampling
- Top-K: Top-k sampling via binary search
- Top-P: Nucleus sampling via binary search

Deadline: March 14, 2026 midnight PST
@msaroufim msaroufim changed the title Add Helion Kernel Challenge competition with 9 problems Add Helion Kernel Challenge competition Feb 26, 2026
@msaroufim msaroufim closed this Feb 26, 2026
@msaroufim msaroufim reopened this Mar 4, 2026
The regex only matched [a-zA-Z]+ for keys, which broke parameters
like group_size, hidden_dim, num_tokens, and use_initial_state.
Also adds true/false boolean value parsing.
@msaroufim
Copy link
Copy Markdown
Member Author

Alright this is working!

Last thing we need to do now is make sure we can get hooked up to the Nebius runners, Modal worked great

  One-time setup:
  # DB + migrations
  createdb kernelbot
  DATABASE_URL="postgresql://$(whoami)@localhost:5432/kernelbot" uv run yoyo apply --database "$DATABASE_URL"
  src/migrations/

  # Test user
  psql "$DATABASE_URL" -c "INSERT INTO leaderboard.user_info (id, user_name, cli_id, cli_valid) VALUES ('999999',
  'testuser', 'test-cli-id-123', true) ON CONFLICT (id) DO UPDATE SET cli_id = 'test-cli-id-123', cli_valid = true;"

  Start server:
  cd ~/Dev/kernelbot/src/kernelbot
  modal profile activate gpu-mode
  DATABASE_URL="postgresql://$(whoami)@localhost:5432/kernelbot" DISABLE_SSL=true ADMIN_TOKEN=<token> GITHUB_TOKEN=dummy
  GITHUB_REPO=gpu-mode/reference-kernels .venv/bin/python main.py --api-only

  Sync problems:
  curl -X POST http://localhost:8000/admin/update-problems \
    -H "Authorization: Bearer <token>" -H "Content-Type: application/json" \
    -d '{"problem_set": "helion", "branch": "helion-competition"}'

  # Update GPU types from NVIDIA (GitHub runner) to B200 (Modal)
  psql -d kernelbot -c "UPDATE leaderboard.gpu_type SET gpu_type = 'B200' WHERE leaderboard_id IN (SELECT id FROM
  leaderboard.leaderboard WHERE name IN ('causal_conv1d','fp8_quant','gated_deltanet_chunk_fwd_h','gated_deltanet_chunk_fw
  d_o','gated_deltanet_recompute_w_u'));"

  Submit:
  curl -N -X POST http://localhost:8000/<leaderboard>/B200/<test|benchmark> \
    -H "X-Popcorn-Cli-Id: test-cli-id-123" \
    -F "file=@path/to/submission.py"

@msaroufim msaroufim merged commit facc675 into main Mar 4, 2026
@msaroufim
Copy link
Copy Markdown
Member Author

Screenshot 2026-03-03 at 5 54 16 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants