GPU optimizations, profiling tooling, and test infrastructure

[jxucoder]

Summary

• GPU training ~7x faster: Wire fit_tree_gpu_native into training loop, add fused prediction updates, histogram subtraction, and GPU array caching
• Profiling tooling: ProfilingCallback produces structured JSON reports identifying bottlenecks for self-recursive improvement loops
• Parallel tests: Add pytest-xdist for 1.4x faster test suite (312s → 226s)
• Benchmark restructuring: Replace scattered scripts with focused modules (compare_cpu, compare_gpu, profile_kernels, profile_loop, check_performance, check_scaling)
• Test infrastructure: Add conftest.py with shared fixtures, 7 new test files

GPU Performance Results (A100)

Task	Before	After	Speedup	vs XGBoost
Regression	4.74s	0.65s	7.3x	0.56x
Binary	4.88s	0.73s	6.7x	0.43x
Poisson	4.39s	0.58s	7.6x	0.53x

Profiling Usage

# CLI runner
uv run python benchmarks/profile_loop.py --summarize

# Env var (zero-code-change)
OPENBOOST_PROFILE=1 uv run python train.py

# Explicit callback
profiler = ProfilingCallback(output_dir="logs/")
model.fit(X, y, callbacks=[profiler])

Test plan

• All 509 tests pass with parallel execution (-n auto)
• Profiler produces valid JSON with phase breakdown
• Env-var profiling activation works
• Comparison with previous runs works
• CI passes on ubuntu + macOS matrix

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 37f8edeb31

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Keep NaN/categorical data off the GPU-native builder

When CUDA is enabled and the model uses the default reg_alpha/subsample/colsample settings, this branch now sends every BinnedArray through fit_tree_gpu_native(). Unlike fit_tree(), that builder never reads BinnedArray.has_missing / is_categorical (src/openboost/_core/_tree.py:354-369) and returns the legacy Tree type with no missing_go_left or categorical metadata (src/openboost/_core/_tree.py:200-234). Any GPU run on data containing NaNs or categorical features will therefore silently learn plain ordinal splits instead of the documented missing/category-aware behavior.

Useful? React with 👍 / 👎.

[jxucoder]

Fixed in bda4781. Added has_missing and has_categorical checks to the use_gpu_native gate — falls back to fit_tree() which handles missing values and categorical features correctly.

[chatgpt-codex-connector]

Preserve TreeStructure entries in GPU-trained boosters

On the default CUDA fast path, tree is a legacy _core._tree.Tree, and this line stores it in self.trees_. The sklearn wrappers call compute_feature_importances() immediately after fitting (src/openboost/_models/_sklearn.py:287-289), but _importance._accumulate_importance() expects every tree to expose n_nodes, features, and left_children (src/openboost/_importance.py:156-158). As a result, OpenBoostRegressor.fit() / OpenBoostClassifier.fit() now fail right after a successful GPU training run with the default hyperparameters.

Useful? React with 👍 / 👎.

[jxucoder]

Fixed in bda4781. The GPU-native path now converts the legacy Tree to TreeStructure (with features, left_children, n_nodes, etc.) before appending to trees_, so compute_feature_importances() and sklearn wrappers work correctly.

[chatgpt-codex-connector]

Keep GPU-native trees compatible with PersistenceMixin

This append also breaks persistence for CUDA-trained models, because PersistenceMixin.save() serializes each entry in trees_ through _tree_to_dict() (src/openboost/_persistence.py:45-64), which assumes a TreeStructure with features/thresholds/left_children/.... A legacy Tree from fit_tree_gpu_native() does not expose those fields, so saving a GPU-trained GradientBoosting—or using ModelCheckpoint, which calls model.save() during training—will now raise instead of writing a checkpoint.

Useful? React with 👍 / 👎.

[jxucoder]

Fixed in bda4781 (same commit as above). Since the legacy Tree is converted to TreeStructure before storing, PersistenceMixin.save() / _tree_to_dict() now gets the fields it expects. ModelCheckpoint callback works correctly as well.