paraprof is a robust and MPI-parallelised algorithm for computing profile likelihood projections – or more generally for mapping out low-dimensional projections of a target function by optimizing over the remaining dimensions.
First an initial global optimization identifies promising starting points. These starting points are used to activate initial grid points in the user-selected projection space. At each active grid point the remaining parameters are profiled (optimized) out using differential evolution (DE) and/or L-BFGS-B. If the optimized target value for a grid point meets the user-set threshold, the neighboring grid points are activated. This way the set of optimized grid points dynamically expands to cover the full region of interest, e.g. the high-likelihood region. Information from already converged grid points is communicated to neighboring grid points for faster convergence.
pip install git+https://github.com/anderkve/paraprof.gitOptional extras: pip install -e ".[viz]", ".[dev]", or ".[all]".
Requires Python 3.10+, NumPy, SciPy, and mpi4py (with an MPI implementation like OpenMPI or MPICH). Matplotlib and scikit-learn are optional.
Below are 1D and 2D projections of three 4D test functions used as example log-likelihood functions. Dimensions not shown on the axes are profiled out. The white star marks the best-fit point; the contours are 68% and 95% confidence regions (assuming Wilks' theorem). Plots come from examples/run_showcase_scan.py followed by examples/make_showcase_plots.py.
Himmelblau 4D: 264,617 target-function evaluations across all three projections.
Rosenbrock 4D: 63,904 evaluations.
Levy 4D: 117,757 evaluations. The last dimension uses sin(2π·w₃) rather than sin(π·w + 1), so the (x₀, x₃) projection has denser horizontal ridges than (x₀, x₁).
from mpi4py import MPI
from paraprof import (
ProfileProjector, run_all_projections, terminate_workers, worker_main,
get_test_function,
)
comm = MPI.COMM_WORLD
myrank = comm.Get_rank()
log_likelihood, bounds, _ = get_test_function("himmelblau_4d")
projections = [
{'dims': [0, 1], 'grid_points': [50, 50]},
]
if myrank == 0:
with ProfileProjector(
target_func=log_likelihood,
bounds=bounds,
projections=projections,
roi_threshold=4.0,
pop_per_grid_point=3,
) as sampler:
comm.bcast(sampler.target_func, root=0)
results = run_all_projections(
comm=comm, sampler=sampler, projections=projections,
save_plots=True,
)
terminate_workers(comm, myrank)
else:
worker_main(comm, myrank)Run with mpiexec -n 4 python your_script.py.
More examples live in examples/. The *_advanced.py scripts show the full advanced_config layout.
Rank 0 is the master: it owns the grid, hands out optimization jobs, and tracks which cells are active. Ranks 1+ are stateless workers that evaluate target_func on demand.
A scan proceeds roughly as follows:
- Lay a regular grid over the projection dimensions; the rest are profiled at each grid point.
- Run a few global L-BFGS-B starts to find initial maxima. On later projections these are seeded from an in-memory pool built up by earlier projections, often skipping the global step entirely.
- Anchor a DE population (or an initial point for L-BFGS-B optimization) at each promising cell. One population slot is filled with the highest-fitness past evaluation nearest the cell (proximity warm-start), so later projections inherit useful starting points.
- Optimize the profiled parameters at each active cell. L-BFGS-B cells also reuse the best already-converged neighbour's quasi-Newton history and its best profiled parameters as an alternative start, propagating local curvature outward.
- Activate the neighbours of high-likelihood cells, expanding the active set into the region of interest.
- Patching: re-test each cell with its neighbours' best profiled parameters and polish any improvement.
- Suspect recheck: cells whose profiled parameters look discontinuous compared to their neighbourhood are re-optimized from diverse seed points. This breaks contiguous wrong-optimum strips that the fitness-only filter of the patching step cannot fix.
- Optionally refine the grid: interpolate the coarse grid to warm-start a finer one.
Key code paths: ProfileProjector (sampler.py) holds state and configuration, master_main() (master.py) is the state machine, worker_main() (worker.py) is the evaluation loop, and jobs/ contains the asynchronous optimization jobs.
Common constructor arguments:
roi_threshold— region-of-interest cutoff in log-likelihood; cells withlogL > global_max - roi_thresholdare inside the ROI. Default 3.0.pop_per_grid_point— DE population size per cell. Default 3.n_initial_optimizations— global L-BFGS-B starts before grid optimization. Defaultmin(100, 20 * n_dims).max_patching_waves— cap on patching iterations. Default 10.lbfgsb_max_iter,lbfgsb_polish— L-BFGS-B iteration cap and whether to polish DE results. Defaults 50 andTrue.initial_points— explicit starting points to activate; useful when you already know where the good regions are.use_clustering— detect multiple modes during refinement. DefaultTrue.refinement_direct_eval— skip optimization in the refinement run and just evaluate the interpolated point. DefaultFalse.samples_output_file— CSV path to log every evaluation.warm_start_file— CSV read at the start of each projection to pre-populateinitial_maxima. Set this equal tosamples_output_fileto feed a run's samples into the next one.grad_func— analytic gradient (see below).
By default L-BFGS-B uses finite differences, which costs 2N (central difference) or N (forward difference) extra calls per gradient. If you have an analytic gradient, pass it via grad_func:
def target(p):
return -float(np.sum(p**2)) # log-likelihood, maximized
def grad(p):
return -2.0 * np.asarray(p) # ∇target_func
sampler = ProfileProjector(target_func=target, grad_func=grad, ...)grad_func returns the gradient of the function being maximized; ParaProf negates internally. Getting the sign wrong sends L-BFGS-B uphill.
You can return either a length-n_dims array or a {dim_index: value} dict for partial gradients. Entries that are NaN/±inf (or dims missing from the dict) fall back to finite differences using lbfgsb.gradient_method.
Only the L-BFGS-B paths use the gradient; DE is gradient-free. sampler.target_calls_saved_by_user_gradient and sampler.user_gradient_errors track usage and fallbacks.
Pass an advanced_config dict for the knobs that actually move solution quality or are real iteration budgets:
| Key | Default | What it does |
|---|---|---|
memory_size |
max(grid_sizes) * 25 |
DE F/CR adaptation memory size |
convergence_threshold |
1e-6 |
DE per-cell convergence cutoff |
de.convergence_window |
3 |
Generations of no-improvement before DE declares convergence |
de.num_generations |
100000 |
Hard cap on DE generations |
de.max_num_to_evolve |
None |
Cap on cells evolved per generation |
lbfgsb.ftol |
1e-9 |
L-BFGS-B function tolerance |
lbfgsb.gradient_method |
'forward' |
'forward' or 'central' (~50% more calls) |
clustering.* |
auto-DBSCAN | Mode detection during refinement |
cross_projection.proximity_warm_start |
True |
Swap one LHS seed for the best nearby past evaluation. |
cross_projection.pool_seeded_initial_maxima |
True |
Seed initial_maxima from the pool on later projections and skip the global L-BFGS-B starts. |
suspect_recheck.enabled |
True |
Run the suspect-cell recheck pass after patching. |
suspect_recheck.max_waves |
3 |
Cap on suspect-recheck waves. |
suspect_recheck.param_k |
3.0 |
MAD multiplier for the discontinuity threshold. Lower flags more cells. |
suspect_recheck.max_fraction |
0.25 |
Hard cap on the fraction of ROI cells flagged per wave. |
suspect_recheck.seeds_k_ring |
3 |
Max Chebyshev radius for extended-neighbour seeds. |
suspect_recheck.seeds_from_pool |
3 |
Cross-projection pool seeds tested per suspect cell. |
suspect_recheck.polish_threshold |
1e-4 |
Min logL improvement to trigger the L-BFGS-B polish. |
See the ProfileProjector docstring for the full structure. Several DE knobs that did not change ROI quality in benchmarking (mutation_strategy, pbest_fraction, neighbor_pull_probability, global_pool_size, patching.n_neighbors, activation.mix_ratios) are module-level constants in sampler.py and are intentionally not user-tunable.
Each projection is a dict. Required: dims (parameter indices) and grid_points (resolution per dimension). Optional: optimization_method ('de' or 'lbfgsb', default 'de'), patch_coarse_grid (default True), patch_refined_grid (default False), grid_refinement_factor (integer > 1 to enable refinement), and refinement_method (default 'linear').
With save_plots=True, paraprof writes 1D line plots, 2D heatmaps with contours, and pairwise 2D slices for higher-dimensional projections (either the max slice or marginalized). It also writes plots of the optimal profiled-parameter values across the projection grid, which is useful for seeing what the profiling actually did.
Override the defaults with a plot_settings dict:
plot_settings = {
'dpi': 300,
'filetype': 'png',
'slice_mode': 'max', # or 'all' for marginalization (3D+)
'vmin': -4.0,
'vmax': 0.0,
'plot_profiled_params': True,
}pytest tests/ -v
pytest tests/ -v --cov=src/paraprof --cov-report=term-missingparaprof/
├── src/paraprof/
│ ├── sampler.py # ProfileProjector (state + config)
│ ├── master.py # Master event loop
│ ├── worker.py # Worker event loop
│ ├── jobs/ # Optimization jobs (LBFGSB, DE, activation, patching)
│ ├── interpolation.py # Grid interpolation + clustering for refinement
│ ├── visualization.py # Plot helpers
│ ├── nuisance_wrapper.py # Wrap test functions with nuisance parameters
│ ├── test_functions.py # Himmelblau, Rosenbrock, Rastrigin, etc.
│ ├── exceptions.py
│ └── logger.py
├── tests/
└── examples/ # `*_advanced.py` exercises advanced_config
MIT — see LICENSE.
@software{paraprof2025,
title = {paraprof},
author = {Kvellestad, Anders},
year = {2025},
url = {https://github.com/anderkve/paraprof}
}Maintainer: Anders Kvellestad.