Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load balancer #807

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Conversation

PhilipDeegan
Copy link
Member

@PhilipDeegan PhilipDeegan commented Jan 16, 2024

Summary by CodeRabbit

  • New Features

    • Added C language support to the project alongside CXX.
    • Introduced new functions for floating-point comparison and particle data handling.
    • Implemented load balancing strategies and estimations for simulations.
    • Enhanced logging with specific log levels for better categorization.
    • Added a RunTimer utility for performance measurement.
  • Bug Fixes

    • Corrected a typo in diagnostic comments.
  • Improvements

    • Improved error handling in simulation methods.
    • Refined initialization logic for regridding scenarios.
    • Updated logging and load balancing operations for better performance.
  • Tests

    • Added new test scripts for load balancing and Harris current sheet simulations.
    • Introduced MPI tests for load balancing scenarios.
  • Chores

    • Updated CMake configurations to include new files and dependencies.
    • Added new template functions and utility methods for improved code functionality.

@PhilipDeegan PhilipDeegan marked this pull request as draft January 16, 2024 21:06
Copy link

coderabbitai bot commented Jan 16, 2024

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

Commits Files that changed from the base of the PR and between 0f2686c and 4f501ca.
Files selected for processing (43)
  • pyphare/pyphare/pharein/init.py (5 hunks)
  • pyphare/pyphare/pharein/load_balancer.py (1 hunks)
  • pyphare/pyphare/pharein/maxwellian_fluid_model.py (2 hunks)
  • pyphare/pyphare/pharein/simulation.py (5 hunks)
  • pyphare/pyphare/pharesee/hierarchy.py (6 hunks)
  • pyphare/pyphare/pharesee/particles.py (1 hunks)
  • pyphare/pyphare/simulator/simulator.py (5 hunks)
  • pyphare/pyphare_tests/test_pharesee/test_geometry_3d.py (1 hunks)
  • res/cmake/def.cmake (4 hunks)
  • res/cmake/dep/samrai.cmake (1 hunks)
  • src/amr/CMakeLists.txt (2 hunks)
  • src/amr/data/particles/particles_data.hpp (1 hunks)
  • src/amr/level_initializer/hybrid_level_initializer.hpp (2 hunks)
  • src/amr/load_balancing/concrete_load_balancer_hybrid_strategy_homogeneous.hpp (1 hunks)
  • src/amr/load_balancing/concrete_load_balancer_hybrid_strategy_nppc.hpp (1 hunks)
  • src/amr/load_balancing/load_balancer_details.hpp (1 hunks)
  • src/amr/load_balancing/load_balancer_estimator.hpp (1 hunks)
  • src/amr/load_balancing/load_balancer_estimator_hybrid.hpp (1 hunks)
  • src/amr/load_balancing/load_balancer_hybrid_strategy.hpp (1 hunks)
  • src/amr/load_balancing/load_balancer_hybrid_strategy_factory.hpp (1 hunks)
  • src/amr/load_balancing/load_balancer_manager.hpp (1 hunks)
  • src/amr/messengers/hybrid_hybrid_messenger_strategy.hpp (11 hunks)
  • src/amr/multiphysics_integrator.hpp (13 hunks)
  • src/amr/wrappers/integrator.hpp (4 hunks)
  • src/core/data/ions/particle_initializers/maxwellian_particle_initializer.hpp (6 hunks)
  • src/core/data/ions/particle_initializers/particle_initializer_factory.hpp (2 hunks)
  • src/core/data/ndarray/ndarray_vector.hpp (1 hunks)
  • src/core/data/particles/particle_array.hpp (1 hunks)
  • src/core/utilities/box/box.hpp (1 hunks)
  • src/hdf5/detail/h5/h5_file.hpp (2 hunks)
  • src/initializer/data_provider.hpp (1 hunks)
  • src/initializer/dictator.cpp (1 hunks)
  • src/simulator/simulator.hpp (6 hunks)
  • tests/core/data/ions/test_ions.cpp (1 hunks)
  • tests/diagnostic/init.py (1 hunks)
  • tests/functional/harris/harris_2d_lb.py (1 hunks)
  • tests/simulator/CMakeLists.txt (1 hunks)
  • tests/simulator/init.py (2 hunks)
  • tests/simulator/test_load_balancing.py (1 hunks)
  • tests/simulator/test_tagging.py.off (1 hunks)
  • tools/config/config.py (1 hunks)
  • tools/python3/init.py (2 hunks)
  • tools/python3/caliper.py (1 hunks)
 ___________________________________
< Colonel Panic is my arch-nemesis. >
 -----------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ

Walkthrough

The update brings significant enhancements to the PyPHARE project, introducing new functionalities and optimizations across various modules. Key changes include integrating load balancing strategies, improving logging and diagnostics, and adding new simulation tests. These modifications enhance project robustness, performance, and flexibility, particularly in handling adaptive mesh refinement (AMR) and floating-point precision comparisons. The updates extend the project's capabilities by supporting both C and C++ languages.

Changes

File Path Change Summary
CMakeLists.txt Added C language support to the project.
pyphare/pyphare/core/phare_utilities.py
pyphare/pypharein/init.py
pyphare/pypharein/diagnostics.py
Introduced new functions and classes for floating-point comparisons, load balancing, and diagnostics.
pyphare/pypharein/load_balancer.py
pyphare/pypharein/maxwellian_fluid_model.py
pyphare/pypharein/simulation.py
Added load balancing management, population adjustments, and simulation enhancements.
pyphare/pypharesee/hierarchy.py
pyphare/pypharesee/particles.py
pyphare/pypharesee/run.py
Implemented equality comparisons, particle data aggregation, and simulation configuration methods.
res/cmake/dep.cmake
res/cmake/dep/caliper.cmake
res/cmake/dep/cppdict.cmake
Updated CMake configurations and added instructions for Caliper integration.
src/amr/CMakeLists.txt
src/amr/data/particles/particles_data.hpp
src/amr/level_initializer/hybrid_level_initializer.hpp
Introduced new files and updated logging scopes for AMR functionalities.
src/amr/load_balancing/...
src/amr/messengers/hybrid_hybrid_messenger_strategy.hpp
src/amr/multiphysics_integrator.hpp
Added load balancing strategies, messaging logic, and enhanced integration for AMR simulations.
src/core/CMakeLists.txt
src/core/data/grid/gridlayout.hpp
src/core/data/ions/...
Included new files, updated loop declarations, and added density cutoff parameters.
src/core/data/ndarray/ndarray_vector.hpp
src/core/data/particles/particle_array.hpp
src/core/def.hpp
Introduced template functions, particle array methods, and macro definitions.
src/core/logger.hpp
src/core/numerics/...
src/core/utilities/...
Updated logging macros, numerical classes, and utility functions.
src/diagnostic/detail/...
src/diagnostic/diagnostic_manager.hpp
src/diagnostic/diagnostic_model_view.hpp
Enhanced diagnostic data writing, dumping logic, and model view methods.
tests/functional/harris/harris_2d_lb.py
tests/simulator/CMakeLists.txt
tests/simulator/init.py
Added test scripts, MPI tests, and functions for simulation testing.
tests/simulator/test_load_balancing.py Introduced test cases for load balancing in simulations.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

pyphare/pyphare/pharesee/particles.py Outdated Show resolved Hide resolved
src/amr/multiphysics_integrator.hpp Outdated Show resolved Hide resolved
coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

@PHAREHUB PHAREHUB deleted a comment from coderabbitai bot Apr 25, 2024
@PHAREHUB PHAREHUB deleted a comment from coderabbitai bot Apr 25, 2024
@PHAREHUB PHAREHUB deleted a comment from coderabbitai bot Apr 25, 2024
@PHAREHUB PHAREHUB deleted a comment from coderabbitai bot Apr 25, 2024
@PHAREHUB PHAREHUB deleted a comment from coderabbitai bot Apr 25, 2024
@PHAREHUB PHAREHUB deleted a comment from coderabbitai bot Apr 25, 2024
@PHAREHUB PHAREHUB deleted a comment from coderabbitai bot Apr 25, 2024
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

Out of diff range and nitpick comments (1)
res/cmake/dep/caliper.cmake (1)

5-6: Consider adding more detailed comments explaining the purpose and effect of the CALI_CONFIG settings for better clarity and maintainability.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 11

Out of diff range and nitpick comments (6)
res/cmake/dep/caliper.cmake (1)

5-6: Consider adding more detailed documentation or examples on how to use the new Caliper configuration for inclusive timings per function. This will help users understand when and why they might want to use this configuration.

pyphare/pyphare/pharein/load_balancer.py (1)

11-20: Ensure that the default values for the LoadBalancer properties are well-documented, especially how they interact with each other (e.g., how auto affects the use of on_init and every).

pyphare/pyphare/simulator/simulator.py (2)

Line range hint 99-99: Avoid using bare except statements as they can mask unexpected errors. Specify the exceptions you expect to handle.

-    except:
+    except Exception as e:
+        # Handle specific exceptions here

Line range hint 119-119: Avoid using bare except statements as they can mask unexpected errors. Specify the exceptions you expect to handle.

-    except:
+    except Exception as e:
+        # Handle specific exceptions here
src/amr/wrappers/integrator.hpp (2)

4-4: Consider organizing imports to group related headers together for better readability.


62-64: The constructor now initializes several new member variables related to load balancing. Ensure these are properly documented to clarify their roles.

Comment on lines 92 to 131
bool _should_rebalance_now();

bool cadence_rebalance_check()
{
return (time_step_idx == 0 and lb_info_.on_init)
or (lb_info_.every > 0 and time_step_idx > 0
and time_step_idx % lb_info_.every == 0);
}

bool tolerance_rebalance_check()
{
std::size_t constexpr static next_rebalance_backoff_multiplier = 2;
std::size_t constexpr static next_rebalance_defaut = 200;
std::size_t constexpr static max_next_rebalance = 1000;

if (rebalance_coarsest_auto_back_off == 0)
{
PHARE_LOG_SCOPE(1, "Integrator::_should_rebalance_now::automatic");

auto workLoads = core::mpi::collect(computeNonUniformWorkLoadForLevel0());
auto const max_value = *std::max_element(workLoads.begin(), workLoads.end());
for (auto& workload : workLoads)
workload /= max_value;
auto const min_value = *std::min_element(workLoads.begin(), workLoads.end());
assert(min_value <= 1);

if ((1 - min_value) > lb_info_.tolerance)
{
rebalance_coarsest_auto_back_off_by = next_rebalance_defaut;
rebalance_coarsest_auto_back_off = rebalance_coarsest_auto_back_off_by;
return true;
}

rebalance_coarsest_auto_back_off_by *= next_rebalance_backoff_multiplier;
rebalance_coarsest_auto_back_off = rebalance_coarsest_auto_back_off_by;

if (rebalance_coarsest_auto_back_off > max_next_rebalance)
rebalance_coarsest_auto_back_off_by = rebalance_coarsest_auto_back_off
= max_next_rebalance;
}

--rebalance_coarsest_auto_back_off;
return false;
};
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The method _should_rebalance_now contains complex logic with multiple conditional checks and mathematical operations. Consider refactoring this method to improve readability and maintainability.

- bool _should_rebalance_now();
+ bool _should_rebalance_now(); // Consider refactoring for simplicity

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
bool _should_rebalance_now();
bool cadence_rebalance_check()
{
return (time_step_idx == 0 and lb_info_.on_init)
or (lb_info_.every > 0 and time_step_idx > 0
and time_step_idx % lb_info_.every == 0);
}
bool tolerance_rebalance_check()
{
std::size_t constexpr static next_rebalance_backoff_multiplier = 2;
std::size_t constexpr static next_rebalance_defaut = 200;
std::size_t constexpr static max_next_rebalance = 1000;
if (rebalance_coarsest_auto_back_off == 0)
{
PHARE_LOG_SCOPE(1, "Integrator::_should_rebalance_now::automatic");
auto workLoads = core::mpi::collect(computeNonUniformWorkLoadForLevel0());
auto const max_value = *std::max_element(workLoads.begin(), workLoads.end());
for (auto& workload : workLoads)
workload /= max_value;
auto const min_value = *std::min_element(workLoads.begin(), workLoads.end());
assert(min_value <= 1);
if ((1 - min_value) > lb_info_.tolerance)
{
rebalance_coarsest_auto_back_off_by = next_rebalance_defaut;
rebalance_coarsest_auto_back_off = rebalance_coarsest_auto_back_off_by;
return true;
}
rebalance_coarsest_auto_back_off_by *= next_rebalance_backoff_multiplier;
rebalance_coarsest_auto_back_off = rebalance_coarsest_auto_back_off_by;
if (rebalance_coarsest_auto_back_off > max_next_rebalance)
rebalance_coarsest_auto_back_off_by = rebalance_coarsest_auto_back_off
= max_next_rebalance;
}
--rebalance_coarsest_auto_back_off;
return false;
};
bool _should_rebalance_now(); // Consider refactoring for simplicity
bool cadence_rebalance_check()
{
return (time_step_idx == 0 and lb_info_.on_init)
or (lb_info_.every > 0 and time_step_idx > 0
and time_step_idx % lb_info_.every == 0);
}
bool tolerance_rebalance_check()
{
std::size_t constexpr static next_rebalance_backoff_multiplier = 2;
std::size_t constexpr static next_rebalance_defaut = 200;
std::size_t constexpr static max_next_rebalance = 1000;
if (rebalance_coarsest_auto_back_off == 0)
{
PHARE_LOG_SCOPE(1, "Integrator::_should_rebalance_now::automatic");
auto workLoads = core::mpi::collect(computeNonUniformWorkLoadForLevel0());
auto const max_value = *std::max_element(workLoads.begin(), workLoads.end());
for (auto& workload : workLoads)
workload /= max_value;
auto const min_value = *std::min_element(workLoads.begin(), workLoads.end());
assert(min_value <= 1);
if ((1 - min_value) > lb_info_.tolerance)
{
rebalance_coarsest_auto_back_off_by = next_rebalance_defaut;
rebalance_coarsest_auto_back_off = rebalance_coarsest_auto_back_off_by;
return true;
}
rebalance_coarsest_auto_back_off_by *= next_rebalance_backoff_multiplier;
rebalance_coarsest_auto_back_off = rebalance_coarsest_auto_back_off_by;
if (rebalance_coarsest_auto_back_off > max_next_rebalance)
rebalance_coarsest_auto_back_off_by = rebalance_coarsest_auto_back_off
= max_next_rebalance;
}
--rebalance_coarsest_auto_back_off;
return false;
};

Comment on lines 461 to 483
def GetAllAvailableQties(self, time=0, pops=[]):
assert self.single_hier_for_all_quantities # can't work otherwise

self.GetParticles(time, pops)
self.GetB(time)
self.GetE(time)
self.GetNi(time)
self.GetVi(time)
def _try(fn, *args, **kwargs):
try:
fn(*args, **kwargs)
except Exception:
... # file not found

_try(self.GetParticles, time, pops)
_try(self.GetB, time)
_try(self.GetE, time)
_try(self.GetNi, time)
_try(self.GetVi, time)

for pop in pops:
self.GetFlux(time, pop)
self.GetN(time, pop)
_try(self.GetFlux, time, pop)
_try(self.GetN, time, pop)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace the ellipsis in the exception handling block with proper logging.

-                ...  # file not found
+                log.error("File not found", exc_info=True)  # Log the exception information

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
def GetAllAvailableQties(self, time=0, pops=[]):
assert self.single_hier_for_all_quantities # can't work otherwise
self.GetParticles(time, pops)
self.GetB(time)
self.GetE(time)
self.GetNi(time)
self.GetVi(time)
def _try(fn, *args, **kwargs):
try:
fn(*args, **kwargs)
except Exception:
... # file not found
_try(self.GetParticles, time, pops)
_try(self.GetB, time)
_try(self.GetE, time)
_try(self.GetNi, time)
_try(self.GetVi, time)
for pop in pops:
self.GetFlux(time, pop)
self.GetN(time, pop)
_try(self.GetFlux, time, pop)
_try(self.GetN, time, pop)
def GetAllAvailableQties(self, time=0, pops=[]):
assert self.single_hier_for_all_quantities # can't work otherwise
def _try(fn, *args, **kwargs):
try:
fn(*args, **kwargs)
except Exception:
log.error("File not found", exc_info=True) # Log the exception information
_try(self.GetParticles, time, pops)
_try(self.GetB, time)
_try(self.GetE, time)
_try(self.GetNi, time)
_try(self.GetVi, time)
for pop in pops:
_try(self.GetFlux, time, pop)
_try(self.GetN, time, pop)

def GetAllAvailableQties(self, time, pops):
assert self.single_hier_for_all_quantities == True # can't work otherwise
def GetAllAvailableQties(self, time=0, pops=[]):
assert self.single_hier_for_all_quantities # can't work otherwise
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider replacing the assert statement with a more robust control flow mechanism.

Assert statements are generally used for debugging purposes and not for controlling program flow. Consider using an if statement and raising an appropriate exception if the condition is not met.

Comment on lines +35 to +49
if self.mode not in allowed_modes:
raise RuntimeError(f"LoadBalancer mode '{self.mode}' is not valid")

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message should include all allowed modes to guide the user correctly. This change was suggested in a previous review and should be implemented to enhance usability.

-            raise RuntimeError(f"LoadBalancer mode '{self.mode}' is not valid")
+            raise RuntimeError(f"LoadBalancer mode '{self.mode}' is not valid. Allowed modes are: {allowed_modes}")

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
if self.mode not in allowed_modes:
raise RuntimeError(f"LoadBalancer mode '{self.mode}' is not valid")
if self.mode not in allowed_modes:
raise RuntimeError(f"LoadBalancer mode '{self.mode}' is not valid. Allowed modes are: {allowed_modes}")

pyphare/pyphare/pharesee/run.py Dismissed Show dismissed Hide dismissed
@PHAREHUB PHAREHUB deleted a comment from coderabbitai bot Apr 25, 2024
@PhilipDeegan PhilipDeegan force-pushed the loadBalancer branch 2 times, most recently from c1cef36 to c911d4d Compare April 25, 2024 16:05
@PhilipDeegan PhilipDeegan force-pushed the loadBalancer branch 2 times, most recently from 749286a to d24d498 Compare May 15, 2024 08:57
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 28

Out of diff range and nitpick comments (10)
tests/diagnostic/__init__.py (1)

Line range hint 16-16: Remove the unused import of numpy.

- import pyphare.pharein as ph, numpy as np
+ import pyphare.pharein as ph
pyphare/pyphare/simulator/simulator.py (3)

Line range hint 110-110: Avoid using bare except statements. Specify the exception types to catch.

-        except:
+        except Exception:

Line range hint 133-133: Avoid using bare except statements. Specify the exception types to catch.

-        except:
+        except Exception:

Line range hint 164-164: Use is not None for comparison to follow Python best practices.

-        if self._auto_dump() and self.post_advance != None:
+        if self._auto_dump() and self.post_advance is not None:
pyphare/pyphare/pharein/maxwellian_fluid_model.py (1)

Line range hint 4-4: Remove the unused import yee_element_is_primal to clean up the code.

- from pyphare.core.gridlayout import GridLayout, yee_element_is_primal
+ from pyphare.core.gridlayout import GridLayout
tests/simulator/__init__.py (3)

Line range hint 77-77: Avoid assigning a lambda expression directly to a variable. Use a def statement instead for better readability and debugging.

- _ = lambda i: 0.1 * np.cos(2 * np.pi * xyz[i] / L[i])
+ def _(i):
+     return 0.1 * np.cos(2 * np.pi * xyz[i] / L[i])

Line range hint 178-178: The Box class is used but not defined or imported. Ensure that Box is defined or imported correctly.

+ from some_module import Box

Also applies to: 184-184, 191-191


Line range hint 77-77: The time_step_nbr parameter is no longer used and should be removed if it's not required for backward compatibility or other reasons.

- "time_step_nbr": 1000,
pyphare/pyphare/pharein/__init__.py (1)

Line range hint 32-47: Move all import statements to the top of the file and remove unused imports to follow Python's best practices.

- from .uniform_model import UniformModel
- from .maxwellian_fluid_model import MaxwellianFluidModel
- from .electron_model import ElectronModel
- from .diagnostics import (
-     FluidDiagnostics,
-     ElectromagDiagnostics,
-     ParticleDiagnostics,
-     MetaDiagnostics,
-     InfoDiagnostics,
- )
- from .simulation import (
-     Simulation,
-     serialize as serialize_sim,
-     deserialize as deserialize_sim,
- )
- from .load_balancer import LoadBalancer
+ # Move all imports to the top of the file
+ from .uniform_model import UniformModel
+ from .maxwellian_fluid_model import MaxwellianFluidModel
+ from .electron_model import ElectronModel
+ from .diagnostics import (
+     FluidDiagnostics,
+     ElectromagDiagnostics,
+     ParticleDiagnostics,
+     MetaDiagnostics,
+     InfoDiagnostics,
+ )
+ from .simulation import (
+     Simulation,
+     serialize as serialize_sim,
+     deserialize as deserialize_sim,
+ )
+ from .load_balancer import LoadBalancer
pyphare/pyphare/pharein/simulation.py (1)

630-630: Update documentation to include loadbalancing.

The wrapper function now accepts loadbalancing as a keyword. Consider updating the documentation to reflect this change and provide examples of how to use this new option.

Review Details

Configuration used: .coderabbit.yaml
Review profile: CHILL

Commits Files that changed from the base of the PR and between dd0fe18 and d24d498.
Files selected for processing (41)
  • pyphare/pyphare/pharein/init.py (5 hunks)
  • pyphare/pyphare/pharein/load_balancer.py (1 hunks)
  • pyphare/pyphare/pharein/maxwellian_fluid_model.py (2 hunks)
  • pyphare/pyphare/pharein/simulation.py (7 hunks)
  • pyphare/pyphare/pharesee/hierarchy.py (6 hunks)
  • pyphare/pyphare/pharesee/particles.py (1 hunks)
  • pyphare/pyphare/simulator/simulator.py (5 hunks)
  • pyphare/pyphare_tests/test_pharesee/test_geometry_3d.py (1 hunks)
  • res/cmake/def.cmake (4 hunks)
  • res/cmake/dep/samrai.cmake (1 hunks)
  • src/amr/CMakeLists.txt (2 hunks)
  • src/amr/data/particles/particles_data.hpp (1 hunks)
  • src/amr/level_initializer/hybrid_level_initializer.hpp (2 hunks)
  • src/amr/load_balancing/concrete_load_balancer_hybrid_strategy_homogeneous.hpp (1 hunks)
  • src/amr/load_balancing/concrete_load_balancer_hybrid_strategy_nppc.hpp (1 hunks)
  • src/amr/load_balancing/load_balancer_details.hpp (1 hunks)
  • src/amr/load_balancing/load_balancer_estimator.hpp (1 hunks)
  • src/amr/load_balancing/load_balancer_estimator_hybrid.hpp (1 hunks)
  • src/amr/load_balancing/load_balancer_hybrid_strategy.hpp (1 hunks)
  • src/amr/load_balancing/load_balancer_hybrid_strategy_factory.hpp (1 hunks)
  • src/amr/load_balancing/load_balancer_manager.hpp (1 hunks)
  • src/amr/messengers/hybrid_hybrid_messenger_strategy.hpp (11 hunks)
  • src/amr/multiphysics_integrator.hpp (13 hunks)
  • src/amr/wrappers/integrator.hpp (4 hunks)
  • src/core/data/ions/particle_initializers/maxwellian_particle_initializer.hpp (6 hunks)
  • src/core/data/ions/particle_initializers/particle_initializer_factory.hpp (2 hunks)
  • src/core/data/ndarray/ndarray_vector.hpp (1 hunks)
  • src/core/data/particles/particle_array.hpp (1 hunks)
  • src/core/utilities/box/box.hpp (1 hunks)
  • src/initializer/data_provider.hpp (1 hunks)
  • src/initializer/dictator.cpp (1 hunks)
  • src/simulator/simulator.hpp (6 hunks)
  • tests/core/data/ions/test_ions.cpp (1 hunks)
  • tests/diagnostic/init.py (1 hunks)
  • tests/functional/harris/harris_2d_lb.py (1 hunks)
  • tests/simulator/CMakeLists.txt (1 hunks)
  • tests/simulator/init.py (2 hunks)
  • tests/simulator/test_load_balancing.py (1 hunks)
  • tests/simulator/test_tagging.py.off (1 hunks)
  • tools/python3/init.py (2 hunks)
  • tools/python3/caliper.py (1 hunks)
Files not reviewed due to errors (1)
  • pyphare/pyphare/pharesee/particles.py (no review received)
Files skipped from review as they are similar to previous changes (24)
  • res/cmake/def.cmake
  • res/cmake/dep/samrai.cmake
  • src/amr/CMakeLists.txt
  • src/amr/data/particles/particles_data.hpp
  • src/amr/level_initializer/hybrid_level_initializer.hpp
  • src/amr/load_balancing/concrete_load_balancer_hybrid_strategy_homogeneous.hpp
  • src/amr/load_balancing/concrete_load_balancer_hybrid_strategy_nppc.hpp
  • src/amr/load_balancing/load_balancer_details.hpp
  • src/amr/load_balancing/load_balancer_estimator.hpp
  • src/amr/load_balancing/load_balancer_estimator_hybrid.hpp
  • src/amr/load_balancing/load_balancer_hybrid_strategy.hpp
  • src/amr/load_balancing/load_balancer_hybrid_strategy_factory.hpp
  • src/amr/load_balancing/load_balancer_manager.hpp
  • src/amr/messengers/hybrid_hybrid_messenger_strategy.hpp
  • src/core/data/ions/particle_initializers/maxwellian_particle_initializer.hpp
  • src/core/data/ions/particle_initializers/particle_initializer_factory.hpp
  • src/core/data/ndarray/ndarray_vector.hpp
  • src/core/data/particles/particle_array.hpp
  • src/core/utilities/box/box.hpp
  • src/initializer/data_provider.hpp
  • src/initializer/dictator.cpp
  • tests/core/data/ions/test_ions.cpp
  • tests/simulator/CMakeLists.txt
  • tests/simulator/test_tagging.py.off
Additional Context Used
Ruff (46)
pyphare/pyphare/pharein/__init__.py (15)

32-32: Module level import not at top of file


32-32: .uniform_model.UniformModel imported but unused; consider removing, adding to __all__, or using a redundant alias


33-33: Module level import not at top of file


33-33: .maxwellian_fluid_model.MaxwellianFluidModel imported but unused; consider removing, adding to __all__, or using a redundant alias


34-34: Module level import not at top of file


34-34: .electron_model.ElectronModel imported but unused; consider removing, adding to __all__, or using a redundant alias


35-41: Module level import not at top of file


36-36: .diagnostics.FluidDiagnostics imported but unused; consider removing, adding to __all__, or using a redundant alias


37-37: .diagnostics.ElectromagDiagnostics imported but unused; consider removing, adding to __all__, or using a redundant alias


38-38: .diagnostics.ParticleDiagnostics imported but unused; consider removing, adding to __all__, or using a redundant alias


39-39: .diagnostics.MetaDiagnostics imported but unused; consider removing, adding to __all__, or using a redundant alias


40-40: .diagnostics.InfoDiagnostics imported but unused; consider removing, adding to __all__, or using a redundant alias


42-46: Module level import not at top of file


43-43: .simulation.Simulation imported but unused; consider removing, adding to __all__, or using a redundant alias


47-47: Module level import not at top of file

pyphare/pyphare/pharein/load_balancer.py (2)

46-46: f-string without any placeholders


49-49: f-string without any placeholders

pyphare/pyphare/pharein/maxwellian_fluid_model.py (1)

4-4: pyphare.core.gridlayout.yee_element_is_primal imported but unused

pyphare/pyphare/pharein/simulation.py (7)

338-338: Comparison to None should be cond is not None


358-358: Ambiguous variable name: l


564-564: f-string without any placeholders


572-572: f-string without any placeholders


692-692: Comparison to None should be cond is not None


949-949: Multiple imports on one line


955-955: Multiple imports on one line

pyphare/pyphare/pharesee/hierarchy.py (1)

668-668: Comparison to None should be cond is None

pyphare/pyphare/pharesee/particles.py (1)

88-88: f-string without any placeholders

pyphare/pyphare/simulator/simulator.py (4)

110-110: Do not use bare except


133-133: Do not use bare except


145-145: pyphare.cpp.cpp_lib imported but unused


164-164: Comparison to None should be cond is not None

pyphare/pyphare_tests/test_pharesee/test_geometry_3d.py (1)

1-1: Undefined name test_geometry_3d

tests/diagnostic/__init__.py (2)

16-16: Multiple imports on one line


16-16: numpy imported but unused; consider removing, adding to __all__, or using a redundant alias

tests/functional/harris/harris_2d_lb.py (1)

74-74: Ambiguous variable name: l

tests/simulator/__init__.py (7)

4-4: Multiple imports on one line


77-77: Do not assign a lambda expression, use a def


178-178: Undefined name Box


184-184: Undefined name Box


191-191: Undefined name Box


217-217: Local variable rank is assigned to but never used


218-218: Local variable env is assigned to but never used

tests/simulator/test_load_balancing.py (1)

73-73: Ambiguous variable name: l

tools/python3/__init__.py (1)

72-72: Module level import not at top of file

tools/python3/caliper.py (2)

6-6: tools.python3.run imported but unused


20-20: subprocess imported but unused

Learnings (1)
pyphare/pyphare/pharesee/hierarchy.py (1)
User: PhilipDeegan
URL: https://github.com/PHAREHUB/PHARE/pull/807

Timestamp: 2024-04-15T11:57:24.673Z
Learning: The `__eq__` method in `ParticleData` uses `self.dataset == that.dataset` for dataset comparison, which might not be sufficient for numpy arrays as it returns an array of booleans. The use of `np.array_equal` is suggested for a more robust comparison.
Additional comments not posted (24)
pyphare/pyphare/simulator/simulator.py (1)

147-147: LGTM!

tests/simulator/test_load_balancing.py (1)

200-283: LGTM!

pyphare/pyphare/pharein/maxwellian_fluid_model.py (2)

75-75: LGTM!


127-127: LGTM!

src/amr/wrappers/integrator.hpp (7)

27-28: Including load_balancer_details.hpp is necessary for the LoadBalancerDetails type used in the class.


47-54: The advance method correctly includes a call to _should_rebalance_now for load balancing.


56-64: The constructor correctly initializes new member variables and sets up the load balancer.


67-73: The new member variables are necessary for the load balancing logic and their initialization is appropriate.


78-90: The computeNonUniformWorkLoadForLevel0 method is well-implemented and necessary for load balancing.


94-99: The cadence_rebalance_check method is straightforward and necessary for load balancing.


156-166: The constructor definition correctly initializes new member variables and sets up the load balancer.

pyphare/pyphare/pharein/__init__.py (2)

228-247: The addition of load balancer parameters in the populateDict function looks good. Ensure that all necessary parameters for the load balancer are included and correctly set.

Verification successful

The LoadBalancer class has the following parameters:

  • active
  • mode
  • tol
  • on_init
  • auto
  • next_rebalance_backoff_multiplier
  • next_rebalance
  • max_next_rebalance
  • every
  • _register

The populateDict function sets the following parameters:

  • active
  • mode
  • tol
  • on_init
  • auto
  • next_rebalance_backoff_multiplier
  • next_rebalance
  • max_next_rebalance
  • every

All necessary parameters for the LoadBalancer are included and correctly set in the populateDict function.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Description: Verify all load balancer parameters are included and correctly set in the populateDict function.

# Test: Search for the load balancer parameters in the populateDict function. Expect: All necessary parameters are included.
rg --type py "load_balancer" ./pyphare/pyphare/pharein/__init__.py

Length of output: 173



Script:

#!/bin/bash
# Locate the LoadBalancer class definition and display its parameters.

# Find the file containing the LoadBalancer class definition.
lb_file=$(fd LoadBalancer.py)

# Display the LoadBalancer class definition and its parameters.
ast-grep --lang python --pattern $'class LoadBalancer($_, $_):\n    def __init__($$$):\n        $$$' $lb_file

Length of output: 132



Script:

#!/bin/bash
# Search for the LoadBalancer class definition across the entire codebase.

# Search for the LoadBalancer class definition.
ast-grep --lang python --pattern $'class LoadBalancer($_, $_):\n    def __init__($$$):\n        $$$'

Length of output: 99



Script:

#!/bin/bash
# Search for the LoadBalancer class definition using rg and display its parameters.

# Search for the LoadBalancer class definition.
rg --type py "class LoadBalancer" > load_balancer_class_location.txt

# Extract the file path from the search result.
lb_file=$(awk -F: '{print $1}' load_balancer_class_location.txt | head -n 1)

# Display the LoadBalancer class definition and its parameters.
if [ -n "$lb_file" ]; then
  ast-grep --lang python --pattern $'class LoadBalancer($_, $_):\n    def __init__($$$):\n        $$$' $lb_file
else
  echo "LoadBalancer class definition not found."
fi

Length of output: 351



Script:

#!/bin/bash
# Display the contents of the file containing the LoadBalancer class.

# Display the contents of pyphare/pyphare/pharein/load_balancer.py
cat pyphare/pyphare/pharein/load_balancer.py

Length of output: 1725


130-132: The add_bool function is correctly implemented and follows the pattern of other add_* functions.

src/simulator/simulator.hpp (4)

Line range hint 4-18: The addition of include statements for load balancing headers is necessary and appropriate for the new functionality.


73-75: The update to the dump method ensures that it returns the result of dMan->dump if dMan is not null. This is a good addition.


266-293: The load balancing setup in the hybrid_init method looks comprehensive. Ensure that all necessary configurations for load balancing are included and correctly set.


Line range hint 392-401: The addition of the logging scope in the advance method is beneficial for debugging and performance monitoring.

pyphare/pyphare/pharein/simulation.py (5)

130-131: Ensure start_time is correctly used in the function.


150-154: The return statement correctly includes start_time in the calculation of final_time.


587-594: The check_loadbalancing function correctly validates the loadbalancing keyword.


651-652: The call to check_loadbalancing ensures the loadbalancing keyword is validated.


849-849: The addition of load_balancer as an attribute in the Simulation class is correct.

src/amr/multiphysics_integrator.hpp (1)

628-628: The declaration of load_balancer_manager_ as a unique_ptr is appropriate for managing the lifetime of the load balancer manager instance. Ensure that all necessary checks for nullptr are in place before dereferencing this pointer to maintain robustness.

pyphare/pyphare/pharesee/hierarchy.py (1)

258-259: LGTM! The __getitem__ method is straightforward and correct.

Comment on lines +91 to +92
def __eq__(self, that):
return self.field_name == that.field_name and self.dataset[:] == that.dataset[:]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use np.array_equal for robust array comparison in numpy arrays.

- return self.field_name == that.field_name and self.dataset[:] == that.dataset[:]
+ return self.field_name == that.field_name and np.array_equal(self.dataset, that.dataset)

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
def __eq__(self, that):
return self.field_name == that.field_name and self.dataset[:] == that.dataset[:]
def __eq__(self, that):
return self.field_name == that.field_name and np.array_equal(self.dataset, that.dataset)

Comment on lines +229 to +230
def __eq__(self, that):
return self.name == that.name and self.dataset == that.dataset
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use np.array_equal for robust array comparison in numpy arrays.

- return self.name == that.name and self.dataset == that.dataset
+ return self.name == that.name and np.array_equal(self.dataset, that.dataset)

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
def __eq__(self, that):
return self.name == that.name and self.dataset == that.dataset
def __eq__(self, that):
return self.name == that.name and np.array_equal(self.dataset, that.dataset)

patch_data_ref = patch_ref.patch_datas[patch_data_key]
patch_data_cmp = patch_cmp.patch_datas[patch_data_key]

if patch_data_cmp != patch_data_ref:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use np.array_equal for robust array comparison in numpy arrays within hierarchy_compare.

- if patch_data_cmp != patch_data_ref:
+ if not np.array_equal(patch_data_cmp, patch_data_ref):

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
if patch_data_cmp != patch_data_ref:
if not np.array_equal(patch_data_cmp, patch_data_ref):

plt.xlabel("timestep")
fig.savefig("timestep_times.png")

cpp_lib().mpi_barrier()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider removing the MPI barrier to avoid unnecessary synchronization unless it is explicitly needed for correctness.

-    cpp_lib().mpi_barrier()

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
cpp_lib().mpi_barrier()

Consider removing the MPI barrier to avoid unnecessary synchronization unless it is explicitly needed for correctness.

-    cpp_lib().mpi_barrier()

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
cpp_lib().mpi_barrier()

rho[np.where(np.isclose(y, 10, atol=0.1))] = 0
return rho

def S(y, y0, l):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename the variable l to length for clarity and to avoid confusion with the number 1.

-    def S(y, y0, l):
+    def S(y, y0, length):

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
def S(y, y0, l):
def S(y, y0, length):

Comment on lines +24 to +70
# Setup desired cali query.
# cali_query = "cali-query"
# grouping_attribute = "function"
# default_metric = "sum(sum#time.duration),inclusive_sum(sum#time.duration)"
# query = "select function,%s group by %s format json-split" % (
# default_metric,
# grouping_attribute,
# )

# # Use ``cali-query`` here to produce the json-split stream.
# with extend_env_path(DEFAULT_CALI_QUERY_PATH):
# cali_json = subprocess.Popen(
# [cali_query, "-q", query, cali_file], stdout=subprocess.PIPE
# )

# # Use hatchet's ``from_caliper`` API with the resulting json-split.
# # The result is stored into Hatchet's GraphFrame.
# gf = ht.GraphFrame.from_caliper(cali_json.stdout)

# # Printout the DataFrame component of the GraphFrame.
# print(gf.dataframe)

# # Printout the graph component of the GraphFrame.
# # Use "time (inc)" as the metric column to be displayed
# print(gf.tree(metric_column="time (inc)"))

# # Setup desired cali query.
# grouping_attribute = "function"
# default_metric = "sum(sum#time.duration),inclusive_sum(sum#time.duration)"
# query = "select function,%s group by %s format json-split" % (
# default_metric,
# grouping_attribute,
# )

# # Use hatchet's ``from_caliper`` API with the path to the cali file and the
# # query. This API will internally run ``cali-query`` on this file to
# # produce a json-split stream. The result is stored into Hatchet's
# # GraphFrame.
# with extend_env_path(DEFAULT_CALI_QUERY_PATH):
# gf = ht.GraphFrame.from_caliper(cali_file, query)

# # Printout the DataFrame component of the GraphFrame.
# print(gf.dataframe)

# # Printout the graph component of the GraphFrame.
# # Use "time (inc)" as the metric column to be displayed
# print(gf.tree(metric_column="time (inc)"))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Large block of commented-out code detected. Consider removing or justifying this to maintain code cleanliness.

+ 1.0 / np.cosh((y - Ly * 0.7) / L) ** 2
)

def S(y, y0, l):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename the variable l to a more descriptive name to avoid ambiguity and improve readability.

- def S(y, y0, l):
+ def S(y, y0, length):

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
def S(y, y0, l):
def S(y, y0, length):

Comment on lines +36 to +41
bool static _is_tagging_refinement(initializer::PHAREDict const& dict)
{
return cppdict::get_value(dict, "simulation/AMR/refinement/tagging/method",
std::string{"none"})
== std::string{"auto"};
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a constant for the string comparison to improve maintainability.

- return cppdict::get_value(dict, "simulation/AMR/refinement/tagging/method", std::string{"none"}) == std::string{"auto"};
+ const std::string AUTO_TAGGING = "auto";
+ return cppdict::get_value(dict, "simulation/AMR/refinement/tagging/method", std::string{"none"}) == AUTO_TAGGING;

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
bool static _is_tagging_refinement(initializer::PHAREDict const& dict)
{
return cppdict::get_value(dict, "simulation/AMR/refinement/tagging/method",
std::string{"none"})
== std::string{"auto"};
}
bool static _is_tagging_refinement(initializer::PHAREDict const& dict)
{
const std::string AUTO_TAGGING = "auto";
return cppdict::get_value(dict, "simulation/AMR/refinement/tagging/method",
std::string{"none"})
== AUTO_TAGGING;
}

return load;
}

bool _should_rebalance_now();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider refactoring the _should_rebalance_now method to improve readability and maintainability.

- bool _should_rebalance_now();
+ bool _should_rebalance_now(); // Consider refactoring for simplicity

Also applies to: 216-222


Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
bool _should_rebalance_now();
bool _should_rebalance_now(); // Consider refactoring for simplicity

Comment on lines +101 to +131
bool tolerance_rebalance_check()
{
if (rebalance_coarsest_auto_back_off == 0)
{
PHARE_LOG_SCOPE(1, "Integrator::_should_rebalance_now::automatic");

auto workLoads = core::mpi::collect(computeNonUniformWorkLoadForLevel0());
auto const max_value = *std::max_element(workLoads.begin(), workLoads.end());
for (auto& workload : workLoads)
workload /= max_value;
auto const min_value = *std::min_element(workLoads.begin(), workLoads.end());
assert(min_value <= 1);

if ((1 - min_value) > lb_info_.tolerance)
{
rebalance_coarsest_auto_back_off_by = lb_info_.next_rebalance;
rebalance_coarsest_auto_back_off = rebalance_coarsest_auto_back_off_by;
return true;
}

rebalance_coarsest_auto_back_off_by *= lb_info_.next_rebalance_backoff_multiplier;
rebalance_coarsest_auto_back_off = rebalance_coarsest_auto_back_off_by;

if (rebalance_coarsest_auto_back_off > lb_info_.max_next_rebalance)
rebalance_coarsest_auto_back_off_by = rebalance_coarsest_auto_back_off
= lb_info_.max_next_rebalance;
}

--rebalance_coarsest_auto_back_off;
return false;
};
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider refactoring the tolerance_rebalance_check method to improve readability and maintainability.

- bool tolerance_rebalance_check();
+ bool tolerance_rebalance_check(); // Consider refactoring for simplicity

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
bool tolerance_rebalance_check()
{
if (rebalance_coarsest_auto_back_off == 0)
{
PHARE_LOG_SCOPE(1, "Integrator::_should_rebalance_now::automatic");
auto workLoads = core::mpi::collect(computeNonUniformWorkLoadForLevel0());
auto const max_value = *std::max_element(workLoads.begin(), workLoads.end());
for (auto& workload : workLoads)
workload /= max_value;
auto const min_value = *std::min_element(workLoads.begin(), workLoads.end());
assert(min_value <= 1);
if ((1 - min_value) > lb_info_.tolerance)
{
rebalance_coarsest_auto_back_off_by = lb_info_.next_rebalance;
rebalance_coarsest_auto_back_off = rebalance_coarsest_auto_back_off_by;
return true;
}
rebalance_coarsest_auto_back_off_by *= lb_info_.next_rebalance_backoff_multiplier;
rebalance_coarsest_auto_back_off = rebalance_coarsest_auto_back_off_by;
if (rebalance_coarsest_auto_back_off > lb_info_.max_next_rebalance)
rebalance_coarsest_auto_back_off_by = rebalance_coarsest_auto_back_off
= lb_info_.max_next_rebalance;
}
--rebalance_coarsest_auto_back_off;
return false;
};
bool tolerance_rebalance_check(); // Consider refactoring for simplicity

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Outside diff range and nitpick comments (1)
pyphare/pyphare/pharein/__init__.py (1)

Line range hint 32-32: Move module-level imports to the top of the file to improve code organization and readability.

+ from .uniform_model import UniformModel
+ from .maxwellian_fluid_model import MaxwellianFluidModel
+ from .electron_model import ElectronModel
+ from .diagnostics import (
+     FluidDiagnostics,
+     ElectromagDiagnostics,
+     ParticleDiagnostics,
+     MetaDiagnostics,
+     InfoDiagnostics,
+ )
+ from .simulation import (
+     Simulation,
+     serialize as serialize_sim,
+     deserialize as deserialize_sim,
+ )
+ from .load_balancer import LoadBalancer
- # All imports moved to the top

Also applies to: 33-33, 34-34, 35-41, 42-46, 47-47

Review Details

Configuration used: .coderabbit.yaml
Review profile: CHILL

Commits Files that changed from the base of the PR and between d24d498 and 0f2686c.
Files selected for processing (6)
  • pyphare/pyphare/core/init.py (3 hunks)
  • pyphare/pyphare/pharein/init.py (5 hunks)
  • pyphare/pyphare/pharein/load_balancer.py (1 hunks)
  • src/hdf5/detail/h5/h5_file.hpp (2 hunks)
  • tests/simulator/test_load_balancing.py (1 hunks)
  • tools/config/config.py (1 hunks)
Files skipped from review due to trivial changes (1)
  • src/hdf5/detail/h5/h5_file.hpp
Additional Context Used
Ruff (21)
pyphare/pyphare/pharein/__init__.py (15)

32-32: Module level import not at top of file


32-32: .uniform_model.UniformModel imported but unused; consider removing, adding to __all__, or using a redundant alias


33-33: Module level import not at top of file


33-33: .maxwellian_fluid_model.MaxwellianFluidModel imported but unused; consider removing, adding to __all__, or using a redundant alias


34-34: Module level import not at top of file


34-34: .electron_model.ElectronModel imported but unused; consider removing, adding to __all__, or using a redundant alias


35-41: Module level import not at top of file


36-36: .diagnostics.FluidDiagnostics imported but unused; consider removing, adding to __all__, or using a redundant alias


37-37: .diagnostics.ElectromagDiagnostics imported but unused; consider removing, adding to __all__, or using a redundant alias


38-38: .diagnostics.ParticleDiagnostics imported but unused; consider removing, adding to __all__, or using a redundant alias


39-39: .diagnostics.MetaDiagnostics imported but unused; consider removing, adding to __all__, or using a redundant alias


40-40: .diagnostics.InfoDiagnostics imported but unused; consider removing, adding to __all__, or using a redundant alias


42-46: Module level import not at top of file


43-43: .simulation.Simulation imported but unused; consider removing, adding to __all__, or using a redundant alias


47-47: Module level import not at top of file

pyphare/pyphare/pharein/load_balancer.py (3)

36-36: f-string without any placeholders


53-53: f-string without any placeholders


56-56: f-string without any placeholders

tests/simulator/test_load_balancing.py (2)

73-73: Ambiguous variable name: l


218-218: Local variable diag_dir is assigned to but never used

tools/config/config.py (1)

144-144: Local variable mpi_type is assigned to but never used

Additional comments not posted (2)
pyphare/pyphare/pharein/load_balancer.py (2)

56-56: Remove the unnecessary f-string as there are no placeholders used.

- f"LoadBalancer is already registered to simulation"
+ "LoadBalancer is already registered to simulation"

Likely invalid or redundant comment.


53-53: Remove the unnecessary f-string as there are no placeholders used.

- f"LoadBalancer cannot be registered as no simulation exists"
+ "LoadBalancer cannot be registered as no simulation exists"

Likely invalid or redundant comment.

Comment on lines 22 to 24
if "PHARE_SKIP_CLI" in os.environ:
return os.environ["PHARE_SKIP_CLI"] == "1"

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using a configuration file or environment variable to define the test directory path to increase flexibility and maintainability.

@@ -34,8 +38,7 @@


def parse_cli_args():
default_off = len(sys.argv) == 1 and disabled_for_testing()
if default_off:
if disabled_for_testing():
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider setting the default value of --write_reports to False to align with common command-line interface practices where optional behaviors are disabled by default.


def __post_init__(self):
if self.auto and self.every:
raise RuntimeError(f"LoadBalancer cannot work with both 'every' and 'auto'")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the unnecessary f-string as there are no placeholders used.

- raise RuntimeError(f"LoadBalancer cannot work with both 'every' and 'auto'")
+ raise RuntimeError("LoadBalancer cannot work with both 'every' and 'auto'")

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
raise RuntimeError(f"LoadBalancer cannot work with both 'every' and 'auto'")
raise RuntimeError("LoadBalancer cannot work with both 'every' and 'auto'")

return

with self.assertRaises(RuntimeError):
diag_dir = self.run_sim(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the unused variable diag_dir to clean up the code.

- diag_dir = self.run_sim(
-     self.unique_diag_dir_for_test_case(diag_outputs, ndim, interp),
-     dict(active=True, mode="nppc", tol=0.01, **lbkwargs),
- )

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
diag_dir = self.run_sim(

@PHAREHUB PHAREHUB deleted a comment from coderabbitai bot May 23, 2024
@@ -575,6 +581,16 @@ def check_clustering(**kwargs):
return clustering


def check_loadbalancing(**kwargs):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

torm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants