"some processor has no bands-row-blocks." occurs when i set nbands and ocp for constraint-DFT

[Jweijian]

Describe the bug

When running tddft by abacus, i set such input file, but error occurs. Here is input file:

And output file：

If i set ncores=5， program works fine！How can I solve such a problem?
Abacus version == 3.8

Expected behavior

No response

To Reproduce

No response

Environment

No response

Additional Context

No response

Task list for Issue attackers (only for developers)

• Verify the issue is not a duplicate.
• Describe the bug.
• Steps to reproduce.
• Expected behavior.
• Error message.
• Environment details.
• Additional context.
• Assign a priority level (low, medium, high, urgent).
• Assign the issue to a team member.
• Label the issue with relevant tags.
• Identify possible related issues.
• Create a unit test or automated test to reproduce the bug (if applicable).
• Fix the bug.
• Test the fix.
• Update documentation (if necessary).
• Close the issue and inform the reporter (if applicable).

[WHUweiqingzhou]

I check the source code, and found it is from the Parallel_Orbitals::set_nloc_wfc_Eij. I am quite curious why we need a WarningQuit if some processors do not have a band since these processors can still accelerate the diagonalization. Do you have any idea? @jinzx10

[jinzx10]

I check the source code, and found it is from the Parallel_Orbitals::set_nloc_wfc_Eij. I am quite curious why we need a WarningQuit if some processors do not have a band since these processors can still accelerate the diagonalization. Do you have any idea? @jinzx10

This behavior is a legacy and I have no knowledge of why it behaves this way. Parallel_Orbitals is a child class derived from Parallel_2D. While the latter has been thoroughly cleaned up in the past few quarters, Parallel_Orbitals remains what it was before.

[mohanchen]

I would suggest removing the warningquit and see what happen?

[jinzx10]

@Jweijian This warning message occurs when there's a resource waste. When a certain number of MPI processes is launched, this number is factored to a product of two numbers as close as possible, and the wavefunction is distributed across that process grid. For example, if you launch abacus with "mpirun -np 16", the wavefunction (whose size is nbase x nbands) will be distributed across a 4x4 grid. A potential problem is, when the number of bands is smaller than the number of columns of the process grid, some process will not be assigned any part of the wavefunction, i.e., they are wasted. In your specific case, it seems you launch abacus with 40 processes, and the process grid is 5x8, but the number of bands is only 5. If ABACUS continues running, 15 processes will be wasted. When you reduce the number of processes to 5, in which case the process grid becomes 1x5, everything should work.
(Of course, in this example, abacus could have created a 8x5 grid to avoid this issue, but currently abacus always create a process grid with number of rows smaller than number of columns. I think this is not the point of this post.)

@mohanchen @WHUweiqingzhou I tried to remove the above WARNING_QUIT and run a similar md/tddft calculation where the number of nbands is greater than the number of columns in the processor grid. Everything seems to work as usual. I haven't gone through all the code to see if this behavior is safe for all types of calculations in all scenarios, but I think the question is, should abacus stop when there's a resource waste?

[WHUweiqingzhou]

@maki49 what is your opinion?

[maki49]

@jinzx10 How about to try 8*5 when 5*8 has a waste? If it's not hard to implement, it will be an improvement of flexibility.
If both of them have such a waste, I feel nothing bad to give a warning and quit to help the user save machine-hours.

[jinzx10]

@jinzx10 How about to try 85 when 58 has a waste? If it's not hard to implement, it will be an improvement of flexibility. If both of them have such a waste, I feel there's nothing bad to give a warning and quit to help the user save machine-hours.

I notice that Parallel_Orbitals current has several members, among which the sizes of wfc and wfc1 are simply related by a transpose. If the decomposition has one number greater than nbands, it seems one of them will inevitably has empty process if they stick to the same process grid.

[jinzx10]

@jinzx10 How about to try 8_5 when 5_8 has a waste? If it's not hard to implement, it will be an improvement of flexibility. If both of them have such a waste, I feel there's nothing bad to give a warning and quit to help the user save machine-hours.

I notice that Parallel_Orbitals current has several members, among which the sizes of wfc and wfc1 are simply related by a transpose. If the decomposition has one number greater than nbands, it seems one of them will inevitably has empty process if they stick to the same process grid.

Oh it seems desc_wfc1 is no longer used anymore so it might be safely ignored.

[jinzx10]

@jinzx10 How about to try 85 when 58 has a waste? If it's not hard to implement, it will be an improvement of flexibility. If both of them have such a waste, I feel nothing bad to give a warning and quit to help the user save machine-hours.

I'm just curious about one thing... why the default is to have a wide grid instead of a tall grid? Since nbands is never larger than nlocal, a tall grid is always a safer choice. Is there a consideration from performance?