From 532b026f7515302c0879fc5dd9666d4a214dde59 Mon Sep 17 00:00:00 2001 From: CFLAG Date: Thu, 14 Dec 2023 11:39:47 +0100 Subject: [PATCH 01/11] Clarify slab decomposition and optional arguments --- docs/source/pages/api_domain.rst | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/docs/source/pages/api_domain.rst b/docs/source/pages/api_domain.rst index 20fc4e5..85bb35a 100644 --- a/docs/source/pages/api_domain.rst +++ b/docs/source/pages/api_domain.rst @@ -156,17 +156,26 @@ The algorithm will choose the closest combination such as n\_proc=n\_col=\sqrt{nproc} In case the root is not exact the closest combitation to have :math:`n\_proc \approx n\_col` with -`n\_proc < n\_col` is used. +`n\_proc < n\_col` is used. +If a 1D slab decomposition is needed instead of a 2D pencil one, it is recommended to set `p_row` to unity and `p_col` to `nproc`. An optional parameter may be passed to this initialisation routine: :: - call decomp_2d_init(nx, ny, nz, p_row, p_col,periodic_bc) + call decomp_2d_init(nx, ny, nz, p_row, p_col, periodic_bc) Here periodic_bc is a 1D array containing 3 logical values that specify whether periodic boundary condition should apply in certain dimensions. Note this is only applicable if halo-cell communication is to be used. +Another optional parameter may be passed at the initialization stage: + +:: + + call decomp_2d_init(nx, ny, nz, p_row, p_col, periodic_bc, comm) + +Here comm is the MPI communicator that the library will use. By default, MPI_COMM_WORLD is used. + A key element of this library is a set of communication routines that actually perform the data transpositions. As mentioned, one needs to perform 4 global transpositions to go through all 3 pencil orientations. Correspondingly, the library provides 4 communication subroutines: From 1d488cfee48e80134d81bff595c93b372a94dd62 Mon Sep 17 00:00:00 2001 From: CFLAG Date: Thu, 14 Dec 2023 11:42:24 +0100 Subject: [PATCH 02/11] Minor update --- docs/source/pages/api_domain.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/pages/api_domain.rst b/docs/source/pages/api_domain.rst index 85bb35a..710d11a 100644 --- a/docs/source/pages/api_domain.rst +++ b/docs/source/pages/api_domain.rst @@ -157,7 +157,7 @@ The algorithm will choose the closest combination such as In case the root is not exact the closest combitation to have :math:`n\_proc \approx n\_col` with `n\_proc < n\_col` is used. -If a 1D slab decomposition is needed instead of a 2D pencil one, it is recommended to set `p_row` to unity and `p_col` to `nproc`. +If a 1D slab decomposition is needed instead of a 2D pencil one, it is recommended to set ``p_row`` to unity and ``p_col`` to ``nproc``. An optional parameter may be passed to this initialisation routine: From 7c065e191d1a168f919899cc53099373f54348ab Mon Sep 17 00:00:00 2001 From: CFLAG Date: Thu, 14 Dec 2023 11:43:37 +0100 Subject: [PATCH 03/11] Minor update --- docs/source/pages/api_domain.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/pages/api_domain.rst b/docs/source/pages/api_domain.rst index 710d11a..a0026ce 100644 --- a/docs/source/pages/api_domain.rst +++ b/docs/source/pages/api_domain.rst @@ -165,7 +165,7 @@ An optional parameter may be passed to this initialisation routine: call decomp_2d_init(nx, ny, nz, p_row, p_col, periodic_bc) -Here periodic_bc is a 1D array containing 3 logical values that specify whether periodic boundary condition +Here ``periodic_bc`` is a 1D array containing 3 logical values that specify whether periodic boundary condition should apply in certain dimensions. Note this is only applicable if halo-cell communication is to be used. Another optional parameter may be passed at the initialization stage: @@ -174,7 +174,7 @@ Another optional parameter may be passed at the initialization stage: call decomp_2d_init(nx, ny, nz, p_row, p_col, periodic_bc, comm) -Here comm is the MPI communicator that the library will use. By default, MPI_COMM_WORLD is used. +Here ``comm`` is the MPI communicator that the library will use. By default, MPI_COMM_WORLD is used. A key element of this library is a set of communication routines that actually perform the data transpositions. As mentioned, one needs to perform 4 global transpositions to go through all 3 pencil orientations. From 6f62440e186af8658820fe0182b2d2610e1af8b8 Mon Sep 17 00:00:00 2001 From: CFLAG Date: Fri, 15 Dec 2023 12:44:03 +0100 Subject: [PATCH 04/11] Typo --- docs/source/pages/api_domain.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/pages/api_domain.rst b/docs/source/pages/api_domain.rst index a0026ce..b9d7e2d 100644 --- a/docs/source/pages/api_domain.rst +++ b/docs/source/pages/api_domain.rst @@ -153,10 +153,10 @@ The algorithm will choose the closest combination such as .. math:: - n\_proc=n\_col=\sqrt{nproc} + n\_row=n\_col=\sqrt{nproc} -In case the root is not exact the closest combitation to have :math:`n\_proc \approx n\_col` with -`n\_proc < n\_col` is used. +In case the root is not exact the closest combination to have :math:`n\_row \approx n\_col` with +`n\_row < n\_col` is used. If a 1D slab decomposition is needed instead of a 2D pencil one, it is recommended to set ``p_row`` to unity and ``p_col`` to ``nproc``. An optional parameter may be passed to this initialisation routine: From 7284c60f2cae3d5f9428514abc7fc99c69f08e79 Mon Sep 17 00:00:00 2001 From: CFLAG Date: Fri, 15 Dec 2023 12:47:07 +0100 Subject: [PATCH 05/11] Minor update --- docs/source/pages/api_domain.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/pages/api_domain.rst b/docs/source/pages/api_domain.rst index b9d7e2d..b27e8c9 100644 --- a/docs/source/pages/api_domain.rst +++ b/docs/source/pages/api_domain.rst @@ -151,12 +151,12 @@ Note that none of the dimensions need to be divisible by ``p_row`` or ``p_col``, In case of ``p_row=p_col=0`` an automatic decomposition is selected among all possible combination available. The algorithm will choose the closest combination such as -.. math:: +```math +n\_row = n\_col = \sqrt{nproc} +``` - n\_row=n\_col=\sqrt{nproc} - -In case the root is not exact the closest combination to have :math:`n\_row \approx n\_col` with -`n\_row < n\_col` is used. +In case the root is not exact the closest combination to have $ n\_row \approx n\_col $ with +$ n\_row < n\_col $ is used. If a 1D slab decomposition is needed instead of a 2D pencil one, it is recommended to set ``p_row`` to unity and ``p_col`` to ``nproc``. An optional parameter may be passed to this initialisation routine: From 05f278f12cafccfbbd82cc16e8993c44b473be92 Mon Sep 17 00:00:00 2001 From: CFLAG Date: Fri, 15 Dec 2023 12:48:18 +0100 Subject: [PATCH 06/11] Minor update --- docs/source/pages/api_domain.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/source/pages/api_domain.rst b/docs/source/pages/api_domain.rst index b27e8c9..9d3c7e0 100644 --- a/docs/source/pages/api_domain.rst +++ b/docs/source/pages/api_domain.rst @@ -151,9 +151,7 @@ Note that none of the dimensions need to be divisible by ``p_row`` or ``p_col``, In case of ``p_row=p_col=0`` an automatic decomposition is selected among all possible combination available. The algorithm will choose the closest combination such as -```math -n\_row = n\_col = \sqrt{nproc} -``` +$$ n\_row = n\_col = \sqrt{nproc} $$ In case the root is not exact the closest combination to have $ n\_row \approx n\_col $ with $ n\_row < n\_col $ is used. From da8c6d964d1de995b70e94922f5e9320559b0807 Mon Sep 17 00:00:00 2001 From: CFLAG Date: Fri, 15 Dec 2023 12:48:59 +0100 Subject: [PATCH 07/11] Update api_domain.rst --- docs/source/pages/api_domain.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/source/pages/api_domain.rst b/docs/source/pages/api_domain.rst index 9d3c7e0..583d1e4 100644 --- a/docs/source/pages/api_domain.rst +++ b/docs/source/pages/api_domain.rst @@ -151,7 +151,9 @@ Note that none of the dimensions need to be divisible by ``p_row`` or ``p_col``, In case of ``p_row=p_col=0`` an automatic decomposition is selected among all possible combination available. The algorithm will choose the closest combination such as -$$ n\_row = n\_col = \sqrt{nproc} $$ +$$ +n\_{row} = n\_{col} = \sqrt{nproc} +$$ In case the root is not exact the closest combination to have $ n\_row \approx n\_col $ with $ n\_row < n\_col $ is used. From cb4b327c16ef21e6592a0291eea662369833699c Mon Sep 17 00:00:00 2001 From: CFLAG Date: Fri, 15 Dec 2023 12:51:32 +0100 Subject: [PATCH 08/11] Update api_domain.rst --- docs/source/pages/api_domain.rst | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/source/pages/api_domain.rst b/docs/source/pages/api_domain.rst index 583d1e4..469d005 100644 --- a/docs/source/pages/api_domain.rst +++ b/docs/source/pages/api_domain.rst @@ -151,11 +151,9 @@ Note that none of the dimensions need to be divisible by ``p_row`` or ``p_col``, In case of ``p_row=p_col=0`` an automatic decomposition is selected among all possible combination available. The algorithm will choose the closest combination such as -$$ -n\_{row} = n\_{col} = \sqrt{nproc} -$$ +$$ n\_{row} = n\_{col} = {nproc}^{1/2} $$ -In case the root is not exact the closest combination to have $ n\_row \approx n\_col $ with +In case the root is not exact the closest combination to have $ n\_{row} \approx n\_{col} $ with $ n\_row < n\_col $ is used. If a 1D slab decomposition is needed instead of a 2D pencil one, it is recommended to set ``p_row`` to unity and ``p_col`` to ``nproc``. From 0191eafdeeb04877b0dbe2a51f947233f5c2e52f Mon Sep 17 00:00:00 2001 From: CFLAG Date: Fri, 15 Dec 2023 12:52:37 +0100 Subject: [PATCH 09/11] Update api_domain.rst --- docs/source/pages/api_domain.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/pages/api_domain.rst b/docs/source/pages/api_domain.rst index 469d005..e6dd3fc 100644 --- a/docs/source/pages/api_domain.rst +++ b/docs/source/pages/api_domain.rst @@ -153,8 +153,8 @@ The algorithm will choose the closest combination such as $$ n\_{row} = n\_{col} = {nproc}^{1/2} $$ -In case the root is not exact the closest combination to have $ n\_{row} \approx n\_{col} $ with -$ n\_row < n\_col $ is used. +In case the root is not exact the closest combination to have $` n\_{row} \approx n\_{col} `$ with +$` n\_{row} < n\_{col} `$ is used. If a 1D slab decomposition is needed instead of a 2D pencil one, it is recommended to set ``p_row`` to unity and ``p_col`` to ``nproc``. An optional parameter may be passed to this initialisation routine: From d8d75f609a13954781a34035701eafdfe2485070 Mon Sep 17 00:00:00 2001 From: CFLAG Date: Fri, 15 Dec 2023 12:53:52 +0100 Subject: [PATCH 10/11] Update api_domain.rst --- docs/source/pages/api_domain.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/source/pages/api_domain.rst b/docs/source/pages/api_domain.rst index e6dd3fc..862fb32 100644 --- a/docs/source/pages/api_domain.rst +++ b/docs/source/pages/api_domain.rst @@ -153,8 +153,7 @@ The algorithm will choose the closest combination such as $$ n\_{row} = n\_{col} = {nproc}^{1/2} $$ -In case the root is not exact the closest combination to have $` n\_{row} \approx n\_{col} `$ with -$` n\_{row} < n\_{col} `$ is used. +In case the root is not exact the closest combination to have $n_{row} \approx n_{col}$ with $n_{row} < n_{col}$ is used. If a 1D slab decomposition is needed instead of a 2D pencil one, it is recommended to set ``p_row`` to unity and ``p_col`` to ``nproc``. An optional parameter may be passed to this initialisation routine: From 9e56f09c1d23705251a071daf36febcc09a2f3b7 Mon Sep 17 00:00:00 2001 From: Paul Bartholomew Date: Fri, 15 Dec 2023 12:02:46 +0000 Subject: [PATCH 11/11] Replace \times -> x Doesn't seem to be supported by github markdown... --- docs/source/pages/api_domain.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/pages/api_domain.rst b/docs/source/pages/api_domain.rst index 862fb32..4706bcb 100644 --- a/docs/source/pages/api_domain.rst +++ b/docs/source/pages/api_domain.rst @@ -146,14 +146,14 @@ using the following call call decomp_2d_init(nx, ny, nz, p_row, p_col) where ``nx``, ``ny`` and ``nz`` are the size of 3D global data to be distributed over -a 2D processor grid :math:`p_row \times p_col`. +a 2D processor grid ``p_row x p_col``. Note that none of the dimensions need to be divisible by ``p_row`` or ``p_col``, i.e. the library can handle non-evenly distributed data. In case of ``p_row=p_col=0`` an automatic decomposition is selected among all possible combination available. The algorithm will choose the closest combination such as $$ n\_{row} = n\_{col} = {nproc}^{1/2} $$ -In case the root is not exact the closest combination to have $n_{row} \approx n_{col}$ with $n_{row} < n_{col}$ is used. +In case the root is not exact the closest combination to $n_{row} = n_{col}$ with $n_{row} < n_{col}$ is used. If a 1D slab decomposition is needed instead of a 2D pencil one, it is recommended to set ``p_row`` to unity and ``p_col`` to ``nproc``. An optional parameter may be passed to this initialisation routine: @@ -233,7 +233,7 @@ different data sets as 2D pencils. call decomp_info_init(n1, n2, n3, new_decomp) Here decomp is an instance of Fortran derived data type DECOMP_INFO encapsulating -the 2D decomposition information associated with one particular global size :math:`n1\times n2 \times n3`. +the 2D decomposition information associated with one particular global size ``n1 x n2 x n3``. The decomposition object can be initialised using the ``decomp_info_init`` routine as: ::