cray_inline/cray_noinline in GPU_ROUTINE never emit their !DIR$ directive on Cray GPU builds

[sbryngelson]

Summary

GPU_ROUTINE(..., cray_inline=True) and GPU_ROUTINE(..., cray_noinline=True) never emit their !DIR$ directive on a Cray GPU build. The directives are emitted only on Cray CPU builds, where a device-inlining hint has no purpose. Every current use of these options is in device code, so in practice both options are no-ops.

Where

src/common/include/parallel_macros.fpp:66-100. The cray_noinline branch expands to:

#ifdef _CRAYFTN
#if MFC_OpenACC
        $:acc_directive              ! acc routine only
#elif MFC_OpenMP
        $:omp_directive              ! omp declare target only
#else
        $:cray_noinline_directive    ! !DIR$ NOINLINE -- Cray CPU only
#endif
#elif MFC_OpenACC
        $:acc_directive
#elif MFC_OpenMP
        $:omp_directive
#endif

Under _CRAYFTN, the !DIR$ line sits in the #else arm, so it is selected only when neither MFC_OpenACC nor MFC_OpenMP is defined. The cray_inline branch immediately below has the same shape and the same problem.

Reproduction

$ printf "#:include 'parallel_macros.fpp'\nsubroutine s_t\n    \$:GPU_ROUTINE(function_name='s_t', parallelism='[seq]', cray_noinline=True)\nend subroutine\n" > /tmp/mt.fpp
$ ./build/venv/bin/fypp -I src/common/include -DMFC_COMPILER='"Cray"' -DMFC_CASE_OPTIMIZATION=False /tmp/mt.fpp

subroutine s_t

#ifdef _CRAYFTN
#if MFC_OpenACC
!$acc routine seq
#elif MFC_OpenMP
!$omp declare target device_type(any)
#else
!DIR$ NOINLINE s_t
#endif
#elif MFC_OpenACC
!$acc routine seq
#elif MFC_OpenMP

Substituting cray_inline=True produces the same structure with !DIR$ INLINEALWAYS s_t in the same unreachable arm.

Impact

• 8 call sites use cray_noinline=True, 43 use cray_inline=True, across m_variables_conversion, m_boundary_primitives, m_riemann_state, m_qbmm, m_bubbles, m_bubbles_EL, m_bubbles_EL_kernels, m_compute_cbc, m_chemistry, m_phase_change, and m_sim_helpers.
• All of them are device routines, so on Cray GPU builds the inlining behaviour is whatever the compiler's heuristics choose. Whatever these options were added to guarantee is not in effect there.
• If any of them was added to work around a Cray codegen issue, that workaround has been silently inactive on Cray GPU builds.

Both !DIR$ INLINEALWAYS and !DIR$ NOINLINE are accepted by Cray ftn alongside !$acc routine / !$omp declare target, so the fix should be to emit the !DIR$ line in addition to the offload directive rather than as an alternative to it:

#ifdef _CRAYFTN
        $:cray_noinline_directive
#if MFC_OpenACC
        $:acc_directive
#elif MFC_OpenMP
        $:omp_directive
#endif
#elif MFC_OpenACC
...

Worth confirming the directive ordering Cray expects relative to !$acc routine before settling on a form.

Notes

Pre-existing; parallel_macros.fpp is unchanged on the current master and by any open PR. Found while investigating a Cray-only NaN regression on Frontier in #1679, where the inlining status of s_convert_species_to_mixture_variables_kernel (declared cray_noinline=True) mattered to the diagnosis. Whether the two are related is not established — filing separately because the macro defect stands on its own.

[sbryngelson]

There is a second, independent defect here: !DIR$ NOINLINE is not a Cray directive. The Cray spelling is INLINENEVER. So cray_noinline=True is a no-op even in the one configuration this issue says it survives to — the Cray CPU build.

CCE 19.0.0 rejects it with ftn-790, the same diagnostic it gives literal gibberish:

NOINLINE         -> ftn-790 ftn: WARNING ... Unknown or unsupported compiler directive or syntax error.
INLINENEVER      -> accepted, no diagnostic
INLINEALWAYS     -> accepted, no diagnostic
TOTALGIBBERISH   -> ftn-790 ftn: WARNING ... Unknown or unsupported compiler directive or syntax error.

!DIR$ NOINLINE victim
               ^
ftn-790 ftn: WARNING VICTIM, File = t_NOINLINE.f90, Line = 4, Column = 16
  Unknown or unsupported compiler directive or syntax error.

explain ftn-790: "A misspelled or unrecognized compiler directive was encountered ... or it may be a different vendor's directives that uses the !DIR$ prefix." NOINLINE is the Intel/LLVM spelling.

cray_inline is unaffected — INLINEALWAYS is correct. Only the cray_noinline path is misspelled.

So the combined state of the 8 cray_noinline=True call sites is that they have never done anything on any build: wrong arm of the #if on GPU, invalid directive on CPU. The 43 cray_inline=True sites are as this issue describes — correct directive, unreachable arm on GPU.

Fixing both together in parallel_macros.fpp:

1. INLINENEVER instead of NOINLINE.
2. Emit the !DIR$ line alongside the offload directive rather than in an #else arm — they are independent, and a Cray GPU build wants both:

#ifdef _CRAYFTN
#if defined(MFC_OpenACC)
        $:acc_directive
#elif defined(MFC_OpenMP)
        $:omp_directive
#endif
        $:cray_noinline_directive
#elif defined(MFC_OpenACC)
        $:acc_directive
#elif defined(MFC_OpenMP)
        $:omp_directive
#endif

Verified emission after the change (fypp + cpp, matching what the build feeds the compiler):

build	before	after
Cray + OpenACC	!$acc routine seq	!$acc routine seq + !DIR$ INLINENEVER <name>
Cray + OpenMP	!$omp declare target device_type(any)	!$omp declare target device_type(any) + !DIR$ INLINENEVER <name>
Cray CPU	!DIR$ NOINLINE (rejected, ftn-790)	!DIR$ INLINENEVER <name>

While changing this it is also worth switching the bare #if MFC_OpenACC / #if MFC_OpenMP in these two branches to #if defined(...), which is what the rest of the file uses.

One caution for whoever lands it: turning on 51 previously-dead directives on Cray GPU is a real change to device codegen, and CMakeLists.txt:170-181 already carries per-file -Oipa0 workarounds for CCE inlining ICEs in m_bubbles_EL, m_phase_change, m_mpi_common, and m_start_up. Those workarounds were tuned against a build where these directives did nothing, so this needs a full Cray GPU build plus regression run on both --gpu acc and --gpu mp, not just a compile check.

[sbryngelson]

Correction: the directive name is also wrong, and that is why the dead arm went unnoticed

Investigating this on Frontier turned up a second defect that the one described above was hiding.

!DIR$ NOINLINE is not a named, routine-definition directive. Per CCE's own man pages:

• man 7 inline — inline_enable/INLINE, inline_disable/NOINLINE, inline_reset/RESETINLINE. Synopsis is bare !DIR$ NOINLINE, with no name list. It is a call-site range directive: it suppresses inlining for calls that follow it, until the opposite directive or the end of the program unit.
• man 7 inlinealways — inline_always/INLINEALWAYS, inline_never/INLINENEVER. Synopsis is !DIR$ INLINENEVER name[, name] .... This is the form that names a routine, and "if the directive is placed in the definition of the function, inlining is never attempted at any call site to name in the entire input file being compiled."

So the '!DIR$ NOINLINE ' + function_name the macro builds is a syntax error, not a valid never-inline request:

$ ftn -hacc -c dirtest.f90
!DIR$ NOINLINE s_a
               ^
ftn-790 ftn: WARNING S_A, File = dirtest.f90, Line = 4, Column = 16
  Unknown or unsupported compiler directive or syntax error.

Reproduced on cce/19.0.0 (the version the f slug currently loads) and on cce/21.0.2. !DIR$ INLINENEVER s_a compiles clean on both, and clean alongside !$acc routine seq and !$omp declare target device_type(any), in either order.

The two bugs interact: because the !DIR$ line was stranded in the #else arm, the malformed directive only ever reached the compiler on Cray CPU builds — where ftn-790 is a warning in a build log nobody reads. That is why an invalid directive survived at 8 call sites. cray_noinline=True has therefore been a no-op on every build configuration, not just GPU ones. (cray_inline/INLINEALWAYS is spelled correctly, so it did work on Cray CPU builds and only GPU builds lost it.)

The directives do change codegen once fixed

ftn -hacc -hlist=a on a two-routine module, one cray_noinline=True and one cray_inline=True, CCE 21.0.2, reading the caller's .lst:

	s_ni (cray_noinline=True)
current master	Leaf "s_ni"(...) was inlined.
INLINENEVER, emitted on GPU	"s_ni"(...) inlining has been disabled with a compiler directive.

So on Cray GPU builds all 8 cray_noinline=True routines have been getting inlined against the explicit request in the source.

Suggested fix

The three GPU_ROUTINE branches are near-identical and collapse into one, which fixes both defects together (about −20 LOC):

    #:if cray_inline or cray_noinline
        #:if not isinstance(function_name, str)
            #:stop "When using cray_inline/cray_noinline, function name must be given and given as a string"
        #:endif
        #:set cray_keyword = 'INLINEALWAYS' if cray_inline else 'INLINENEVER'
        #:set cray_directive = ('!DIR$ ' + cray_keyword + ' ' + function_name).strip('\n')
#ifdef _CRAYFTN
        $:cray_directive
#endif
    #:endif
#if MFC_OpenACC
    $:acc_directive
#elif MFC_OpenMP
    $:omp_directive
#endif

Emission stays gated on the caller's cray_inline=/cray_noinline= kwarg; nothing is emitted for call sites that pass neither, and nothing reaches non-Cray compilers.

Validated on Frontier with cce/19.0.0 + rocm/6.3.1: pre_process/simulation/post_process build clean for both --gpu acc and --gpu mp, and a 10% suite sample passes 63/63 on each (--rdma-mpi, 8 GPUs). fypp expansion checked across Cray/non-Cray × acc/omp/cpu × kwarg-present/absent.

Unrelated blocker found along the way

The original plan was to validate this on the newest CCE. That is currently impossible, for reasons independent of this issue — every CCE newer than 19 aborts in lld while linking MFC's device code for gfx90a:

• cce/20.0.2 — Running pass 'Infer address spaces' on function '@"s_cbc$m_cbc_$ck_L596_5"' then malloc_consolidate(): unaligned fastbin chunk detected. Deterministic, 3/3 runs.
• cce/21.0.0 and cce/21.0.2 — Running pass 'AMDGPU Rewrite AGPR-Copy-MFMA' on function '@"s_compute_bubble_ee_source$m_bubbles_ee_$ck_L185_6"' then SlotIndexes.h:96: Assertion 'isValid() && "Attempt to compare reserved index."' failed.

Both reproduce on unmodified master with no macro change, for both acc and mp, so they are not caused by the fix above. Filing separately.