Add Max Occupancy Fraction Option

[MrBurmark]

Add GPU Max Occupancy Fraction Option

Add an option to change the max grid size in for_all policies that use the occupancy calculator.

The most important thing to be aware of is which policy gets the best performance with reducers, so an alias RAJA::hip_exec_rec_for_reduce<BLOCKS_SIZE> now exists which is the policy that performed best with reducers in our testing.

Another thing to note is that the meaning of the default occupancy calculator policy is changing, (hip|cuda)_exec_occ_calc<BLOCKS_SIZE>, is now the policy using the occupancy calculator that performed best in our testing for non-reducer loops.

More occupancy calculator policies exist now as well. (hip|cuda)_exec_occ_max<BLOCKS_SIZE> policy lets you run with the max occupancy. (hip|cuda)_exec_occ_fraction<BLOCKS_SIZE, Fraction<size_t, numerator, denominator>> policy lets you run with a fraction of the max occupancy. (hip|cuda)_exec_occ_custom<BLOCKS_SIZE, Concretizer> policy lets you use any concretizer.

This is implemented by adding a concretizer class to the exec policies used with for_all. When grid size of block size is not specified this class is used to pick the value. For example RAJA::CudaMaxOccupancyConcretizer uses the occupancy calculator to get the highest occupancy possible with each kernel. Another example 'RAJA::CudaFractionOffsetOccupancyConcretizer<Fraction, BLOCKS_PER_SM_OFFSET>` lets you use a fraction of and/or offset the max occupancy.

• This PR is a feature
• It does the following:
  • Adds hip_exec_rec_for_reduce at the request of people using reducers

[rhornung67]

@MrBurmark Please add some simple, introductory user guide documentation for this. If you put some basic stuff in, I will help expand it as needed.

[MrBurmark]

@rhornung67 I'll add documentation.
I think the design could use some more thorough thought though.

[tomstitt]

Thanks @MrBurmark !

[MrBurmark]

I thought more thoroughly on the design with @rhornung67 help and the implementation is now a little different. I'll update the description above.

[rhornung67]

Overall, this is looking good. I made a variety of comments, suggestions, etc. about documentation, code structure, etc.