Skip to content

v3.3.0

Choose a tag to compare

View all tags
@UCHINO-Yuki UCHINO-Yuki released this 04 Sep 08:37
· 7 commits to main since this release
v3.3.0
9131544

Feat: Add memory-saving mode and handle-local configuration

Changes and New Features

  • Add a memory-saving mode with a default workspace-size limit of 12 GiB.

    • The mode can be enabled or disabled with:

      • void set_memory_saving(cublasHandle_t handle, bool enable)
      • void set_memory_savingLt(cublasLtHandle_t handleLtLt, bool enable)

    • The workspace-size limit can be changed with:

      • void set_max_worksize(cublasHandle_t handle, size_t bytes)
      • void set_max_worksizeLt(cublasLtHandle_t handleLt, size_t bytes)

    • The current settings can be queried with:

      • bool get_memory_saving(cublasHandle_t handle)
      • bool get_memory_savingLt(cublasLtHandle_t handleLt)
      • size_t get_max_worksize(cublasHandle_t handle)
      • size_t get_max_worksizeLt(cublasLtHandle_t handleLt)

    • The HIP variants are also supported.

  • Change TRSM block-size configuration from global to handle-local settings.

    • [Breaking change] The following APIs are no longer supported:

      • void set_block_size_trsm(int nB)
      • int get_block_size_trsm()

    • The TRSM block size can be configured with:

      • void set_block_size_trsm(cublasHandle_t handle, int nB)
      • void set_block_size_trsmLt(cublasLtHandle_t handleLt, int nB)

    • The current settings can be queried with:

      • int get_block_size_trsm(cublasHandle_t)
      • int get_block_size_trsmLt(cublasLtHandle_t)

    • The HIP variants are also supported.

  • Add functions to clear handle-local configuration state.

    • Configurations set through set_memory_saving*, set_max_worksize*, and
      set_block_size_trsm* must be cleared before destroying the corresponding
      handle:

      • clear_config(cublasHandle_t handle)
      • clear_configLt(cublasLtHandle_t handleLt)

    • In hook mode, handle destruction is intercepted and the corresponding configuration is cleared automatically.

  • Add support for device scalars in TRSM.

  • Change the APIs for querying the required workspace size for TRSM.

    • Update workSizeTrsm() to take a cuBLAS/cuBLASLt handle so that the handle-local TRSM block size can be used.

    • [Breaking change] The following API is no longer supported:

      • size_t workSizeTrsm(cublasSideMode_t side, size_t m, size_t n, int num_moduli)

    • The required workspace size for TRSM can be queried with:

      • size_t workSizeTrsm(cublasHandle_t handle, cublasSideMode_t side, size_t m, size_t n, int num_moduli)
      • size_t workSizeTrsmLt(cublasLtHandle_t handleLt, cublasSideMode_t side, size_t m, size_t n, int num_moduli)

    • The HIP variants are also supported.

  • Fix workspace-size queries for the skip-scaling feature in fast mode.

    • For operations other than SYRK and HERK, enabling the skip-scaling feature (enable_skip_scalA = true and/or enable_skip_scalB = true) does not require additional workspace when fast mode (fastmode = true) is used.

    • In previous versions, the workspace-size query included this additional workspace even in fast mode, resulting in an overestimation of the required workspace size.

    • Add a fastmode argument to workSize() so that the required workspace size is computed correctly:

      template <bool is_Complex = false, Backend BACKEND = Backend::INT8, Func FUNC = Func::gemm>
size_t workSize(
    size_t m, size_t n, size_t k,
    int num_moduli,
    bool enable_skip_scalA = false,
    bool enable_skip_scalB = false,
    size_t *workSizeA      = nullptr,
    size_t *workSizeB      = nullptr,
    bool fastmode          = false
);

// Alternative template-argument order:
template <Backend BACKEND = Backend::INT8, bool is_Complex = false, Func FUNC = Func::gemm>
size_t workSize(
    size_t m, size_t n, size_t k,
    int num_moduli,
    bool enable_skip_scalA = false,
    bool enable_skip_scalB = false,
    size_t *workSizeA      = nullptr,
    size_t *workSizeB      = nullptr,
    bool fastmode          = false
);

  • Add test-program options for configuring memory-saving mode.

    • memory_saving=0|1 enables or disables memory-saving mode.
    • max_memory=<bytes> specifies the workspace-size limit.
    • If memory_saving=0 or max_memory=0, the test program uses the full workspace size without nabling memory-saving mode.

Usage notes

  • Memory-saving mode is disabled unless explicitly enabled.
  • When memory-saving mode is enabled, the default workspace-size limit is 12 GiB.
  • Configuration is maintained independently for each handle.
  • Before destroying a handle, call clear_config() or clear_configLt() if handle-local settings have been configured.
  • In hook mode, this cleanup is performed automatically.

See include/config.hpp and README.md for detailed descriptions of the configuration APIs and their behavior.

Assets 2
Loading