Tpetra: use KokkosKernels addition (and remove redundant copies of these kernels in Tpetra) #2444

brian-kelley · 2018-03-23T05:16:42Z

Using KokkosKernels sparse matrix addition kernels in
TpetraExt::MatrixMatrix::add(). Removed the now redundant
copies of these kernels in Tpetra.

@trilinos/tpetra

Description

Motivation and Context

How Has This Been Tested?

Screenshots

Checklist

My commit messages mention the appropriate GitHub issue numbers.
My code follows the code style of the affected package(s).
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the code contribution guidelines for this project.
I have added tests to cover my changes.
All new and existing tests passed.
No new compiler warnings were introduced.
These changes break backwards compatibility.

Additional Information

Using KokkosKernels sparse matrix addition kernels in TpetraExt::MatrixMatrix::add(). Removed the now redundant copies of these kernels in Tpetra.

trilinos-autotester · 2018-03-23T05:24:40Z

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request.

trilinos-autotester · 2018-03-23T05:26:28Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

Build Num: 389
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PULLREQUESTNUM	2444
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	TpetraWork
TRILINOS_SOURCE_REPO	https://github.com/brian-kelley/Trilinos
TRILINOS_SOURCE_SHA	`df99b1f`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`85f88b2`

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

Build Num: 106
Status: STARTED

Jenkins Parameters

Parameter Name	Value
COMPILER_MODULE	sems-gcc/4.9.3
JENKINS_BUILD_TYPE	Release
JENKINS_COMM_TYPE	MPI
JENKINS_DO_COMPLEX	OFF
JENKINS_JOB_TYPE	Experimental
MPI_MODULE	sems-openmpi/1.8.7
PULLREQUESTNUM	2444
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	TpetraWork
TRILINOS_SOURCE_REPO	https://github.com/brian-kelley/Trilinos
TRILINOS_SOURCE_SHA	`df99b1f`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`85f88b2`

Using Repos:

Repo: TRILINOS (brian-kelley/Trilinos)

Pull Request Author: brian-kelley

trilinos-autotester · 2018-03-23T07:57:04Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

Build Num: 389
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PULLREQUESTNUM	2444
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	TpetraWork
TRILINOS_SOURCE_REPO	https://github.com/brian-kelley/Trilinos
TRILINOS_SOURCE_SHA	`df99b1f`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`85f88b2`

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

Build Num: 106
Status: PASSED

Jenkins Parameters

Parameter Name	Value
COMPILER_MODULE	sems-gcc/4.9.3
JENKINS_BUILD_TYPE	Release
JENKINS_COMM_TYPE	MPI
JENKINS_DO_COMPLEX	OFF
JENKINS_JOB_TYPE	Experimental
MPI_MODULE	sems-openmpi/1.8.7
PULLREQUESTNUM	2444
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	TpetraWork
TRILINOS_SOURCE_REPO	https://github.com/brian-kelley/Trilinos
TRILINOS_SOURCE_SHA	`df99b1f`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`85f88b2`

trilinos-autotester · 2018-03-23T07:57:23Z

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
WARNING: NO REVIEWERS HAVE BEEN REQUESTED FOR THIS PULL REQUEST!

trilinos-autotester · 2018-03-23T07:57:29Z

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

mhoemmen · 2018-03-23T17:32:03Z

@brian-kelley Thanks for working on this! Could you please give me some context for the code that you added?

mhoemmen · 2018-03-23T17:32:22Z

That is, I can see that you removed code, but you also added code; I'm asking about the latter :-) .

brian-kelley · 2018-03-23T17:42:03Z

@mhoemmen I didn't want to change anything in TpetraExt::MatrixMatrix::add(), so add() still calls one of 3 internal functions in AddDetails::AddKernels: addSorted(), addUnsorted(), and convertToGlobalAndAdd(). I replaced the internals of these with code to make KK handles and call spadd_symbolic and spadd_numeric.

Also the TimeMonitor just measures symbolic and numeric now.

mhoemmen

Thanks for working on this! I think it's OK. Does it pass CUDA tests? Does it require some experimental kokkos-kernels macros?

mhoemmen · 2018-03-23T17:29:35Z

packages/tpetra/core/ext/TpetraExt_MatrixMatrix_decl.hpp

@@ -138,6 +138,7 @@ struct AddKernels
  typedef Tpetra::Map<LocalOrdinal, GlobalOrdinal, Node> map_type;
  typedef typename Node::device_type device_type;
  typedef typename device_type::execution_space execution_space;
+  typedef typename execution_space::memory_space memory_space;


Does this build clean with -Wall? Lots of typedefs make me nervous sometimes ;-) .

mhoemmen · 2018-03-24T02:02:35Z

packages/tpetra/core/ext/TpetraExt_MatrixMatrix_def.hpp

@@ -561,7 +562,6 @@ makeColMapAndConvertGids(GlobalOrdinal ncols,
  return rcp(new map_type(Teuchos::OrdinalTraits<GlobalOrdinal>::invalid(), colmap, 0, comm));


Technically should be Teuchos::OrdinalTraits<Tpetra::global_size_t>::invalid() but that's OK :)

mhoemmen · 2018-03-24T02:03:27Z

packages/tpetra/core/ext/TpetraExt_MatrixMatrix_def.hpp

-  ColInds Ccolinds;
-};
+  using Teuchos::TimeMonitor;
+  TEUCHOS_TEST_FOR_EXCEPTION(Arowptrs.dimension_0() != Browptrs.dimension_0(), std::runtime_error, "Can't add matrices with different numbers of rows.");


Is this a new test?

or was it there before?

It was there before. Git thinks it's new because the indentation changed for this function.

mhoemmen · 2018-03-26T22:28:33Z

@brian-kelley Once this is ready for pull request testing, be sure to remove the "AT: WIP" label. Thanks!

mhoemmen · 2018-04-02T22:34:15Z

@brian-kelley The title matters less than the "AT: WIP" label; I just removed the latter.

trilinos-autotester · 2018-04-02T22:44:15Z

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ mhoemmen ]!

trilinos-autotester · 2018-04-02T22:44:22Z

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - Master Automerge is disabled (in .cfg file)

mhoemmen · 2018-04-02T22:55:15Z

I added the retest label to force a rebase and retest, since it's been a few days.

trilinos-autotester · 2018-04-02T23:03:42Z

Status Flag 'Pull Request AutoTester' - User Requested Retest - Resetting Testing Status

trilinos-autotester · 2018-04-02T23:05:17Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

Build Num: 420
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PULLREQUESTNUM	2444
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	TpetraWork
TRILINOS_SOURCE_REPO	https://github.com/brian-kelley/Trilinos
TRILINOS_SOURCE_SHA	`df99b1f`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`85f88b2`

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

Build Num: 137
Status: STARTED

Jenkins Parameters

Parameter Name	Value
COMPILER_MODULE	sems-gcc/4.9.3
JENKINS_BUILD_TYPE	Release
JENKINS_COMM_TYPE	MPI
JENKINS_DO_COMPLEX	OFF
JENKINS_JOB_TYPE	Experimental
MPI_MODULE	sems-openmpi/1.8.7
PULLREQUESTNUM	2444
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	TpetraWork
TRILINOS_SOURCE_REPO	https://github.com/brian-kelley/Trilinos
TRILINOS_SOURCE_SHA	`df99b1f`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`85f88b2`

Using Repos:

Repo: TRILINOS (brian-kelley/Trilinos)

Pull Request Author: brian-kelley

trilinos-autotester · 2018-04-03T01:38:22Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

Build Num: 420
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PULLREQUESTNUM	2444
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	TpetraWork
TRILINOS_SOURCE_REPO	https://github.com/brian-kelley/Trilinos
TRILINOS_SOURCE_SHA	`df99b1f`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`85f88b2`

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

Build Num: 137
Status: PASSED

Jenkins Parameters

Parameter Name	Value
COMPILER_MODULE	sems-gcc/4.9.3
JENKINS_BUILD_TYPE	Release
JENKINS_COMM_TYPE	MPI
JENKINS_DO_COMPLEX	OFF
JENKINS_JOB_TYPE	Experimental
MPI_MODULE	sems-openmpi/1.8.7
PULLREQUESTNUM	2444
TEST_REPO_ALIAS	TRILINOS
TRILINOS_SOURCE_BRANCH	TpetraWork
TRILINOS_SOURCE_REPO	https://github.com/brian-kelley/Trilinos
TRILINOS_SOURCE_SHA	`df99b1f`
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`85f88b2`

trilinos-autotester · 2018-04-03T01:38:40Z

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - Master Automerge is disabled (in .cfg file)

…evelop * 'develop' of https://github.com/trilinos/Trilinos: (560 commits) Disabling Stefan Boltzmann tests 1 and 2 due to an unresolved hang. Also, resetting the default problem size for Helmholtz to 16x16. Disable 3 Anasazi tests that randomly fail in debug builds on white/ride (trilinos#2473) TrilinosCouplings: Output iteration count Tpetra: use KokkosKernels addition (trilinos#2444) Tank solve and value correspond for all parameters TrilinosCouplings: OK. Now compiling TrilinosCouplings: More RTC updates Disabling failing test. Stokhos: Allow mean-based PCE preconditioner with double scalar type. (Painstakingly) reimplemented every tank equation individually. Now have solve and value working correctly together. TrilinosCouplings: Turning off file default output Kokkos: fix compilation for GCC 4.8.4 TrilinosCouplings: Adding block / RTC materials support to Tpetra example (take2) Kokkos: disable failing CUDA+DEBUG test TrilinosCouplings: Adding block / RTC materials support to Tpetra example adding doxygen for nd method Added comment Fixed warnings. Panzer: fix race condition in unit test exodus writer for CurlLaplacian example Fixed some problems in tank example. Solve and value are at least consistent when theta=1 ...

Tpetra: use KokkosKernels addition

df99b1f

Using KokkosKernels sparse matrix addition kernels in TpetraExt::MatrixMatrix::add(). Removed the now redundant copies of these kernels in Tpetra.

bartlettroscoe added the stage: in progress Work on the issue has started label Mar 23, 2018

brian-kelley requested a review from mhoemmen March 23, 2018 16:29

brian-kelley changed the title ~~Tpetra: use KokkosKernels addition (and remove redundant copies of these kernels in Tpetra)~~ [WIP] Tpetra: use KokkosKernels addition (and remove redundant copies of these kernels in Tpetra) Mar 23, 2018

mhoemmen added the AT: WIP Causes the PR autotester to not test the PR. (Remove to allow testing to occur.) label Mar 23, 2018

mhoemmen approved these changes Mar 24, 2018

View reviewed changes

brian-kelley changed the title ~~[WIP] Tpetra: use KokkosKernels addition (and remove redundant copies of these kernels in Tpetra)~~ Tpetra: use KokkosKernels addition (and remove redundant copies of these kernels in Tpetra) Apr 2, 2018

mhoemmen removed the AT: WIP Causes the PR autotester to not test the PR. (Remove to allow testing to occur.) label Apr 2, 2018

mhoemmen added the AT: RETEST Causes the PR autotester to run a new round of PR tests on the next iteration label Apr 2, 2018

trilinos-autotester removed the AT: RETEST Causes the PR autotester to run a new round of PR tests on the next iteration label Apr 2, 2018

mhoemmen merged commit c81b16c into trilinos:develop Apr 3, 2018

trilinos-autotester mentioned this pull request Dec 13, 2018

Tpetra: Minor Tpetra testing and interface tweaks #4049

Merged

		@@ -561,7 +562,6 @@ makeColMapAndConvertGids(GlobalOrdinal ncols,
		return rcp(new map_type(Teuchos::OrdinalTraits<GlobalOrdinal>::invalid(), colmap, 0, comm));

Tpetra: use KokkosKernels addition (and remove redundant copies of these kernels in Tpetra) #2444

Tpetra: use KokkosKernels addition (and remove redundant copies of these kernels in Tpetra) #2444

Conversation

brian-kelley commented Mar 23, 2018 • edited

Description

Motivation and Context

How Has This Been Tested?

Screenshots

Checklist

Additional Information

trilinos-autotester commented Mar 23, 2018

trilinos-autotester commented Mar 23, 2018

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

Jenkins Parameters

Using Repos:

trilinos-autotester commented Mar 23, 2018

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

Jenkins Parameters

trilinos-autotester commented Mar 23, 2018

trilinos-autotester commented Mar 23, 2018

mhoemmen commented Mar 23, 2018 • edited

mhoemmen commented Mar 23, 2018

brian-kelley commented Mar 23, 2018

mhoemmen left a comment

Choose a reason for hiding this comment

mhoemmen Mar 23, 2018

Choose a reason for hiding this comment

brian-kelley Apr 2, 2018

Choose a reason for hiding this comment

mhoemmen Mar 24, 2018

Choose a reason for hiding this comment

mhoemmen Mar 24, 2018

Choose a reason for hiding this comment

mhoemmen Mar 24, 2018

Choose a reason for hiding this comment

brian-kelley Mar 26, 2018

Choose a reason for hiding this comment

mhoemmen commented Mar 26, 2018

mhoemmen commented Apr 2, 2018

trilinos-autotester commented Apr 2, 2018

trilinos-autotester commented Apr 2, 2018

mhoemmen commented Apr 2, 2018

trilinos-autotester commented Apr 2, 2018

trilinos-autotester commented Apr 2, 2018

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

Jenkins Parameters

Using Repos:

trilinos-autotester commented Apr 3, 2018

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

Jenkins Parameters

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

Jenkins Parameters

trilinos-autotester commented Apr 3, 2018

brian-kelley commented Mar 23, 2018 •

edited

mhoemmen commented Mar 23, 2018 •

edited