jitify 2 support - #7372
Merged
Merged
Conversation
github-actions
Bot
added
CMake
kkraus14
added
improvement
cwharris
changed the title
devavret
approved these changes
Apr 1, 2021
robertmaynard
approved these changes
Apr 1, 2021
Contributor
Author
|
rerun tests |
trxcllnt
reviewed
Apr 1, 2021
Codecov Report
@@ Coverage Diff @@
## branch-0.19 #7372 +/- ##
===============================================
+ Coverage 81.86% 82.74% +0.88%
===============================================
Files 101 103 +2
Lines 16884 17700 +816
===============================================
+ Hits 13822 14646 +824
+ Misses 3062 3054 -8
Continue to review full report at Codecov.
|
kkraus14
approved these changes
Apr 2, 2021
Contributor
|
I believe @cwharris has one more addition related to the in-memory cache before we should merge this |
Contributor
Author
|
ready for re-review. |
cwharris
added
5 - Ready to Merge
trxcllnt
approved these changes
Apr 5, 2021
Contributor
|
@gpucibot merge |
shwina
pushed a commit
to shwina/cudf
that referenced
this pull request
Apr 7, 2021
This pr upgrades to jitify2. To accomplish this, we adopt jitify's new `ProgramCache` API, which handles in-memory and disk caching, allowing us to delete our jit caching code, and simplifying our jit kernel launches. Kernels and headers which were previously stringified at compile time and assembled together in cudf code at run time are now preprocessed using a new `jitify_preprocess` utility at compile time, which produces a single file per kernel that can be de-serialized in to a jitify program. kernel source files which have been serialized with jitifiy_preprocess include all associated headers, and jitify resolves these automatically at jit time. It also allows us to override individual headers on a per-compilation basis. This override is what we can use to inject runtime code (udfs), rather than manually concatenating strings. I'm currently working on a bug where jitify is unable to obtain a file lock - disk caching is disable until that is fixed. Hoping to get that fixed here in the next few hours. Authors: - Christopher Harris (https://github.com/cwharris) Approvers: - Robert Maynard (https://github.com/robertmaynard) - Devavret Makkar (https://github.com/devavret) - Keith Kraus (https://github.com/kkraus14) - Paul Taylor (https://github.com/trxcllnt) URL: NVIDIA#7372
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pr upgrades to jitify2. To accomplish this, we adopt jitify's new
ProgramCacheAPI, which handles in-memory and disk caching, allowing us to delete our jit caching code, and simplifying our jit kernel launches. Kernels and headers which were previously stringified at compile time and assembled together in cudf code at run time are now preprocessed using a newjitify_preprocessutility at compile time, which produces a single file per kernel that can be de-serialized in to a jitify program. kernel source files which have been serialized with jitifiy_preprocess include all associated headers, and jitify resolves these automatically at jit time. It also allows us to override individual headers on a per-compilation basis. This override is what we can use to inject runtime code (udfs), rather than manually concatenating strings.I'm currently working on a bug where jitify is unable to obtain a file lock - disk caching is disable until that is fixed. Hoping to get that fixed here in the next few hours.