Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CRIU Verbose Logging Reinit/Reconfig for Restore #16845

Merged
merged 1 commit into from
Mar 15, 2023

Conversation

RSalman
Copy link
Contributor

@RSalman RSalman commented Mar 7, 2023

Implementation to support CRIU Verbose GC logging.

Verbose GC logging must be properly initialized and reconfigured, taking into account both restore options and initial verbose configuration at startup. There are multiple configuration scenarios possible depending on whether logging is enabled/disabled (pre/post restore) and the options provided at each phase.

  • Introduced reinitalizationInProgress GC Extensions API, this is queried by GC to check if we're in the middle of restore reinit
  • Introduced checkOptsAndInitVerbosegclog in verbose function table, high level method to check opt args and init verbose
  • fileOpenMode implemented for VerboseManagerJava to signify that verbose file writers must append during reinit

Related: eclipse/omr#6888 (comment)
Depends on: eclipse/omr#6920

Signed-off-by: Salman Rana salman.rana@ibm.com

@RSalman RSalman force-pushed the criu-vb-rework branch 3 times, most recently from d3e423e to 4fcd3b8 Compare March 7, 2023 22:02
RSalman added a commit to RSalman/omr that referenced this pull request Mar 7, 2023
Verbose GC OMR API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 7, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
@dmitripivkine dmitripivkine added comp:vm comp:gc criu Used to track CRIU snapshot related work labels Mar 8, 2023
@RSalman RSalman marked this pull request as ready for review March 8, 2023 20:01
@RSalman
Copy link
Contributor Author

RSalman commented Mar 8, 2023

@amicic please review

cc @tajila

RSalman added a commit to RSalman/omr that referenced this pull request Mar 8, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event and verbose handler
- Added tearDown call to MM_VerboseWriterFileLogging::reconfigure to
free memory before a reinit

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 9, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event and verbose handler
- Added tearDown call to MM_VerboseWriterFileLogging::reconfigure to
free memory before a reinit

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 9, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event and verbose handler
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit
- GC Extensions Base class reinitalizationInProgress introduced,
reimplemented by derived class downstream
- VerboseManagerBase fileOpenMode introduced, reimplemented by derived
class downstream
- Verbose File Writers now query verbose manager with fileOpenMode to
determine the file should be appended to

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 9, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event and verbose handler
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit
- GC Extensions Base class reinitalizationInProgress introduced,
reimplemented by derived class downstream
- VerboseManagerBase fileOpenMode introduced, reimplemented by derived
class downstream
- Verbose File Writers now query verbose manager with fileOpenMode to
determine the file should be appended to

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 10, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event and verbose handler
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit
- GC Extensions Base class reinitalizationInProgress introduced,
reimplemented by derived class downstream
- VerboseManagerBase fileOpenMode introduced, reimplemented by derived
class downstream
- Verbose File Writers now query verbose manager with fileOpenMode to
determine the file should be appended to

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 10, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event and verbose handler
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit
- GC Extensions Base class reinitalizationInProgress introduced,
reimplemented by derived class downstream
- VerboseManagerBase fileOpenMode introduced, reimplemented by derived
class downstream
- Verbose File Writers now query verbose manager with fileOpenMode to
determine the file should be appended to

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 10, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event and verbose handler
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit
- GC Extensions Base class reinitalizationInProgress introduced,
reimplemented by derived class downstream
- VerboseManagerBase fileOpenMode introduced, reimplemented by derived
class downstream
- Verbose File Writers now query verbose manager with fileOpenMode to
determine the file should be appended to

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 10, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event and verbose handler
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit
- GC Extensions Base class reinitalizationInProgress introduced,
reimplemented by derived class downstream
- VerboseManagerBase fileOpenMode introduced, reimplemented by derived
class downstream
- Verbose File Writers now query verbose manager with fileOpenMode to
determine the file should be appended to

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 10, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event and verbose handler
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit
- GC Extensions Base class reinitalizationInProgress introduced,
reimplemented by derived class downstream
- VerboseManagerBase fileOpenMode introduced, reimplemented by derived
class downstream
- Verbose File Writers now query verbose manager with fileOpenMode to
determine the file should be appended to

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 10, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event. This is required by the
verbose handler to print init details upon snapshot restore.
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit.
- GC Extensions Base class reinitializationInProgress introduced, this
is to be implemented by downstream J9 derived class.
- VerboseManagerBase fileOpenMode query introduced, this is to be
implemented by downstream J9 derived class.
- Verbose file writers now query the verbose manager (fileOpenMode) to
determine if an opened file should be appended to.

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 14, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event. This is required by the
verbose handler to print init details upon snapshot restore.
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit.
- GC Extensions Base class reinitializationInProgress introduced, this
is to be implemented by downstream J9 derived class.
- VerboseManagerBase fileOpenMode query introduced, this is to be
implemented by downstream J9 derived class.
- Verbose file writers now query the verbose manager (fileOpenMode) to
determine if an opened file should be appended to.

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 14, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event. This is required by the
verbose handler to print init details upon snapshot restore.
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit.
- GC Extensions Base class reinitializationInProgress introduced, this
is to be implemented by downstream J9 derived class.
- VerboseManagerBase fileOpenMode query introduced, this is to be
implemented by downstream J9 derived class.
- Verbose file writers now query the verbose manager (fileOpenMode) to
determine if an opened file should be appended to.

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 14, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event. This is required by the
verbose handler to print init details upon snapshot restore.
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit.
- GC Extensions Base class reinitializationInProgress introduced, this
is to be implemented by downstream J9 derived class.
- VerboseManagerBase fileOpenMode query introduced, this is to be
implemented by downstream J9 derived class.
- Verbose file writers now query the verbose manager (fileOpenMode) to
determine if an opened file should be appended to.

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/omr that referenced this pull request Mar 14, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event. This is required by the
verbose handler to print init details upon snapshot restore.
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit.
- GC Extensions Base class reinitializationInProgress introduced, this
is to be implemented by downstream J9 derived class.
- VerboseManagerBase fileOpenMode query introduced, this is to be
implemented by downstream J9 derived class.
- Verbose file writers now query the verbose manager (fileOpenMode) to
determine if an opened file should be appended to.

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
@@ -1124,9 +1125,10 @@ j9gc_reinitialize_for_restore(J9VMThread *vmThread, const char **nlsMsgFormat)
{
MM_EnvironmentBase *env = MM_EnvironmentBase::getEnvironment(vmThread->omrVMThread);
MM_GCExtensionsBase *extensions = env->getExtensions();
MM_VerboseManagerBase *verboseGCManager = extensions->verboseGCManager;
J9JavaVM* vm = vmThread->javaVM;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
J9JavaVM* vm = vmThread->javaVM;
J9JavaVM *vm = vmThread->javaVM;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed + reverted file header dates to fix merge conflicts

Implementation to support CRIU Verbose GC logging.

Verbose GC logging must be properly initialized and reconfigured, taking
into account both restore options and initial verbose configuration at
startup. There are multiple configuration scenarios possible depending
on whether logging is enabled/disabled (pre/post restore) and the
options provided at each phase.

- Introduced reinitalizationInProgress GC Extensions API, this is
queried by GC to check if we're in the middle of restore reinit
- Introduced checkOptsAndInitVerbosegclog in verbose function table,
high level method to check opt args and init verbose
- fileOpenMode implemented for VerboseManagerJava to signify that
verbose file writers must append during reinit

Related:
eclipse/omr#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
@amicic
Copy link
Contributor

amicic commented Mar 15, 2023

Jenkins test sanity xlinuxcriu jdk17

@amicic
Copy link
Contributor

amicic commented Mar 15, 2023

Jenkins test sanity win jdk11

@amicic
Copy link
Contributor

amicic commented Mar 15, 2023

Jenkins compile aix jdk8

RSalman added a commit to RSalman/omr that referenced this pull request Mar 15, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event. This is required by the
verbose handler to print init details upon snapshot restore.
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit.
- GC Extensions Base class reinitializationInProgress introduced, this
is to be implemented by downstream J9 derived class.
- VerboseManagerBase fileOpenMode query introduced, this is to be
implemented by downstream J9 derived class.
- Verbose file writers now query the verbose manager (fileOpenMode) to
determine if an opened file should be appended to.

Related:
eclipse#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
RSalman added a commit to RSalman/openj9-omr that referenced this pull request Mar 15, 2023
Verbose GC API changes to support CRIU Verbose GC logging:
eclipse-openj9/openj9#16845

- Introduced J9HOOK_MM_OMR_REINITIALIZED event. This is required by the
verbose handler to print init details upon snapshot restore.
- Added missing tearDown call to
MM_VerboseWriterFileLogging::reconfigure to free memory before a reinit.
- GC Extensions Base class reinitializationInProgress introduced, this
is to be implemented by downstream J9 derived class.
- VerboseManagerBase fileOpenMode query introduced, this is to be
implemented by downstream J9 derived class.
- Verbose file writers now query the verbose manager (fileOpenMode) to
determine if an opened file should be appended to.

Related:
eclipse/omr#6888 (comment)

Signed-off-by: Salman Rana <salman.rana@ibm.com>
@amicic amicic merged commit 8cc902b into eclipse-openj9:master Mar 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:gc comp:vm criu Used to track CRIU snapshot related work
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants