SAMZA-2210: Initial majority migration for injecting classloader when doing reflection #1047
Conversation
… doing reflection
… more consistent with other ReflectionUtil methods
…er to be more consistent with other ReflectionUtil methods" This reverts commit ad96515. The commit being reverted should have been made on the move_reflection_utils branch, so will re-apply through that branch.
…ther PR comments" This reverts commit bb99c36. The commit being reverted should have been made on the move_reflection_utils branch, so will re-apply through that branch.
… more consistent with other ReflectionUtil methods
… more consistent with other ReflectionUtil methods
…ith getObjWithArgs
… migrate unnecessary usages of getObjWithArgs
mynameborat
left a comment
There was a problem hiding this comment.
Looks good.
One general question around code style, I noticed some of the methods/callsite have one argument per line while others have them crunched together to fit the width limit. This is prevalent across the code base but I noticed you changed some of them too. So wondering what the guideline is?
samza-core/src/main/scala/org/apache/samza/coordinator/JobModelManager.scala
Outdated
Show resolved
Hide resolved
My personal preference is that "long" argument lists have the format of one argument per line. "Long" can be pretty subjective, but I usually consider argument lists that take up 3 or more lines as "long". The reason why I prefer this style is because future changes to the argument lists are easier to diff (e.g. diff has a single-line addition instead of possibly having multiple lines updated due to overflow of line length). I will sometimes make those style changes when I need to change an argument list. |
|
+1 for Cameron's suggestion on long param list formatting. My "long" threshold is higher (6-7 params / 1-2 lines), but I also prefer to keep/make longer lists multi-line. |
|
+1;
|
Sounds good! We have a coding guideline that touches some areas but not comprehensive. At least that is the my motivation behind these guidelines/style questions. I'd be happy to consolidate our guidelines into a comprehensive wiki and share it. |
prateekm
left a comment
There was a problem hiding this comment.
General questions about whether naming the fields / params / variables more explicitly (fwkClassloader vs pluginClassloader) help clarify the expected usage.
Some other general questions about whether a fwk or plugin classloader should be used.
Otherwise LGTM, thanks for the cleanup as well.
samza-core/src/main/java/org/apache/samza/clustermanager/AbstractContainerAllocator.java
Outdated
Show resolved
Hide resolved
samza-azure/src/main/java/org/apache/samza/coordinator/AzureJobCoordinator.java
Show resolved
Hide resolved
samza-core/src/main/java/org/apache/samza/runtime/LocalApplicationRunner.java
Show resolved
Hide resolved
samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala
Show resolved
Hide resolved
samza-core/src/main/scala/org/apache/samza/coordinator/JobModelManager.scala
Show resolved
Hide resolved
samza-core/src/main/scala/org/apache/samza/storage/ContainerStorageManager.java
Outdated
Show resolved
Hide resolved
samza-log4j/src/main/java/org/apache/samza/logging/log4j/StreamAppender.java
Show resolved
Hide resolved
samza-log4j2/src/main/java/org/apache/samza/logging/log4j2/StreamAppender.java
Show resolved
Hide resolved
prateekm
left a comment
There was a problem hiding this comment.
LGTM, thanks. Feel free to merge after updating with the 'extract the side inputs processor' comment.
When doing reflection, we will now explicitly specify the classloader to use. This will allow us to inject a custom classloader in the future, in order to do things like isolate Samza infrastructure from application code.
From a functionality perspective, this should not change anything. Ultimately, for this PR, the classloader that is passed through is the same classloader that is already being used. The purpose of this PR is to do the wiring to allow easier evolution in a follow-up PR.
This does not cover all reflection cases. There are several other follow-up PRs which will handle other reflection cases. This PR is already very significantly large, so trying to keep it from being even larger.
This PR includes #1044.