Add MXNet prediction tool #4167
Conversation
|
A new Pull Request was created by @hqucms for branch IB/CMSSW_10_2_X/gcc630. @cmsbuild, @smuzaffar, @gudrutis, @mrodozov can you please review it and eventually sign? Thanks. |
|
@gouskos who would also like to follow this thread. |
|
please test |
|
The tests are being triggered in jenkins. |
|
@hqucms , please also update cmssw-tool-conf.spec to depend on mxnet-predict-toolfile. This is needed to get it within cmssw env. |
|
@mrodozov , can you please check if this builds correctly on slc7/aarch64 and , gcc7/gcc8? |
|
Pull request #4167 was updated. |
|
@smuzaffar Done. |
|
Comparison job queued. |
|
Comparison is ready Comparison Summary:
|
|
tested on slc6_amd64_gcc630,700 slc7_amd64_gcc700,810 slc7_aarch64_gcc700
|
|
please test |
|
The tests are being triggered in jenkins. |
|
Comparison job queued. |
|
+externals |
cmsbuild
added
externals-approved
fully-signed
and removed
externals-pending
pending-signatures
labels
|
This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_10_2_X/gcc630 IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2) |
|
Comparison is ready Comparison Summary:
|
|
@smuzaffar @fabiocos |
|
@slava77 as the build looks functional I think that we have no technical obstacle to integrate it, @smuzaffar please comment in case. Anyway, looking at the discussions at the root of this request, I would like to better understand the overall strategy, as this is the 3rd deep learning-oriented external tool that we are integrating to my knowledge, besides tensorflow and lwtnn. From the slides of @Dr15Jones and @makortel at the november O&C meeting I understand that the idea is to use the DeepAK8 tagger based on MXNet as a testbed for that approach. As I did not take part to that discussion, I would like to understand whether this request is part of an overall strategy, as it seems. Do we want to have the 3 of them within CMSSW for comparison, with the idea of possibly moving in the longer term towards a single approach? |
|
My feeling (could be wrong) is that at the moment we don't have enough experience on these tools on the inference side to make a clear choice (are there any other downsides than supporting yet another external?). Also, if we start restricting the inference tools, it should be clearly communicated to the developer community. |
|
Hi
I agree. If we want to move out of the “let 1000 flowers bloom” mode in this problem space, a pr is not the place to decide (nor is first come first served the approach to take). I haven’t seen any research to suggest that one solution is now covering the problem space, so am biased toward continuing to integrate data science supported tools.
Cheers,
David
On 11 Jul 2018, at 22:14, Matti Kortelainen <notifications@github.com<mailto:notifications@github.com>> wrote:
My feeling (could be wrong) is that at the moment we don't have enough experience on these tools on the inference side to make a clear choice (are there any other downsides than supporting yet another external?). Also, if we start restricting the inference tools, it should be clearly communicated to the developer community.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#4167 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEzyw_byfnF50XCV9XKaPyeE3ZvCEHSLks5uFk5ygaJpZM4VBJSo>.
|
|
@davidlange6 I agree that general long term strategies goes beyond the single PR discussion. But the problem is practically posed through PRs. This tool was already discussed and mentioned, so ok, but in case more arrives, I think we need to understand where we want to go |
|
+1 |
This is to add MXNet to CMSSW externals following the discussions in cms-sw/cmssw#21314. It is needed for the integration of the DeepAK8 tagger into CMSSW.
This includes both the C and C++ APIs. MXNet is built in prediction-only mode (
MXNET_PREDICT_ONLY=1) here, as so far we only intend to use it for the evaluation of the neural networks, not for training (which is typically done on a GPU w/ the python API). By compiling in the prediction-only mode, MXNet is also set to run in Native Engine mode such that it runs in the master thread instead of creating its own thread pool.