Skip to content

Conversation

@guitargeek
Copy link
Contributor

@guitargeek guitargeek commented Apr 14, 2025

This makes it possible to avoid memory allocations during inference, also also will help when differentiating through the code with Clad.

Here is a showcase of how the output code of SOFIE would look like with this change. Note that I chose the name doInfer for the new function to better distinguish it from the existing infer method, but I'm happy to rename it to anything including infer as well.

struct Session {
   ...
   // Inference implementation is forward declared:
   void doInfer(float *tensor_x,
                float *tensor_theory_params,
                std::vector<float> &output_tensor_linear_3);
   ...

   std::vector<float> infer(float *tensor_x, float *tensor_theory_params)
   {
      std::vector<float> output_tensor_linear_3;
      doInfer(tensor_x, tensor_theory_params, output_tensor_linear_3);
      return output_tensor_linear_3;
   }
}; // end of Session

void Session::doInfer(float *tensor_x,
                      float *tensor_theory_params,
                      std::vector<float> &output_tensor_linear_3)
{
   // inference code dumped here:
   ...

   // outputs are filled at the end
   using TMVA::Experimental::SOFIE::UTILITY::FillOutput;
   FillOutput(tensor_linear_3, output_tensor_linear_3, 1);
}

@guitargeek guitargeek requested a review from sanjibansg April 14, 2025 13:03
@guitargeek guitargeek self-assigned this Apr 14, 2025
@guitargeek guitargeek requested a review from lmoneta as a code owner April 14, 2025 13:03
Copy link
Member

@lmoneta lmoneta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
Thank you Jonas for this contribution!

This makes it possible to avoid memory allocations during inference,
also also will help when differentiating through the code with Clad.

Here is a showcase of how the output code of SOFIE would look like with
this change:
```c++
struct Session {
   ...
   // Inference implementation is forward declared:
   void doInfer(float *tensor_x,
                float *tensor_theory_params,
                std::vector<float> &output_tensor_linear_3);
   ...

   std::vector<float> infer(float *tensor_x, float *tensor_theory_params)
   {
      std::vector<float> output_tensor_linear_3;
      doInfer(tensor_x, tensor_theory_params, output_tensor_linear_3);
      return output_tensor_linear_3;
   }
}; // end of Session

void Session::doInfer(float *tensor_x,
                      float *tensor_theory_params,
                      std::vector<float> &output_tensor_linear_3)
{
   // inference code dumped here:
   ...

   // outputs are filled at the end
   using TMVA::Experimental::SOFIE::UTILITY::FillOutput;
   FillOutput(tensor_linear_3, output_tensor_linear_3, 1);
}
```
@guitargeek guitargeek added this to the 6.36.00 milestone Apr 14, 2025
@github-actions
Copy link

Test Results

    18 files      18 suites   4d 3h 51m 37s ⏱️
 2 738 tests  2 736 ✅ 0 💤 2 ❌
47 611 runs  47 609 ✅ 0 💤 2 ❌

For more details on these failures, see this check.

Results for commit 6ccaf8e.

@dpiparo dpiparo merged commit 76f671d into root-project:master Apr 15, 2025
19 of 22 checks passed
@guitargeek guitargeek deleted the sofie_dev branch April 15, 2025 05:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants