Optimize tensor copies on inference path

[aobolensk]

No description provided.

[aobolensk]

@AndreySorokin7 could you please verify the patch on real scenarios? Unit tests are accelerated significantly

[codecov]

Codecov Report

❌ Patch coverage is 76.92308% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.59%. Comparing base (f5408c0) to head (359024f).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
include/graph/graph.hpp	76.92%	1 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #245      +/-   ##
==========================================
- Coverage   81.63%   81.59%   -0.04%     
==========================================
  Files          47       47              
  Lines        2385     2391       +6     
  Branches     1409     1413       +4     
==========================================
+ Hits         1947     1951       +4     
- Misses        212      213       +1     
- Partials      226      227       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[AndreySorokin7]

Вот до
Elapsed inference time:101168
Вот после
Elapsed inference time:90479
На alexnet_mnist

Вот на yolo было
Elapsed inference time:22116
Стало
Elapsed inference time:19003

Выигрыш есть я думаю можно Мёржить. Запускал раз пять и всегда вот такие различия во времени.

[aobolensk]

Вот до Elapsed inference time:101168 Вот после Elapsed inference time:90479 На alexnet_mnist

Вот на yolo было Elapsed inference time:22116 Стало Elapsed inference time:19003

Выигрыш есть я думаю можно Мёржить. Запускал раз пять и всегда вот такие различия во времени.

Thank you for the verification!