detach output from teacher

[twmht]

When using Self Distiller with Channel Wise Distill, there would be a backward exception due to the tensor from teacher is not detached.

The PR fixed this bug.

[twmht]

Or we can detach the tensor in cwd loss (https://github.com/open-mmlab/mmrazor/blob/master/mmrazor/models/losses/cwd.py) , what do you think?

[pppppM]

I'm so sorry, I haven't found this PR due to my negligence.

There are some users who will train both teacher and student at the same time.
Maybe adding a flag in SelfDistiller is better.
In SingleTeacherDistiller, use teacher_trainable to control.

mmrazor/mmrazor/models/distillers/single_teacher.py

Lines 172 to 176 in e4e9513

	if self.teacher_trainable:
	output = self.teacher(**data)
	else:
	with torch.no_grad():
	output = self.teacher(**data)

[pppppM]

Would you like to reopen this pr and complete this feature together?

[twmht]

yup it would be better.

[codecov]

Codecov Report

Merging #100 (c19a8ec) into master (0dd407a) will increase coverage by 9.72%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #100      +/-   ##
==========================================
+ Coverage   56.58%   66.31%   +9.72%     
==========================================
  Files          83       92       +9     
  Lines        2932     3369     +437     
  Branches      540      613      +73     
==========================================
+ Hits         1659     2234     +575     
+ Misses       1197     1033     -164     
- Partials       76      102      +26     

Flag	Coverage Δ
unittests	66.28% <100.00%> (+9.69%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmrazor/models/distillers/self_distiller.py	100.00% <100.00%> (ø)
mmrazor/utils/__init__.py	100.00% <0.00%> (ø)
mmrazor/models/ops/__init__.py	100.00% <0.00%> (ø)
mmrazor/models/losses/__init__.py	100.00% <0.00%> (ø)
mmrazor/models/ops/darts_series.py	81.72% <0.00%> (ø)
...els/architectures/components/backbones/__init__.py	100.00% <0.00%> (ø)
.../architectures/components/heads/no_bias_fc_head.py	0.00% <0.00%> (ø)
...chitectures/components/backbones/darts_backbone.py	88.63% <0.00%> (ø)
mmrazor/models/ops/mobilenet_series.py	95.65% <0.00%> (ø)
mmrazor/apis/mmdet/inference.py	51.51% <0.00%> (ø)
... and 47 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0dd407a...c19a8ec. Read the comment docs.