Clean directory after script testing

[JetRunner]

#6421 #6433

This PR cleans the directory after each script testing to prevent bugs like this.

[codecov]

Codecov Report

Merging #6453 into master will decrease coverage by 2.51%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #6453      +/-   ##
==========================================
- Coverage   79.89%   77.37%   -2.52%     
==========================================
  Files         153      153              
  Lines       27902    27902              
==========================================
- Hits        22291    21588     -703     
- Misses       5611     6314     +703     

Impacted Files	Coverage Δ
src/transformers/optimization.py	25.55% <0.00%> (-70.00%)	⬇️
src/transformers/modeling_tf_distilbert.py	34.11% <0.00%> (-63.30%)	⬇️
src/transformers/pipelines.py	26.98% <0.00%> (-52.81%)	⬇️
src/transformers/optimization_tf.py	33.33% <0.00%> (-24.33%)	⬇️
src/transformers/modeling_tf_auto.py	48.79% <0.00%> (-18.08%)	⬇️
src/transformers/modeling_auto.py	64.16% <0.00%> (-14.46%)	⬇️
src/transformers/data/processors/squad.py	13.76% <0.00%> (-14.38%)	⬇️
src/transformers/modeling_tf_gpt2.py	65.68% <0.00%> (-6.16%)	⬇️
src/transformers/modelcard.py	82.71% <0.00%> (-2.47%)	⬇️
src/transformers/modeling_distilbert.py	96.19% <0.00%> (-1.64%)	⬇️
... and 17 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4ffea5c...05950f1. Read the comment docs.

[JetRunner]

I think cleaning the output dir seems to be a better solution than modifying output_dir, since people may still copy those output_dir in the future. What do you think? @LysandreJik

[sgugger]

Looks good to me!