Add Huggingface-cli usage #4027

karthik19967829 · 2022-02-03T10:31:08Z

This PR adds Huggingface-cli usage to upload models to Huggingface Hub

codecov · 2022-02-03T10:49:37Z

Codecov Report

Merging #4027 (354b981) into master (a3e1543) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #4027   +/-   ##
=======================================
  Coverage   80.94%   80.94%           
=======================================
  Files         435      435           
  Lines       37429    37429           
=======================================
  Hits        30297    30297           
  Misses       7132     7132

Flag	Coverage Δ
test_integration_espnet1	`67.13% <ø> (ø)`
test_integration_espnet2	`52.44% <ø> (ø)`
test_python	`66.61% <ø> (ø)`
test_utils	`24.45% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3e1543...354b981. Read the comment docs.

ftshijt

Many thanks!

CONTRIBUTING.md

sw005320 · 2022-02-04T20:31:55Z

I think we can remove the zenodo description (or at least emphasize that this is deprecated and we can move this part in the later part).

ftshijt · 2022-02-04T21:02:17Z

Make sense, maybe just add a note (Not recommended) or something. It may still be interested by someone, so leaving it seems OK

karthik19967829 · 2022-02-14T02:06:09Z

I think we can remove the zenodo description (or at least emphasize that this is deprecated and we can move this part in the later part).

Yes this makes sense, I have added this info next to the zenodo description in the doc

CONTRIBUTING.md

ftshijt · 2022-02-14T05:55:01Z

egs2/TEMPLATE/asr1/asr.sh

@@ -1478,7 +1478,7 @@ fi
 if ! "${skip_upload_hf}"; then
    if [ ${stage} -le 16 ] && [ ${stop_stage} -ge 16 ]; then
        [ -z "${hf_repo}" ] && \
-            log "ERROR: You need to setup the variable hf_repo with the name of the repository located at HuggingFace" && \
+            log "ERROR: You need to setup the variable hf_repo with the name of the repository located at HuggingFace, follow the following steps described here https://github.com/espnet/espnet/pull/4027/files#diff-b9f381af6d3ca8c14a68dca2a24de815830013fd69a3fe6a113e108c7aaf0d2dR1490-R1505" && \ 


The link here is not the correct link to file, but the pull info. Can you fix it as well?

Sure , updated with a permanent link

Could you make it not stick to a specific commit? It won't reflect future changes with a fixed commit.

Hey in the latest commit the link is updated as

espnet/CONTRIBUTING.md

Line 68 in 5cc31d7

To upload models using Huggingface-cli follow the following steps:

, if this is not the permalink could guide me how to get it ?

The link can be `https://github.com/espnet/espnet/blob/master/CONTRIBUTING.md#132-espnet2-recipes·

updated the link

CONTRIBUTING.md

Co-authored-by: Shinji Watanabe <sw005320@gmail.com>

ftshijt · 2022-02-15T03:39:51Z

LGTM. I will merge it after passing the CI tests

karthik19967829 · 2022-02-18T13:27:00Z

@ftshijt if I apply black locally, it says says no file to be reformatted. How to fix this CI error?

sw005320 · 2022-02-18T20:44:04Z

can you check that the black version was correct?
@roshansh-cmu had a similar issue recently. @roshansh-cmu, can you give us some advice?

roshansh-cmu · 2022-02-18T21:21:51Z

@karthik19967829 @sw005320 - I recommend creating an environment with python=3.6 or 3.7, and installing all the CI tools there. Install black with python=3.6/3.7 and redo the black formatting. That worked for me

karthik19967829 · 2022-02-19T06:10:17Z

@karthik19967829 @sw005320 - I recommend creating an environment with python=3.6 or 3.7, and installing all the CI tools there. Install black with python=3.6/3.7 and redo the black formatting. That worked for me

Thank you this helped .

ftshijt · 2022-02-19T10:30:02Z

Many thanks for the contribution!

Add Huggingface-cli usage

6a46783

sw005320 requested a review from ftshijt February 3, 2022 12:52

sw005320 added the Documentation label Feb 3, 2022

sw005320 added this to the v.0.10.6 milestone Feb 3, 2022

ftshijt reviewed Feb 3, 2022

View reviewed changes

CONTRIBUTING.md Outdated Show resolved Hide resolved

kan-bayashi modified the milestones: v.0.10.6, v.0.10.7 Feb 8, 2022

karthik19967829 added 2 commits February 13, 2022 20:51

Update CONTRIBUTING.md

9d9295a

Update asr.sh

8ced059

mergify bot added the ESPnet2 label Feb 14, 2022

ftshijt reviewed Feb 14, 2022

View reviewed changes

CONTRIBUTING.md Show resolved Hide resolved

Update asr.sh

8484ff9

ftshijt reviewed Feb 14, 2022

View reviewed changes

sw005320 reviewed Feb 14, 2022

View reviewed changes

karthik19967829 and others added 9 commits February 14, 2022 15:33

Update CONTRIBUTING.md

9b49a3d

Co-authored-by: Shinji Watanabe <sw005320@gmail.com>

Update CONTRIBUTING.md

4a4d87d

Co-authored-by: Shinji Watanabe <sw005320@gmail.com>

Update CONTRIBUTING.md

d0bf6ce

Co-authored-by: Shinji Watanabe <sw005320@gmail.com>

Update CONTRIBUTING.md

4a0e631

Co-authored-by: Shinji Watanabe <sw005320@gmail.com>

Update CONTRIBUTING.md

88c1021

Co-authored-by: Shinji Watanabe <sw005320@gmail.com>

Update CONTRIBUTING.md

992e184

Co-authored-by: Shinji Watanabe <sw005320@gmail.com>

Update CONTRIBUTING.md

5cc31d7

Co-authored-by: Shinji Watanabe <sw005320@gmail.com>

Update asr.sh

94eaab3

Update asr.sh

baa0974

Add black

3861a96

mergify bot added the ESPnet1 label Feb 15, 2022

mergify bot added the ASR Automatic speech recogntion label Feb 15, 2022

karthik19967829 added 2 commits February 15, 2022 07:57

Merge branch 'espnet:master' into patch-1

b89dc4d

Update asr.sh

f87a7b1

Fix black formatting

354b981

ftshijt merged commit d4eb886 into espnet:master Feb 19, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Huggingface-cli usage #4027

Add Huggingface-cli usage #4027

karthik19967829 commented Feb 3, 2022 •

edited

codecov bot commented Feb 3, 2022 •

edited

ftshijt left a comment

sw005320 commented Feb 4, 2022

ftshijt commented Feb 4, 2022

karthik19967829 commented Feb 14, 2022

ftshijt Feb 14, 2022

karthik19967829 Feb 14, 2022

ftshijt Feb 14, 2022

karthik19967829 Feb 14, 2022

ftshijt Feb 15, 2022

karthik19967829 Feb 15, 2022

ftshijt commented Feb 15, 2022

karthik19967829 commented Feb 18, 2022

sw005320 commented Feb 18, 2022

roshansh-cmu commented Feb 18, 2022

karthik19967829 commented Feb 19, 2022

ftshijt commented Feb 19, 2022

Add Huggingface-cli usage #4027

Add Huggingface-cli usage #4027

Conversation

karthik19967829 commented Feb 3, 2022 • edited

codecov bot commented Feb 3, 2022 • edited

Codecov Report

ftshijt left a comment

Choose a reason for hiding this comment

sw005320 commented Feb 4, 2022

ftshijt commented Feb 4, 2022

karthik19967829 commented Feb 14, 2022

ftshijt Feb 14, 2022

Choose a reason for hiding this comment

karthik19967829 Feb 14, 2022

Choose a reason for hiding this comment

ftshijt Feb 14, 2022

Choose a reason for hiding this comment

karthik19967829 Feb 14, 2022

Choose a reason for hiding this comment

ftshijt Feb 15, 2022

Choose a reason for hiding this comment

karthik19967829 Feb 15, 2022

Choose a reason for hiding this comment

ftshijt commented Feb 15, 2022

karthik19967829 commented Feb 18, 2022

sw005320 commented Feb 18, 2022

roshansh-cmu commented Feb 18, 2022

karthik19967829 commented Feb 19, 2022

ftshijt commented Feb 19, 2022

karthik19967829 commented Feb 3, 2022 •

edited

codecov bot commented Feb 3, 2022 •

edited