failed to get job logs

[mbarnig]

I deployed my own voice-builder on GCP and started one job with Festival and another with Merlin with the default voice parameters. After 12 hours the status of both jobs is still "Started Training Voice Model".

When I look for more details in
"Pipeline API log | Stdout log | Stderr log"
I receive an error message "Failed to get job logs (jobId: x)".

I see the same problems in the public voice-builder (http://tinyurl.com/voice-builder) for the latest jobs with the status "Exporting resources".

[pasindud]

"Failed of get job logs" mean data was exported but the voice building never got started.

We run the voice building jobs using docker, they are scheduled and orchestrated by Google Genomics Pipeline API

Goto
https://console.cloud.google.com/genomics/operations?project=<project_name>

• This will give list of pipeline api operations names (they should be red)

Get a operation name that failed, then run the following in the terminal using gcloud

gcloud alpha genomics  operations  describe <operation name>

In the result there should be some information about why it failed.

• The issue in the demo server is because quotas we have put in place.

[mbarnig]

Thank you Pasindu for your quick and comprehensive answer.

In the mean time I executed two additional jobs, the last one with the data of the public Sinhala example stored in my own Google storage bucket. Both jobs were running fine.

If I look at the Genomics console I see only the two last API operations which are green (status: success). My two first jobs are not listed.

Perhaps it was only a temporary problem on the platform or an issue related to quota, because I started the two first jobs at the same time.

In a next step I will try to run a job with my own data to create a luxembourgish voice (low-resourced language). In the past I did several trials to create a luxembourgish voice in Festival, Merlin and MaryTTS on my own development system. All my attempts failed due to technical complexity.

Therefore I appreciate your great Voice-Builder project and I hope to move forward with my experiments.

best greetings,
Marco Barnig

[pasindud]

If those two are green, check in the GCS bucket gs://<project name>-jobs/<number>/ there should be a file called built-voice.tar.gz . If that is there it means the voice was built. You just have to click deploy on the jobs UI to deploy the model to a server (it should be already done automatically).

• When building a new language make sure the phonology is in the right format doc

Many examples can be found at [1] under each language's festvox folder example si/festvox/ipa_phonology.json

• Make sure the lexicon is in the correct format, you can use this tool [2] to convert a TSV lexicon to festival format SCM.

• Note that these are parametric voices and not unit selection

1 - https://github.com/googlei18n/language-resources
2 - https://github.com/googlei18n/language-resources/blob/master/festival_utils/festival_lexicon_from_tsv.py

Hope you train a luxembourgish TTS

[mirfan899]

How did you solve the issue @mbarnig ?