pretrained_model_dir not implemented

[ekuznetsov139]

layers/common_hparams.py mentions a hyperparameter "pretrained_model_dir":

"Directory containing a checkpoint for a pretrained model. This will only be used if a new run is being started. Parameters not found in the pretrained model will be randomly initialized. Superfluous parameters in the pretrained model will be ignored."

It sounds like it could be useful, however, as far as I can tell, the hyperparameter is not actually hooked up anywhere. The name of the parameter does not appear anywhere else in the sources of tensor2tensor, tensorflow, or tensorflow_estimator.

Is this feature obsolete or not yet implemented?

[dreamingo]

maybe option warm_start_from is what you need;

[ekuznetsov139]

Possibly, but that's not the point.

[sauravjoshi]

@ekuznetsov139 @lukaszkaiser Any update on this?

[ekuznetsov139]

I don't believe this repo is active any more.
From the readme, it is now in maintenance mode and they encourage people to use Trax instead.
Though the last time I tried to use Trax, I had to give up because it lacked some essential features like the tokenizer, and it was not clear whether they expected me to use t2t for that or to write my own code, or they simply neglected to open source the necessary bits.
Maybe things have changed since then.

[sauravjoshi]

@ekuznetsov139 Thanks pal, for the info and time.