Pre-Trained Machine Translation

[shankarthebest]

Hi ,

Do you have pre-trained models to translate English to French. If yes, it would be of great help.
Please let me know.

Thanks,
Shankar R

[w4nderlust]

Hi Shankar, at the moment we don't have a model zoo with pre-trained models. It would be a nice addition in the future. You may start it by contributing your translation model if you end up building one yourself.

[justinepdevasia]

Hi,

When I tried to do machine translation from english to hindi, I am getting the following error,

I am getting the same for italian also

The config file is given below

`input_features:
-
name: english
type: text
level: word
encoder: rnn
cell_type: lstm
reduce_output: null
preprocessing:
word_tokenizer: english_tokenize

output_features:
-
name: hindi
type: text
level: word
decoder: generator
cell_type: lstm
attention: bahdanau
reduce_input: null
loss:
type: sampled_softmax_cross_entropy
preprocessing:
word_tokenizer: multi_tokenize

training:
batch_size: 96
`

`2021-03-06 09:47:08.422559: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
/usr/local/lib/python3.7/dist-packages/typeguard/init.py:804: UserWarning: no type annotations present -- not typechecking tensorflow_addons.layers.max_unpooling_2d.MaxUnpooling2D.init
warn('no type annotations present -- not typechecking {}'.format(function_name(func)))
███████████████████████
█ █ █ █ ▜█ █ █ █ █ █
█ █ █ █ █ █ █ █ █ █ ███
█ █ █ █ █ █ █ █ █ ▌ █
█ █████ █ █ █ █ █ █ █ █
█ █ ▟█ █ █ █
███████████████████████
ludwig v0.3.3 - Experiment

2021-03-06 09:47:09.938591: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-03-06 09:47:09.939327: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2021-03-06 09:47:09.953477: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-03-06 09:47:09.954112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:00:04.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.90GiB deviceMemoryBandwidth: 681.88GiB/s
2021-03-06 09:47:09.954158: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-03-06 09:47:09.956661: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2021-03-06 09:47:09.956750: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2021-03-06 09:47:09.958380: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-03-06 09:47:09.958744: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-03-06 09:47:09.960439: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2021-03-06 09:47:09.961020: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2021-03-06 09:47:09.961214: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2021-03-06 09:47:09.961307: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-03-06 09:47:09.961915: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-03-06 09:47:09.962437: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
Experiment name: experiment
Model name: run
Output directory: results/experiment_run_10

ludwig_version: '0.3.3'
command: ('/usr/local/bin/ludwig experiment --dataset translation.csv --config_file '
'config.yaml')
random_seed: 42
dataset: 'translation.csv'
data_format: 'csv'
config: { 'combiner': {'type': 'concat'},
'input_features': [ { 'cell_type': 'lstm',
'column': 'english',
'encoder': 'rnn',
'level': 'word',
'name': 'english',
'preprocessing': { 'word_tokenizer': 'english_tokenize'},
'proc_column': 'english_o_tfkJ',
'reduce_output': None,
'tied': None,
'type': 'text'}],
'output_features': [ { 'attention': 'bahdanau',
'cell_type': 'lstm',
'column': 'hindi',
'decoder': 'generator',
'dependencies': [],
'level': 'word',
'loss': { 'class_similarities_temperature': 0,
'class_weights': 1,
'confidence_penalty': 0,
'distortion': 0.75,
'labels_smoothing': 0,
'negative_samples': 25,
'robust_lambda': 0,
'sampler': 'log_uniform',
'type': 'sampled_softmax_cross_entropy',
'unique': False,
'weight': 1},
'name': 'hindi',
'preprocessing': { 'word_tokenizer': 'multi_tokenize'},
'proc_column': 'hindi_x50Q5l',
'reduce_dependencies': 'sum',
'reduce_input': None,
'type': 'text'}],
'preprocessing': { 'audio': { 'audio_feature': {'type': 'raw'},
'audio_file_length_limit_in_s': 7.5,
'in_memory': True,
'missing_value_strategy': 'backfill',
'norm': None,
'padding_value': 0},
'bag': { 'fill_value': '',
'lowercase': False,
'missing_value_strategy': 'fill_with_const',
'most_common': 10000,
'tokenizer': 'space'},
'binary': { 'fill_value': 0,
'missing_value_strategy': 'fill_with_const'},
'category': { 'fill_value': '',
'lowercase': False,
'missing_value_strategy': 'fill_with_const',
'most_common': 10000},
'date': { 'datetime_format': None,
'fill_value': '',
'missing_value_strategy': 'fill_with_const'},
'force_split': False,
'h3': { 'fill_value': 576495936675512319,
'missing_value_strategy': 'fill_with_const'},
'image': { 'in_memory': True,
'missing_value_strategy': 'backfill',
'num_processes': 1,
'resize_method': 'interpolate',
'scaling': 'pixel_normalization'},
'numerical': { 'fill_value': 0,
'missing_value_strategy': 'fill_with_const',
'normalization': None},
'sequence': { 'fill_value': '',
'lowercase': False,
'missing_value_strategy': 'fill_with_const',
'most_common': 20000,
'padding': 'right',
'padding_symbol': '',
'sequence_length_limit': 256,
'tokenizer': 'space',
'unknown_symbol': '',
'vocab_file': None},
'set': { 'fill_value': '',
'lowercase': False,
'missing_value_strategy': 'fill_with_const',
'most_common': 10000,
'tokenizer': 'space'},
'split_probabilities': (0.7, 0.1, 0.2),
'stratify': None,
'text': { 'char_most_common': 70,
'char_sequence_length_limit': 1024,
'char_tokenizer': 'characters',
'char_vocab_file': None,
'fill_value': '',
'lowercase': True,
'missing_value_strategy': 'fill_with_const',
'padding': 'right',
'padding_symbol': '',
'pretrained_model_name_or_path': None,
'unknown_symbol': '',
'word_most_common': 20000,
'word_sequence_length_limit': 256,
'word_tokenizer': 'space_punct',
'word_vocab_file': None},
'timeseries': { 'fill_value': '',
'missing_value_strategy': 'fill_with_const',
'padding': 'right',
'padding_value': 0,
'timeseries_length_limit': 256,
'tokenizer': 'space'},
'vector': { 'fill_value': '',
'missing_value_strategy': 'fill_with_const'}},
'training': { 'batch_size': 96,
'bucketing_field': None,
'decay': False,
'decay_rate': 0.96,
'decay_steps': 10000,
'early_stop': 5,
'epochs': 100,
'eval_batch_size': 0,
'gradient_clipping': None,
'increase_batch_size_on_plateau': 0,
'increase_batch_size_on_plateau_max': 512,
'increase_batch_size_on_plateau_patience': 5,
'increase_batch_size_on_plateau_rate': 2,
'learning_rate': 0.001,
'learning_rate_warmup_epochs': 1,
'optimizer': { 'beta_1': 0.9,
'beta_2': 0.999,
'epsilon': 1e-08,
'type': 'adam'},
'reduce_learning_rate_on_plateau': 0,
'reduce_learning_rate_on_plateau_patience': 5,
'reduce_learning_rate_on_plateau_rate': 0.5,
'regularization_lambda': 0,
'regularizer': 'l2',
'staircase': False,
'validation_field': 'combined',
'validation_metric': 'loss'}}
tf_version: '2.4.1'

Found hdf5 and meta.json with the same filename of the dataset, but checksum don't match, if saving of processed input is not skipped they will be overridden
Using full raw csv, no hdf5 and json file with the same name have been found
Building dataset (it may take a while)
Loading NLP pipeline
2021-03-06 09:47:10.411369: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
Loading NLP pipeline
Writing preprocessed dataset cache
Writing train set metadata
Training set: 2047
Validation set: 282
Test set: 586
2021-03-06 09:47:14.369377: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-03-06 09:47:14.369588: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-03-06 09:47:14.370201: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:00:04.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.90GiB deviceMemoryBandwidth: 681.88GiB/s
2021-03-06 09:47:14.370280: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-03-06 09:47:14.370321: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2021-03-06 09:47:14.370347: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2021-03-06 09:47:14.370371: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-03-06 09:47:14.370393: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-03-06 09:47:14.370419: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2021-03-06 09:47:14.370441: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2021-03-06 09:47:14.370463: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2021-03-06 09:47:14.370560: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-03-06 09:47:14.371143: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-03-06 09:47:14.371650: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-03-06 09:47:16.766733: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-03-06 09:47:16.766796: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] 0
2021-03-06 09:47:16.766815: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0: N
2021-03-06 09:47:16.767034: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-03-06 09:47:16.767658: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-03-06 09:47:16.768229: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-03-06 09:47:16.768775: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14445 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0)

╒══════════╕
│ TRAINING │
╘══════════╛

Epoch 1
Training: 0% 0/22 [00:00<?, ?it/s]
y
Traceback (most recent call last):
File "/usr/local/bin/ludwig", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.7/dist-packages/ludwig/cli.py", line 146, in main
CLI()
File "/usr/local/lib/python3.7/dist-packages/ludwig/cli.py", line 72, in init
getattr(self, args.command)()
File "/usr/local/lib/python3.7/dist-packages/ludwig/cli.py", line 92, in experiment
experiment.cli(sys.argv[2:])
File "/usr/local/lib/python3.7/dist-packages/ludwig/experiment.py", line 571, in cli
experiment_cli(**vars(args))
File "/usr/local/lib/python3.7/dist-packages/ludwig/experiment.py", line 239, in experiment_cli
debug=debug,
File "/usr/local/lib/python3.7/dist-packages/ludwig/api.py", line 1054, in experiment
debug=debug,
File "/usr/local/lib/python3.7/dist-packages/ludwig/api.py", line 487, in train
save_path=model_dir,
File "/usr/local/lib/python3.7/dist-packages/ludwig/models/trainer.py", line 550, in train
self.regularization_lambda
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 828, in call
result = self._call(*args, **kwds)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 871, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 726, in _initialize
*args, **kwds))
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 2969, in _get_concrete_function_internal_garbage_collected
graph_function, _ = self._maybe_define_function(args, kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 3361, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 3206, in _create_graph_function
capture_by_value=self._capture_by_value),
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py", line 990, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 634, in wrapped_fn
out = weak_wrapped_fn().wrapped(*args, **kwds)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 3887, in bound_method_wrapper
return wrapped_fn(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py", line 977, in wrapper
raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:

/usr/local/lib/python3.7/dist-packages/ludwig/models/ecd.py:177 train_step  *
    loss, all_losses = self.train_loss(
/usr/local/lib/python3.7/dist-packages/ludwig/models/ecd.py:201 train_loss  *
    of_train_loss = of_obj.train_loss(targets[of_name],
/usr/local/lib/python3.7/dist-packages/ludwig/features/base_feature.py:214 train_loss  *
    return self.train_loss_function(targets, predictions)
/usr/local/lib/python3.7/dist-packages/ludwig/modules/loss_modules.py:131 call  *
    loss = sampled_softmax_cross_entropy(
/usr/local/lib/python3.7/dist-packages/ludwig/modules/loss_modules.py:274 sampled_softmax_cross_entropy  *
    sampled_values = sample_values_from_classes(labels, sampler, num_classes,
/usr/local/lib/python3.7/dist-packages/ludwig/modules/loss_modules.py:427 sample_values_from_classes  *
    sampled_values = tf.random.log_uniform_candidate_sampler(
/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:201 wrapper  **
    return target(*args, **kwargs)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/candidate_sampling_ops.py:154 log_uniform_candidate_sampler
    seed2=seed2, name=name)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_candidate_sampling_ops.py:662 log_uniform_candidate_sampler
    seed2=seed2, name=name)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/op_def_library.py:750 _apply_op_helper
    attrs=attr_protos, op_def=op_def)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py:592 _create_op_internal
    compute_device)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py:3536 _create_op_internal
    op_def=op_def)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py:2016 __init__
    control_input_ops, op_def)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py:1856 _create_c_op
    raise ValueError(str(e))

ValueError: Shape must be rank 2 but is rank 3 for '{{node train_loss/LogUniformCandidateSampler}} = LogUniformCandidateSampler[num_sampled=25, num_true=1, range_max=2904, seed=42, seed2=0, unique=false](train_loss/Cast)' with input shapes: [96,28,1].

Training: 0% 0/22 [00:08<?, ?it/s]`

[jimthompson5802]

@justinepdevasia There is a known error we are working to address involving the sampled_softmax_cross_entropy loss function. Please monitor this issue for status updates: #1096

[w4nderlust]

@justinepdevasia the issue was solved in #1103 Let me know is you still encouter the issue