sampled_softmax_cross_entropy broken for machine translation example

[tgaddair]

Config:

input_features:
    -
        name: english
        type: text
        level: word
        encoder: rnn
        cell_type: lstm
        reduce_output: null
        preprocessing:
          word_tokenizer: english_tokenize

output_features:
    -
        name: italian
        type: text
        level: word
        decoder: generator
        cell_type: gru
        attention: bahdanau
        reduce_input: null
        loss:
            type: sampled_softmax_cross_entropy
        preprocessing:
          word_tokenizer: italian_tokenize

training:
    batch_size: 96

Error (eager execution enabled):

Traceback (most recent call last):
  File "/Users/tgaddair/.venv/ludwig/py38/bin/ludwig", line 11, in <module>
    load_entry_point('ludwig', 'console_scripts', 'ludwig')()
  File "/Users/tgaddair/repos/ludwig/ludwig/cli.py", line 146, in main
    CLI()
  File "/Users/tgaddair/repos/ludwig/ludwig/cli.py", line 72, in __init__
    getattr(self, args.command)()
  File "/Users/tgaddair/repos/ludwig/ludwig/cli.py", line 92, in experiment
    experiment.cli(sys.argv[2:])
  File "/Users/tgaddair/repos/ludwig/ludwig/experiment.py", line 571, in cli
    experiment_cli(**vars(args))
  File "/Users/tgaddair/repos/ludwig/ludwig/experiment.py", line 216, in experiment_cli
    ) = model.experiment(
  File "/Users/tgaddair/repos/ludwig/ludwig/api.py", line 1029, in experiment
    ) = self.train(
  File "/Users/tgaddair/repos/ludwig/ludwig/api.py", line 475, in train
    train_stats = trainer.train(
  File "/Users/tgaddair/repos/ludwig/ludwig/models/trainer.py", line 601, in train
    loss, all_losses = model.train_step(
  File "/Users/tgaddair/.venv/ludwig/py38/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 814, in __call__
    return self._python_function(*args, **kwds)
  File "/Users/tgaddair/.venv/ludwig/py38/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 3936, in bound_method_wrapper
    return wrapped_fn(weak_instance(), *args, **kwargs)
  File "/Users/tgaddair/repos/ludwig/ludwig/models/ecd.py", line 184, in train_step
    loss, all_losses = self.train_loss(
  File "/Users/tgaddair/repos/ludwig/ludwig/models/ecd.py", line 208, in train_loss
    of_train_loss = of_obj.train_loss(targets[of_name],
  File "/Users/tgaddair/repos/ludwig/ludwig/features/base_feature.py", line 214, in train_loss
    return self.train_loss_function(targets, predictions)
  File "/Users/tgaddair/.venv/ludwig/py38/lib/python3.8/site-packages/tensorflow/python/keras/losses.py", line 151, in __call__
    losses = call_fn(y_true, y_pred)
  File "/Users/tgaddair/repos/ludwig/ludwig/modules/loss_modules.py", line 131, in call
    loss = sampled_softmax_cross_entropy(
  File "/Users/tgaddair/repos/ludwig/ludwig/modules/loss_modules.py", line 274, in sampled_softmax_cross_entropy
    sampled_values = sample_values_from_classes(labels, sampler, num_classes,
  File "/Users/tgaddair/repos/ludwig/ludwig/modules/loss_modules.py", line 427, in sample_values_from_classes
    sampled_values = tf.random.log_uniform_candidate_sampler(
  File "/Users/tgaddair/.venv/ludwig/py38/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py", line 206, in wrapper
    return target(*args, **kwargs)
  File "/Users/tgaddair/.venv/ludwig/py38/lib/python3.8/site-packages/tensorflow/python/ops/candidate_sampling_ops.py", line 152, in log_uniform_candidate_sampler
    return gen_candidate_sampling_ops.log_uniform_candidate_sampler(
  File "/Users/tgaddair/.venv/ludwig/py38/lib/python3.8/site-packages/tensorflow/python/ops/gen_candidate_sampling_ops.py", line 636, in log_uniform_candidate_sampler
    _ops.raise_from_not_ok_status(e, name)
  File "/Users/tgaddair/.venv/ludwig/py38/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 6873, in raise_from_not_ok_status
    six.raise_from(core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: true_classes must be a matrix [Op:LogUniformCandidateSampler]

[jimthompson5802]

@tgaddair While working on a PR #1103 to address Issue #1093, I encountered an issue with the sampled_softmax_cross_entropy. Although the failure symptoms I see are different, I think our issues may be related. I'll keep you posted on what I find.

[w4nderlust]

#1103 fixed this