TypeError in forward() in language model

[geoffbacon]

Describe the bug
I'm trying to train language models using the built-in LanguageModelingReader and LanguageModel. I get TypeError: forward() got an unexpected keyword argument 'input_tokens', even though the LanguageModelingReader yields Instances with 'input_tokens' Fields (see here).

The full stack trace is:

2019-10-04 09:24:00,362 - INFO - allennlp.training.trainer - Training
  0%|          | 0/34685 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/Users/bacon/miniconda/bin/allennlp", line 10, in <module>
    sys.exit(run())
  File "/Users/bacon/miniconda/lib/python3.7/site-packages/allennlp/run.py", line 18, in run
    main(prog="allennlp")
  File "/Users/bacon/miniconda/lib/python3.7/site-packages/allennlp/commands/__init__.py", line 102, in main
    args.func(args)
  File "/Users/bacon/miniconda/lib/python3.7/site-packages/allennlp/commands/train.py", line 124, in train_model_from_args
    args.cache_prefix)
  File "/Users/bacon/miniconda/lib/python3.7/site-packages/allennlp/commands/train.py", line 168, in train_model_from_file
    cache_directory, cache_prefix)
  File "/Users/bacon/miniconda/lib/python3.7/site-packages/allennlp/commands/train.py", line 252, in train_model
    metrics = trainer.train()
  File "/Users/bacon/miniconda/lib/python3.7/site-packages/allennlp/training/trainer.py", line 478, in train
    train_metrics = self._train_epoch(epoch)
  File "/Users/bacon/miniconda/lib/python3.7/site-packages/allennlp/training/trainer.py", line 320, in _train_epoch
    loss = self.batch_loss(batch_group, for_training=True)
  File "/Users/bacon/miniconda/lib/python3.7/site-packages/allennlp/training/trainer.py", line 261, in batch_loss
    output_dict = self.model(**batch)
  File "/Users/bacon/miniconda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'input_tokens'

To Reproduce

1. My config file is:

{
    "dataset_reader": {
        "type": "language_modeling",
        "tokenizer": {
            "type": "word",
            "word_splitter": {
                "type": "just_spaces"
            }
        },
        "token_indexers": {
            "input_tokens": {
                "type": "single_id"
            }
        },
        "lazy": false
    },
    "train_data_path": "corpus.txt",
    "model": {
        "type": "language_model",
        "text_field_embedder": {
            "type": "basic",
            "token_embedders": {
                "input_tokens": {
                    "type": "embedding",
                    "embedding_dim": 100
                }
            }
        },
        "contextualizer": {
            "type": "lstm",
            "input_size": 100,
            "hidden_size": 200
        },
        "dropout": 0.2,
        "initializer": {},
        "regularizer": {}
    },
    "iterator": {
        "type": "basic"
    },
    "trainer": {
        "optimizer": {
            "type": "adam"
        },
        "patience": 2,
        "num_epochs": 10
    }
}

which was slightly manually edited from the output of the configuration wizard.

2. Run allennlp train config.json -s model, with a corpus in "corpus.txt" and an empty directory "model". As you can see from the traceback, everything up until the actual training works fine, but then it produces the error above.

Expected behavior
I would have expected that naming the token_indexer and and token_embedder "input_tokens" would be all you need to do in order to use the out-of-the-box model with the out-of-the-box dataset reader.

System (please complete the following information):

• OS: OSX
• Python version: 3.7.4
• AllenNLP version: v0.9.0
• PyTorch version: v1.2.0 from AllenNLP

Additional context
This issue is very similar to the discussion in #2528. I googled around, checked StackOverflow and other related issues in allennlp but did not find any solutions. The documentation and tutorial does make it very clear that "The forward method expects dicts of tensors as input, and it expects their names to be the names of the fields in your Instance.", but in this case I think I'm doing that and I'm still getting an error.

[brendan-ai2]

As you can see from https://github.com/allenai/allennlp/blob/master/allennlp/models/language_model.py#L248, the forward method of the language model actually wants an argument called source. The confusion here is the result of us having multiple dataset readers for the LM task. You actually need to use the one named simple_language_modeling which you can view at https://github.com/allenai/allennlp/blob/master/allennlp/data/dataset_readers/simple_language_modeling.py.

Sorry for the confusion! I'm going to delete/deprecate the https://github.com/allenai/allennlp/blob/master/allennlp/data/dataset_readers/language_modeling.py as it's not compatible with any models we currently have.

[brendan-ai2]

Also, working config for LanguageModel which you can tweak as desired can be found here: https://github.com/allenai/allennlp/blob/master/training_config/bidirectional_language_model.jsonnet.

[geoffbacon]

Thanks for your quick reply! Switching to the simple_language_modeling dataset reader fixed this.