punctuation-restoration-for-structure-understanding

This is our code that accompanies our RepL4NLP Paper Punctuation restoration improves structure understanding without supervision, where we apply punctuation restoration as an unsupervised structure learning objective that complements masked language modeling in T5 models. The additional training in punctuation restoration results in improved in-distribution and out-of-distribution performance in various structure-related NLP tasks such as named entity recognition, open information extraction, and semantic role labeling.