Home

#Complex Word Identification Project #####Keywords: NLP, Word Difficulty, Complex Words, DKPro

##About The aim of this Project is to write a tool, capable of identifying Complex Words as defined in SemEval2016 Task 11.

Complex Word Identification consists in determining which words in a given sentence can challenge the readers of a certain target audience. The goal of the proposed Complex Word Identification shared task is to provide a framework for the evaluation of methods for this first step in a Lexical Simplification pipeline.

In order to achieve this, potential measures (see the concept section for more details on those) that could influence word difficulty were generated from intuition. These measures were then tested using a tool, developed with the aid of different Frameworks of the DKPro Family.

---

####Table of Contents Setup

• Run Configuration
• Output

Evaluation

Code Commentary & Resource Listing

Project Report

• Concept
• Challenges
• Future Outlook

---

###Discussion

#######Future Improvements

There are three main areas, which could be improved significantly in the future. Chief amongst them the Project's Theoretical Foundation, Methods for Complex Word Identification and (self written) Code.

• More Research (SLA, Linguistics, LA in children, ML, similar projects & best practices) -> Better complexity prediction
• Realise all original Measures (Context& Ambiguity missing)
• Appropriate Classifiers / Regression Models, what are differences, which are good for tasks like this?
• Change specific preprocessing to feature extraction. Improves performance significantly. (Selection of individual Annotations hampers performance.)

###Outlook

The results show that this project is nothing more than a first step in the right direction. There is still a long way to go, in Complex Word Identification. The main merit of this project for the author is a more advanced understanding of and heightened interested in Computational Linguistics. This project can be used as a basis for future experiments in the field of Natural Language Processing.

back to navigation