Recognising legal characteristics of the judgments of the European Court of Justice: Difficult but not impossible

Short paper released for JURIX 2022, conference on Legal Knowledge and Information Systems.

Machine learning has improved significantly during the past decades, with computers performing remarkably in formerly difficult tasks. This article reports the preliminary results on the prediction of two legally relevant characteristics of judgments of the European Court of Justice, doctrinal outcome and deference. The variables require the knowledge of concepts and doctrines of European Union law and judicial decision-making. The analysis relies on 1704 manually labelled judgments and trains a set of classifiers based on word embedding, LSTM, and convolutional neural networks. The preliminary findings suggest that all classifiers exceed simple baselines, learning meaningful representations of the judgments. Yet, the overall performance is rather weak, suggesting that the usual limitations of machine learning, including small training data, significant class imbalance and the characteristics of the variables requiring external knowledge pose a significant challenge to machines learning the law. The article also outlines directions for future research.

Please refer to our working paper here for further and detailed explantions about the approach and the results.

Authors: Alessandro Contini, Sebastiano Piccolo, Lucia Lopez Zurita and Urska Sadl

December 2022

NOTE: the code in this repository is an example of the final neural networks used for the predictions. Please, contact one of the authors for the full code containing datasets, algorithms for paramaeter tuning and the final models.