-
Notifications
You must be signed in to change notification settings - Fork 0
Document Metadata
For each document in the corpus, we tried to obtain the following metadata:
- spatial scale
- temporal scale
- country
- funding source
After careful investigation of the topics resulting from the LDA, we found that the spatial scale as well as the components of the water budget can both be obtained by using the LDA topics.
The problem at hand is a classification task. This classification is characterized by having multiple labels. In other words, one document can have the spatial scales labels: "Day" and "Week" and "Year" at the same time. This problem of classification is different of the simpler problem of having to predict one label with different values: a multi-class classification problem. For example, the current problem could be conceptualized as one label "Timescale" taking the values "Day", "Week", "Year" and so on.
Two main approaches exist to solve such a multi-label classification problem. The first one is to use algorithm adapted for such a task. The second one is to transform the problem to make it tractable. Such a transformation can be achieved through two schemes: Binary Relevance (BR) and Label Powerset (LP). In the LP approach, the multilabel problem is transformed into a multi-class classification of the labelsets. In the BR approach, the problem is decomposed in a set of binary problems. In addition, we tested more refined BR approaches: classifier chains were the binary classifiers are trained in a given order to predict the labels; nested stacking were the labels are approximated successively by learners and dependent binary relevance where every other labels are used as the input for each label-specific binary learners and stacking which combine dependent binary relevance and nested stacking.
The main performance metric used is the multilabel Hamming loss corresponding to the proportion of incorrectly predicted labels. We also use the subset 0 1 metric corresponding to the proportion of observations for which the complete set of label is correctly predicted.