Detecção de Hiperônimos com BERT e Padrões de Hearst

Abstract

Hypernym relation (also known as is-a relation) is a relevant semantic relation between words that is useful to tasks like coreference resolution, relation extraction, textual entailment, among others. A hypernym is a generic word, while a hyponym is a specific word. For example, city is a hypernym of rome, and dog is a hyponym of animal. In this work, we propose an unsupervised algorithm for hypernym detection that combines Hearst patterns with the BERT language model. Hearst patterns are linguistic patterns such as banana is a kind of fruit, which indicates that fruit is hypernym of banana. An important limitation of such methods is its sparsity, a common problem for pattern-based methods. The BERT language model is a contextual representation model trained to predict masked words within an input sequence. We combine this aspect of BERT with Hearst patterns to create a novel algorithm for hypernym detection which achieves the state-of-the-art performance on 7 out of 13 evaluated datasets. Among these datasets, there are three new datasets in Portuguese, which were developed during this work and are the first for this language. We compare our method to the DIVE algorithm, an extension of the well-known word2vec algorithm. DIVE retained the best results for most of the datasets in English. Our method outperforms DIVE by 3 points on average for the thirteen considered datasets.