Uma visão geral sobre a aplicação de Grandes Modelos de Linguagem na obtenção de informações de editais da Universidade Federal de Mato Grosso do Sul

Abstract

In the era of Artificial Intelligence applications, it is hard to go by a day without using, or at least interacting with, something based on Large Language Models (LLMs). Although these tools can provide impressive results, they are often limited by the scope of information they have been trained on, which poses a big challenge for specific use cases. With that in mind, this work aims to implement and test a technique designed to be used in these types of scenarios, no re-training or fine-tuning necessary, called Retrieval-Augmented Generation (RAG), to facilitate students access to information regarding University notices. The performance of three different models was tested, using advanced techniques in both retrieval and augmentation to find the optimal trade-off between reliable answers and implementation cost. Experiments shows that, Llama 3.1 with 8 billion parameters outperforms the other models in both answer faithfulness and relevance in most tests. Additionally, the model shows competitive results against larger variants of the Llama 3 family, establishing itself as a trade-off in terms of computational cost. Finally, we make available a Q&A set, used to test our RAG implementation, as well as reference contexts for each question.