LitLat BERT

  Authors

Ulčar, Matej ; Robnik-Šikonja, Marko

 Project URL

https://embeddia.eu

 Date issued

2020-12-24

 Type

toolService

 Language(s)

Lithuanian , Latvian , English

 Description

Trilingual BERT-like (Bidirectional Encoder Representations from Transformers) model, trained on Lithuanian, Latvian, and English data. State of the art tool representing words/tokens as contextually dependent word embeddings, used for various NLP classification tasks by fine-tuning the model end-to-end. LitLat BERT are neural network weights and configuration files in pytorch format (i.e. to be used with pytorch library). The corpora used for training the model have 4.07 billion tokens in total, of which 2.32 billion are English, 1.21 billion are Lithuanian and 0.53 billion are Latvian. LitLat BERT is based on XLM-RoBERTa model and comes in two versions, one for usage with transformers library (https://github.com/huggingface/transformers), and one for usage with fairseq library (https://github.com/pytorch/fairseq). More information is in the readme.txt.

 Publisher

University of Ljubljana, Faculty of Computer and Information Science

 Acknowledgement

European Union

Project code: EC/H2020/825153

Project name: EMBEDDIA - Cross-Lingual Embeddings for Less-Represented Languages in European News Media

 Subject(s)

BERT RoBERTa embeddings multilingual model contextual embeddings word embeddings

 Collection(s)

CLARIN-LT

Show full item record