AI TrackerFrontier desk · AEST
Back to news
normaltechai

GoiBERT: Compact Vietnamese Language Models via Two-Stage Knowledge Distillation

arXiv cs.CL · 22 Oct 2026, 03:24 AEST

GoiBERT introduces two compact Vietnamese encoders with 26M and 49M parameters that operate directly on raw text. The models achieve high performance across various benchmarks while being more efficient than existing compact baselines.

Key facts

Read aloud

GoiBERT introduces two compact Vietnamese encoders with 26M and 49M parameters that operate directly on raw text. The models achieve high performance across various benchmarks while being more efficient than existing compact baselines. Vietnamese pre-trained encoders have improved rapidly, but the strongest ones are large, multilingual-vocabulary models or depend on external word segmentation, which makes them costly to deploy. GoiBERT learns from two complementary teachers. Across eight benchmarks covering text classification, aspect-based sentiment analysis, natural language inference, and named entity recognition, GoiBERT-large achieves the highest average score among non-teacher models. GoiBERT-base retains ninety nine percent of that baseline's performance at a quarter of its size.

Sources in this cluster