Can an Agent Debug a Web Corpus? Agentic Auditing of Slovak FineWeb2
arXiv cs.CL · 23 Oct 2026, 03:52 AEST
A new approach using an LLM as a corpus debugger has been developed to audit the Slovak portion of FineWeb2. This method identified significant issues in the corpus, including a large amount of machine-generated content that passed filters.
Key facts
- Multilingual pipelines such as FineWeb2 apply one recipe across a thousand languages, leaving the failure modes of individual languages unanticipated.
- On the 26.5M-document Slovak portion of FineWeb2, three runs costing $2.21-$2.84 each found seven findings over four phenomena.
- The largest is a machine-generated content farm of 1,408,437 documents (5.32%) that passed the pipeline's filters, 86.6% of it passing even stricter thresholds.
- The compiled suite scans the full corpus in 0.7 s of wall-clock time without a GPU, against $12k-$90k per LLM classification pass.
Read aloud
A new approach using an L L M as a corpus debugger has been developed to audit the Slovak portion of FineWeb2. This method identified significant issues in the corpus, including a large amount of machine-generated content that passed filters. Multilingual pipelines such as FineWeb2 apply one recipe across a thousand languages, leaving the failure modes of individual languages unanticipated. On the twenty six.5M-document Slovak portion of FineWeb2, three runs costing two dollars and twenty one cents-two dollars and eighty four cents each found seven findings over four phenomena. The largest is a machine-generated content farm of one million four hundred eight thousand four hundred thirty seven documents five point three two percent that passed the pipeline's filters, eighty six point six percent of it passing even stricter thresholds. The compiled suite scans the full corpus in zero point seven s of wall-clock time without a G P U, against twelve dollarsk-ninety dollarsk per L L M classification pass.
Sources in this cluster
- Can an Agent Debug a Web Corpus? Agentic Auditing of Slovak FineWeb2
arXiv cs.CL · 23 Oct 2026, 03:52 AEST