Large Language Models as a Universal Searcher for Open-World Automated Machine Learning
arXiv cs.CL · 10 Nov 2026, 03:58 AEST
A new framework using large language models aims to optimize machine learning configurations without task-specific fine-tuning. This universal AutoML approach demonstrates impressive performance across various benchmarks and showcases superior scalability.
Key facts
- Developing effective machine learning models requires complex manual tuning, while current Automated Machine Learning (AutoML) frameworks remain domain-specific and isolate model architecture, training hyperparameter, and data ratio optimization.
- We propose NCode, a unified numerical encoding scheme that translates diverse AutoML design choices into universal discrete sequences and effectively reframes the optimization process as a unified ranking task.
- Trained solely on optimization trajectories derived from a limited, single-domain vision task, the proposed LLM-based searcher demonstrates impressive cross-task, cross-modality and joint search-space optimization generalizability, achieving state-of-the-art performance across image classification, image segmentation, speaker verification and text-to-image benchmarks.
- Crucially, it exhibits superior scalability in the large-scale training and joint optimization of a billion-parameter Diffusion Transformer, requiring 896 GPU-days and attaining a state-of-the-art GenEval score of 0.91.
Read aloud
A new framework using large language models aims to optimize machine learning configurations without task-specific fine-tuning. This universal AutoML approach demonstrates impressive performance across various benchmarks and showcases superior scalability. Developing effective machine learning models requires complex manual tuning, while current Automated Machine Learning AutoML frameworks remain domain-specific and isolate model architecture, training hyperparameter, and data ratio optimization. We propose NCode, a unified numerical encoding scheme that translates diverse AutoML design choices into universal discrete sequences and effectively reframes the optimization process as a unified ranking task. Trained solely on optimization trajectories derived from a limited, single-domain vision task, the proposed L L M-based searcher demonstrates impressive cross-task, cross-modality and joint search-space optimization generalizability, achieving state-of-the-art performance across image classification, image segmentation, speaker verification and text-to-image benchmarks. Crucially, it exhibits superior scalability in the large-scale training and joint optimization of a billion-parameter Diffusion Transformer, requiring eight hundred ninety six G P U-days and attaining a state-of-the-art GenEval score of zero point nine one.
Sources in this cluster
- Large Language Models as a Universal Searcher for Open-World Automated Machine Learning
arXiv cs.CL · 10 Nov 2026, 03:58 AEST