AI TrackerFrontier desk · AEST
Back to news
normalaitech

Fast Rates for Offline Q-Learning with KL Regularization

arXiv cs.AI · 14 Oct 2026, 03:56 AEST

This paper investigates offline Q-learning with Kullback-Leibler (KL) regularization. It proposes KL-regularized Q-Learning and KL-regularized Pessimistic Q-Learning, achieving improved sample complexity in offline decision-making problems.

Key facts

Read aloud

This paper investigates offline Q-learning with Kullback-Leibler KL regularization. It proposes KL-regularized Q-Learning and KL-regularized Pessimistic Q-Learning, achieving improved sample complexity in offline decision-making problems. This paper investigates offline Q-learning for finite-horizon Markov decision processes with Kullback-Leibler KL regularization. We first propose KL-regularized Q-Learning KLQ, a simple variant of classical Q-learning. We prove that KLQ achieves a sample complexity of $ ilde O one/ε $ without explicit pessimism. To address this limitation, we further propose KL-regularized Pessimistic Q-Learning KLPQ, which incorporates pessimism into KLQ. We prove that KLPQ retains the same $ ilde O one/ε $ rate while removing the explicit exponential dependence.

Sources in this cluster