Programming Prescience: Issues in Human Construction of Forecasting Bots
arXiv cs.AI · 10 Nov 2026, 04:33 AEST
AI models are increasingly being used for forecasting based on historical data. A study of 15 builders of forecasting bots reveals significant challenges in tuning, reasoning, and performance evaluation.
Key facts
- AI models' growing capacity for complex reasoning has made them attractive tools for forecasting, or the systematic prediction of the future based on past and present data.
- In this paper, we explore what this work looks like with a qualitative interview study with 15 builders of forecasting bots, sampled from among successful participants in a recent AI forecasting competition.
- Our analysis surfaces the challenges around (1) tuning a bot to acquire factual research, (2) implementing coherent reasoning, and (3) understanding how well one's bot actually does.
- During their work, builders frequently introduced complexity that did not pay off, and tested in ways that were opportunistic and piecemeal.
Read aloud
A I models are increasingly being used for forecasting based on historical data. A study of fifteen builders of forecasting bots reveals significant challenges in tuning, reasoning, and performance evaluation. A I models' growing capacity for complex reasoning has made them attractive tools for forecasting, or the systematic prediction of the future based on past and present data. In this paper, we explore what this work looks like with a qualitative interview study with fifteen builders of forecasting bots, sampled from among successful participants in a recent A I forecasting competition. Our analysis surfaces the challenges around one tuning a bot to acquire factual research, two implementing coherent reasoning, and three understanding how well one's bot actually does. During their work, builders frequently introduced complexity that did not pay off, and tested in ways that were opportunistic and piecemeal.
Sources in this cluster
- Programming Prescience: Issues in Human Construction of Forecasting Bots
arXiv cs.AI · 10 Nov 2026, 04:33 AEST