SonexAI: Real-Time Autonomous Play in a Tactical Shooter via Deep Reinforcement Learning
Abstract
We present SonexAI, an end-to-end deep learning agent that plays the tactical first-person shooter Valorant autonomously in real time. SonexAI unifies a convolutional-transformer perception stack, a recurrent world model, and an actor-critic policy into a single C++ runtime with a CUDA inference backend that sustains sub-10ms decision latency at 144 FPS. The policy is trained via a three-stage curriculum: behavioral cloning from 1.2M ranked matches, on-policy reinforcement learning with Proximal Policy Optimization against a growing pool of self-play opponents, and a fine-tuning phase that shapes human-like motor dynamics. On held-out evaluation, SonexAI reaches Radiant-tier combat statistics while producing movement and aim trajectories that are statistically difficult to distinguish from expert human play.
1. Introduction
Tactical shooters combine partial observability, fast reaction requirements, long-horizon economic decisions, and multi-agent coordination. Prior game-playing systems have targeted either perfect-information board games or simplified control tasks. SonexAI addresses the full loop of a commercial title running unmodified, learning directly from rendered frames and producing low-level mouse and keyboard actions.
2. Perception
Frames are captured through a present-hook and preprocessed on the GPU. A ResNet-50 backbone with a feature-pyramid neck feeds a YOLOv8-style detection head for enemies, a pose head for hitbox localization, and a minimap encoder. The outputs are fused into a structured game-state tensor consumed by the policy. Perception runs in 4.1ms on an RTX 4070.
3. World Model & Memory
A gated recurrent world model integrates the per-frame state into a temporal belief that tracks occluded enemies, spike timers, and utility cooldowns. This gives the agent object permanence: a player seen rotating is remembered and its position decayed under a learned uncertainty model rather than forgotten between frames.
4. Policy & Control
The actor-critic policy outputs a factorized action distribution over aim deltas, fire control, movement, and ability usage. Aim is realized through a minimum-jerk trajectory generator with calibrated noise injection, reproducing the micro-corrections and overshoot characteristic of human tracking. A separate value head estimates round-win probability for economic reasoning.
5. Training
Stage one clones behavior from 1.2M ranked matches. Stage two applies PPO with a reward that blends combat, objective, and survival signals against a self-play opponent pool sampled to prevent overfitting. Stage three fine-tunes motor dynamics against human-likeness discriminators. Training consumed roughly 2,400 GPU-hours to a locked 127M-parameter checkpoint.
6. Results
SonexAI achieves a 71% round-win rate against Radiant opponents and a mean time-to-headshot of 118ms in the competitive reaction profile. A human-study panel of coaches classified agent demos as human at chance level for the humanized configuration, indicating the motor model closes much of the behavioral gap.
7. Limitations & Ethics
SonexAI is a research artifact demonstrating autonomous play. It is not affiliated with or endorsed by Riot Games, and the authors do not condone use that violates the terms of service of any title. We publish these results to advance the study of real-time embodied decision making.