OpenAI
OpenAI's latest frontier model with improved reasoning, creativity, and factual accuracy.
This model is still tracked for research and discovery, but it is excluded from default public rankings until it returns to active status.
60.4
Quality Score
1413
Arena ELO
Undisclosed
Parameters
128K
Context
Sign in to join the discussion
0
Downloads
0
Likes
Feb 2025
Released
Launches
3
Benchmarks
16
Research
3
General
4
Recent launch, pricing, benchmark, and API signals linked to this model or its provider.
People are generating over 1.5 billion images a week in ChatGPT. Researcher @kenjihata joins Product lead @adele__li and host @AndrewMayne to explore the new use cases and trends emerging since the launch of Images 2.0. https://t.co/INhLS7TDri
View source
Arena-Hard-Auto official Gemini-2.5 judged score 50.0 with CI -1.9/2
Introducing OpenAI Guaranteed Capacity: a new offering that enables customers to guarantee long-term access to OpenAI compute. We’ve made long-term investments in infrastructure, partnerships, and capacity planning to help customers scale reliably. Now, Guaranteed Capacity
View sourceArena-Hard-Auto official Gemini-2.5 judged score 50.0 with CI -1.9/2
View sourceMMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
View sourceToday, we share a breakthrough on the planar unit distance problem, a famous open question first posed by Paul Erdős in 1946. For nearly 80 years, mathematicians believed the best possible solutions looked roughly like square grids. An OpenAI model has now disproved that https://t.co/j2g3Ze0zEG
People are generating over 1.5 billion images a week in ChatGPT. Researcher @kenjihata joins Product lead @adele__li and host @AndrewMayne to explore the new use cases and trends emerging since the launch of Images 2.0. https://t.co/INhLS7TDri
Introducing OpenAI Guaranteed Capacity: a new offering that enables customers to guarantee long-term access to OpenAI compute. We’ve made long-term investments in infrastructure, partnerships, and capacity planning to help customers scale reliably. Now, Guaranteed Capacity
Large language model (LLM) agents increasingly operate over long and recurring external contexts, like document corpora and code repositories. Across invocations, existing approaches preserve either the agent's trajectory, passive access to raw material, or task-level strategies. None of them preserves what we argue is most needed for repeated same-context workloads: reusable orientation knowledge (e.g., what the context contains, how it is organized, and which entities, constants, and schemas have historically been useful) about the recurring context itself. We introduce PEEK, a system that caches and maintains this orientation knowledge as a context map: a small, constant-sized artifact in the agent's prompt that gives it a persistent peek into the external context. The map is maintained by a programmable cache policy with three modules: a Distiller that extracts transferable knowledge from inference-time signals, a Cartographer that translates it into structured edits, and a priority-based Evictor that enforces a fixed token budget. On long-context reasoning and information aggregation, PEEK improves over strong baselines by 6.3-34.0% while using 93-145 fewer iterations and incurring 1.7-5.8x lower cost than the state-of-the-art prompt-learning framework, ACE. On context learning, PEEK improves solving rate and rubric accuracy by 6.0-14.0% and 7.8-12.1%, respectively, at 1.4x lower cost than ACE. These gains generalize across LMs and agent architectures, including OpenAI Codex, a production-grade coding agent. Together, these results show that a context map helps long-context LLM agents interact with recurring external contexts more accurately and efficiently.
The design of modern neural architectures has converged through incremental empirical choices, yet the mechanisms governing their training dynamics remain only partially understood. We identify and analyze a negative weight drift induced by the interaction between standard losses and positively biased activation functions. We prove that under MSE or cross-entropy loss, the gradient with respect to positive pre-activations is non-negative in expectation at initialization, driving downstream weights toward negative values during early training. The drift is intrinsic to optimization rather than data, and persists across architectures (MLP, ResNet, ViT, GPT-nano, MP-SENe) and asymmetric activation functions (ReLU, GELU, SiLU). Coupled with ReLU, weight drift produces activation sparsity reaching up to 90\% in GPT-nano. We characterize the sparsity-accuracy tradeoff across 79 configurations and identify a sharp accuracy cliff above sim70\% activation sparsity. While ReLU^2 achieves a good sparsity--accuracy ratio in GPT-nano, it pathologically amplifies identified activation spikes in intermediate transformer layers. Clipping resolves this while preserving the representational benefits of squaring: clipped ReLU^2 outperforms its unclipped version, and GELU^2 achieves the lowest validation loss on GPT-nano. Code is available at https://github.com/On-Point-RND/BugOrFeature.
GPU kernel optimization is increasingly critical for efficient deep learning systems, but writing high-performance kernels still requires substantial low-level expertise. Recent AI coding agents can iteratively read code, invoke compilers and profilers, and refine implementations, yet existing kernel benchmarks evaluate single LLM calls rather than full agent workflows, and none include both kernel-to-kernel optimization and unseen-configuration generalization testing. We present AgentKernelArena, an open-source benchmark for measuring AI coding agents on GPU kernel optimization. The benchmark contains 196 tasks spanning HIP-to-HIP optimization, Triton-to-Triton optimization, and PyTorch-to-HIP translation, and evaluates complete agent workflows in isolated workspaces using gated compilation, correctness, and performance checks, centralized scoring and an unseen-configuration generalization protocol that tests whether optimizations transfer to input configurations the agent never observed. Across production agents including Cursor Agent, Claude Code, and Codex Agent, we find near-perfect compilation and high correctness rates on most task categories, with the strongest configurations achieving mean speedups of up to 6.89x on PyTorch-to-HIP, 6.69x on HIP-to-HIP, and 2.13x on Triton-to-Triton tasks. Our unseen-configuration evaluation shows that HIP-to-HIP and Triton-to-Triton optimizations largely transfer to unseen input shapes, while PyTorch-to-HIP exhibits substantial correctness drops, indicating that agents generating kernels from scratch frequently hardcode shape-specific assumptions. AgentKernelArena is designed as a modular, extensible framework for rigorous evaluation of agentic GPU kernel optimization across agents, tasks, and hardware targets.
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0
MMMU_VAL: 72.1 | MathVista: 70.5 | OCRBench: 84.5 | MMBench_TEST_EN_V11: 83.5 | AI2D: 87.2 | MMStar: 69.3 | HallusionBench: 60.0