o4 Mini High

As AI takes on longer, higher-stakes tasks, we want models to carry beneficial and safe behavior into new domains beyond their training—and maintain it under pressure. That’s the idea behind our new r

As AI takes on longer, higher-stakes tasks, we want models to carry beneficial and safe behavior into new domains beyond their training—and maintain it under pressure. That’s the idea behind our new research on training models to be broadly and persistently beneficial.

Improving health intelligence in ChatGPT

Introducing LifeSciBench, a benchmark for measuring and improving how well AI supports real-world life science research. Developed with 173 scientists from biotechnology and pharmaceutical research, L

Introducing LifeSciBench, a benchmark for measuring and improving how well AI supports real-world life science research. Developed with 173 scientists from biotechnology and pharmaceutical research, LifeSciBench includes 750 expert-authored tasks across seven biological research https://t.co/JDkKWcnL9F

Introducing LifeSciBench

We’re sharing new research on a method for anticipating how models may behave in real-world use before release: simulating deployment with recent, de-identified user requests and studying candidate mo

We’re sharing new research on a method for anticipating how models may behave in real-world use before release: simulating deployment with recent, de-identified user requests and studying candidate model responses. https://t.co/7RJzBfNniQ

Let’s talk about evals. We’re always looking for better ways to measure and forecast model progress, especially as benchmarks get saturated or gamed. @tejalpatwardhan, who leads our frontier evals tea

Let’s talk about evals. We’re always looking for better ways to measure and forecast model progress, especially as benchmarks get saturated or gamed. @tejalpatwardhan, who leads our frontier evals team, spoke to @andrewmayne about why evals matter and what models need to be https://t.co/Q3oRCuNxYB

Predicting model behavior before release by simulating deployment

Introducing the OpenAI Partner Network

o4-mini (high) Benchmark Update

Quality: 25.6/100 | Price: $1.925/M tokens | Output: 189.761 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 25.6/100 | Price: $1.925/M tokens | Output: 183.724 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 25.6/100 | Price: $1.925/M tokens | Output: 196.937 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 194.832 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 179.819 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 162.545 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 163.401 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 163.401 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 177.332 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 162.074 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 160.141 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 153.914 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 152.701 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 153.684 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 165.029 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 158.061 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 148.564 tok/s | MMLU: 0.832% | HumanEval: 0.859%

o4-mini (high) Benchmark Update

Quality: 33.1/100 | Price: $1.925/M tokens | Output: 174.319 tok/s | MMLU: 0.832% | HumanEval: 0.859%