TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
OpenAI and biotech company Ligo introduced BioBasicsBench, a 100-task benchmark for testing whether AI agents can interpret everyday wet-lab evidence, spot errors and recognize when results are inconclusive. In the reported evaluation, GPT-6 Astra averaged 66.7% of available credit at the “xhigh” reasoning setting, compared with 43.8% for GPT-5.6 Sol; the results also point to continuing challenges with images, controls and experimental context.
OpenAI and biotech company Ligo have introduced BioBasicsBench, a benchmark of 100 routine wet-lab analysis tasks intended to test whether AI agents can interpret raw experimental evidence, detect errors and recognize when results do not support a conclusion. In the reported evaluation, GPT-6 Astra averaged 66.7% of available credit at the “xhigh” reasoning setting, compared with 43.8% for GPT-5.6 Sol, showing progress alongside substantial room for improvement.
The benchmark covers five areas of routine laboratory work: protein methods; imaging and cytometry; quantification and assays; nucleic acid methods; and molecular cloning. Its tasks range from reading a DNA sample’s molecular weight from a gel to analyzing more detailed assays. Inputs include gel and blot images, microscopy data, sequencing files, instrument exports, photographs of lab equipment and videos of bench procedures. A single task may combine several kinds of evidence.
According to OpenAI, scientists built the questions from data from their own experiments and paired each with expert-written grading criteria. Ligo scientists who had not authored a task reviewed it internally; a wet-lab scientist with a PhD conducted an external review; and a final internal review followed. Responses receive credit for individual rubric criteria, which lets the scoring distinguish a correct conclusion from partially correct analysis or reasoning.
OpenAI says it tested seven models across 37 model and reasoning-effort settings, running each setting on all 100 tasks five times, for 18,500 results in total. The reported comparison at the “xhigh” setting gives GPT-6 Astra an average of 66.7% of available credit and GPT-5.6 Sol 43.8%. These are benchmark scores, not measures of how often either model would independently produce a successful outcome in a working laboratory.
Why Routine Lab Errors Matter
Wet-lab research depends on interpreting evidence that can be incomplete, ambiguous or affected by experimental problems. Misreading an image or missing a flaw in a control can lead a scientist to draw the wrong conclusion and spend time on follow-up work that does not answer the research question. A benchmark that includes raw experimental material and asks agents to identify limitations tests a different capability from answering questions based on clean charts selected for publication.
The results matter as a measure of current capability, but they do not establish that AI agents are ready to run laboratory research without supervision. An average of 66.7% of available credit means the evaluated model did not earn all rubric credit across these tasks, and scores alone do not show which types of mistakes would be most consequential in a particular lab. The benchmark’s focus on error detection and insufficient evidence is relevant because a useful research assistant must sometimes withhold a conclusion, not just produce one.
For researchers and developers, the task-level approach may help identify where assistance is more or less reliable: image interpretation, numerical measurement, protocol understanding or use of experimental controls. The source describes BioBasicsBench as a measurement tool; it does not report that the benchmark has been independently validated as a predictor of performance across laboratories or research programs.
Products Worth Considering
DJI Osmo Pocket 3 Creator Combo 4K Vlogging Camera
Anker SOLIX C1000 Gen 2 Portable Power Station, 1,024Wh
Jackery Explorer 300 Portable Power Station with 40W Solar Panel
As an affiliate, we earn on qualifying purchases.
How BioBasicsBench Is Scored
OpenAI says existing benchmarks built around charts and images chosen for scientific papers may not capture the ambiguity and troubleshooting common in day-to-day laboratory work. BioBasicsBench was developed with Ligo, which OpenAI describes as a biotech company focused on enzyme design, to assess routine analysis using evidence from experiments rather than only polished presentation materials.
Each task’s grading rubric awards credit for specified criteria, which may include a conclusion, measurements, supporting reasoning or recognition that evidence is insufficient. The report says tasks often combine multiple skills, including biological knowledge, data and image interpretation, error detection and protocol understanding. Its counts for different data types and skills can overlap: one task may contain several inputs or test several abilities, so those category totals should not be added to infer a separate task count.
OpenAI reports testing seven models—GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna and GPT-6 Astra—using five attempts per task in each model-and-effort setting. The supplied source material gives the Astra-versus-Sol comparison but does not provide the full score table or scores for every model and setting.
What the Scores Do Not Show
The provided report excerpt does not include the complete results table, so it is not possible to compare all seven models or all 37 settings from the supplied figures. It also does not specify confidence intervals or other measures of variation for the reported averages. The 66.7% and 43.8% figures are average shares of available credit at one named reasoning setting, not percentages of tasks fully solved.
It remains unclear from the supplied material how closely performance on the 100 benchmark tasks predicts performance in different laboratories, with different equipment, protocols or biological systems. The report does not establish whether agents can safely make experimental decisions or operate lab equipment, nor does it provide evidence of improvements to research outcomes or time saved in practice. Those questions require evidence beyond benchmark scoring.
Further Testing Needed in Labs
The immediate next step is for researchers and model developers to use the benchmark to examine performance across task types, especially image interpretation, control checks and experimental context, the areas OpenAI identifies as needing improvement. A fuller assessment would require detailed results across all model and reasoning settings, as well as scrutiny of which rubric criteria models miss.
The report, as supplied, does not announce a date for another evaluation or specify a deployment plan. Whether benchmark gains translate into reliable assistance in working laboratories remains an open question; testing across varied experimental settings and independent reporting of results would help clarify that.
Key Questions
What is BioBasicsBench?
BioBasicsBench is a 100-task benchmark developed by OpenAI and Ligo to test AI agents on routine wet-lab analysis, including interpreting experimental data, finding errors and judging when evidence is insufficient.
What score did GPT-6 Astra receive?
OpenAI reports that GPT-6 Astra averaged 66.7% of available credit at the “xhigh” reasoning setting. That is an average score, not a claim that it fully solved 66.7% of tasks.
How did GPT-5.6 Sol compare?
At the same “xhigh” setting, OpenAI reports that GPT-5.6 Sol averaged 43.8% of available credit. The supplied source excerpt does not include scores for every other model or setting.
Does the benchmark show that AI can run a lab independently?
No. The reported evaluation measures performance on selected analysis tasks. It does not establish that an agent can safely operate a laboratory, make experimental decisions independently or improve research outcomes in practice.
Source: openai.com
![GPT-5.6 Sol Ultra Produces Proof Of The Cycle Double Cover Conjecture [Pdf]](https://deepintellica.com/wp-content/uploads/2026/07/gpt-5-6-sol-ultra-produces-proof-of-the-cycle-double-cover-conjecture-pdf-featured-560x300.jpg)
