OpenAI says it has improved its coding models’ ability to work with legacy programming languages, positioning them as potential aids for organizations trying to understand, maintain and eventually move old software to newer platforms. The immediate focus is COBOL, a language still used in systems that support banking work, alongside other long-established languages including Fortran and Visual Basic.
The pitch is not that an AI model can safely press a button and replace a bank’s core systems. OpenAI’s own framing is more cautious: a migration succeeds only when every test passes, and production use still requires expert review and validation against an organization’s business rules and software dependencies. That caveat is the heart of the story. In a field where the edge cases are often the rules that matter most, better benchmark numbers can be useful without being a substitute for accountable engineering.
Why COBOL remains a difficult modernization problem
COBOL is often described as a legacy language, but “legacy” should not be mistaken for irrelevant. In this context, it means that an organization has long-running software built on an older technology stack and still depends on it for important work. OpenAI says 40 percent of the world’s banks continue to rely on COBOL systems.
Those systems are not necessarily simple applications that can be swapped out like a web page. A banking program can embody years of operational decisions: how numbers are treated at a specific precision, what occurs when a record is missing, and how one task connects to other systems. Rebuilding or translating that behavior requires more than producing code that looks modern. The replacement must do the same job under the same relevant conditions.
That is why the cost of modernization can run from millions to hundreds of millions, as OpenAI describes it, while also taking substantial time. There is also a staffing concern. Programmers who work with COBOL are aging out of the workforce, making it harder for institutions to depend indefinitely on a shrinking pool of specialists. The combination creates pressure to migrate, but pressure does not lower the standard for correctness.
For teams responsible for these systems, AI could be valuable in narrower, practical roles: helping developers read unfamiliar code, draft documentation, identify patterns in large codebases, generate candidate translations and accelerate test creation. None of those uses removes the need for people who understand the bank’s actual rules. They may, however, reduce some of the slow, labor-intensive work involved in mapping an older system before changes begin.
What OpenAI measured
OpenAI developed an evaluation suite called MultiLangBench to assess coding models across languages. The task asks a model to create an SQLite-compatible database engine in a designated programming language. SQLite compatibility here refers to meeting the behavior expected by SQLite, a widely used database software library, within the boundaries of the evaluation.
The company reports that GPT-6 Astra achieved an average test pass rate of 87.4 percent across 20 languages. That compares with 26.8 percent for GPT-5.6 Sol in the same broad measurement. COBOL was still the toughest language in the test set: Astra reached a 72 percent pass rate there.
Those figures indicate progress, but they also put an important boundary around it. A 72 percent result means COBOL remains materially more difficult than the cross-language average in this benchmark. And a benchmark is a controlled way to compare performance on a defined task, not proof that a model can take responsibility for an individual bank’s production environment. A live environment can contain organization-specific rules, integrations and historical assumptions that no generic test fully captures.
The distinction matters especially when people hear a high percentage and instinctively translate it to “solved.” In technical evaluation, a pass rate is only meaningful alongside the task being tested. A model that performs well at creating a database engine is demonstrating a useful capability. It is not automatically demonstrating that it understands every rule embedded in a decades-old financial workflow.
Translation is about behavior, not just syntax
OpenAI also tested migration tasks more directly. It had models translate COBOL programs into C++, Java, Kotlin, Python or Rust while preserving what the original program does. This is commonly called behavior preservation: the new implementation should produce the outcomes required by the old one, including in unusual situations that may be easy to overlook.
The company uses examples that show why this is a demanding requirement. A translated program must preserve how the original handles number rounding when there are extra decimal digits. It must also behave correctly when records are missing. These might sound like small implementation details, but they can change an outcome. If a replacement changes a rounding rule or responds differently to absent data, then it may no longer be an equivalent replacement even if its code is cleaner and written in a newer language.
In this evaluation, GPT-6.1 Sol reached 97.9 percent accuracy with high reasoning effort. At low reasoning effort, its accuracy was 86.6 percent. OpenAI’s terminology suggests that reasoning effort is a setting associated with how much work the model devotes to a request. The reported gap is a reminder that model configuration can affect outcomes, rather than being a mere speed preference.
It also introduces an operational choice for teams considering such tools. If stronger results depend on higher reasoning effort, organizations would need to decide where that added effort is justified and how to validate every output. For a high-stakes migration, a faster but less accurate first draft may create additional review work rather than save it. For lower-risk preparatory tasks, a different balance may make sense. The evidence presented supports performance comparisons in OpenAI’s tests; it does not establish a universal configuration for every organization.
Why “all tests pass” is the key line
OpenAI says a migration is successful only if all tests pass. This is a much stricter standard than showing that most outputs look right. In software testing, a test is a defined check that verifies whether a program behaves as expected for a particular condition. A comprehensive set of tests aims to cover normal flows, error conditions and edge cases—the uncommon inputs or situations where hidden differences can emerge.
For banking-related software, a single failure may be unacceptable depending on what rule it affects. That is why preserving behavior is not simply a code-conversion exercise. Developers need to determine which existing behaviors are intentional business rules, which are dependencies on other systems and which may be defects that should be corrected only through a deliberate, reviewed change process.
That last distinction is easy to miss. An older program may contain behavior that seems awkward to a modern developer. Yet if other processes rely on it, removing it during translation can cause a failure elsewhere. Conversely, an old behavior may be undesirable but still require a carefully planned correction. AI can generate a proposed implementation; it cannot independently decide what a financial institution ought to preserve, change or approve.
What expert review means in practice
OpenAI explicitly says expert review and testing against business rules and dependencies remain necessary. In practical terms, that means a proposed migration must be evaluated by people with the right technical and organizational knowledge. They need to compare the original and translated programs, assess results through the relevant test suites and investigate differences rather than assuming generated code is correct because it compiles.
Business rules are the specific rules an organization uses to carry out its work. Dependencies are the other components, systems or expected behaviors a program relies on. Both can be central to a migration because a program rarely operates alone. A translation may accurately reproduce one piece of logic while still creating problems if it no longer interfaces as expected with the rest of the environment.
That is also why feedback from developers and enterprise customers guided the work on these legacy-language capabilities. Organizations dealing with old code are likely to judge usefulness through the details: whether a model recognizes the language, retains behavior and supports the painstaking verification required to move safely. Broad coding fluency is useful, but it is not the same as dependable modernization support.
What this development does—and does not—signal
The reported results suggest that AI-assisted legacy modernization is becoming more credible as a development aid, particularly for generating and reviewing candidate migrations. OpenAI’s cross-language and COBOL-specific measurements show meaningful improvement over the referenced earlier model results, while the direct migration evaluation targets the more relevant question of whether behavior survives a translation.
At the same time, the remaining gap between a benchmark result and a finished deployment is substantial. COBOL is still described by the results as the hardest language in the MultiLangBench group. Even the high migration-accuracy result is not framed as permission to skip testing. The company’s insistence on expert review is appropriate to the stakes of software that handles banking tasks.
For technology teams, the sensible takeaway is not “AI will replace COBOL expertise.” It is that AI may help make scarce expertise go further—if teams use it inside rigorous engineering processes. The model can assist with exploration, translation and iteration. Human specialists must still set the requirements, interpret the meaning of legacy behavior, verify outcomes and decide whether a change is safe to release.
The bigger technology context is also worth watching. As more products build AI features into paid services and workplace tools, understanding the limits attached to those features matters as much as understanding their headline capabilities. Our guide to AI limits and subscription perks in Google One explores that broader question from a consumer-service angle.
For banks facing an aging COBOL workforce and expensive modernization choices, the potential value lies in careful augmentation rather than automation theater. The benchmark gains are a sign that legacy code is getting more attention from AI developers. The real measure of success, however, remains the unglamorous one OpenAI identifies: every required test passes, and qualified people have verified that the system still does what it must do.
