Rewriting COBOL in Java? The Good Idea That Isn't

Every few years, the same slide shows up in a steering committee. On the left, a gray box labeled “Mainframe · COBOL · 1987”. On the right, a shiny cloud of Java or C# microservices. In between, an arrow and a budget. Lately the arrow has a new label: “AI-powered migration”.

It's a seductive story. For most core systems, it's also the wrong one.

I'm not saying legacy systems should stay frozen in time. Quite the opposite. I'm saying that rewriting them in another language is usually the most expensive, riskiest and least useful way to modernize them. There's a better path: keep the code that works and change everything around it. Use AI to understand what you have, not to translate it. Then transform the platform, the data and the interfaces with proven, deterministic methods, which AI now makes much faster to apply.

Your legacy code is an asset, not debt

That COBOL program nobody wants to touch has been processing payments, calculating premiums or closing accounting periods for thirty years. It has survived regulatory changes, mergers, the switch to the euro and every edge case your customers could invent. Each of those edge cases left a trace: an IF here, a special rate table there, a patch from 1998 that fixed a rounding issue for one product line.

That's not technical debt. That's your business, written down in the only place where it's still fully documented. Reuters put the amount of COBOL in production at 220 billion lines, and 95% of ATM swipes still rely on it. Those systems aren't still running out of inertia. They're running because they work.

A rewrite asks you to rediscover every one of those rules, reimplement them in a new language, and prove you didn't miss any. Keep that last part in mind.

COBOL is the right tool for its job

There's a persistent myth that COBOL survives only because nobody dares to replace it. In reality, it was designed for exactly the work it still does: business data processing. Banks, insurers, pension funds and public administrations didn't pick it by accident.

COBOL is extremely fast at what matters to them: high-volume batch runs, transactional workloads, millions of fixed-format records processed overnight. And it's remarkably readable when you handle data that comes in many formats, sizes, precisions and rounding rules. Decimal arithmetic is built into the language. You declare a field with two decimal places, and that's what you get, every time, with no surprises.

01 WS-BALANCE  PIC S9(11)V99  COMP-3.
01 WS-RATE     PIC S9V9(6)    COMP-3.
01 WS-INTEREST PIC S9(9)V99   COMP-3.

COMPUTE WS-INTEREST ROUNDED =
    WS-BALANCE * WS-RATE / 12.

var balance = new BigDecimal("125000.00");
var rate    = new BigDecimal("0.034500");

var interest = balance.multiply(rate)
    .divide(BigDecimal.valueOf(12),
            2, RoundingMode.HALF_UP);

// forget the scale → may throw ArithmeticException
// use double instead → wrong cents

None of this is impossible in Java or C#. BigDecimal and decimal exist for exactly this. But every operation becomes a decision: scale, rounding mode, intermediate precision. Even the COBOL compiler's rules for intermediate results have to be reproduced exactly. Every decision is a chance to differ from the original by one cent, and one cent times ten million accounts is an audit finding.

And what about AI translation?

This is the new pitch: let an AI translate the whole thing. To be fair, for small applications it works well. A few thousand lines, a handful of programs, simple data, a clear scope: AI-assisted migration can get you a clean, maintainable result in days.

Core banking is not a small application, and on large systems the story has two chapters.

The first one, many teams still remember: rule-based converters that translated COBOL to Java line by line. They produced what developers call “JABOL”: Java structured exactly like the COBOL it came from. Paragraphs became methods, PERFORM became a call, working storage became giant classes full of fields. Java developers couldn't read it because it wasn't idiomatic Java. COBOL developers couldn't read it because it wasn't COBOL anymore. You kept all the complexity and lost the language designed to express it, along with the precision and performance described above.

JABOL: Java that nobody can read, generated by a machine, describing rules nobody remembers.

The second chapter is AI, and it has changed the game. AI doesn't have to translate line by line. It can genuinely redesign an application, restructure it and produce code that looks like a Java team wrote it. That's real progress. But it raises a much harder question: when the new code looks nothing like the old one, how do you know it still does exactly the same thing?

The testing wall

That question sinks most rewrite projects, whether the new code is written by people, converters or AI, and usually late and at great cost: how do you prove that the new system does exactly what the old one did? You need test scenarios on both sides, and both sides are hard.

On the source side, the knowledge is fading. The people who wrote the rules have retired, specifications were lost three reorganizations ago, and nobody remembers why account type 7 is handled differently on the last business day of February. This part, at least, has improved. AI can now analyze code, data and production logs to recover business rules and rebuild realistic test scenarios. A few years ago that meant months of archaeology. Today it's a realistic first step.

On the target side, it's a different story. If you change the language, the architecture, the database and the runtime all at once, you lose every point of comparison. Batch becomes events, records become objects, one program becomes twelve services.

That's what makes it so crippling. Your existing tests, if you have any, can't be reused: they were written for JCL, 3270 screens and fixed-format files that no longer exist. Old and new outputs can't be compared directly either: a sequential file on one side, JSON messages and database rows on the other. Before you can test anything, you have to build a reconciliation layer that maps one world onto the other, and that's a project in itself.

Then come the differences nobody planned for:

• Rounding and intermediate precision: one cent here, one cent there, across millions of accounts.
• Sort order: on the mainframe, EBCDIC sorts letters before digits. ASCII does the opposite. Every report, key range and file merge can come out in a different order.
• Data quirks: REDEFINES, packed decimals, LOW-VALUES and dates stored in a dozen formats that the old code handled silently.
• Bugs the business relies on: downstream processes were built around the old system's quirks. Reproduce them, or fix them and break everything that depends on them? Each one is a business decision.

Every difference has to be investigated by someone who understands both the business and the old code, and those are the scarcest people you have. You also need near-total behavioral coverage, built from decades of production data that first has to be extracted and anonymized. Whatever you miss shows up in production.

This is why so many AI conversion projects I've seen blow up. AI makes the conversion itself fast: the new code is there in weeks, and the project looks almost done very early. But the cost has only moved. The bottleneck is now proof, and proof doesn't speed up the same way. Months of parallel runs on two infrastructures, reconciliation teams, a list of differences that never quite converges. Go-live slips, the budget explodes, and many of these projects are quietly stopped, with the old system still running.

The real problem isn't the language

Ask IT leaders why they want to leave COBOL and the answers are rarely about the language itself. They're about:

• The platform: mainframe, AS400 or GCOS licensing, MIPS costs and hardware lock-in.
• Isolation: systems that can't talk to the web, mobile apps or the cloud.
• The data: locked in VSAM files, IMS or proprietary databases that analytics teams can't reach.
• The tooling: green screens, no version control, no CI, no automated tests.
• The skills: fear that nobody will be able to maintain it in ten years.

Look closely and every item on that list comes down to the same thing: cost. Platform licenses and MIPS billed every month. Point-to-point integrations built by hand every time a new channel needs the data. Exports and duplicate databases just to feed reporting. Slow, manual releases. Scarce specialists billed at a premium. The language isn't what's expensive. Everything around it is.

That's good news, because every one of those costs can be cut without rewriting a single business rule. A rewrite, on the other hand, adds the biggest cost of all on top: years of building a second system that has to do exactly what the first one already does.

Let AI read, let proven methods write

This is the distinction that matters most, and it's where most “AI migration” pitches get it backwards.

AI is extraordinary at understanding. It can read millions of lines of COBOL, copybooks and JCL, connect programs to the data they touch, explain what a batch chain really does, spot dead code and pull business rules out of nested IFs. Work that used to take a team months of manual analysis now takes weeks. For the first time, you can see your whole system clearly before deciding what to do with it. That alone changes every modernization project.

Changing a production system that moves money is a different job. There you want transformations that are deterministic: the same input gives the same output, every time. And the good news is that the transformations you actually need aren't new problems. Rehosting COBOL, migrating data out of VSAM or IMS, exposing transactions as web services: there are established methods for each of them, driven by your copybooks and data definitions, and they've run on real production systems for years. They're repeatable, auditable and testable, which is exactly what your auditors and regulators will ask for.

The problem with AI translation isn't AI. It's asking a probabilistic tool to write millions of lines of critical code that nobody can fully review. Run it twice and you may get two different answers. That's fine for a draft. It's not fine for a ledger.

Deterministic doesn't mean safe by default: the JABOL converters were deterministic too. Their problem was the job they were given, turning one language into another line by line. The methods here do something narrower. They leave the business logic in COBOL and only change what surrounds it: the platform, the data format, the interface. That's a job rules do well.

So AI doesn't replace those methods. It dramatically speeds up putting them in place.

Setting up a deterministic transformation used to be the slow part: analyzing every copybook, writing the mapping rules by hand, configuring the converters, defining each service interface, building the test harness and the reconciliation scripts. That's months of specialist work before the first record moves. This is exactly the kind of work AI is good at. It drafts the mapping rules, the converter configuration, the service definitions and the tests, and your experts review them. The deterministic engine then applies those rules, the same way every time, as often as you need.

The difference is what gets reviewed. With AI translation, humans have to check millions of lines of generated code. With AI-assisted deterministic methods, they check a set of rules and mappings that fits in a document. AI drafts the rules, experts approve them, the engine applies them. You get the speed of AI and the guarantees of a method that has already worked in production.

AI to understand the system. Proven, deterministic methods to transform it. AI to set those methods up in weeks instead of months.

What to do instead

Modernization doesn't have to be a big bang. It can be a sequence of small, verifiable steps, with the business running the whole time.

“But nobody knows COBOL anymore”

That's the objection I hear most often, and it's the one that has changed the most in the last few years.

COBOL itself was never the hard part. It's a small, verbose, explicit language, and a developer comfortable with Java or Python can read it within a few weeks. What was hard was the system: thousands of programs, undocumented flows, company-specific conventions, knowledge locked in a few heads.

That's exactly where AI helps. System maps, dependency graphs, plain-language explanations of a program, documentation generated from the code, business rules extracted and cross-referenced: none of this was available at a reasonable cost a few years ago. A new developer can now ask “what happens to a loan when the customer moves abroad?” and get an answer that points to the right programs. The same goes for RPG, PL/I, Natural and every other legacy language.

And a modernized environment with Linux, Git, APIs and a modern IDE is a much easier place to hire into than a green screen.

Keep what works, change what doesn't

The choice isn't between “keep the mainframe forever” and “rewrite everything in Java”. That's a false dilemma, and it has cost the industry a great deal of money.

The pragmatic path is to use AI to understand your existing system: its rules, its data, its tests. Then transform it with established, deterministic methods, and let AI speed up putting them in place: move it to modern platforms, free the data, and open it up with web services so it coexists with new systems, built in whatever language makes sense for them. Then let old and new evolve together, each doing what it does best.