Why we should take the progress of generative AI seriously without forgetting how reliable financial analysis actually works.
I recently came across an article by René Kampschulte with the rather alarming title Fellow CFOs, we are f**.* In his article, René describes how, after years of scepticism about the practical applications of generative AI in finance, his recent experiences with Claude led him to a radically different conclusion. The technology has become so capable that he expects the roles of CFOs, accountants and tax professionals, at least as we know them today, to largely disappear within five years.
I am certainly not going to argue that the capabilities of generative AI are unimpressive. I use the technology extensively myself and am regularly surprised by the speed at which language models are developing. In particular, their ability to interact with software, write scripts, structure information and make complex technical tasks more accessible is genuinely interesting.
What surprised me about René’s article, however, was not his enthusiasm. It was how quickly he appeared to abandon several perfectly reasonable concerns he had previously held about generative AI. He mentions the limitations of language models when it comes to calculations, processing structured data, producing reliable outcomes and understanding financial processes. He then argues that larger context windows, the ability to use Python and the emergence of AI agents have largely resolved these problems.
I think there is a fundamental misunderstanding here. We increasingly confuse what generative AI can actually do itself with what existing, often deterministic technology can now make accessible through a language model.
And that distinction matters when we want to use these technologies in a professional finance environment.
1. If Claude uses Python, who is actually doing the calculations?
One of René’s original concerns was that language models are not particularly reliable when performing financial calculations. That concern now appears to have disappeared because modern models can use Python, process spreadsheets and call specialised calculation tools.
But why would we consider this evidence that language models themselves have become better at mathematics? If I ask a language model to perform a revenue analysis, it can generate a Python script that subsequently processes thousands of transactions. The calculations are then performed by Python, not by the language model. I could have performed exactly the same analysis using SQL, KNIME or other analytical technologies that have been available for many years.
The progress lies in the fact that I no longer necessarily need to write the code myself to have such an analysis developed. That is a significant improvement in the accessibility of technology, but it does not change the underlying principles of reliable data analytics.
It also introduces a new dependency. Who verifies that the AI-generated script actually performs the calculations the user intended?
Suppose a CFO asks for an analysis of gross margins by product category. The language model writes a script, processes the transactions and produces a perfectly presented report. All the arithmetic may be correct, while the analysis itself is fundamentally wrong because credit notes have been handled incorrectly, cost prices do not reconcile or intercompany transactions have not been eliminated.
This is not a new problem introduced by AI. It is a familiar issue in data analytics. The reliability of an outcome depends not only on the accuracy of the calculations but also on data selection, definitions and the logic applied.
The fact that a language model can operate Python does not make these issues any less relevant.
2. Being able to read more data does not mean understanding that data
A second argument is that modern language models, thanks to their much larger context windows, can now process significantly greater volumes of financial information. Entire spreadsheets, annual reports and other documents can be provided in a single session.
This is useful and creates interesting possibilities. However, I think it is quite a leap to conclude that the limitations surrounding structured financial data have therefore largely disappeared.
Any CFO who has ever been involved in an ERP implementation or the development of management reporting knows how difficult it can be to bring financial and operational information together consistently.
Revenue is recorded in the financial accounting system. The CRM system contains contract values and sales opportunities. The operational system records deliveries and services performed. All three systems contain information about the same business, but they do not necessarily represent the same economic reality. You can feed all three datasets into a language model. The model may then produce a convincing explanation or even develop an impressive dashboard. But that does not establish which definitions have been applied, which information is authoritative or whether the different systems actually reconcile.
At Coney Minds, we prefer to start by examining the underlying data architecture. How do we extract data from the original source systems? Which transformations are applied? How do we establish consistent definitions, and how do we ensure that different reports are based on the same reliable information?
From that foundation, organisations can support finance reporting, tax reporting and business reporting. The information requirements may differ, but that does not mean each reporting function should develop its own version of reality.
Generative AI can certainly help design, document and make this information architecture more accessible. But providing a language model with a large volume of data is not an alternative to building a sound data architecture.
3. Why would we use probabilistic technology for deterministic calculations?
This is perhaps my most important concern about much of the current discussion surrounding AI in finance. For decades, we have had technology capable of processing large volumes of structured data reliably and reproducibly. Relational databases, SQL, Python, KNIME and other analytical applications are familiar examples.
Of course, these technologies have limitations too. A poorly written SQL query produces incorrect results, and a Python script containing programming errors does not magically become reliable simply because its execution is deterministic.
The important difference is that we can explicitly define, test and reproduce the logic being applied. Generative AI works differently. Language models are particularly well suited to interpreting information, formulating instructions, generating code and explaining results. However, the model’s own outputs are not governed by the same explicitly defined calculation rules as those of a controlled analytical process.
This distinction becomes blurred when we conveniently describe every technology operated through Claude or ChatGPT as AI.
When a language model generates a SQL query that is subsequently executed against a database, the calculation is reproducible because the database executes that query according to defined rules. Not because the language model itself has suddenly become deterministic.
I therefore see little reason to replace established, well-functioning analytical technologies with a language model. A much more interesting question is how generative AI can help us use those technologies more effectively and make them more accessible. For me, the value lies in combining them rather than replacing one with the other.
4. An AI agent checking its own work does not automatically provide assurance
The emergence of AI agents also plays an important role in René’s argument. Modern agents can independently perform multiple steps, collect information, develop scripts, evaluate results and initiate follow-up actions.
This is a meaningful development because increasingly complex parts of financial processes can be automated. However, it also requires us to distinguish between performing checks and actually obtaining assurance.
Suppose an AI agent performs a revenue analysis and then compares the calculated totals with those in the original dataset. When both figures match, the agent has established an arithmetic reconciliation.
But what exactly have we verified? We still do not know whether the dataset is complete, whether the revenue has been recognised in the correct period or whether the underlying transactions actually occurred. If both the analysis and the subsequent check are based on the same incorrect assumptions, the system may simply confirm its own mistakes.
As an auditor, I am accustomed to approaching these questions from the perspective of risks and financial statement assertions. What assurance are we trying to obtain, which data do we need and which procedures will provide sufficient appropriate evidence?
The answers do not fundamentally change simply because an AI agent performs the work.
An AI agent can certainly help automate audit and control procedures. But the quality of those procedures depends on their design, the reliability of the underlying sources, the independence of the checks and the documentation of the results.
That requires a professionally designed process in which AI can play a useful role, rather than an assumption that an agent is reliable simply because it performs multiple steps and repeatedly reviews its own output.
5. We still judge AI too often by how impressive the results look
I increasingly see demonstrations in which a language model is given a spreadsheet and produces a comprehensive financial analysis within minutes.
That can be impressive, particularly when the model also creates a dashboard and writes a clear explanation of the findings. But I regularly find myself wondering what exactly has been demonstrated.
That the technology can produce a convincing result quickly? Certainly.
But have we also established that the results are accurate, complete and reproducible?
For a professional application, I want to know more. Which transactions have been processed? Which definitions have been applied? How were exceptions handled? Which calculations were performed, and how can we verify that those calculations are correct? And perhaps more importantly, what happens when the input is incomplete or when financial data is contradictory?
These are questions we also ask when using traditional data analytics. It would be rather strange if, simply because generative AI looks so impressive, we suddenly accepted a lower standard of evidence. For me, the audit trail is therefore not an administrative afterthought that can be added later. It is an essential part of designing a reliable analytical process.
I want to be able to reconstruct how we arrived at a conclusion from the original source data. Which records were selected, which transformations were applied, which exceptions were identified and which decisions were made along the way?
A well-designed AI application can record and make this information accessible. But that is a feature of the process we have designed, not an inherent guarantee provided by the language model.
6. The truly interesting development is the combination of existing technology and generative AI
Perhaps we should approach this discussion differently. Instead of constantly asking which activities generative AI will take over, we should investigate how different technologies can complement one another.
In our own practice, we have been using data analytics, scripting and visualisation technologies for many years to make financial and operational information more accessible. The underlying needs of organisations have not fundamentally changed.
They want to understand where money is being made and lost, why performance differs from expectations, which risks are emerging and which decisions need to be made.
What is changing is how we can answer those questions. A financial information environment can be built around consistent data sources, controlled data transformations and reproducible analyses. Reporting and dashboards can then provide users with reliable insights into business performance.
Generative AI can help make that information environment more accessible. A CFO may no longer need to navigate several dashboards but could simply ask, in natural language, why the gross margin of a particular product group declined.
A properly designed system could translate that question into existing analytical procedures, retrieve the relevant results and support their interpretation. An AI agent could coordinate the different applications and organise the necessary steps.
I find this a much more interesting development than the idea that a language model should independently take over financial analysis. The quality of the process still depends on the underlying data architecture, the calculations performed and the way the results are validated. Generative AI therefore becomes an additional layer on top of a reliable information environment rather than an alternative to that environment.
7. And what about the CFO who supposedly will not exist in five years?
I understand the temptation to make bold predictions when technology is developing so quickly. But over the past thirty years, I have heard more than once that existing professions were about to disappear. The introduction of ERP systems, business intelligence, process automation and data analytics has fundamentally changed the finance function. Many activities that were previously performed manually have now been automated.
That has not resulted in the disappearance of the CFO, although it has certainly changed the role, the available information and the expectations organisations place on finance professionals. Generative AI may accelerate these developments. I expect that developing analyses, programming reports and making information accessible will become significantly easier. But a CFO is not merely someone responsible for producing spreadsheets or preparing financial reports.
The profession is also about understanding the business, evaluating risks, making judgements and accepting responsibility for the quality of information used to support decisions.Technology can support many of these responsibilities and undoubtedly automate particular activities. But the prediction that CFOs, accountants and tax professionals will largely disappear within five years seems insufficiently supported by the examples currently being presented.
In fact, as organisations automate more analyses and reporting activities, it may become even more important to have professionals who understand how the underlying information environment works and recognise the limitations of the technology being used. That seems to me a much more interesting development for our profession.
8. My advice: invest in a reliable information environment first, then decide where AI adds value
If I were reviewing a company’s financial information environment as a CFO today, I would not start by asking which AI agent to purchase. I would want to understand how reliable the existing data is, how information is extracted from source systems and which analyses can already be performed reproducibly. I would also investigate how much time finance professionals spend collecting and correcting data before they can even begin analysing the business.
Many of the problems that generative AI is now expected to solve have existed for years and have very little to do with AI itself. Inconsistent revenue definitions, poorly integrated systems, manual Excel adjustments and reports whose underlying calculations nobody fully understands will not disappear simply because we place a language model on top of them. Once the information environment is sufficiently reliable, we can meaningfully explore where generative AI creates the greatest value.
Perhaps in automatically developing and maintaining analytical procedures. Perhaps in providing access to financial information through natural language, identifying developments that require attention or supporting management reporting.
And I would certainly not limit that exploration to generative AI. In many situations, existing analytical technology and relatively straightforward automation can already deliver substantial improvements. Technology choices should follow from the business problem we are trying to solve, rather than from the desire to do something with AI.
Final thoughts
I appreciate that René has exchanged his original scepticism for enthusiasm. His experiences illustrate just how accessible sophisticated technology has become and how many opportunities this may create for finance professionals.
But I believe his original concerns were more valuable than he now appears to give them credit for.
The fact that a language model can operate Python does not mean the language model itself has become reliable at calculations. The ability to process thousands of pages of financial information does not establish that the underlying data is accurate and consistent. And the fact that an AI agent performs several checks does not automatically mean we have obtained sufficient assurance over the results.
These are not merely theoretical distinctions. They go directly to how we design financial analyses, management information and control processes.
I am therefore not sceptical about the capabilities of generative AI. I am critical of the conclusions we sometimes draw from impressive technological demonstrations.
If we take these developments seriously, we should also continue to think seriously about how reliable financial information is actually produced.
Over the years, the finance profession has developed considerable expertise and technology to address precisely these challenges. I would rather build on that foundation than suggest, with every new generation of language models, that we need to start all over again.
