From project rescue to vibe-code rescue
Vibe coding is transforming how fast businesses build internal tools. But before an AI-built app touches client data, it needs proper assessment. Working is not the same as safe.

For years, a software project rescue looked much the same. A business had something it needed to deliver, it had run short of people who lacked the necessary skills, and it called in a software house. A squad would float in alongside the in-house team, steady things and get the project over the line.
Those calls still come in. But we are now inundated with a different kind of request.
The business has already built the product. Usually someone with a good idea and little or no software development experience has sat down with an AI tool and prompted their way to a working app. They have clicked through it, it does what they wanted, and now they would like a review before they switch it on, put client data through it and, quite often, sell it to other businesses.
I want to be clear from the start: I am not here to discourage anyone from building with AI. Getting from an idea to something that works, without writing code, is impressive, and it is changing how quickly businesses get value. My point is narrower. A vibe-coded app should not be treated as production ready, or as safe for confidential, sensitive or client data, until it has been properly assessed. What most of these businesses have built is not yet what they think it is.
What lurks behind the glossy front end
The pattern is remarkably consistent. The code lives on one person’s laptop, with no shared repository, no version history and no backup. API keys, database passwords and cloud credentials are written straight into the source. Individual files run to thousands of lines, because the AI kept adding to what was there rather than structuring it, and nobody can now change one part without breaking another. Code quality, put kindly, is poor.
What worries me most is not any single finding, it is that the person who built the app can rarely tell me anything about how it works: which framework it uses, what it depends on, where it is hosted, which third parties it talks to. The answers are usually a shrug. Somewhere along the way, the AI wired in a hosted database, an email platform or another AI service because it was the quickest route to a working feature, and nobody noticed. If the owner does not know those services are there, their clients do not either.
And yet they are confident it is secure, because they asked the AI to make it secure. That confidence is the real risk. Asking a model to “make it secure” is not the same as anyone checking that it is, and the model will happily tell you it has done the job. An app can look finished and work perfectly on screen while nobody, including the person who built it, knows what is underneath. That is the heart of the problem.
Working is not the same as safe
This might sound odd from someone who spends a good part of his week reviewing these apps, but businesses should be embracing vibe coding, not being warned off it. For internal tools, it has completely changed how fast a team can fix a clunky process, automate a manual task or test an idea with the people who will use it. Work that once sat in an IT backlog for months can now be in someone’s hands in days. That is real value, and I would encourage any business to explore it.
The moment an app is going to hold confidential, sensitive or client information, or be sold to other businesses, the question changes from “does it work?” to “is it safe?”. The mistake I see is businesses treating a working app as a finished product. What most of them have is a convincing, clickable representation of their idea, but without a proper assessment, it is unlikely to be ready to handle that data securely. Part of the problem is language. A prototype, a proof of concept, a minimum viable product and a finished product are four different things, each with a different job and a different bar for safety, yet the terms get used interchangeably. Knowing which one you actually have is where good decisions start.
Professional software is built through a secure software development lifecycle, or SDLC, which goes as follows: requirements, design, build, test, release and ongoing support, with security built into every stage, not bolted on at the end. The secure SDLC exists because skipping any of those steps eventually costs someone. In my view, once an app is heading for sensitive data, AI coding tools should be used to speed up the SDLC, not to replace it. They are very good at the first but cannot do the second. The businesses that get into trouble are the ones that skip the assessment, not the ones that use AI.
What regulators and researchers are finding
This is not just my frustration. The UK’s National Cyber Security Centre has published guidance it calls the “vibe coding spectrum”1. Its message is balanced. Letting the AI run is often fine for prototypes and demos, but anything that handles personal data, logins or credentials needs experienced people reviewing, understanding and testing what the AI produced. That is precisely where most of the apps I see have failed without anyone noticing.
McKinsey’s latest research is a useful corrective to the hype. Its Technology Trends Outlook 2026 adds agentic software development as a new trend, and warns plainly that “AI is producing code faster than human systems can review, test, and deploy it securely”2. The full report found that just a quarter of companies using agentic software development tools had achieved meaningful acceleration, and that in 30% of companies, productivity actually fell after teams adopted them. In McKinsey’s words, almost everyone seems to be “vibe coding”, but that does not always lead to value. In one study cited by McKinsey, AI tools increased coding activity by 180% while shipped releases rose by only 30%, and 46% of developers actively distrust the accuracy of AI tools, against 33% who trust them3.
Independent research points in the same direction. A peer-reviewed study published in Empirical Software Engineering in March 2026 analysed code that ChatGPT had written during real developers’ conversations. It found AI-generated code was about as likely to contain vulnerabilities as developer-written code, and that AI models could confidently give incorrect information, a particular risk for less experienced developers4. Confident and wrong is a dangerous combination when the person reading the answer has no way to check if it is correct, which is exactly the position many vibe coders find themselves in. The same study found the latest models now spot and fix around 80% of those flaws when explicitly asked to look, up from around half in the same tests a year earlier. That is the real lesson: AI is a strong tool inside a review process, not a substitute for one. Google’s DORA research, drawn from nearly 5,000 technology professionals, found AI adoption is linked to faster delivery but also less stable delivery, and describes AI as an amplifier of whatever process is already in place5. If there is no process, there is nothing substantial to amplify.
The security industry’s figures tell the same story. Veracode’s latest testing found AI models are improving, but around 38% of code generation tasks still produced a known vulnerability, even though almost all of the code ran perfectly well6. GitGuardian found that commits made with Claude Code leaked passwords and keys at roughly twice the baseline rate7.
Why it becomes a legal problem
The moment client data goes through one of these apps, UK GDPR applies in full. The law does not care how the code was written. The ICO has been blunt that autonomy is “not an excuse for poor compliance”8.
The sub-processor issue is where I see businesses most exposed without them realising it. If you handle personal data on behalf of clients, Article 28 of UK GDPR requires their written authorisation before you engage another processor9. Every service the AI quietly wired in that touches that data counts. Nobody has told the clients, and often nobody could have, because the owner did not know either. That can put a business in breach of its own client contracts before an attacker has even turned up.
Software suppliers are not exempt from enforcement either. In 2025 the ICO fined software provider Advanced £3.07 million for basic security failings, its first fine against a data processor10, and once you start selling your app to other businesses, the government’s Software Security Code of Practice sets out what those customers can reasonably expect of you as a software vendor11.
Brexit does not keep the EU AI Act out
Article 2 of the Act applies to providers and deployers anywhere, including the UK, once their AI system’s output is used in the EU12.
Scope matters here. Using an AI tool to write your code does not, on its own, bring your app under the Act. Building AI into the app does, whether that is a chatbot, AI-written content, or recommendations and decisions about people. Plenty of vibe-coded apps do exactly that, often through an AI service the builder barely noticed being added. If your app does, be aware that some uses of AI have been banned outright since February 2025, and transparency rules have applied since 2 August 2026, so people must be told when they are dealing with AI12. High-risk uses, such as recruitment or credit scoring13, face the full regime from 2 December 2027 after this year’s Digital Omnibus delay14. A delay is not an exemption, and governance built in now costs far less than governance bolted on later.
The part of the Act I find most useful sits in Recital 2715. It points back to seven ethical principles from the Ethics Guidelines for Trustworthy AI, published in 2019 by the EU’s independent High-Level Expert Group on AI: human agency and oversight; technical robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; societal and environmental well-being; and accountability16. They are not binding on their own, but the Act says they should shape how AI is designed and used.
I think the seven ethical principles deserve a far wider audience than compliance teams. They are the obvious standard for any AI features inside an app, but they are also a fair test of the app itself, because vibe coding hands the whole build to an AI. Put the principles to a typical vibe-coded app as plain questions and the gaps quickly become apparent. Has a qualified person reviewed what the AI built, and could they overrule it? Is it secure, tested and resilient to attack? Do you know where every piece of personal data goes? Can you explain how it works, and do users know when AI is involved? If it makes judgements about people, has anyone tested it for bias? Who could be harmed if it fails? And who owns it, and could you show a regulator how it reaches its decisions?
Most of the apps I review cannot yet answer the questions on oversight, robustness, privacy and accountability. Even if you never sell into the EU, those seven questions are a sound test of whether something built with AI is ready for real people and real data.
It’s already happening
Early in 2026, the security firm Wiz found that Moltbook, a platform whose founder said he had not written the code himself, had left its production database open. A key sat in the site’s JavaScript and the database’s row-level security had never been switched on. Wiz reported that around 1.5 million API tokens and 35,000 email addresses were exposed17. The app worked perfectly well, but nobody had checked whether the data was locked.
Before you switch it on
If you have built something with AI and plan to put client data through it, start by being honest about what you have. If it will handle personal data, logins or credentials, it needs a secure SDLC and experienced review, not more prompting.
Get the code off the laptop and into a managed repository. Move every secret out of the code and change it, because anything that has ever sat in the code or been pasted into the chat should be treated as exposed. Make a list of every third party the app sends data to, and tell your clients before go-live rather than after. If the app uses AI and reaches anyone in the EU, work out where it stands under the AI Act, and test it against the seven principles either way. Then get an independent review, and name one person who owns patching and incidents once real users arrive.
Build fast, launch carefully
Keep vibe coding. Use it to build the internal tools that make work faster and better, because the gains are real. But do not mistake a working app for a safe one. Before confidential, sensitive or client data goes in, someone with the right experience needs to assess what the AI built and decide whether it is ready.
That is what our application reviews at RiverAI are for. We look at code quality, security, infrastructure, data protection and governance, and give you a clear, prioritised plan written for the people who have to act on it. A review is not a certification, and we will always tell you plainly what it does and does not cover.
If you have built something and are about to switch it on, talk to us first. Before you do, ask yourself one honest question: is what you have a prototype, a proof of concept, an MVP or a product? Your answer changes what you need to do before any real data is passed through it.
This is general information, not legal advice.
References
- Toby W (2026) The ‘vibe coding spectrum’ approach to AI-assisted software development. National Cyber Security Centre, 18 June 2026. (Accessed: 6 November 2026).
- Chui, M., Roberts, R. and Catlin, T. (2026) McKinsey Technology Trends Outlook 2026. McKinsey & Company, 15 September 2026. (Accessed: 6 November 2026).
- Chui, M., Roberts, R. and Catlin, T. (2026) McKinsey Technology Trends Outlook 2026, full report (PDF), chapter 01: ‘Agentic software development’. McKinsey & Company, September 2026. (Accessed: 6 November 2026).
- Belozerov, V., Barclay, P.J. and Sami, A. (2026) ‘Secure coding with AI: from detection to repair’, Empirical Software Engineering, 31, article 93. Published 13 March 2026. (Accessed: 7 November 2026).
- Harvey, N. (2025) Announcing the 2025 DORA Report: State of AI-assisted Software Development. Google Cloud Blog, 23 September 2025. (Accessed: 7 November 2026).
- Veracode (2026) GenAI Code Security Fall 2026: The Ceiling Moves, But on the Low-Hanging Fruit. Veracode Blog, 29 October 2026. (Accessed: 7 November 2026).
- GitGuardian (2026) GitGuardian Reports an 81% Surge of AI-Service Leaks as 29M Secrets Hit Public GitHub. Press release, CyberNewswire via HackerNoon, 17 March 2026. (Accessed: 8 November 2026).
- Freshfields (2026) AI agents and UK data protection law: ICO sets out its expectations and calls for evidence. Freshfields Technology Quotient, October 2026. (Accessed: 6 November 2026).
- UK Health Security Agency (n.d.) Approval standards and guidelines: engaging a data processor. GOV.UK. (Accessed: 6 November 2026).
- Clifford Chance (2025) ICO fines processor after inadequate security measures lead to widespread disruption. Talking Tech, April 2025. (Accessed: 8 November 2026).
- Department for Science, Innovation and Technology and National Cyber Security Centre (2025) Software Security Code of Practice. GOV.UK, 7 May 2025, updated 15 January 2026. (Accessed: 8 November 2026).
- Phillips, N. and Clifford, S. (2026) EU AI Act: The need for transparency, why UK businesses cannot ignore Article 50. Edwin Coe Insights, 17 August 2026. (Accessed: 7 November 2026).
- European Parliament and Council of the European Union (2024) Regulation (EU) 2024/1689 (Artificial Intelligence Act), Annex III: High-risk AI systems. Via the EU AI Act Explorer. (Accessed: 6 November 2026).
- Hunton Andrews Kurth (2026) EU Digital Omnibus on AI enters into force. Privacy & Cybersecurity Law Blog, July 2026. (Accessed: 8 November 2026).
- European Parliament and Council of the European Union (2024) Regulation (EU) 2024/1689 (Artificial Intelligence Act), Recital 27. Via the EU AI Act Explorer. (Accessed: 6 November 2026).
- High-Level Expert Group on Artificial Intelligence (2019) Ethics Guidelines for Trustworthy AI. European Commission, 8 April 2019. (Accessed: 6 November 2026).
- Nagli, G. (2026) Hacking Moltbook: The AI Social Network Any Human Can Control. Wiz Blog, February 2026. (Accessed: 8 November 2026).
