Several people I know have been saying for some time that we need to stop looking at emerging AI capabilities as isolated pieces of technology because the real risk rarely comes from one tool existing on its own. The issue is what happens when multiple capabilities converge and suddenly something that previously required significant skill, time and resources becomes available to almost anyone with a smartphone.
Let me be clear from the start: AI itself does not wake up one morning and decide to harm someone – software does not have intent. It does not decide that someone’s privacy, dignity or safety no longer matters; the technology is not making moral choices – this is being done by some people. That point is important because, as I wrote about recently, I think we are in danger of having the wrong conversations. We keep asking whether AI is good or bad, as though technology comes with its own personality and intentions – it does not. The more useful question is whether we are prepared for what happens when powerful tools are placed into the hands of people who choose to use them irresponsibly.
The reason image-to-video technology was developed was not to create deepfakes or enable abuse. There are genuine and valuable reasons for wanting to transform still images into moving content. A teacher can bring history to life through animated photographs; museums can create more engaging experiences about our ancestors or dinosaurs, a small business can produce marketing content without needing a business loan, a micro indie game producer can get a great idea out to market for a much reduced cost and compete with the bigger players; all these are positive uses that matter and should not be dismissed.
However, and this is where I believe we need to have a much more honest conversation, technology does not stay within the boundaries of its original intention – we have seen this repeatedly. The internet was built to connect people and share information, but it also created new opportunities for fraud and abuse. Social media gave millions a voice, but it also became a vehicle for harassment, monetisers and manipulation. Smartphones gave us incredible convenience, but they also changed the way privacy works in public spaces; AI is no different.
What concerns me is not image-to-video AI sitting in isolation; what concerns me is the combination of technologies that are now becoming available at the same time. Smart glasses capable of discreet image capture, powerful mobile devices capable of running AI locally, face manipulation technology, voice cloning and synthetic media generation are all moving forward together – the result is a very different risk landscape and as a result, we now have to ask is what happens when someone can capture an image without consent, process it privately on a device in their pocket and generate convincing “fake” content without ever needing to upload anything to a platform.
This is where safeguarding becomes much harder because for years, much of online safety has relied on the idea that harmful content will eventually pass through a platform where it can be detected/spotted and hopefully reported and removed (but don’t hold your breath). But what happens when the harmful content is created privately and locally before anyone else even knows it exists? That is the part of this conversation that worries me and this is also why I think we need to be more nuanced when discussing AI applications such as nudification tools. It is not enough to say “the software is neutral” and leave the conversation there. Yes, the underlying technology may be neutral, but not every application of that technology is the same. A general-purpose AI image tool can be used for education, creativity, business and accessibility; the fact that someone might misuse it does not mean the technology itself should be condemned, but we need to be aware.
But an application whose main purpose is taking an image of a real person and generating fake intimate image(s) without their consent is a different category of problem. The foreseeable harm is not an unexpected side effect; it is central to the way the product is used. This is where I think we need to move beyond simplistic arguments. We already understand that intent and purpose matter in other areas of society and do not treat every object that could potentially cause harm in exactly the same way. We look at why something exists, how it is marketed, how it is likely to be used and what risks it creates. I believe that AI should not be treated differently and the question should not be “can this technology be misused?” because almost every technology can. The question should be “what is this application primarily enabling, and are we taking responsibility for the predictable consequences?” a question for platforms, app stores and search engines. If an application exists primarily to create non-consensual intimate imagery, why should it sit alongside tools designed for creativity, education or business? At what point does providing access and visibility become part of the responsibility conversation? I understand that enforcement is complicated, because echnology operates globally and currently, legislation does not. Developers can change names, move services and attempt to avoid restrictions. Removing one version of an application does not necessarily remove the problem. But difficulty does not mean impossibility, surely and why not enable AI to track and identify these rogue elements?
The mistake we often make with technology is waiting until harm becomes widespread before deciding that safeguards were needed. We then look backwards and ask why nobody anticipated the risks; with AI, we have the chance to do better. This is not about stopping innovation or about saying that AI has no value , on the contrary – the reason this conversation matters is precisely because these technologies are powerful and useful. But power without responsibility creates risk…
The challenge ahead is making sure we are capable of recognising the difference between innovation that expands opportunity and applications where the foreseeable outcome is harm. Because currently, AI is not the thing making the decision, a small group (I presume) of people are. Our responsibility is to make sure that when powerful technology reaches people’s hands, the safeguards protecting others are not an afterthought added once the damage has already been done.
FAQs:
What are the risks of AI image-to-video technology?
AI image-to-video technology creates risks when combined with malicious intent, particularly around deepfakes, impersonation, harassment and non-consensual imagery.
Is AI itself responsible for harmful content?
No. AI does not have intent. Harm occurs when people choose to misuse technology or when safeguards fail to address foreseeable risks.
Why are AI nudification apps a safeguarding concern?
Many AI nudification applications are designed around creating synthetic intimate images of real people without consent, creating serious privacy and abuse concerns.
Why do smart glasses increase AI safety concerns?
Smart glasses combined with local AI processing could make it easier to capture, manipulate and generate synthetic content without the knowledge of the person involved.
Should all AI tools be regulated in the same way?
No. Regulation should consider the purpose of the application, foreseeable misuse and the level of potential harm rather than treating all AI systems as identical.
References:
NIST – Artificial Intelligence Risk Management Framework
OECD – AI Principles
European Union – Artificial Intelligence Act
European Union – Digital Services Act
UK Government – Online Safety Act 2023
Ofcom – Online Safety Codes and Guidance
UK AI Security Institute – AI Safety Research
UNESCO – Recommendation on the Ethics of Artificial Intelligence
Internet Watch Foundation – Research on AI-generated harmful imagery
