Putting a leash on AI

Apr 30, 2024
ai

Ai is an exciting topic. The entire field is fraught with potential. Even in it’s current infancy we see startups forming every day, not to mention the more mundane and pragmatic uses cropping up in everyday life. Quick document preparation, summarization, rough draft generation, chat bots, content thumbnail generation and many more. In our lifetime the ability to direct, narrate, score and edit an entire film in an accessible format to many people. But all this excitement comes with a looming anxiety about these technologies.

Who controls them? What are the rules for accessing them? How do they screen for dangerous content? How do we screen for content produced by them? These are just the simple questions, what about questions of laying off workers to automate their work with AI, or the dangers of the inherently incestuous relationship of learning off their own generated data? These questions are important, this article seeks to bring up some of these questions in specific contexts and use cases, as well as provide current lawsuits, draft legislation, and legal precedence in various countries to address some of these anxieties.

Deepfakes

Starting with one of the most contentious topics, deepfakes are a system of generative AI that allows you to do face and body-swaps of people into existing footage. In it’s positive uses it can be used by film companies to edit in actors to shots that could be dangerous, or even just for some fun¹ ². There are obvious concerns with this, most notably it’s use to propagate fake information³ (I will talk about this in the fake information section), and the ability of others to create deepfake porn.

The porn problem

Starting with the second point first, it seems to me that there’s no way around the potential issues of deepfake porn. In general models don’t do well at culling NSFW prompts, at least not reliably (2/3 in optimistic cases⁴). Likewise there’s not much research I’m aware of to just cull specifically deepfake content, and/or what that would look like. Though, admittedly the technology could prove possible eventually with the rate it’s developing, if this becomes the case I may change my perspective.

Most “mainstream” AI platforms should probably ban NSFW content in general and especially face/body swapping/undressing (and do⁵, though people think they shouldn’t⁶ ) . My opinion on this comes largely from 4 things.

The datasets being used have been demonstrated to have a ton of CSAM (Sexual Abuse Material) ⁷ ⁸ ⁹ ¹⁰, and NSFW prompts that are allowed to go through can obviously pull from that data and use it
We are seeing it used frequently for creating non-consensual content for celebrities¹¹ ¹²
Last and probably the most concerning is that the technology has gotten good enough that normal people are being used in the content¹³ ¹⁴ ¹⁵
There’s enough consensually shot porn in the world. By this I mean you can already find what you want, shot by people who are paid and willing participants. Beyond insidious purposes I don’t see much of a need for this feature on mainstream platforms considering how involved of a vetting process you would need to do on each request to do it responsibly

Unfortunately banning it from most easy-to-use generative AI systems will not get rid of it entirely¹⁶ ¹⁷ ¹⁸ ¹⁹, so additionally I would say prosecution akin to regular non-consensual porn should probably come into play (and additional charges for distribution). This seems to be in line with new draft law proposals from the UK and Wales²⁰ ²¹. Hopefully other countries will follow suit.

Fake information

For as long as the internet has existed there’s been people posting misinformation on it. It’s a time-honored legacy that even without AI has left a chilling effect on the world, from state-sponsored attacks²², to lying about science²³ it doesn’t take an AI to be awful. The concern now however is that AI can help lend an air of credibility to misinformation that previously was not possible. Mundane and annoying situations such as incorrect bug disclosures²⁴, to more malicious circumstances like companies being scammed with real-time recreations of their board members²⁵, or disgruntled ex-employee’s posting falsified racial screeds²⁶ ²⁷. Not to mention the “improvement” to the aforementioned state sponsored attacks and “hacktivist” groups²⁸.

Sadly these problems are going to likely require a multifaceted approach to resolve. On a platform basis fact-checking has become a crowd-sourced endeavor with platforms like X (twitter)²⁹ ³⁰ ³¹, and YouTube³² ³³ including fact-check boxes of crowd-sourced information. These have several problems including people “brigading” them³⁴ ³⁵ ³⁶. There are also third-party fact-checking websites³⁷ ³⁸ ³⁹ ⁴⁰, however all of this is assuming there are correct answers, and that these groups can be trusted. Searching on search engines now is so heavily influenced by demographics that you can often never escape your own biases. Another interesting approach I’ve seen is with groups like ground news who’s subscription plan instead shows you each “side” of stories. This still has the issue of bias, but I think gives an opportunity to somewhat escape ones own eco chamber of looking things up in our demographics-driven world of searches. Doing this and holding people more accountable for their beliefs, and the standard of information they use for developing those beliefs seems to be the only path forward I can see. Requiring media-literacy in education to understand these landscapes better would also help. Additionally more ways to escape a potential demographics-based echo chamber seems to be the sort of tooling we should invest in and/or legislate for.

Ownership

This one is a relatively simple point, all current forms of generative AI are currently breaking most copyright laws and most terms of service(s) for the sites they are pulling their data for. Every major generative AI that is in the mainstream has been trained on copyrighted works, and produces derivatives of those works. This can be seen most evidently in art generators reproducing image watermarks⁴¹ ⁴². However legally speaking it’s actually not this simple. As they cover in their blog post (though it is bias) open AI (makers of Chat GPT) have precedent in the US to continue their training on copyrighted materials⁴³. Time will tell whether these lawsuits pan out or not, but it is very clear that content creators are not being properly compensated for their work⁴⁴.

There have been some technical attempts to mitigate this with “watermarks”⁴⁵ ⁴⁶, to little avail⁴⁷ ⁴⁸. Other companies such as DeviantArt take the bizarre approach of allowing you to request your name to be blacklisted as a “style”⁴⁹, all but admitting it will be used to replace you. The indifference to the art community by these companies is astonishing, and the fact that situations where models are copying works of clearly-defined styles⁵⁰. Problem is that most models can’t just have the requested work removed, so there is no way around this once a model has been trained. Even more concerning is the move towards companies selling private data for datasets⁵¹. Ultimately the legislation will move too slowly on this, but the only fair resolution would be to force companies to retrain on explicitly opt-in datasets. This solution is basically untenable however because until forced to these companies will not acquiesce. So a more malicious solution might be better.

Some people have taken matters into their own hands and have tried data poisoning attacks⁵² ⁵³ ⁵⁴. These combined with prompt injection attacks⁵⁵ ⁵⁶ including providing fake prices, and using them to generate complex scripts. The intention being to cost the companies using them money, and ultimately that might be the best solution right now. Legislation is unlikely to catch up, and so some malicious messing with the systems might be the most reasonable and cathartic way to slow down this sort of adoption. It’s unclear whether companies will be forced to uphold the deals given on these sites, but hey may as well make it a headache for them, after all it’s not illegal! (double check that’s true in your country before following this advice).

Generative intimacy

Everybody gets a little lonely, and AI services are here to help⁵⁷ ⁵⁸. Don’t want to touch grass, but still want to get a girl, there’s dozens of services that will emulate one for you! With some incredible levels of tact, and very fine tuning, it would be possible to have AI companions that don’t cause psychological harm to people. Unfortunately we rarely get the good ending these days, and the reality is that AI companion apps are primarily, thinly veiled sex chatbots that happen to occasionally show a bit of personality. Many of the same AI companion concerns would also be applicable to OnlyFans⁵⁹ ⁶⁰, and it’s messaging system, with a few crucial exceptions.

Did your boyfriend die and you want to avoid the grieving process and pay a company to replace the hole in your heart, we’ve got you⁶¹ ⁶²! Many of these platforms play into very sinister aspects of dark-patterns including playing on people’s grief and loneliness. Concerning situations could arise in the future such as a commercialized version of a situation where a south Korean group “brought a daughter back to life”⁶³. This used an actor instead of AI, but this sort of thing being mass-market capable is concerning. However legislation around this would be difficult, especially when practices like mediumship are still allowed commercially. Likewise using someone’s (ostensible) partner to market to them is something to be highly concerned over. Depending on how prevalent these situations end up being there may need to be outright bans on certain interactions, like marketing, assuming another personality etc.

Generative academia

It is highly concerning to me just how quickly generative AI’s found a footing in academia. Anecdotally most of the students I know in computer science don’t write their own code anymore. Beyond that papers are getting less and less human⁶⁴ ⁶⁵ ⁶⁶, this is concerning in how much learning is no longer required from students. There’s a lot of skipping out on work that I suspect will lead to more at-time assignments (essays written in class, no laptops for exams etc.). Even in the research world chat GPT is being used to generate papers. This paper for example forgot to remove the GPT response in the introduction⁶⁷. This doesn’t necessarily invalidate the work, but it is a red-flag that should be a cause for concern.

Additionally we need to be skeptical of work being done using generative AI for research itself, such as recent CRISPR advances⁶⁸ ⁶⁹. This sort of work is nothing new⁷⁰, but again should raise some red flags. We need better systems in place for monitoring the work being done to ensure it’s up to standard. At minimum that should include a defined set of standard generative AI models that can be used in research this way, governance around requiring people to mention they used AI, and additional testing to validate results (as much as possible).

Wearable bugs

There has been an increase in the number of AI voice assistants becoming standalone products in recent years. Humane, the Rabbit R1 and I’m sure no end of similar clones. The problem with these systems that I see most prevalent is the same issues we already find with existing voice assistants like Siri, Bixby, Alexa and Google Assistant, the data. For years now conversations with these voice assistants has constantly leaked⁷¹ ⁷² ⁷³ ⁷⁴, and had data retention issues⁷⁵ ⁷⁶. Audio is notoriously hard to process⁷⁷ because of the density of the data, so it’s often trained more manually than something like image generation. People are used to label conversations effectively, which inevitably causes privacy issues. Even without this human intervention these AI pins will have other issues. When you’re using the assistants out and about none of the people you’re potentially recording have agreed to the terms of service. Every person who buys one of these and uses it frequently has the potential to leak the data of people around them at all times. Intense voice isolation is an option to limit this potential, but nothing is perfect.

I suspect these systems would also in some cases violate all-party consent requirements. For anyone who doesn’t know there are two different sets of laws for recordings, single-party consent, and all-party consent. Single-party locations allow people to record others without their consent, all-party locations require everyone to agree to being recorded. These laws differ in different countries around when these rules apply, however it is possible these types of assistants are just generally illegal in many places already.

Boundaries of evolution

When I was in high school I came across an incredible project called MarI/O (an interesting follow up here), at this point in my life I had never even programmed. I remember finding the mutation rate and testing to see how modifying it at different parts of the training would effect the result.

Seeing this project in action, and the work of creators like carykh was incredibly interesting and was an additional nudge to my interest in computer science. They were at the time working on an approach to deep learning called evolutionary neural networks. The terms get somewhat nebulous, but in essence it’s a system where you tell it things it can do, tell it what “succeeding” is (fitness function), and then it will mutate and modify it’s behavior to better attain that goal. With MarI/O that included telling it what buttons it could press, and allowing it to pick when to press them. Over time it builds a sort of “script” that it follows to learn when to press what buttons. This all sounds interesting, so what’s the problem.

Remember that whole bit about telling it what “success” looks like, it turns out that’s really hard to do. Imagine I have a semi-sentient robot, and I tell a robot to make ice cream, what could go wrong. If I’m an ice cream salesman it will just make ice cream for me right? Well we didn’t limit how it creates that ice cream, so it will just do whatever gets it more ice cream. If it determines beating cows to an inch of their life produces more milk (and thus more ice cream), it will do it, animal cruelty laws be dammed. If it determines it can threaten surrounding farmers and take their milk it will. Essentially because there is no well-defined boundary the typical moral intuitions we rely on, and the contextual understanding of the world does not apply.

While the examples seem like a far-fetch I-robot-esque plot consider a simpler case. What if I get a system to design a train system to get people from one place to the other faster. How long before it cuts safety regulations that weren’t given to it as context? What if we make a rule to say when people are on board it needs to be safe? Well then when people aren’t on board how long before it “cheats” and starts sending the train hundreds of miles an hour faster than it should to game the system? This is a nearly intractable problem, and as such this sort of system should really only be applied in simulations, and then human-verified.

Conclusion

AI poses an incredible dialectic of possibility and danger. There is so much potential to do incredible never-before-seen things with it, yet some things are better left never seen. Hopefully some of these suggestions can help get the ball rolling on ideas for where the lines should be to limit AI, and help start discussions for where future problems may need to be addressed.