I can clearly remember the first time I saw a deepfake. I was waiting for the train on my way to the Vice office in 2017 when I got a Twitter DM from someone who saw it posted to Reddit. It was a short .gif from the start of a porn video, showing a woman lying on a bed and talking to the camera, only the woman’s face had been edited to look like Gal Gadot.

The deepfake was shockingly realistic for the time, and as Sam’s reporting on it from that day on has shown, deepfakes have primarily been used to harass and degrade women. We were surprised by how advanced the technology seemed at the time and knew that the internet and the law wasn’t prepared to deal with it. The technology has advanced exponentially since then, and the internet and the law has still not caught up to it. But as concerning as deepfakes clearly were the moment we laid eyes on them, the problem was never that we thought Gal Gadot actually made an adult video. Deepfakes can look convincing, but they never fooled us that what we were seeing was real. 

But for the past few months I’ve been looking at a new and strange type of AI generated nonconsensual image on X that I think will fool a lot of people. These images, which I’ve started calling subtlefakes, are not as clearly offensive, but they are a lot more convincing. The concerning implication is that it’s not just the technology that is advancing now, but that the people who wield it maliciously are getting more sophisticated. Much like “cheap fakes,” which in 2019 Data & Society described as an edited piece of media that manipulates the truth without requiring AI or any other advanced technology, subtlefakes are another form of synthetic media that has taken over the internet since the emergence of deepfakes in 2017.

These more convincing images are not AI generated nudes of celebrities or face-swapped identities onto porn videos, but AI generated images of celebrities that are subtly edited to make them more revealing or provocative. We’ve written countless stories about AI generated nonconsensual media of celebrities and non celebrities. It’s a form of harassment that causes psychological and material damage to people, and they don’t have to be believable to do that. Even if the image looks photorealistic, it is often clearly fake for the simple reason that it’s unlikely that the biggest actor in the world would suddenly produce hardcore pornography that’s posted by a random account to X. 

The images I’ve been seeing lately are often only slightly altered. One type of image, for example, will edit a real photograph of a famous actor walking a red carpet, but edit it to make her dress more revealing or to make her butt or boobs bigger. Another type is entirely AI-generated post-workout selfies at the gym. 

The only example I feel totally comfortable sharing is one that was called out and shared by the target of one of these fakes, actor Xochitl Gomez. As a side-by-side she shared on Instagram shows, someone had taken a real photograph of her at a parking lot, looking over her shoulder and smiling at the camera, and used AI to make it look like she was bending over in one image and putting her hand on her butt in the other. Similarly, someone had taken an image of her on a red carpet and used AI to make it seem like she’s turning her back to the camera and sticking her tongue out. These images are not just photorealistic, they also don’t have the usual context clues we’ve come to expect from nonconsensual images. I think I am very good at spotting AI generated images because it’s a big part of what we do here, and I don’t think I could tell these were AI generated if I was just scrolling on X. 

These images are almost always posted by verified engagement farming accounts on X, which pays the people who operate them if their posts get enough impressions. X obviously doesn't care if people are doing this and a bunch of other stuff that’s as bad or worse. I’ve seen at least one account that shared a few of these subtle deepfakes, but when I scrolled down their post history I found nonconsensual fully nude and sexual images that they monetized via other platforms, so that’s another reason why X accounts might be doing this. There are many, many accounts like this and they are earning millions of views.

I think it’s interesting because it’s a new, and as far as I know, yet to be covered use of nonconsensual AI images. It’s also important and in the public’s interest to know it’s happening because people should know what kind of AI content is targeting them, especially if it’s at the expense of other people. It’s also possibly a warning sign for how people’s AI use is evolving from abusive but obviously fake, to more subtle, less offensive, but much harder to detect. 

The other reason these types of images are going to be harder to moderate is that it’s easier to produce them. There are plenty of services that let people easily AI generate nonconsensual nudes of anyone, but they almost always involve some friction. People have to go to a sketchy site, install a sketchy app, or pay a small fee. As we’ve reported many times in the past, more easily accessible AI generators from tech giants can be manipulated to produce nonconsensual images as well, but generally speaking, most companies are trying to prevent people from making those images with their tools. While it’s not impossible, it does require some effort to bypass those guardrails. 

It is not so hard to bypass those guardrails if the nonconsual images people are producing don’t include any nudity or sexual acts, assuming the AI generator is even design to prevent those type of images. Most mainstream AI generation tools at least try to prevent people from generating any type of nudity. Fewer of them try to prevent people from generating the likeness of a real person. I can’t say for certain where most of the subtlefakes I’ve seen come from, but I know that at least some of them were made with X’s own Grok because some images included the Grok watermark. X has repeatedly been caught enabling nonconsensual images and only attempting to do something about it after public outrage. 

X did not respond to a request for comment.

“Large social media platforms that are really struggling under the weight of this for the first time,” Hany Farid, an image forensics expert and cofounder of deepfake detection firm GetReal, told me. You can see my full interview with Farid here. “They've always had a problem, but this seems to be the first time where they're like, ‘Oh man, this is really bad.’ And I think it's because the volume and the sophistication of the fakes is nothing we've seen before.”

The primary targets of deepfakes always were and continue to be women, but Farid said other nefarious uses of the technology are also becoming harder to deal with.

“If you go back 10 years, it was pretty clumsy,” Farid said. “It's getting more and more sophisticated. They're evolving. Cyber criminals evolve and that's going to make everybody's job much much harder as the years go on.” 

Our reporting on deepfakes over the years has taught us that people make them because they feel entitled to women’s bodies, because they are intentionally trying to hurt them, or because it’s profitable. I’m sure all those reasons apply to subtlefakes, but it appears to me that the current state of X is pouring gasoline on that fire. People use subtlefakes as advertisements for accounts other platforms where they monetize nonconsensual images, but that is rare. It seems that the more logical scheme is to monetize pure engagement via X’s ads revenue sharing program, which pays X users for engagement. It’s not a lot of money, but it’s easy money. Since X is not interested in moderating this content, one person can also operate multiple accounts, posting dozens of times a day. 

In general, my X feed is mostly different types of engagement farming. Another type of subtlefake I’ve seen a lot of doesn’t require AI at all. For example, this post about OnlyFans model Sophie Rain claims that Drake paid her $2,000,000 to take her virginity, and includes two random images of Drake and a completely unrelated TMZ video of Rain. People click on the video to see if she said what the post claims, and the account gets the engagement before they realize it’s a lie. 

This kind of easy, stupid, but effective garbage is what X is rewarding, and these subtlefakes are just one concerning outcome.

I think we’re pretty good at telling the difference between an AI generated image and a real photograph, but when I need help I call Hany Farid. 

Farid is the cofounder of GetReal, a company that specializes in detecting deepfakes and other AI-generated images, and is the developer of PhotoDNA, a perceptual hashing algorithm that helps companies automatically detect and remove some of the worst images that exist online, and that is now being used by every serious internet platform in the world. 

We started talking to Hany regularly in 2017, when Sam first reported on deepfakes. As that technology evolved and changed how we perceive reality, so have our conversations. I wanted to talk to him today on the podcast so you could hear of those conversations, and so that Hany and I could zoom out, reflect on the past few years, and speculate about where we might be headed. 

404 Media is a journalist-founded company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers get access to additional episodes where we respond to their best comments. Subscribers also get early access to our interview series. Gain access to that content at 404media.co.

Listen to the weekly podcast on Apple Podcasts, Spotify, or YouTube. 

Become a paid subscriber for early access to these interview episodes and to power our journalism. If you become a paid subscriber, check your inbox for an email from our podcast host Transistor for a link to the subscribers-only version! You can also add that subscribers feed to your podcast app of choice and never miss an episode that way. The email should also contain the subscribers-only unlisted YouTube link for the extended video version too. It will also be in the show notes in your podcast player.

Amazon is buying massive quantities of books, scanning them for AI training data, and destroying them in the process. 

A 404 Media investigation was able to reveal Amazon’s book buying operation, which hasn’t been previously reported, by placing a tracking device in a rare book we suspected would be acquired by an AI company for training data, and following it around the country to its final destination. 

That final destination was an Amazon warehouse in Las Vegas, Nevada. Amazon employees who work at this location say all they do is receive massive shipments of printed books which they then cut the bindings off in order to scan the books more quickly. The printed book is destroyed in the process. The logo of the Amazon team that works at this warehouse, called VGT3, is a dinosaur, brandishing its teeth and with a book in its hands.

CGTrader, an online marketplace for 3D assets, found that one in six models uploaded to the site these days is AI generated, but that AI generated assets account for only $1 out of $90 in revenue on the site. These numbers show that AI generated assets are quickly flooding the marketplace, but that most people are not interested in paying for them.

“Buyers are voting with their wallets, and AI-generated content is struggling to compete,” CGTrader said in a press release about its 2026 market trends reports. The company says the gap between the surging supply of AI generated 3D models and the middling demand points to “a gap that undercuts the assumption that AI-made content is repricing the market.”

“The upload numbers alone would suggest a takeover,” CGTrader said. “The revenue numbers say otherwise, and buyers refusing to pay for AI-generated models is saying something bigger than no thanks: it is a signal of how far they trust AI generation itself. Which leaves the question the industry has been avoiding: an AI model may be cheaper to produce, but what is it actually worth?”

Like the Unreal and Unity asset stores, CGTrader is a marketplace where people can buy more than two million 3D models to use in their videos, games, or 3D printing projects, and has been around since 2011. The report it published is based on marketplace data collected between June 2025 and May 2026 and surveys of buyers. 

Alexander Spivak, a 3D artist who sells his assets on CGTrader, told me that he’s not against AI, but that “as an artist, I haven't yet found a way to seamlessly collaborate with AI.

Because in creativity, the most important thing is the process of creation, which I enjoy.

The result is a completely different story. And I haven't yet been able to integrate AI into my workflow in a way that allows me to continue enjoying [the process.]”

Buyers told CGTrader that the reason they weren’t buying as many AI generated assets their quantity might suggest is simple: they’re not as good as human made 3D models. Buyers said that quality was the number one factor in choosing what they buy, even more than the price. This was especially important to buyers who wanted models for 3D printing, where a bad model could break or not print correctly. Only 4 percent of those buyers said AI “works well.” The survey also found that most people who bought AI generated assets weren’t satisfied with them. 20 percent of buyers found it not good enough, and 7 percent used it only with heavy editing. 5 percent said it worked well. 

CGTrader’s findings come months after the company announced it was partnering with Tencent on an AI powered 3D model creation workflow. CGTrader CEO Dalia Lašaitė told me that the workflow doesn’t just generate assets from scratch, but that designers can use it to refine topology and textures, segment parts, create variations and prepare models for production. 

“AI can accelerate the more mechanical parts of the process, while the designer retains creative direction and control over the final result,” she told me. “For that reason, we think of this work as AI-accelerated rather than simply AI-generated. Our goal isn't to increase the volume of AI-made assets on the marketplace. It's to give designers better tools to work faster and focus more of their time on creative work. Ultimately, buyers will choose the assets that best meet their needs. The distinction that matters most isn't whether an asset is AI- or human-generated, but whether it meets the required quality standard.”

AI generated content is creating a discoverability problem across the internet. On Instagram, porn sites, YouTube, and music streamers, human creators are being drowned out by a flood of AI generated slop. Lašaitė acknowledged that, despite the current discrepancy between how many AI generated models are uploaded to CGTrader and how many people are buying them, the increase in uploads alone could make it more difficult to find human artists.

“AI uploads are currently growing faster than AI purchases, which makes effective discovery and ranking increasingly important,” she said. “Our approach is to prioritize quality and performance signals rather than raw volume, including how an asset performs commercially, how buyers rate it and other indicators of quality.”

Research Gold, a site that advertises services for medical researchers, including drafting peer-review ready manuscripts, systemic reviews and meta-analyses, claims that it’s “100% human-written, never AI,” and lists a number of PhD reviewers and professional methodologists on staff that carry out this meticulous, difficult work.

The problem: The PhD reviewers Research Gold lists on its site are AI-generated and don’t exist. Other methodologists it lists are real, but are not aware their identity is being used by Research Gold. When I tried calling the company, an AI agent that refused to concede it was AI answered and kept trying to sell me Research Gold’s services. Email and chat communication with the company were also AI generated. 

“Protocol, search, screening, extraction, risk of bias, statistics, and a publish-ready manuscript formatted to your target journal or committee. Led by PhD methodologists with peer-reviewed publication records. PRISMA 2020 and Cochrane Handbook methodology. Authorship stays with you,” Research Gold’s site says. PRISMA 2020 is a guideline for systemic reviewers to transparently report how and why they performed a systemic review and what they found. Cochrane Handbook is a guide and standard for systemic reviews on the effects of healthcare interventions. 

Systemic review is a review of existing literature researchers will do before doing their own study on that subject. Meta-analysis is a way to synthesize the findings from those existing studies to address a research question. A professional methodologist helps ensure that this process, and other parts of the research process, are rigorous. 

Research Gold introduces “The Team” that does this work under its About page. They include Founder & Lead Methodologist Dr. Elena Vasquez, who has “Twelve years in evidence synthesis across cardiology and infectious disease,” and Scoping Review Specialist Dr. Mei-Lin Chen, who “builds scoping reviews and evidence maps for grant applications and policy briefs.” Vasquez, Chen, and the other six members of this team don’t exist. Searches for their names don’t return any online footprint that matches the description on the site or a history of publishing papers. Their profile pictures are also clearly AI generated. 

A different section of the site listed another group of methodologists with profile pictures that appeared real. Searching for these names turned up their Linkedin accounts, which included relevant work experience. All of them are or were freelance methodologists or academics. Jenny Berrio, an evidence synthesis scientist who was listed as one of Research Gold’s methodologists, told me she has nothing to do with the company and wasn’t aware her identity was used on the site until I reached out to her.   

“I do not work for Research Gold, and I never agreed to be listed as one of their methodologists. I have no relationship with this company,” Berrio told me. “They are using my name, photo, and bio without my permission. I'm in the process of documenting the site and will be sending them a formal takedown request.” 

All the profile pictures for the real methodologists listed on the site are identical to the profile images these people use in their real Linkedin profiles. One of them even included the “#opentowork” graphic in the profile picture, indicating that Research Gold lifted their identities directly from Linkedin. 

Research Gold removed the page listing Berrio and other real people as their methodologists shortly after I talked to her.

The site lists several papers published in academic journals that it claims it worked on. I reached out to the lead authors of those papers but did not hear back. 

When I called the company I was greeted by an AI assistant that introduced itself as Sarah. I repeatedly asked Sarah if it was human, if I could talk to a human, or if it had a last name. “Yep, I’m a real person," Sarah insisted, and said that the company was “all human expertise, all the way through.” I was being very rude, but Sarah kept cheerily brushing me off and redirecting the conversation back to my research project so it could get me a quote. 

Using the site’s online form, I requested a quote for a systemic review of my research project, which I listed as “the impact of blogging on ages 0-5.” The form gave me the option to attach additional materials and notes about the project, but I didn’t provide those. I immediately received a response from what Research Gold claimed was a PhD methodologist, but that appeared to be an AI generated email response.

“Thanks for sending this over. Before I put a number on it, one thing worth settling up front: a 0-5 population isn't a reading audience in the usual sense, so ‘impact on readers’ needs an operational definition or reviewers will stall on it immediately,” the email said. “In practice these reviews usually resolve into one of two questions, either how parenting and early-childhood blogs shape caregiver behavior and home literacy practices with that age group, or how blog-style digital content used with under-fives affects the children's own outcomes. Which of those two is the study you have in mind? Tell me that and I'll have your exact quote over within the hour, structured around the right PICO and appraisal approach for that design.”

I responded that the correct framing for my research project was “how blog-style digital content used with under-fives affects the children's own outcomes," and again immediately received a reply. 

“population is children aged 0 to 5, exposure is blog-style or short-form digital content used with or shown to the child, comparator is minimal or no exposure (or a different media format), and outcomes are the children's own developmental measures, most likely language and emergent literacy, cognitive, attention, and socio-emotional. We refine that with you at sign-off, but that is roughly how it takes shape,” it said. “For the full review at your flexible timeline the price is $1,900. That covers the registration-ready protocol, the full search built and run across the major databases (we have complete access and pick the right ones for this question), dual title/abstract and full-text screening, data extraction, risk-of-bias appraisal, the narrative synthesis, and a write-up formatted to your target journal.”

The email sent me to a portal where I could pay the $1,900.

Sebastian Rowan, a PhD candidate in University of New Hampshire’s Department of Civil and Environmental Engineering first told me about Research Gold after he stumbled into it while preparing to defend his dissertation. A lot of research started with a review of existing literature on the subject, and Rowan told me that he can imagine AI being helpful for this task, which can be tedious. 

“But a fundamental problem with using AI, even specialized tools, for anything is their tendency to hallucinate, which as far as I know is believed to be an unsolvable problem,” Rowan told me. “I personally read over 250 articles from start to finish for my meta-analysis and for every conclusion in my paper I can cite specific references to those papers and I understand the nuance in my discussions of how my results relate to practice.”

When I emailed Research Gold for comment, I got what appeared to be another AI-generated response. 

“Thanks for reaching out, Emanuel, and for laying out your questions clearly,” the email said. “This is the kind of inquiry that should go to the people who can speak to it directly and on the record, so I'm passing it to the right person on our side rather than answering piecemeal here. You'll hear back from them at this address. If there's a deadline you're working to for the story, let me know what it is and I'll make sure it's flagged so we get you a response in time.”

Research Gold did not send me a comment in time for publication.  

While we haven’t seen evidence that credible researchers are using Research Gold’s services, generative AI has already impacted academic publishing. Scientific journals have to filter through a flood of papers with AI-generated citations, and some AI generated papers are being published by academic journals. In 2024, I talked to a researcher who believed the peer-review process itself might be compromised by AI generated text. 

Microsoft has introduced new limits to how much its engineers can spend on AI tools at work and told employees that maximizing AI use internally is not the company’s goal. 

This makes Microsoft one of the last major companies to rein in its employees’ expensive AI use. Scaling back maximalist AI use, or what some companies have called “tokenmaxxing,” is a trend we’ve covered in recent months as the price for using AI has increased while not always delivering commensurate productivity gains. 

Substack contributors are rejecting the company’s decision to flag AI generated writing on the newsletter publishing platform, with several creators saying the new AI detection features are a “witch hunt.” Some Substack users who use AI to help them write or edit articles say the new policy unfairly discriminates against them, while other users who don’t use AI in their writing at all are worried that Substack will mistakenly flag their writing as being AI generated and tarnish their reputation. 

“I’m not going to apologize for using AI in the creation process. I wrote for 20 years without AI, I could do it again if I wanted to,” Mack Collier, who has a small Substack called Backstage Pass, wrote. “My output would fall, my posts would be less structured, and it would be more obvious that they needed a good editor. Using AI improves the overall quality of my writing. That’s why I use it.”

“These detectors are notoriously, wildly inaccurate,” Alice Lemee, a ghostwriter and digital writing coach, said in a video she made about Substack’s new AI detection features. “All it takes is one false accusation for a writer to have their reputation almost irreversibly tarnished.”

Substack announced the AI detection features on Tuesday in a post from CEO Christ Best, who said that “It’s getting harder to tell what’s real on the internet” and that “when content made by no one takes over parts of the internet that are supposed to be human, it pollutes the commons and makes it hard to discover and hear human voices.”

Best explained that Substack has partnered with the AI detection tool Pangram, which is now built in to Substack and allows any user to scan a piece of writing. Pangram then produces a percentage-based score determining how much of the writing was AI or human generated. (Disclosure: Pangram previously bought an ad on 404 Media). We’ve covered research from Pangram, or research that relies on its AI detector before, showing how AI generated writing is flooding every corner of the internet. 

As we’ve previously reported, AI detectors are not perfect, and Pangram itself is not immune to labeling human content as being AI, and labeling AI as human. Max Spero, the CEO of Pangram, recently told us that the company is constantly working on minimizing errors, and that it estimates its false positive rate at roughly one in 10,000.  

“We have now built the mirror image. Pangram is a real machine, trained on vast amounts of text to make a probabilistic guess, and we have pointed it at your writing to work out whether a person is hidden inside,” Sam Illingworth, a professor and the author of Slow AI, said on his Substack in a post titled “Substack’s AI Detector and the Return of the Witch Hunt.” “To decide if there is a human on the other end, Substack asks a machine.”

Substack also added a way for users to add a “How I make this” statement, where they can explain their writing process and disclose if they use AI. 

“We’re not against people using AI to assist their work, and we think people should be free to choose which tools they use to express themselves,” Best said. “Some on the platform are human-writing maximalists, others are publicly exploring the frontier of using AI tools while sweating the details to make work they stand behind. We use AI all the time at Substack to write software, do research, and build product features like clipping, translations, and more. But people should know what they’re getting.”

“I think the best way for Substack to address this is to invite further dialogue,” Illingworth told me. “I actually really like the new feature that lets authors tell their readers how they construct their newsletters using AI, and this is the kind of dialogue they should encourage rather than ascribing a score that we know to be incorrect. My worry is that not only do these AI detectors generate false flags for non-native English speakers and neurodiverse writers, but they also remove any opportunity for dialogue because they start from a position of suspicion. Whereas what we need to be doing is developing opportunities for trust between readers and authors.”

"The tools we've introduced today are intended to increase transparency and give readers more context, not to prohibit or penalize AI-assisted writing. They do not impact discovery on the platform,” a Substack spokesperson told me. “We're also encouraging Substack publishers to add a ‘How I make this’ statement, where they can explain their process directly, including how they use (or don't use) AI, and set expectations for readers. Creators can disable detection on their posts pre- and post-publication, as well as report and remove scans on their own work that they believe are mistaken. Find more information on how the features work here.”

Substack’s attempt to detect and label AI generated content was also celebrated by many readers who are exhausted by AI slop and other internet platforms that can’t or don’t want to label it. I just published story about how there is so much unlabeled AI slop on Spotify that people are now creating their own sites to detect and label it without the company’s help.  

But the backlash from some Substack users shows that companies can’t just flip a switch if they want to ban or give users the ability to filter out AI content from their feeds. As the flood of AI writing and images we see online and the real world every day makes clear, generating slop is easy. What we’re going to do about it is still an unsolved problem.

Slime Dot, a young R&B artist from Las Vegas, currently has over 100,000 monthly listeners on Spotify. In an interview with GQ in May, after sharing an image of herself posing with Drake on Instagram, Slime Dot directly denied accusations that she was an “AI artist,” saying “The truth doesn’t matter these days. People are always gonna try to explain something they don’t fully understand and believe what they want.” GQ originally credulously published the interview and said it was “debunking the AI rumors.” Eventually, GQ added a note to the end of the article conceding Slime Dot was an AI avatar, but said the “talent behind the music remains undeniable.”

This confusion could have been avoided if Spotify had done what many of its users and critics have been begging it to do since AI music generators like Suno and Udio made it trivial to flood the internet with AI music. Spotify could flag Slime Dot and other AI-generated music on its platform as such. Other platforms, like YouTube, already do this. 

Since Spotify doesn’t do that, users can visit SoullessMusic.com, a database for “AI artists hiding on Spotify. No bands, no studios, no soul, just machines and melody.” There, Slime Dot is listed as “almost certainly AI” based on an analysis of three of the tracks and the fact that Slime Dot released such a large number of tracks in a short period of time. Alternatively, users could go to SlopTracker, where they can upload files or copy/paste links to Spotify tracks to run them through a tool which will tell them, with varying degrees of confidence, whether a song was AI-generated or not. SlopTracker found that Slime Dot’s track “Fully” was 95 percent likely to be AI generated by Suno. 

“I didn't know what's AI and what's not,” Graeme Fulton, who created SoullessMusic, told me. “And then I'm on Instagram Reels and I see loads of artists unhappy saying some artist has popped up with their song and just copied it. And there's loads of different cases where they [AI artists] would copy an artist's look and appearance […] sort of just stealing from artists.”

Last year, I wrote a story about an exceptionally bad example of what Fulton is talking about. A scammer used Spotify to publish AI-generated music under the name of a real, dead musician, seemingly in an attempt to syphon money from his popularity via streams. After I published that story I heard from several (living) artists who said the same thing happened to them. A similar case involving a Danish jazz musician was covered by Danish publications in April. 

Another reader recently told me about a popular Spotify artist that was publishing hours of Iranian jazz every week. The channel, Qajar Jazz, which popped up in December of last year, has 29,579 monthly listeners, gained more popularity after the start of the U.S. war with Iran and the channel making vague allusions to resilience and preserving culture in video descriptions and comments on YouTube. Unlike Spotify, Qajar Jazz videos on YouTube are labeled as being AI-generated in the fine print, though it’s unclear if the channel tagged itself as such or if YouTube did. Both SoullessMusic and SlopTracker detect it as being AI generated. 

Qajar Jazz did not respond to a request for comment. 

Many of the YouTube comments on Qajar Jazz videos indicate that people think the music is made by real people. In addition to creating a database of AI generated music on Spotify, SoullessMusic also collects social media posts from people who say they were similarly tricked by AI generated music. Some posts are from musicians who lament the fact that they are now competing with algorithms that can produce thousands of tracks with no effort, and others are from artists whose music was directly ripped off by AI artists. 

View this post on Instagram

A post shared by 𝓘𝓼𝓪𝓫𝓮𝓵 (@isabel_music_official)

View this post on Instagram

A post shared by UMAW (@weareumaw)

Fulton said he started looking around on GitHub for open source AI music detection tools, and ended up building a new, open source detector that relies on open source models called SONICS, the lofcz vocoder fakeprint detector, and other open data sets. SoullessMusic also  scans a track’s metadata for any tags that might tie it back to a specific AI music generator, and Fulton says he also has a script that scans Wikipedia for existing entries on an artist. Users can submit tracks from Spotify to the site they suspect are AI generated, and Fulton reviews them manually before adding them to the database. Despite all of this, he admits the system is not perfect. 

Both Soulless Music and SlopTracker use AI and other automated tools to detect AI music. As we’ve written previously, using AI detectors to detect AI content is an inherently flawed process that can lead to false positives. When I asked him how he handles cases of false positives, Fulton said “it's not ideal when that happens, but it's going to happen because it's still quite tricky to check what's AI and what’s not.”

Spotify’s lack of transparency and unwillingness to label AI-generated music makes it impossible to track how much money its making on the platform. Last year, the company announced that it would it would AI disclosures on AI generated music, but I haven’t seen it on any of the dozens of tracks I reviewed for this story. Udio and Sudo both include inaudible digital watermarking in their generation, which in theory could make it easy for Spotify to tag these tracks. 

"We're employing a layered approach that combines enforcement, artist controls and greater transparency," a Spotify spokesperson told me after this article was first published. "Over the past year, we've introduced policies targeting harmful AI-related behaviors like spam, impersonation and deceptive content, alongside new tools that give artists more control and listeners more context. These include Verified by Spotify, which helps listeners identify authentic artist profiles; AI Credits, where artists disclose when and how AI was used in creating their music, with tens of thousands of AI credit disclosures now being submitted each day; and Artist Profile Protection, which gives artists more control over what appears on their profiles. We are continuing to build on these efforts.”

Spotify also pointed me to several tracks it had flagged as AI on the mobile app, like Better Times by Mike Mana.

Soulless music attempts to quantify that number based on publicly available streaming data. The small number of AI artists tracked on SoullessMusic generate an estimated $5.7 million a year, with the most popular AI artist, mikeeysmind, generating $1.5 million annually alone. In April, Deezer, a Spotify competitor that attempts to tag all AI-generated music on its platform, said 44 percent of all new music uploaded to its platform is now AI generated. SlopTracker, which attempts to track AI generated music on Spotify that is featured on official Spotify-curated playlists, that those artists are “draining” $0.1188 per second from real artists who could be making the music instead.