Vue normale

  • ✇404 Media
  • Anthropic’s Text Watermarking Proves AI Companies Do Not Care at All About Writing
    Earlier this month, Anthropic announced that future versions of Claude will generate text that includes watermarks showing it was AI-generated. At the time, Anthropic did not explain how this would work, leaving us to speculate on the podcast: Would it somehow encode this into the text? Include invisible characters? Do something with the metadata? We now know, thanks to a blog post over the weekend, that Anthropic will do this by changing how its AI writes altogether. “Nothing is added to the te
     

Anthropic’s Text Watermarking Proves AI Companies Do Not Care at All About Writing

18 août 2026 à 11:54
Anthropic’s Text Watermarking Proves AI Companies Do Not Care at All About Writing

Earlier this month, Anthropic announced that future versions of Claude will generate text that includes watermarks showing it was AI-generated. At the time, Anthropic did not explain how this would work, leaving us to speculate on the podcast: Would it somehow encode this into the text? Include invisible characters? Do something with the metadata? We now know, thanks to a blog post over the weekend, that Anthropic will do this by changing how its AI writes altogether. 

“Nothing is added to the text and there are no hidden characters,” Anthropic wrote in that company blog post. “The difference between watermarked and un-watermarked text will not be distinguishable to readers.” The way it will work, the post explained, is that Anthropic will subtly alter the word choices in AI-generated text in a way that is only known to Anthropic and its algorithms. Anthropic will know the watermarking algorithm, which will change “the source of the randomness used to pick among words” and thus can write a tool to detect whether something has been AI-generated.

This research and approach is interesting in a data science kind of way, but Anthropic’s layperson explanation for how this will work shows how little the company thinks about the craft of writing or the subtle differences between words a human author might want to use to convey their thoughts. 

Anthropic asks us to consider the difference between two sentences: “Take the sentence ‘The weather today was cold and…’. The next word is very unlikely to be ‘sugary.’ But it is quite likely to be ‘overcast’ or ‘grey.’ Under most circumstances, it doesn’t matter much to the reader which of these latter two words the model ultimately chooses—the meaning of the sentence is largely the same either way. In cases like this, the choice is settled by a random number,” Anthropic writes. “Watermarking uses low-stakes choices like these—which occur many times over a piece of generated text—to leave a pattern in Claude’s responses. That pattern is undetectable to the reader, but is detectable to anyone who has a key that encodes it. When watermarking is used, choices are still made at random, but the source of the randomness is different.”

Anyone who has written anything would, I hope, understand that the difference between the sentences “The weather today was cold and grey” and “The weather today was cold and overcast” are sometimes “low stakes,” as Anthropic describes, but not always. “Grey,” and “overcast” are different words, and there are any number of reasons why a human author might pick one over the other in a given context. In this example, however, Anthropic’s algorithm sees these words as totally interchangeable and thus its watermarking algorithm has decided that it can “nudge” the word choice one way or the other for the purposes of watermarking. 

Anthropic continues: “Instead of using an arbitrary random number generator to pick the next word, watermarking uses the key and a few words that come before to settle what word the model should pick. That is, the words that Claude picks are still random, but now, one can check the sequence of words and see if it’s consistent with the choices Claude would make if it was using the key. If it is, one can assign a probability that the text was generated by Claude.”

Anthropic claims “Watermarking does not impact the quality of Claude’s output. To a reader, a watermarked response is indistinguishable from an unwatermarked one,” and that “in internal testing, we’ve seen no impact of watermarking on the content, level of creativity, or readability of Claude’s text.”

People are quite mad about Anthropic’s watermarking system, and understandably so. Synonyms are sometimes interchangeable, but not always, as is pointed out in this excellent essay by John Gruber of Daring Fireball, and by journalism academic Jeff Jarvis, in which he claims Anthropic “devalues writing.” In making this choice, “Anthropic declares words fungible, language random, choice meaningless,” Jarvis writes. 

When I sat down to write this post, I was mad because it seems like Anthropic is  putting its thumb on the scale, messing with the outputs of its machine and saying that the resulting text is qualitatively just the same as the other AI text it was probably going to output. But as I began writing this, I realized that my problem is not necessarily with text watermarking but with AI-generated text altogether. It does not matter to me, necessarily, whether the output of Claude’s garbage AI text is one way or is a slightly different way. But it does matter to me that AI data scientists at huge tech companies think that word choice doesn’t matter, or that it is possible to statistically use synonyms wherever without fucking with the meaning of a sentence.  

Throughout the blog post, Anthropic describes the act of writing as being akin to a probabilistic game of chance. In Anthropic’s own words, its writing is sometimes the result of an “arbitrary random number generator,” and “random” whenever its systems encounter a situation where its tool believes, based on pattern recognition, that the choice between several possible next words isn’t all that important. That may be true for LLM garbage, but is not true for the human experience of writing, which is why human writing almost always feels different than AI writing.

This watermarking approach, and Anthropic’s blog post about it, highlights something that should already be clear about a company that famously scanned and destroyed huge numbers of printed books and has trained its LLMs on stolen content: Anthropic does not care about the craft or effort of writing, and sees words as fungible and unimportant. Anthropic says it is making this change as part of the European Union’s new AI regulations, which are well-intentioned but problematic. While it can definitely be useful to have additional ways of detecting AI-generated content, the carelessness with which Anthropic has announced this decision highlights the broader problem with using LLMs to write: They are, as Anthropic notes, probabilistic tools that do not “write” in the way that humans do, rather, they mimic their training data which is, by definition, things that have already happened and been ingested. 

Contrast this with how Anthropic sees code, something where it says an “exact output is required.” In writing, meanwhile, Anthropic suggests different words are often “equally good.” Over and over again, Anthropic and the researchers who work on this type of watermarking claim that text can be “nudged” in this way without being noticeable to humans or without impacting “quality.” 

But it is worth noting that the people judging the “quality” of the AI-generated outputs are either data scientists or people asking AI tools to do their writing for them, not, say, people who care about reading or writing. The scientific paper that Anthropic cites was done by Google researchers on a Google watermarking tool called “SynthID,” which Anthropic’s watermarking is based on. 

In the SynthID study, quality was assessed by randomly putting watermarking on some Gemini outputs, then asking Gemini users to either thumbs-up or thumbs-down the response: “A random fraction of queries were routed to a watermarked model and an equivalent number to the unwatermarked counterpart. The Gemini user interface allows users to provide feedback on model responses via a thumbs-up (good response) and a thumbs-down (bad response). We analysed approximately 20 million watermarked and unwatermarked responses and computed the thumbs-up and thumbs-down rates (both as a fraction of the total number of thumbs-up and thumbs-down feedback received). We found that the thumbs-up rate for the two models differed by 0.01%.”

I hope it is clear to anyone who has clicked on this article that asking someone who asked a chatbot something to thumbs up or thumbs down a response is not a very good way of assessing the “quality” of “writing.” The other human assessment that Google did was to ask people to assess side-by-side watermarked and unwatermarked text for quality. Here are examples given in an appendix of the study; apparently people did not really have a preference one way or the other:

Anthropic’s Text Watermarking Proves AI Companies Do Not Care at All About Writing

One could argue that these passages are two different ways of explaining something, yes. But they are definitively not the “same,” and it is unclear to any reader why one version is one way and the other version is another way. Why did the LLM write “respiratory failure” in one example and “cessation of breathing” in the other? The answer for both is an “arbitrary random number generator” and proprietary black box algorithmic weighting systems controlled by the AI company. In the watermarked version there’s been an additional “nudging” or messing with the machine that’s already just a pattern matcher. 

The point is, there is no conscious thought or decision-making process happening here, so perhaps watermarked AI text is not all that much more offensive than regular AI text. But to see it laid out in such stark terms by the companies building these machines shows how little they actually care about writing. If you asked me, on the other hand, why I used one word instead of another, I might not be able to tell you exactly why, but I could probably explain to you what I was going for, the style of writing I do, my intended audience, my mood that day, whether my heart was racing or not, where I was, what I was doing, what I did earlier that morning and what I did later that day. Maybe it was a word my third grade teacher used all the time or which I read in an article last week or is an inside joke with my friends or which I have recently become obsessed with or tend to overuse. Why I wrote what I wrote or why I did anything at all is the result of my some mix of human experiences dating back to when I first acquired language as a baby and continuing on to this very moment that I may or may not be able to explain, but which result in a certain style of writing that is mine.

This is the case even when I’m working fast or carelessly dashing off text messages, when the thoughts just kind of flow from my brain to my fingers to my keyboard where I don’t know if what I’m saying is making sense at all but is probably legible because it’s coming from a human brain and not a random number generator. 

This is why short passages of AI-generated text feel soulless and generic, as we have written about repeatedly. And there are many AI tools that use AI to make AI writing seem less generic (yo dawg, we heard you like AI so we put AI in your AI) by using synonyms that are supposed to make a passage sound more human — or less plagiarized — by picking words that are less commonly used. The text outputted by these tools, which are called “spinners” or “humanizers” are often just as uncanny and weird as AI writing itself. Or, when applied to things where, to use Anthropic’s own language, “an exact output is required” such as quotes in a news article, the output is often factually inaccurate, libelous, or just plain garbage.  

  • ✇404 Media
  • X's Algorithm Feeds Off Ragebait and Impacts Democrats More, Study Finds
    X’s algorithm learns what you hate and shows you more of it, according to a new study just published in the Proceedings of the National Academy of Sciences (PNAS). The paper, titled Value misalignment of X’s feed algorithm is a reflection of value tensions in engagement, found that the site’s algorithm prioritized engagement above all else when it generated a user’s For You Page. It also showed that X serves more ragebait to people who say they are Democrats, although the exact reason for tha
     

X's Algorithm Feeds Off Ragebait and Impacts Democrats More, Study Finds

18 août 2026 à 09:27
X's Algorithm Feeds Off Ragebait and Impacts Democrats More, Study Finds

X’s algorithm learns what you hate and shows you more of it, according to a new study just published in the Proceedings of the National Academy of Sciences (PNAS). The paper, titled Value misalignment of X’s feed algorithm is a reflection of value tensions in engagement, found that the site’s algorithm prioritized engagement above all else when it generated a user’s For You Page. It also showed that X serves more ragebait to people who say they are Democrats, although the exact reason for that is unclear.

“In 2026 that’s maybe not the most surprising headline ever,” Ziv Epstein, a postdoctoral researcher at Stanford University, and co-author of the paper, told 404 Media. “So we actually dug in a little deeper to figure out why this is actually happening, and it turns out that X's feed algorithm, like a lot of these social media algorithms, is optimized for engagement [but] it turns out that not all types of engagement are considered equally.”

The study’s goal was to understand how a user’s self-professed values system might shape what they see on X. “We recruited a nationally representative sample of N = 715 Americans who are active users of X in September and October 2024, quota matched on ethnicity, gender and partisanship, to install a browser extension to collect their [For You Page] and Following feeds,” the study said.

Epstein said the study was observational and meant to get people asking questions about what they want to see on social media, how their feed is designed, and by whom. “There are these social media algorithms that have enormous amounts of power in our lives, they shape the information that we consume, and we have very little transparency into how they operate and what their implications are,” he said. “And so, we were very interested in trying to understand the particular effects of this particular algorithm, and so I think that has kind of important implications for civil society and just fighting some of the technofeudalistic tendencies of platforms to control these algorithms.”

For the study, researchers collected a “values inventory” of the volunteers using a research tool called the Schwartz Theory of Basic Values. The values inventory in the study is presented as a wheel with 19 points that corresponded to features like “tolerance,” “dominance,” “hedonism,” and “openness to change.” Users also reported their political alignments.

Then researchers watched how users engaged with posts on X and how those posts reflected their self-reported values. “We observe that the inventory of posts from followed accounts reflects users’ self-stated values — but that there is an overall negative correlation (misalignment) between users’ explicit values and the values in content that the algorithm is more likely to amplify,” the study said.

When a user on X sees a post that makes them mad — like a press release from a politician from a political party they don’t like — sometimes they’ll fight about the post in the replies. It doesn’t matter who you follow or what your stated values are, X reads replying as engagement and will send more of the infuriating posts the user’s way.

“When we look at commenting, the act of replying to posts, that's where we actually see some kind of meaningful misalignment between people’s values and the values of the content they’re replying to,” Epstein explained.

Most of the participants liked and reposted content on X and got served more of the same sort of content. Replying was rare, just 6.8% of the interactions according to the study, but had an outsized impact on the algorithm. “Replying is only a fraction of engagement, but there does seem to be some evidence that these algorithms are prioritizing and learning more from this kind of rarer form of engagement,” Epstein said. “So it’s this feedback loop of outrage baiting. The algorithm learns that you get outraged and then continues to serve more content in that direction and that seems to be particularly true of the Democratic users of our study.”

Though this happened across the political spectrum, the study found that ragebaiting occurred more for users that identified themselves as Democrats. “Democrat users confront the abundant value-misaligned content by replying to it, which the algorithm in turn preferentially learns from and continues to feed them,” the study said. “This highlights a core tension with how engagement-maximizing algorithms operate on social media: frictions between users’ stated preferences and their behaviors of reactive confrontation are exploited by engagement-maximizing algorithms to create runaway feedback loops of increasing value misalignment.”

Epstein said he’d need to do more research to find out why X seems to serve ragebait to Democrats more often than Republicans. It could be that there’s more rightwing content on X overall or it could be that Democrats tend to engage with posts they disagree with more often. “There might be some kind of differential effects on information diets there, or it might be something more psychological about how different you know partisan identities are triggering different kinds of actions and reactions, but ultimately I don't want to speculate too much,” he said.

I asked Epstein if he worried that prioritizing “values” in a social media algorithm might lead to more siloed user bases and more echo chambers. “I do think that if we go kind of down this path of thinking through and imagining value line social media feeds, we do have to be very aware of the potentials of value echo chambers, right?” he said. “Where people just, you know, they have the certain values that they have, and all the content they see is just aligned with those values. I think that is a very scary and dark reality.”

But he also said that values are intentional and that thinking about the kind of stuff you want to see on a social media site before you pull up your feed is a positive. “You're pumping the brakes, you're taking a breath, and you're thinking about what you actually really care about. In this world — and I don't have the data for this — but I would speculate that a lot of people actually do care about seeing a diverse set of content and engaging meaningfully across these lines, and you know maybe that isn't a particular value on my 19-dimensional wheel, but this is just a starting point of thinking about our intentions and thinking very deliberately about the kind of information we want to be exposed to, versus the the knee-jerk reaction that we're kind of learning in this very kind of short, shallow attention span, emotion and negative affect-driven model of these very myopic forms of engagement.”

X did not return 404 Media’s request for comment, but Nikita Bier — X’s former head of product — confirmed that the site’s algorithm had at one time been set to favor replies. “This is no longer true,” Bier said in a post on X. “The largest contributor of seeing ragebait was the reply predictor and we were aware that angry replies were causing people to see more of that content. So last month, we gave the reply predictor a 15x boost if it’s a friend’s post — and it reduced ragebait by [an] order of magnitude.”

Update 8/18/26 at 1:30PM: This story has been updated to include a comment from X’s former head of product

  • ✇404 Media
  • The National Park Service Is Using Flock. Rangers Are Pissed
    The National Park Service (NPS), the agency tasked with managing the U.S. national parks and monuments, has purchased Flock cameras and had them installed at parks including in Yosemite.Current and former NPS rangers are not happy about it.“Flock protects property, not people or parks. People should be free to recreate in ways that are respectful to the ecosystem — but the surveillance is a disrespectful invasion of privacy,” a current NPS ranger told 404 Media. 404 Media granted the ranger a
     

The National Park Service Is Using Flock. Rangers Are Pissed

18 août 2026 à 09:21
The National Park Service Is Using Flock. Rangers Are Pissed

The National Park Service (NPS), the agency tasked with managing the U.S. national parks and monuments, has purchased Flock cameras and had them installed at parks including in Yosemite.

Current and former NPS rangers are not happy about it.

“Flock protects property, not people or parks. People should be free to recreate in ways that are respectful to the ecosystem — but the surveillance is a disrespectful invasion of privacy,” a current NPS ranger told 404 Media. 404 Media granted the ranger and others anonymity because they weren’t permitted to speak to the press.

💡
Do you know anything else about Flock? I would love to hear from you. Using a non-work device, you can message me securely on Signal at joseph.404 or send me an email at joseph@404media.co.

Another current NPS ranger said, “I just think that every American should be free to visit our national parks without their location being traced and tracked by the federal government. Wild places are at the core of American freedom and nobody should be afraid to visit them.”

A former NPS ranger added, “I think if the public knew that their public lands were being surveilled, they would push back.”

Flock cameras constantly scan the license plate, color, model, make, and other identifying features of every vehicle that drives past, creating a timestamped record of where a vehicle was at a particular time, and by extension, a person. This data is then available to Flock’s law enforcement customers and is usually queried by police without a warrant. Many of Flock’s cameras are part of a national network that other law enforcement officials can then query. For example, a cop in Texas might query the network to look for a specific vehicle and search cameras across the country. This, among increased reporting of cops abusing their access to Flock to stalk people with no legitimate investigative purpose, and local cops performing lookups for ICE, has contributed to a nationwide conversation around Flock and whether communities want the company’s cameras in their neighborhoods.

Flock cameras are installed inside Yosemite, WBTW News 13 reported earlier this month. The outlet referenced data from DeFlock, an open source map of Flock and other automatic license plate reader (ALPR) locations. 

There is also a Flock camera on a road towards the Golden Gate National Recreation Area, according to the data.

NPS also plans to install cameras from Verkada, a controversial company with a history of abusing its own products and whose products sometimes have facial detection capabilities, according to an internal NPS document viewed by 404 Media.

The NPS told 404 Media in a statement: “Yosemite National Park uses a traffic-monitoring system to measure vehicle counts, travel times and entrance-station wait times. The system helps the park better understand traffic conditions and provide visitors with information to help them make informed decisions about when to visit and where parking is available. The cameras are not connected to law enforcement or DMV databases.”

One of the current rangers said, “Last year Yosemite changed the rules about hanging flags on El Capitan and tried to backdate the rule change to make it look like the people involved had committed a crime. What happens when that sort of malfeasance is combined with the ability to track any park visitor's location, anywhere in the country? If that happens, we are no longer free in this country.”

They added, “I fear for the day when a visitor's National Park experience is interrupted by being pulled over and held at gunpoint because a license plate reader misread their plate.”

❌