I wonder how many people will be identified as AI because they used AI so much they started constructing sentences like AI.
This particular watermarking would be effectively impossible for a person to end up replicating.
As a neurodivergent (possibly AuDHD), I wonder and worry if this wouldn’t end up increasing false positives when it comes to neurodivergent way of speaking.
It won’t, unless you normally talk almost exactly like the model in question and then alter your word choice distribution according to the specific secret key entropy.
I’m not convinced that they even know 100% how Anthropic is doing it. I can think of an easier way that doesn’t corrupt the text: just find a bunch of tokens where there is a good spread of token possibilities, and the more often the most likely one is chosen, the more likely it’s AI.
That being said, it doesn’t seem much different from what any of us do to identify AI text — it has lots of tells anyway.
That’s how AI testers work and its why they don’t. Most forms of formal writing are predictable by design. If the AI can predict predictable formulaic writing, it doesn’t mean its AI, its probably just any form of professional writing other than fiction.
Famous public domain works will always be considered AI by those tests, because of course your LLM knows the american national constitution. It was in the training data, so it can predict it with 100% accuracy, therefore your test wrongly calls it AI.
Testing for AI writing that way does not work.
The difference between what you describe and what I describe, is that a 100% match isn’t a hit. Nor is a 90/7/2/1. You need something with meaningful variability. Even within formal papers there are places where word choice is arbitrary as the article explains.
Of course, you’re lacking the context of the full prompt and just feeding in the raw text. Again it gets way more reliable the more text you have.
But it’s moot because the more text you have the more tells will sneak in and you probably don’t even need an AI checker. Those phrases that AI loves but humans use comparatively rarely. It’s not a tell — it’s the whole game!
do to identify AI text — it has lots of tells anyway
I see what you did there.
Although this looks like a clever approach, a kind of stochastic key, I do not see how this guarantees to distinguish text written by big babble machines versus humans. Humans also have a certain pattern of writing, a given distribution of how some words are more likely to appear than others. How can one tell them really apart?
As an indicator, yeah, might be usable. But I wouldn’t read too much into it before seeing results of a study that runs actual tests.
It’s not about the variation of the words, it’s about the variation of the words from the model baseline.
Like if your word choice was almost the exact same as Claude’s normally, maybe you just talked to them a lot and picked up their phrases like it’s not nothing.
But if you managed to be almost exactly like Claude and yet varied the possible words exactly according to a hidden entropy key, they’d know it was actually Claude with the SymthID-Text watermarking applied, as no human would end up falling into that statistical bucket.
Yeah, still, I wouldn’t claim “as no human would end up falling into that”, given that it may not be that unlikely to find at least one human who displays similar writing the more humans you involve.
Until a formal analysis is presented and an experimental study is published, which covers the most important influencing factors, the reliability of this concept is limited.
Yeah, still, I wouldn’t claim “as no human would end up falling into that”, given that it may not be that unlikely to find at least one human who displays similar writing the more humans you involve.
No, it is actually statistically impossible for a human to replicate this on sufficiently long runs of text.
This is not about replicating writing like a model. This is basically about guessing which words to pick from the list of suitable words based on a rule that you don’t know (because the key is secret).
To reduce this to the simplest possible example, imagine you are writing a “text” from just two letters: “a” and “b”. Let’s say for convenience that the text is supposed to be random. So the text would look something like “ababaaabbababbbabababaabbbabaababbaaabbabaabbaaaaabaaabbbaabaabababbabbbbbbbbabbabaabbbbbbbaabbabaab”
(generated with
'''.join(random.choice(['a', 'b']) for i in range(0, 50)))The watermarking works as follows: the model owner holds a key, and then uses that key to influence the random choices between “a” and “b” somehow, in a context-dependent way. The actual algorithm is quite complicated, but for simplicity let’s just say we have a secret pattern which biases the random choice towards it. In order to see the exaggerated results, let’s say the secret key is “aaaabbbb” (of course this is a bad secret key, once again just an example), and that the bias is strong (let’s say 80%). So this would mean that the first four letters in our text are more likely to be “a”, the next four letters are more likely to be “b”, then the next four letters are more likely to be “a”, and so on.
Then the text would look something like “aaaabaabaabaabbbabaaaabbaaaababbbaaaabbbabaabbbbaaabaabbaaaaabbbabababbaaaaabbbbaaabbbbbaaaaababaaba”.
(generated with
''.join(random.choice(['a', 'b'] + ([key[i % len(key)]] * 3)) for i in range(0, 100)))You can see visually that the secret key has affected the text. Of course in this example even if you didn’t know the secret key you could probably figure it out, in reality the algorithm is way more complicated than that, relying on cryptography, so you wouldn’t be able to know the secret key or see that the string has been biased at all.
If the text is long enough, and you know the secret key, you can guarantee that the text was generated with it. In our examples, the letters in the text match our key 77% of the time. The probability of an actual random algorithm generating a text like that is already very low, despite the base entropy being only 100 bits. If my math is correct, for our example the p-value is 2.7 * 10⁻⁸, or about 0.00000027%. I would bet a hungy that the text was generated by our watermarking algorithm, with odds like these!
Of course we did exaggerate the bias and our base algorithm was random. In reality the bias is smaller, the algorithm for determining the likelihoods of possible next tokens is very complicated (it’s the LLM itself), and the algorithm for determining which token to bias is also way more complicated (involving cryptography and real secret keys). That said, hopefully it should help you understand why, for sufficiently long texts, this fingerprinting is just not possible to be replicated by humans.
I do not have the time to work through every part of the example, but imo the main claim is still overstated. Showing that a result would be extremely unlikely under a particular null model is not the same as showing that it is “statistically impossible” for a human to produce. It also does not give a guarantee how the text was written. A tiny p-value is still a probability under assumptions and not a proof of provenance.
Furthermore, a human does not even need to know the secret key. By pure chance a human written text can display an unusually high alignment with the detector’s secret partitioning.
The published watermark work, which is also cited by the article, appears to be much more careful about this (based on a quick skim). It reports false positive/negative rates, thresholds, length requirements, and more. Those can be very strong results, provided the assumed conditions apply. They do not turn a detector into an infallible test. Moreover longer text only helps if the assumptions and watermark signal actually remain intact, which can fail in general.
In such controlled settings, sure, I do not have much issues there. But the claims of “a human cannot replicate this” or “we can guarantee the text was generated with the watermark” are much stronger than the statistics, and especially the cited literature, actually appear to support.
I thought the article explained that pretty reasonably on a scale of probability and weight. The longer the text, the more reliable the scoring.
But it does not show a sufficient formal proof and no experimental validation. Many important questions to evaluate the concept are left unanswered, which limits the interpretability and condenses it to “just trust me, bro, it’s a good idea, because I say so”.
I’m not sure we’ve read the same article. There are literally interactive demonstrations within the page to demonstrate how the concept works.
Interactive demonstrations are not the same as a formal proof or experimental validation. So we shouldn’t attribute more to this technique than the available evidence can really support.
I found some time to quickly skim through the sources they have listed. And from that it became pretty clear that this is not realiable in detecting LLM generated versus human output in general. Under very tight assumptions specific error rates were reported that appeared rather low. However, these assumptions do not hold in general, even with more text if no relevant signal remains. There is currently no scientifically validated general purpose way of reliably detection.
More importantly in the context of Claude, the production watermarking scheme is undisclosed. Therefore, the cited experiments on known watermarking schemes can neither establish how reliably text generated by Claude can be detected, nor how reliably the technique described in the article removes the actual watermark.
It can be treated as an indicator at best, but not as validated proof.
Gish Gallop… if you’re going to start questioning whether the technique clearly demonstrated has validity, then you need to specifically state what your objections are, as opposed to vague statements. For emphasis, the demonstration isn’t on AI detecting, but rather AI watermarking. You wouldn’t use this tool to check if text was written by AI, but rather if the text was written by one singular LLM vs literally everything else.
Interesting stuff. My own far less scientific reading of the article itself seems to fittingly suggest it too is largely if not entirely AI generated, which I guess would make sense.
95% AI text agree. It reeks.
- Only the key-holder can check. Your teacher, editor, or favourite “AI detector” website cannot run this test; a genuine check needs the provider’s secret key, or a checking service the provider runs. Google runs an early-access detector portal for SynthID; Anthropic says detection tooling is forthcoming.
I am not so sure about that. The amounts of words is finite and with enough text, you will see that certain words are used more often, especially in certain combinations. I believe people will brute force this and then create a way to destroy the watermark again.
Yeah, they can’t easily rotate keys, because the text can’t tell you which key was used.
They could switch to a new key e.g. every month and then just check every previous key during detection. But that would slowly increase the likelihood of false positives, so no idea if that’s a good idea either.
with enough text, you will see that certain words are used more often
Which is also a thing humans do.
Absolutely! We all basically do fingerprinting. We’re just not really conscious about the key we are using. But with a bit of statistics, you could identify people.
But how reliable? With which guarantees? What are the prerequisites for this to work at all? Telling people from each other apart is one thing, the other is telling them reliably apart from a machine generated text.
Very interesting read.
I wonder if they did this to appease the EU or just to have a way to prove in court that a specific competitor distilled their model using claude
Sell access to Turnitin and the likes for a small fortune. They are all but required to pay whatever the price is.
Great article. I wonder if the same markers can be used to detect AI generated code (if you suppressed comments).
As, code requires a much more rigid syntax, compared to free flowing docs.
Definitely. It might require significantly more input to gain the same certainty, but all it’s doing is reweighting the possible next tokens before choosing, and code output is still just token output. The rigid syntaxes probably means that the next token probabilities are much more sharply divided (maybe a random sentence the top 1 choice is just 40%, top 3 are 80%, but for a line of code, the top 1 choice might be 90% probability and top 3 hit 99%)
You still have choices.
Variable names are pretty free form.
A switch statement or if/else might be a choice that achieves the same thing. Waffling between them would be highly suspect, since a person isn’t going to be so wishy washy. So you could use structural choices too.
AI code tends to look more obviously AI than prose anyway.
I mean 50% of the time you just use the name intellij or vs code suggests and the other 48% are slight variances
Unless I missed something, that seems pretty brittle. Wouldn’t any minor editing break it because the watermark is derived from the preceding text? Eg. Find + replace “it is” to “it’s”
The section “4. What editing does to the mark” talks about that. Probably best to look at that illustration again, but basically those edits would interrupt consecutive runs of detectable text, but if a run is long enough, it can still be detected with statistical significance.
So, it doesn’t have to check the ‘color’ of the words from start to end uninterrupted, but rather can also detect color sequences in the middle of the text.
Yes you missed at least one whole section including a graphic that shows the breakdown of the watermark with typo fixing, light paraphrasing, moderate and heavy editing.
What if you copy it into word or notepad and then open another program and post as text only?
The watermark is in the word choice, it’s not in hidden characters or Unicode characters.
What is the statistical likelihood of an individual possessing access to a thesaurus inadvertently precipitating the activation of the artificial-intelligence revelation watermark?
It’s not like it’s a secret invisible Unicode character flag or something, it’s just a series of word choices. To me, this seems extremely unreliable. It’s only one step removed from those “unreliable A.I. detection tools” that scan for common word choices A.I. uses. You’ve just biased your own A.I. to use specific word choices and then told your own A.I. to check for those words. This doesn’t seem special or interesting to me. There’s still going to be false positives, but now with even more false confidence.
Lots of people say my writing looks like AI because i use em dashes (learned about them in 8th grade) and semicolons (6th grade). My only saving grace is my extremely long sentences; AIs tend to have shorter, more poignant sentences with obvious-ish tells once you know what to look for.
Maybe also helps that i changed keyboards recently so i type weird words like “knkw” instead of “know” and dont always double check my spelling
How does this survive variation in temperature? Also I wonder if it’s possible to fine tune this behavior out.
Chortle my balls, #nannythropic
And the lower used one might actually be for the “posh” kind.
At least copy the work. Even when you had someone else do your homework, you still had to copy what they put down so you don’t get caught.
