I have this recurring nightmare where a lawyer in a very expensive suit asks me to explain, in detail, how an AI model "knows" what it knows. I wake up sweating every time. And the chardet situation is basically that nightmare, playing out in public, for real.
Here's what happened. It's messy, and it might change how open source licensing works.
The old school way: clean rooms before AI
Back in 1982, Compaq wanted to make a portable computer that could run all the IBM software people were already using. Problem: IBM owned the BIOS, and you can't just copy someone's copyrighted firmware. Their solution was a "clean room."
Two teams. Team A studied the IBM BIOS and wrote a detailed spec of what it did, without writing any code. Team B took that spec and wrote fresh code from scratch, without ever seeing the original. Team B never saw a single line of IBM code. They just knew what the thing needed to do.
That process took teams of engineers weeks or months. Today? You can ask Claude to do a version of it in an afternoon. I tried this myself with a small project last year, and the speed scared me.
What actually happened with chardet
chardet is a Python library that guesses character encodings. Mark Pilgrim created it in 2006 under the LGPL license. He stepped away from public internet life in 2011, and Dan Blanchard took over maintenance.
Earlier this year, Dan released chardet 7.0.0 with a bombshell: a complete rewrite, now MIT licensed. Faster, more accurate, same API, drop-in replacement.
Mark Pilgrim came back to open an issue titled "No right to relicense this project." His argument was straightforward:
Their claim that it is a "complete rewrite" is irrelevant, since they had ample exposure to the originally licensed code (i.e. this is not a "clean room" implementation). Adding a fancy code generator into the mix does not somehow grant them any additional rights.
This is the core tension. Dan has been maintaining chardet for over a decade. He knows that codebase inside and out. There is no way to un-know something.
Dan's actual process
Here's the part that makes this complicated. Dan didn't just tell Claude "rewrite this." He did something closer to a legitimate clean room:
- Started in an empty repository with no access to the old source tree
- Explicitly instructed Claude not to base anything on LGPL/GPL-licensed code
- Built tests first, then iterated on the implementation
- Every piece was reviewed and tested before it went in
He also ran the new code through JPlag, a plagiarism detection tool. The new release showed a max similarity of 1.29% with the old chardet. For comparison, earlier minor releases between versions showed 80-93% similarity.
That 1.29% is mostly metadata: lists of character encoding names and their properties, which is basically factual data that nobody can copyright anyway.
Dan's own words on this are worth reading:
A traditional clean-room approach involves a strict separation between people with knowledge of the original and people writing the new implementation, and that separation did not exist here. However, the purpose of clean-room methodology is to ensure the resulting code is not a derivative work of the original. It is a means to an end, not the end itself. In this case, I can demonstrate that the end result is the same through direct measurement rather than process guarantees alone.
The complicating factors
This case has a lot of gray area. Several twists make the right answer hard to call.
Twist one: Dan has been immersed in chardet for over a decade. It's not just that he saw the code. He lived in it. Every assumption he has about how character detection should work was shaped by that codebase. Can you clean room your own brain?
Twist two: There's at least one documented instance where Claude referenced parts of the old codebase during the rewrite. It looked at a file listing character sets and their properties. Dan argues these are essentially factual data, not creative expression, but the line is blurry.
Twist three: Claude itself was almost certainly trained on chardet's code. It's a popular Python library that has been on GitHub for nearly two decades. You cannot meaningfully instruct a model to "forget" something in its training data. So even if Dan never showed Claude the old code, Claude might have been drawing on it anyway. Can a model trained on a codebase produce a defensible clean-room implementation?
Twist four: Mark Pilgrim's original code was itself a port of Mozilla's MPL-licensed character detection library, translated from C to Python. So the copyright chain is already tangled.
Twist five: The rewrite was released under the same PyPI package name. If Dan had released it as a new package under a new name, would anyone have cared? Probably not. The name reuse is what made this a direct confrontation.
What the actual lawyers are saying
Richard Fontana, one of the authors of the GPLv3 and LGPLv3 licenses, weighed in with a TINLA (This Is Not Legal Advice) comment. His take:
I don't currently see any basis for concluding that chardet 7.0.0 is required to be released under the LGPL. AFAIK no one including Mark Pilgrim has identified persistence of copyrightable expressive material from earlier versions in 7.0.0 nor has anyone articulated some viable alternate theory of license violation.
He said he personally would not have used MIT here, but that's a different question from whether the license change was legally valid.
What this means beyond chardet
chardet isn't a one-off; it's the opening salvo of a much bigger conflict.
Armin Ronacher described it as the Ship of Theseus problem for software. When the cost of reimplementing something drops to near zero, what happens to licensing? Will we see a mass migration of GPL code to MIT, as people rewrite around licenses they don't like? Maybe proprietary software reappears as open source through the same mechanism, or companies use it to strip-mine competitors' IP.
I don't know. My gut says the chardet rewrite was legitimate, but the arguments on both sides are strong enough that reasonable people can disagree.
The uncomfortable question nobody wants to answer
Here's what keeps me up at night, and it's not really about chardet:
Once commercial companies see that their proprietary code can be reimplemented by an LLM that was trained on it, they will sue. They will sue hard. The legal theory will be something like: "Your model ingested our copyrighted code during training, and now it's producing functionally equivalent code for our competitors."
The open source world is the canary. The coal mine is corporate IP litigation, and it's coming.
Mark Pilgrim was not wrong to be angry. He wrote that code. The LGPL was his choice, and someone used a tool he did not anticipate to change that choice. Whether or not the law ultimately agrees with him, the feeling of betrayal is real.
At the same time, Dan is not wrong either. He invested a decade of volunteer labor into maintaining someone else's project, and he found a way to produce a better version using modern tools. Saying he can't work on character encoding at all because of his own unpaid maintenance work is an absurd conclusion.
The law will catch up eventually. Court cases will set precedents. But for now, we are in the messy, uncomfortable period where nobody really knows, and everyone is guessing.
I do know one thing: I am never going to be the person a lawyer in an expensive suit asks to explain how an LLM works. I have updated my LinkedIn accordingly.