Distillation is a superficial step and doesn't need a lot of data, it's not "stealing the model" like they want everyone to believe. 99% of work is already done by that point. That said, it's pretty clear K3 has Claude's data in the training set (either Opus or Fable), as it repeats Anthropic's prompt injections. (not that it matters to anyone besides Anthropic themselves)
If by "distill" they mean "used it for fine-tuning" then they might have used it in the final stages of fine-tuning of Kimi K3. I image they might have already been using Opus, and when Fable became available it was easy to switch over to it
It would have been a tiny part of the overall training, given the timeline
The line "how could they do it in such a short time" is absolutely idiotic. The infrastructure was already there, it's massively parallel, and it's not like the only thing being distilled on is Fable.
A month seems plenty long enough. They're not rebuilding the entire model from scratch. It's just getting Fable to act as a teacher model for some of the final reinforcement learning on the base that Kimi already had.
I think that would also be a bad idea, as all models opus 4.6 got increasingly smarter, but also crappier at following instructions or genuinely assisting.
They just try to figure out what the goal is and hyper focus on solving it.
Heh, even just telling fable don't commit doesn't work half the times, let alone more complex instructions.
Honestly, it wouldn't surprise me if they just found evidence of distillation once in 2025 against some Chinese AI lab, and they've been lying about the rest to create a narrative.
