That solves nothing because that's already what it's doing. The problem of training is adding new data beyond what's currently validated and into the future. And it'll get harder to validate it.
Huh? With the approach outlined above you can continuously keep adding new validated data.
> And it'll get harder to validate it.
Sure, and that's true in general: advances become harder, because we pick the low hanging fruit first anyway. Nothing new about that.
The arms race I described is also only one approach. Model trainers will also want to investigate economising on training data, making their approaches more robust to model collapse, multi-modal training, and a million other strategies and tricks that I can't think of in thirty seconds.
