> If all these AI models were trained on copyrighted materials for which the trainers had no right to, is it wrong to steal their models and use them however we want?
If (which the courts seem to be pretty consistently finding) training models on copyright-protected works generally is fair use, though using models to produce works which would violate copyright if made by other means with reference to the source material is still a copyright violation, then training has no bearing on the legality of copying the models. (Even if it wasn't, then copying and using the models at all would violate the copyright of the original owners of the training material again and be illegal irrespective of the “license” offered by the model trainer.)
Morally? Well, pretty much the same dichotomy applies; if training the model isn't a violation of the source material's creators' rights, then the fact it was trained without permission has no bearing on the morality of using the model without the trainers permission, if it is a violation of the source material's creators' rights, then so is using the model irrespective of the trainer's “license”, as the trainer has no right to permit further use of the material they had no right to create.
The idea that the model is an intrusion on the rights of the creators of the materials used in training and that this makes use of the model more rather than less permissibly, legally or morally, takes some bizarre mental gymnastics.
If (which the courts seem to be pretty consistently finding) training models on copyright-protected works generally is fair use, though using models to produce works which would violate copyright if made by other means with reference to the source material is still a copyright violation, then training has no bearing on the legality of copying the models. (Even if it wasn't, then copying and using the models at all would violate the copyright of the original owners of the training material again and be illegal irrespective of the “license” offered by the model trainer.)
Morally? Well, pretty much the same dichotomy applies; if training the model isn't a violation of the source material's creators' rights, then the fact it was trained without permission has no bearing on the morality of using the model without the trainers permission, if it is a violation of the source material's creators' rights, then so is using the model irrespective of the trainer's “license”, as the trainer has no right to permit further use of the material they had no right to create.
The idea that the model is an intrusion on the rights of the creators of the materials used in training and that this makes use of the model more rather than less permissibly, legally or morally, takes some bizarre mental gymnastics.