Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I can't speak to GP, but AI is exactly what I had in mind when I read their comment. With AI transforming website content, the sites themselves just become data sources.

You could be right though, especially with more and more sites blocking robots. Legitimate companies might just have to shrug and say, "this site prohibits me from reading it. You'll have to go there instead) though that seems like a legal problem to me rather than a technical one.



Sites that try to use robots.txt to stay relevant are going to fail spectacularly. All their information is going to leak or be provided by other sources, and they're going to get forgotten. The only people who will be able to make a living selling data will be the ones who can produce data so unique that nobody else can claim to have created it, and for which training on it leaves an unmistakable signature that makes litigation easy.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: