What about the situation where the first players got to scrape, then all the content companies realize what’s going on so they lock their data up behind paywalls?

Not a fan, but I'm not sure what can be done.

Assets like the Internet Archive, though, should be protected at all costs.

Wholeheartedly agree.