#1 is a well-established technique called "online learning", and I'd bet money that it's how many of these "industrial scale" ML algorithms are trained.

#2 makes no sense from a business perspective.

#3 is also well-established. This is how Google's Captchas work, for example.