That may be true of neural net research, but I think there are still tons of places where automatic derivative calculation can be a big win. In protein simulation/design, for example, people spend a lot of time and effort coming up with derivatives for functions that do things like calculating the change in potential/kinetic energy of a protein side-chain atom, given a perturbation in one of the backbone angles. It's not always trivial to come up with efficient methods for derivatives in these problems.
The one real limitation here seems to be that you have to know that your function is differentiable (over the domain of interest) to use autodiff software. That can be difficult to determine. However, some of these packages say that they're able to detect non-differentiability, so even that point may be moot, if they can do it reliably, in advance.
To my knowledge, most autodiff tools will tend to silently ignore discontinuities, as long as you don't compute them at a non-differentiable point. E.g. if you have
y = abs(x)
you will get back the derivative
g = sign(x).
This works as long as you don't try x=0. Similar things would happen for floors, rounding, if statements, etc. In general, as long as each local operations is differentiable, the whole program will be. That's isn't too hard to check.
The one real limitation here seems to be that you have to know that your function is differentiable (over the domain of interest) to use autodiff software. That can be difficult to determine. However, some of these packages say that they're able to detect non-differentiability, so even that point may be moot, if they can do it reliably, in advance.