The flashlight behaviour is odd, I just tried it and I got the same delay the first time I turned it on, but no delay after turning it on multiple times. It's almost as if the code was swapped out to a slow disk and you have to wait for it to be swapped back in before the first use. There's still no good reason why it should take such a noticeable time, though.
There's no swapping on iOS. The delay is probably caused by iOS lazily waking up some hardware. The same happens when you use haptic feedbacks:
>Calling the generator’s prepare() method puts the Taptic Engine in a prepared state. To preserve power, the Taptic Engine stays in this state for only a short period of time (on the order of seconds), or until you next trigger feedback.