Gemini models are written in Jax which through the XLA compiler can be compiled either to TPU or GPU hardware.

Performance may differ but Google (and Nvidia) are very interested in having good performance on both platforms.