This is exactly why there is a lot of opposition to high speed rail in the US. It primarily helps those who live inside major cities. When constituents feel like they are paying for something that they are not benefiting from they will oppose it.