There are even some foods that are just unsafe organic. Pork (less interesting to you, being vegetarian, bear with me) is not nearly as safe, there's been a rise in trichinosis cases at least related to organic pork. I know I've seen evidence suggesting organic celery is worth avoiding too, but I can't find it now.

In any case, agreed that you don't have to spend an arm and a leg to get healthy food. Expensive often doesn't even indicate better food, just more aggressively marketed at suckers that'll pay more.