Every time you make a change to your yaml it requires a recompile. I think now ESP Home allows you to change configurations and upload a bin compiled on your main machine so it's really not a limitation at all. Plus, once it recompiles it automagically uploads so just make your changes and forget about it