You can do that! But you may run into a problem where one of the crates you use detects that you're using nightly, and then opts in to nightly-only features. Except it may be using features that are only available on the latest version of nightly, or an ancient version that you don't have, and there is no version that makes both your code and the dependent package happy.
It's much better to just use stable. Then you're guaranteed to have forward compatibility, and you only have one place where you need to deal with the nightly weirdness.