Most likely Apple is still working out the internals of the Swift compiler. Refactoring requires a thorough knowledge of code paths and object lifetimes.
Once Swift 2 is nailed down, I'm sure refactoring will follow.
Even in Objective-C, the refactorings available are pretty limited and don't always work. You can do Rename and Extract Method. In a few corner cases, Extract method will break. Forget about extracting a variable or a parameter, of pulling a mtehod up toa superclass. Also, the fact that you have Rename but you can't find usages for a variable of method seems to me jsut lazy or ill-thought.
It is a very odd feeling to miss Eclipse, but back when I worked in XCode, that would frequently be how I felt.
Once Swift 2 is nailed down, I'm sure refactoring will follow.