I think it still requires your normal password as well, so the server wouldn't be lost unless someone had both.