There's no clear argument for why direct-election became so popular. There were mostly vague arguments given such as direct-election would somehow fight against the cozy relationships between industry and the Senate, or end bribery in Senatorial politics.[1]
It seems that what was behind the movement to the popular vote was a strong populist sentiment without clear reason; or at least without a reason that made sense.[1]
Last I checked, bribery and cozy industry relationships in the Senate were not stopped by the 17th amendment. I doubt they will ever stop. Power and money tend to mingle, and that part of human nature will probably never go away.
There are some strong arguments that a senate operating behind closed doors would be more effective at ignoring special interests and working together than otherwise. The writers of the constitution did just this (they swore secrecy of anything they talked about in the convention to avoid voter backlash) [1] This is why they initially wanted health care debates behind closed doors - the lobbyists can make much more use of transparency than the uninterested voter can. Fareed Zakaria wrote a lot about this in The Future of Freedom: Illiberal Democracy at Home and Abroad.
[A Brilliant Solution: Inventing the American Constitution by Carol Berkin]
This shouldn't be downvoted just because it's disagreed with: it's a cogent and well-sourced argument, which should be met with a rebuttal rather than silent disapproval.
In Canada, our upper house (also called the Senate), is not directly elected. The members are selected by the Prime Minister (formally the Governor General, but this is done on "advice" of the PM, meaning in practice the PM makes the call). These positions are usually filled with party cronies or other such favoured individuals.
The current government, when they were the opposition, had called for Senate reforms, including direct elections. Once they got into power, they continued the status quo once they saw how beneficial it would be to them to do so. Funny how that works.
I'm no expert on the history (or the topic), though I did find the article I linked persuasive. The author did get into an argument over the history, though. You can read what he had to say about it here:
If the Congress is corrupt, than the state legislatures are a corpulent sewer of corruption and graft. Anything that takes power from them is generally a good thing.
The upside of direct election is that at least there's a chance that good Senators, not subject to the petty whims of the state houses might make better policy. State legislatures were/are generally the products of political machines and tended to appoint Senators who would advance their aims.
In the 21st century, the power of the corporations, unions and trade associations to manipulate the electorate via mass media is at least as powerful as the political machinery that the 17th amendment attempted to fight.
