At our school, by the time you reached 2nd year most everyone disappeared, male and female. Fall semester sophomore year was about 25% of fall semester freshmen.
"I want to make games!"
to
"This shit is hard, I'm gonna get a business degree."
As it happens when you have projects that take about 4x (or more) the amount of time as your friends majors that takes a toll. People would rather be partying than writing code. Maybe our CS1 was a bit too hard (we've had many discussions with the profs about this and the attrition rate and what to do about it), but everyone who finished freshmen year graduated in our class. It prepared us well, but it was too hard for most.
Yeah I think that seems to be a pretty common scenario. It's pretty unfortunate that there are people who only think of programming in relation to game development rather than the rest of the spectrum.