Yes, not installing X windows would be one of those things. But not one of the main issues. Removing lots of stuff that is installed on a fresh system, configuring the services that the server will be running in, that intrincated "debian like" /etc, making shure no ubuntu specific configuration tool will destroy some configuration file edited by hand, etc...
About the LTS, I would like to remind you that it mean Long Time Support, not Long Time Server. As for the Fedora Core, I myself don't use it, but my previous experience with Red Hat lead me to believe it is easier for the Linux beginner than Debian.But you're probably right about that, Cento OS would probably be a better choice that a Fedora Core... but don't quote me on that cause my experience with Cent OS is almost null.
Any changes made by hand to a configuration file on a debian-like system are noted and respected by aptitude and apt-get, so your configuration files will not be over-written (unless you want them to.)
All services are "off" by default on the LTS server and debian tries to conform to the Linux Standards Base so what you find in /etc/ is most likely similar to other distros.
The Cent OS community is not anywhere near as large as the debian community, and when you add in commercial support from Canonical, that makes for a pretty good value for money when you chose Ubuntu.
I'm not talking about apt or any debian configuration tool, I'm speaking of those graphical configuration applications that come with ubuntu, some of them use specific files to overwrite default configuration files on boot, etc...
About the ease of configuration on /etc, I can tell you that I've lost a full day to try an make apache work with vhosts and mod_python on ubuntu because of the debian specific way of scatering the configuration files, I usually do this in 5 minutes on gentoo or on apache compiled by me using the official distributed tar packages.
Even this last week I had do configure a qmail with vpopmail on a debian system and it was again a pain in the ass.
Maybe I've been unluck with the services I've needed to run on a Debian, but as far as I can say, I would not advice Debian.
A big difference on more "low level" distros where the user contrlos everything, as gentoo or slackware to more "automated" ones, like Debian or Fedora Core are the services on by default. A "ps ax" after finishing a base installation on gentoo fits on a 80x25 screen, on a Debian/Fedora this could fit on two or more screens.
That is actually a good argument for running the same OS on the desktop as well as servers. Experience gained while configuring a development envrionment can be used towards maintaining a production server.
About the LTS, I would like to remind you that it mean Long Time Support, not Long Time Server. As for the Fedora Core, I myself don't use it, but my previous experience with Red Hat lead me to believe it is easier for the Linux beginner than Debian.But you're probably right about that, Cento OS would probably be a better choice that a Fedora Core... but don't quote me on that cause my experience with Cent OS is almost null.