Sysadmins would say that developers could learn a lot from them. Like, documentation, maintaining backwards compatibility, sane upgrade processes ...

Don't throw stones in glass houses and all that. We all could stand to learn things from other people.

Sure, sysadmins could stand to be better programmers, but software developers tend to forget that software is just a tool, and sysadmins use it like a tool. They just want to get in, get out, and move on to the next problem, so for them, certain best practices in software development aren't all that attractive.

> Shell scripting used to be amazing, but as a language these days it's behind the times.

It was never amazing, was given far too much credit, used too widely, and brought up a generation of people thinking that this is the way to manage a complex web of dependencies.

On the other hand, constraints like the above tend to foster creativity, so perhaps that's why people have been so nostalgic for this way of solving the system startup problem.

You should take a look at modern init scripts in FreeBSD. Common functions are sourced in, and then most "scripts" are just a series of variables that define various aspects of a daemon. Scripts are about the same length as systemd scripts, yet you have all the power of shell scripting right there if you need to do something special. In contrast, the systemd method means that if you need to do anything extraordinary, you have to call a shell script, and you've suddenly lost all the functionality given by the init system. So, for all intents and purposes, systemd ends up giving you the worst of both worlds if you need to call out to do housekeeping on startup/shutdown.

>I'm not saying systemd is the best answer, but I really think >sysadmins could learn a thing or two from developers.

>Nearly every custom init script I've ever seen has bugs, and >isn't portable between systems.

Nearly every custom init script is written by a developer for the product you desperately have to run because development wanted it, not bothering to give us sys admins time to look into the: packaging, configurability, operability and if a custom module/cookbook/playbook needs to be written.

>Where are the unit tests? Code reuse? >Focus on maintainability/readability?

If we were given the time, it would be there. My deadline for almost all work I do is "yesterday".

> My deadline for almost all work I do is "yesterday".

Having more or less worn both hats, it's exactly the same for developers. :)

A lot of us sysadmins are or were developers too, it's a lot easier to maintain a system you understand. Plus if it can be automated in code, it will be automated in code, we're lazy like that (well some of us, if I have to do the same thing more than once, I'm not doing it by hand the second time) and end up writing non-trivial amounts of code. Nearly every piece of software I've ever seen has bugs, and what do you mean I can't run the software without xyz framework, abc runtime and libraries a-zzz being installed on the system. Where are the performance specs? Resource reuse? Documentation? Focus on deployment/maintainability? Security audits? (I'm only mostly sarcastic)

Shell scripting is still amazing... when you use it for shell scripting.

Now I do like systemd more, but not because init scripts are bad, I just prefer how it does things, and can always call out to a script if need be anyways. And I suppose making it more friendly to developers is going to make my life easier too, the atrocities I've seen written in every language imaginable to replicate something that would be trivial to do in an init script. I didn't think it was possible (well possible yes, but why, WHY) to essentially rewrite the userland tools and the job of the init system in (very poorly written) php, but I've been proven wrong. Even something like forever for nodejs doesn't really have a need to exist.

But point is developers and sysadmins are both more than capable of writing shit code, lacking tests, code reuse, and all the rest of the best practices. At least we have to deal with the fallout from our code ourselves ;)

The shell has always been awful. It is underpowered and overfeatured with a deeply crappy API. It is more like Windows 3.1 than any sane development environment.

The good news is that you don't have to implement initscripts as shell scripts. I recall at least one GNU/Linux distro that used Python for all its initscripts, for example. Really, as long as the "script" is executable, a SysVInit-based or BSD-rc-based system doesn't really care (unless it's doing something strange like calling a specific interpreter on each script - i.e. calling `sh /etc/rc.d/rc.*` or somesuch).

Actually, a FreeBSD or NetBSD rc system does indeed care, for that very reason. rc.d scripts are sourced by a /bin/sh shell, using the . command. See run_rc_script on the rc.subr(8) manual page for details. Ironically, it's daemontools and daemontools-like systems where what you say holds the most widely. Tools such as execline can be and are in practice used in "run" programs.

I learn something new every day; thanks!

It doesn't look like OpenBSD has such a requirement (the start_daemon() routine in /etc/rc just calls each configured daemon's initscript as-is from /etc/rc.d without specifying sh or ksh to execute with); I had (incorrectly) assumed that this was a general BSDism rather than an OpenBSDism.