Hacker Timesnew | past | comments | ask | show | jobs | submitlogin
Apache Removed from OpenBSD Base (undeadly.org)
83 points by zdw on March 15, 2014 | hide | past | favorite | 50 comments


Note this is about OpenBSD dropping support for OpenBSD's own fork of the skanky old Apache 1.3 line, which hasn't been officially developed in aeons, and no official security updates since 2010.

The title implies some vote of no confidence by the OpenBSD team regarding Apache, when in actual fact they're just cleaning up their own mess, due to objecting to an earlier licensing change in the 2.x line.


> The title implies some vote of no confidence by the OpenBSD team regarding Apache

No it doesn't. It says what it says: Apache was removed from base. Now that there is a reasonable alternative with an acceptable license, it is time to move on.

> when in actual fact they're just cleaning up their own mess

You choice of words sure reeks of negativity towards the project. The fact is, skanky old Apache 1.3 has served the project well for a long long time, and it has had a good track record as far as security goes. Keep in mind that code in the base system has been audited, patched and maintained by the project.


So I can see this getting boring pretty fast, but whatever.. on reading the title I wondered "why!", and on reading the article I find a recommendation for nginx. The reason for this appears to be entirely due to licensing in modern (and more comparable) versions of Apache 2.x than anything technical. The article does not make that clear at all.

Apologies if you read so much negativity into my choice of words, it wasn't meant as an attack on OpenBSD, forking unmaintained software, or whatever else, just intended to point out a correction in the way probably half or more of readers will interpret the article – as some kind of indication that Nginx is technically somehow better than Apache


Its not entirely due to licensing. OpenBSD has security patches to Apache that weren't accepted plus the license change. OpenBSD has been maintaining their own branch until the switch.


While I understand that if the choice were between the OpenBSD fork of Apache and nginx the only sane choice would be nginx (the OpenBSD fork is silly). But, I see people making the argument that nginx is "more secure", which would fit into the OpenBSD mindset. But, the reality is that nginx and Apache both have imperfect security records; Apache is pretty damned good on the security front and has been for a decade or more. Also, nginx makes it much easier for a sysadmin to screw up their security...including with some example configurations that are commonly found floating around on the web.


Aha, "silly". I see your comment is full of arguments why this was the case all these years. Most Apache security issues didn't affect or had lesser impact on OpenBSDs fork.

The only thing silly here is your opinion.


The OpenBSD fork was ancient. The only people using it were those who didn't actually need a modern web server...which is to say, not even OpenBSD users were using it, if they were using their OpenBSD server for serious web service. While I understand the motivation behind it, I believe it would be silly to build a web service on top of an effectively 15 year old version of Apache (1.3 was first released in 1998 and was EOL, even for security releases, years ago).

What is also silly is for a project that has so few serious developers to take on a project as massive as maintaining a fork of a project as large as Apache.

When Apache had security issues that didn't affect the OpenBSD fork it was often because the OpenBSD fork didn't have the module in question (most security issues in Apache in the past decade have been in modules, often not frequently used ones). People not using those modules wouldn't have had those security issues, and people using those modules on OpenBSD (after building from source to get those modules and a modern version of Apache) would have also been affected. I'll admit that there is a certain charm to software that has few bugs because it does very little, but Apache is what it is because people need it to be those things. It is certainly not through incompetence.

So, my opinion may be silly. But, it is not unconsidered. I've thought a lot about web service for the past 15 years, as it's a core part of my business. I could very well be wrong. Maybe OpenBSD would have been even less successful for web service had they opted to include a modern version of Apache. (And, before you say "popularity isn't a measure of being right", I will point out that no matter how secure software is, it is useless if no one uses it.)


Yeah, in my OpenBSD work, we all skipped httpd and put in something else.

Personally, lighttpd for a long time, but then it failed to be maintained in keeping with OpenSSL libraries, so it was dropped for nginx, and no looking back since.


I always get funny feelings when people use "ancient" and "modern" as argument, without further explanation. Not that it is an invalid point, mind you. But certainly one that would bear expanding on.


I mentioned one reason above. The Apache developers are far from incompetent, and all the changes in Apache since 1.3 have been because somebody (usually a lot of people) needed or wanted those changes.

In most things I agree with you; I use vim, bash, shell scripts for automation, Perl, etc. All things that get the ancient vs. modern argument used against them...but, when it comes to software security old nearly always means "exploitable". Since Apache 1.3 became EOL in 2010, OpenBSD developers have been solely responsible for finding and fixing 1.3 security issues. That's a lot of work. If I were responsible for Apache packages (and I am; we ship a custom build of Apache for some of our supported operating systems in Virtualmin repos), I would want to leverage the knowledge and efforts of the people who primarily work on Apache.

To continue...

There have been major architectural changes in Apache from 1.3 to 2.x. Performance on a number of use cases has nearly doubled in those 15 years, for example.

Modern web standards are more widely supported by recent versions, as well, either through modules or core HTTP protocol support changes. Wanna use SPDY? Yeah...gotta have 2.x.

A ton of security and QoS oriented capabilities are also "new" in 2.x. Given OpenBSDs security focus, I believe tools to help prevent exploits and DoS of web apps should also be considered...not just whether the web server has any directly exploitable bugs. Very few people are just serving flat files with their web server today. The code interacting with clients is often mostly in Python, Ruby, Perl, PHP, etc. Those are as much potential attack vectors as the web server itself, and 2.x has better tools for managing that risk.


In somewhat related news, OpenSMTPD finally replaced sendmail as the default mailer on OpenBSD.

http://undeadly.org/cgi?action=article&sid=20140313052817

http://www.opensmtpd.org/


This is medium news. I'd rather use postfix. There's a ton of stuff missing from opensmtpd at the moment. Recently it had some serious crashing bugs when pipes were getting severed between the various privsep processes as well. It's getting there but it'll be a while yet before it should be in production on anything than really simple single nodes as a relay/local delivery.


Postfix was a no-go for OpenBSD because of the license. The only existing mail server with an acceptable license was qmail, which had its own issues, to put it lightly.


Gotcha - makes sense. First thing I do though is install postfix from ports still!


This is a good thing. Apache is a swiss-army chainsaw of last-resort, not a minimal webserver.


Apache is very modular. You enable what you need.

Apache's biggest problem is that it isn't cool and hip anymore. It's config file is too easy to understand.

Good { thing we = have; nginx; }

Phew!


Sounds like it's time for an anecdote!

In 2008, my web site saw some decent traffic, and I was stuck in a spiral of buying more and more expensive servers to keep up with the demand. Some of the media files were large, and slow downloads would tie up a thread, and corresponding chunk of memory.

I spent a long time working through the documentation that was available at the time, and I couldn't get anything to work stably with Django.

Then I found a guide for lighttpd, and was floored when the resources used by my web server dropped to single digits. Much later, I'd learn that even saturating a gigabit pipe with just html/css/image traffic ran fine.

Nginx has since replaced lighttpd, since the latter doesn't seem to be well maintained these days.

As I understand it, Apache now is capable of high performance, but once bitten twice shy. The whole thing just seemed to be a big, bloated mess to me. Like you can do everything imaginable, except what you actually want to do.

I couldn't imagine a feature that would exist to convince me to move back. I expect quite a few web developers are in the same boat.

Your criticism of the config file is nonsensical. I'd use nginx with Apache style configuration over the inverse in a heart beat - though I disagree it would be an improvement.


I like servers that work reliably so the "pager" doesn't go off in the middle of the night nor get calls as to why our service is down.


It does have too easy of a config file. Who needs per directory config anyway? Or the ability to create redirects without a restart? I love nginx, but it has some flaws. The reality is it just has different flaws and strengths than Apache.


You can do per-directory config in nginx with the location block, it's just that you just can't put the config inside the directory itself and have nginx automatically load it.

You can tell nginx to reload its configuration without restarting the entire server process by running "sudo /etc/init.d/nginx reload", or on ubuntu "sudo service nginx reload".


'service' is not just on Ubuntu - i believe it started life on Red Hat, and has now spread everywhere. Including FreeBSD:

http://www.freebsd.org/cgi/man.cgi?query=service&sektion=8


Oh, I thought it was part of upstart for some reason. Thanks for the the clarification.


I find the lack of dynamically loaded modules more aggravating, personally. Having to recompile and optionally repackage the damn thing because you need a module not included in the base nginx is a pain.


I <3 nginx and apache. Tech is a toolbox. Pick the closest tool for the job.

Per directory config is extremely beneficial for static hosting / VPS, but sketchy to get the security of values just right.


Bikeshedding on config syntax isn't a valid critique.

Apache uses tons of memory regardless of which worker model is used.

Nginx is reliable and stable, we've never had memory leaks or weird shit happen. This is why it's often uses as a front-end reverse proxy.

More often, nginx is put out in front of apache where other dependencies require it (dav svn).


No, Apache's biggest problem is its architecture that limits quite a few things you can do. Case in point: the farce around mpm_event. It took ages to implement it to solve a central issue Apache always had (i.e. that it could be DoS'd by slow but long-living connections), and even now, it's not exactly in a great state, compared to nginx's async I/O internals.


mpm_event in 2.4 is quite fast and let's you keep all of your terrible mod_rewrite hacks in place. :)


> Apache's biggest problem is that it isn't cool and hip anymore. It's config file is too easy to understand.

Yeah, mod_rewrite sure is the epitome of a readable config file syntax.

Why have one or two simple to understand "if" statements when you could have a pile of incomprehensible regexps with random one letter control flow flags?

Simple. Ha.


The fork of Apache 1.3 that OpenBSD had wasn't really that big.

EDIT:

The C source files and headers under src/usr.sbin/httpd are about 100k lines. Under src/usr.sbin/nginx you have closer to 170k lines. Of course this isn't exactly apples to apples comparison as both have optional modules and at least nginx has some OS-specific code.

In any case though, it is really hard to argue that nginx is "light, small, and fast" while httpd is a bloated swiss army chainsaw with a kitten sink...


Code size is in almost no way an indicator of performance or bloat.

Ik can write a program that will use 100% CPU in about 10 LOC, while there can be very well-optimized pieces of software can be thousands to millions LOC.

e.g. the Quake 3 engine is probably as well optimized as you can make it and is about 300K LOC: https://github.com/id-Software/Quake-III-Arena


In my book bloat and performance are two entirely different things, and I think there is a strong correlation between code size and bloat.

I agree about highly optimized code being typically larger than the simpler alternative.

Now nginx has interesting optimizations in some places (e.g. the http parser), but if you read the code, it's not like 90% of the code in nginx is there due to optimization.

I'd like to add that from an OpenBSD perspective, excess optimizations would no doubt be considered bloat as well. One of the goals is that the code is clean, correct, and secure. I know for a fact that the developers are not going to carefully audit and maintain millions of lines of micro-optimizations. What these might get you is a tick in a hypothetical feature checklist ("performance"). But if another daemon does the job and performs well enough, then that feature is unneeded complexity, hence bloat. Not something you want to include in what aims to be the world's most secure general purpose operating system.


can I ask why?

I am not an openbsd guy obviously but know httpd is popular. Is it because adding it to base install adds too many security concerns?


OpenBSD forked Apache 1.3, when the Apache 2.0 license was introduced. One argument could be that maintaining, patch and extend a web server on top of writing an operating system is to much work, especially if there's a better alternative, with a license you can accept.

Including a web server isn't necessarily a security concern, it's not running by default. The advantages is that there's a web server included, with privsep, chrooting and a sane default configuration.


What was the incompatibility?


I not completely sure. One complaint was, if I recall correctly simply the length and complexity, compared the BSD license. The Apache 2.0 license is very much lawyer talk and a little hard to read.

More likely it was the patent clause of the license that made it so bad that the OpenBSD developers didn't want it. I can sort of see where they're coming from. If some company puts a bit of code into Apache 2, covered by one of their patents, that's okay, anyone using the Apache 2 gets a license. However it's pretty much impossible for anyone to tell which parts of the code is covered by the patent, so you could get in trouble, without knowing it, if you yank out that part of the code an use it elsewhere. The patent could also cover a protocol, an algorithm, whatever, the people using the Apache licensed code don't care, they where given a license for the pattern, but nobody else can do an implementation.

It's not for me to say, but I believe that the OpenBSD developers wanted an http server that came unencumbered by patent and was ensured to be free of any software patent. Instead the Apache 2.0 license presented them they the problem that you can actually tell if a given piece of code was covered under some patent.


The OpenBSD policy page is quite useless[0] on this, but I _believe_ the issue is that the Apache License 2.0 contains the patent clause that states something like "you can't sue apache if someone contributes stuff that infringes your patent", which limits the freedom of the openbsd users to sue apache.

The relevant thread[1] from the time may be of use, but a specific message sums it up:

"We've been clear: Their new license contains more stuff, and we do not accept MORE STUFF in licenses."

[0] http://www.openbsd.org/policy.html [1] http://www.monkey.org/openbsd/archive/misc/0406/msg00412.htm...


On the one hand being against MORE STUFF is a perfectly reasonable position. There's too much accumulated cruft in legal documents that has no real purpose besides make-work for lawyers.

On the other hand it's absurd to dogmatically oppose MORE STUFF regardless of what the stuff is. The added terms in Apache-2 make sense for a US-based collaborative software project that accepts contributions from businesses that hold software patents. They may be pointless legalese for a Canadian-based project whose contributors are mostly individual developers, but that doesn't make them an assault on developers' freedoms.


Is simple, unconditional freedom really that absurd?

We want to make available source code that anyone can use for ANY PURPOSE, with no restrictions.

That's what we call a free gift.

That is one of the project's goals.

You're right, the patent clause might not necessarily be an attack against the OpenBSD developers' freedoms. But they do not care about their own freedoms only; they want it to be free for anyone (this includes corporations who might hold patents, and also individual developers working for such corporations), for any purpose. It is really as simple as that.


Let me quote Theo --

  You guys keep missing the point really.

  We know what a free license should say.

  It should say

  Copyright foo

  I give up my rights and permit others to:
		distribute
		sell
		give
		modify
		use

  I retain the right to be known as the author/owner

  When it says something else, ask this:

  - is it 100% gauranteed fluff which cannot ever affect anyone?

  - is it giving away even more rights (the author right)?

  If not, then it must be giving someone more rights, or by the same
  token -- taking more rights away from someone else!

  Then it is _less_ free than our requirements state!

  So why even BOTHER wasting your time trying to understand what they
  say?

  Who cares if it is legal or not!  We're not going to want to go
  quibble in a court!  We're trying to make it so simple that something
  can't even GO to court, because it's free and, anyone can tell that it
  is free because the language used to say so is SIMPLE.
(From http://marc.info/?l=openbsd-misc&m=103283218106749&w=2)


That's noble and I respect that. Unfortunately it fails to address software patents at all, and that doesn't make the problem go away.

Copyrights are essentially about copying. Patents aren't - you can write a clean-room program that's entirely your own work, absolutely guaranteeing that you aren't infringing anyone's copyright, but it could still infringe a patent you've never heard of. Or you could accept code from a friendly company which has a few defensive patents on it, and some years later the friendly company comes under less friendly ownership and the defensive patents turn offensive.

Apache, being based in the US (which has many software patents) and accepting code from many companies that have patents, has to deal with these scenarios. OpenBSD, being based in Canada (with far fewer - but not zero! - software patents), isn't as affected by it so Theo calls it bullshit. But it's not so black-and-white.


You are right, copyrights are about copying. Copyright licenses cannot address software patents any more than they can address US export restrictions, trade marks, contracts, or any other international or local laws that might deny you some freedoms. Theo cannot grant you rights to Microsoft's or Cisco's patents. What he can grant you is rights to his code.

Now you can try to "address" software patents e.g. by obligating distributors to grant rights to their patents, but that is a restriction; it might be giving someone more rights, and by the same token is taking more rights away from someone else! More importantly, that would be a restriction imposed by the copyright holder, rather than an external restriction imposed on you by a third entity the copyright holder does not control.

OpenBSD does not want to impose such restrictions on you or anyone else. They make their code free.

You seem to think that OpenBSD or Theo simply disregard patent issues because they hail from Canada. Maybe you should learn about CARP[1][2], or figure out why many of the ports' Makefiles have a line like this:

  PERMIT_PACKAGE_CDROM=  patents
[1] http://www.openbsd.org/lyrics.html#35 [2] https://en.wikipedia.org/wiki/Common_Address_Redundancy_Prot...


(I lean on the *BSD side of anarchistic, DWTFYL.)

Patents + OSS is a huge, under-served point that unfortunately Lessig is no longer working on.

I and others that open source tons of code will be the first to wave the FOSS banner all day long, but it's not going change the reality of the present landscape.

As such, open source projects and startup founders repeatedly fail to anticipate the consequences of nonparticipating in the patent scheme. If the other side (a company) holds a (shitty) patent, and a project doesn't... they can still squish them like the cockroaches they are should the project become significant. So there's options: either play the game, fight city hall or get fucked by pretending it doesn't exist. Not choosing an action is still a choice.

Perhaps one of the larger OSS orgs would show leadership on this to help projects manage this tricky minefield.


> "On the other hand it's absurd to dogmatically oppose MORE STUFF regardless of what the stuff is. " I don't think that it is absurd to oppose stuff written a in language you can't understand, you will never be able to learn and you actually don't care about anyway.


The incompatibility is with their beliefs, it's not technical. You can get Apache 2 in ports and packages. They disagre with the license, so it is not acceptable to them in base. It's the same reason they ship an ancient GCC in base, but once again, the newer ones are in ports. What they include in base OpenBSD is very special to them.


It's the base system, so it makes more sense to include a small, fast, and secure HTTP server than one that includes everything and the kitchen sink. Note that it's still easy to install Apache from the OpenBSD ports.


It is being replaced by Nginx


How good is OpenBSD's USB support on recent motherboards? A USB bug that affects motherboards with Intel chipsets in FreeBSD 9.2-RELEASE makes me think of switching; it hit print servers bad.


Have you tried 9-STABLE to see if the xhci bug is fixed there? It looks like some patches were committed for it, but not until after the change freeze for 9.2-RELEASE: http://www.freebsd.org/cgi/query-pr.cgi?pr=181159&cat=

If the problem is fixed in 9-STABLE, but you're only deploying RELEASE builds, 9.3-RELEASE should be available in a few months: http://www.freebsd.org/releases/9.3R/schedule.html


Not sure if it's in STABLE already but I was aware of the patch. We have rolled back to 9.1-RELEASE for now and will wait for 9.3-RELEASE.


I've had mixed results, though my sample set is small, and I can't say for certain that the problems I've had are a software issue. Also, there have been some recent cleanups and fixes in the USB stack that I have not tried yet. For these, you need to grab a -current snapshot or wait for 5.5. Note that the xhci work is post 5.5.

Your best bet is to find some spare hardware and give it a try. I wouldn't switch a production box over before that.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: