Open source under attack?

My boss mentioned this to me the other day, and like the cat that got the cream (he's a Windows/IIS diehard) mentioned that it was Apache web servers that were the vector.

Thanks for posting this, it make interesting and rather scary reading.

From the articles, the Apache vulnerabilities used are very limited: Known (somehow!) user/pass combinations and dynamic module loading. Not even root-kits on the Apache servers!

Fascinating. It hadn't even occurred to me that it might be being done to discredit F/OSS, since the end-target vulnerabilities are all Windows.

If this is as good as a mass Linux attack can get, I'm very happy.

Now, how about Apache quickly making dynamic loading _not_ the default, and convincing these administrators to change all their passwords?

> Known (somehow!) user/pass combinations...

Usually that means somebody didn't bother changing the default.

> Usually that means somebody didn't bother changing the default.

First thing I do on any machine, remove FTP and telnet, RPC and every other such thing I can find, then enable only SSH.

SSH has file transfer just as good as FTP, it's cross platform and F/OSS available for everything, no reason to use passwords or even standard ports.

Oh well. "Security Is Inconvenient."

> Usually that means somebody didn't bother changing the default.

What default? By default the account passwords are disabled on any distro I've seen.

There is something fishy about this so-called report. Too much missing information, too vague about the details.

> First thing I do on any machine, remove FTP and telnet, RPC and every other such thing I can find, then enable only SSH.

Ouch. an nfs or nis server without rpc is going to be sort of crippled ;) Hey, just kidding, maybe you don't need a server. In any case, I just use firewall rules to protect access to such services. Actually I don't even enable telnet, no need for that sort of thing in this millennium.

Personally I'd like to see a little more info on the cpanel connection, since 100% of the compromised systems reviewed according to the article were all running it. It may not actually be cpanel but it is common to all.

Not knocking cpanel, I have used it and like it, especially with the php mysql plugin (cant remember the name off the top of my head).

> What default? By default the account passwords are disabled on any distro I've seen.

In the distro, yes. But what about in the handy-dandy packages that install 40 different pieces of software to give you a working widget?

Mind you, I don't know this is the case. I'm guessing. But based on past experience I think it's a pretty good guess. Anyone know if cpanel is this way?

I spent some time reading the various forums on this. Looks like this has been roaming since mid november at least. There is a rootkit, a linux kernel module that rewrites anything that comes out of apache to add the IE exploits and that remembers IP addresses of visitors it already served the exploit so it doesn;t server someone twice. It's stored somewhere in the kernel as well.

The problem isn't the rootkit. The problem is how they got the rootkit on the machines in the first place.

wow tracyanne, how lucky you are to have such a mature, reasonable boss. Sheesh. I guess he missed the part where Apache is merely the vector, and Windows and Windows apps are the targets. Oh, and how Linux in general is still about a million exploits behind windoze :)

Quoting: There is a rootkit, a linux kernel module that rewrites anything that comes out of apache to add the IE exploits and that remembers IP addresses of visitors it already served the exploit so it doesn;t server someone twice. It's stored somewhere in the kernel as well.

This is news - everything I'd read so far sounds like maybe default cpanel passwords were used to gain access to hosted linux servers, nothing more. Do you have any URLs with more information about this supposed rootkit?

This thread on WebHostingTalk: http://www.webhostingtalk.com/showthread.php?t=651748

There's some other good info in the comments on El Reg: http://www.theregister.co.uk/2008/01/11/mysterious_web_infec...

There's more links leading away from those two. The rootkit writes the IP addresses of visitors to /dev/kmem which is usually writable in default kernel configurations. A quick way to stop this thing is to compile a monolitic kernel (no mocule loading) and prevent writing to /dev/kmem by using grsec: http://www.grsecurity.net/ After that, monitor the kernel and watch for failed module loads.

hmm still inconclusive - I read some speculation about kernel modules and /dev/kmem but have seen nothing so far that requires super-user access - and nothing that can't be accounted for if a bad guy were simply to log into cpanel with a stolen password, and load a clever custom apache module which then serves up the attacks.

AFACT from the threads, they didn't find any apache module on the effected servers. Only a kernel module.

Quoting:wow tracyanne, how lucky you are to have such a mature, reasonable boss.

No no no, I'm the unreasonable immature fanatical one.... I like Linux. He does however hope that the "Linux People" don't try to sweep this under the carpet, like Microsoft so unfairly gets accused of doing.

Heh...I wonder if you could compile a list of current unpatched exploits for Apache vs IIS.... That might be interesting. I mean..it's one thing to have an exploit that requires a kernel module (and thus most likely root privs), it's another to have an exploit that allows you to do the same thing by say sending a bad url to the server or some such...

> AFACT from the threads, they didn't find any apache module on the effected servers. Only a kernel module.

It seems odd that they would implement that apache-specific functionality at such a low level - but then again, maybe that functionality is complex enough that it made more sense to do it in the kernel.

Like you said, the exploit in and of itself, if it is a kernel module, shows what we already knew: that the super user can load modules to do all sorts of clever tricks with the linux kernel - the real question is how the bad guys got root.

I still suspect a default cpanel password, or something of the sort, meaning there was no remote root exploit, only what amounts to a stolen password.

That's possible but not very likely. The WHT thread stated that many of the servers that were rootkitted were shared hosting servers from big name webhosters. I find it unlikely that a major hoster would leave his cpanel open to such an attack, although my latest "adventures" with ipower and ipowerweb made it painfully clear just how dumb they can be.

That said, such shared servers are a *very* attractive target to crackers. Load in one rootkit and you've just infected 300+ websites to dole out your IE malware. It's entirely possible that such large webhosters don't keep up with the latest version of everything on their rather large server park. They may have had local priviledge exploits that are already fixed.

Rootkit? It looks like someone forgot to chroot their Apache when they set it up.

All of them?

chroot isn't a cure-all by a long shot.

http://www.bpfh.net/simes/computing/chroot-break.html

chroot is not a security device. It's too bad it's taught that it is, because it never has been, and was never intended as such.

That's not what they say in the OpenBSD FAQ,tuxchick. Unless I'm mistaken chrooting apached is a key security precaution when running a web site.

Shagbag: If you follow the article links to the various forum threads you can see it's a rootkit. A kernel module to be exact. The question is how they got the rootkit on the servers in the first place.

Shagbag, this is what Alan Cox says:

Quoting: "chroot is not and never has been a security tool. People have built things based upon the properties of chroot but extended (BSD jails, Linux vserver) but they are quite different."

http://kerneltrap.org/Linux/Abusing_chroot

info chroot says nothing about using it as a security device: http://www.cs.vassar.edu/cgi-bin/info.cgi?(coreutils)chroot+invocation

You need root privs to rootkit anything, so once you have that, game over, chroot or no.

Thanks for clearing that up for me, tuxchick. I see your point about rootkit having root privileges ergo it can chroot itself out of any 'jail'. This is the OpenBSD FAQ I was talking about:

10.16 - Tell me about this chroot(2) Apache?

In OpenBSD, the Apache httpd(8) server has been chroot(2)ed by default. While this is a tremendous boost to security, it can create issues, if you are not prepared. What is a chroot? A chroot(2)ed application is locked into a particular directory and unable to wander around the rest of the directory tree, and sees that directory as its "/" (root) directory. In the case of httpd(8), the program starts, opens its log files, binds to its TCP ports (though, it doesn't accept data yet), and reads its configuration. Next, it locks itself into /var/www and drops privileges, then starts to accept requests. This means all files served and used by Apache must be in the /var/www directory. In the default configuration of OpenBSD, all the files in the /var/www directory are read-only by the user Apache runs as, http://www. This helps security tremendously -- should there be a security issue with Apache, the damage will be confined to a single directory with only "read only" permissions and no resources to cause mischief with.

hmm, that sounds pretty ingenious. Rather like how SELinux and AppArmor are supposed to work. I wonder if it works the same way on Linux?

Trying to get details on the actual exploit is frustrating. I haven't found a single bit of concrete information yet, just a lot of vague scary stuff.

FreeBSD has a similar concept call 'Jails':

http://www.freebsd.org/doc/en/books/handbook/jails.html

Is there not the same facility in linux?

Quoting:should there be a security issue with Apache, the damage will be confined to a single directory with only "read only" permissions and no resources to cause mischief with.

Isn't this the same as ACL (Access Control Lists)?

As I recall, there was a security feature in VMS where certain privileges are granted to the application instead of the user account using ACL, sort of like the S (sticky) bit in Linux. I am not sure but, isn't that the same?

Access Control Lists are a much more generic concept and basically a replacement for the User|Group|World permissions. They control user or process access to a single file, folder, or directory object (i.e. LDAP.)

ACLs, however, do not hamper a user from seeing any part of the file system to which they have permissions.

A chroot or jail, OTOH, makes a sizable chunk of the file system disappear.

Quoting:Access Control Lists are a much more generic concept and basically a replacement for the User|Group|World permissions. ...

@NoDough,

I believe an ACL doesn't replace User|Group|World permissions, it simply amends them for designated objects that carry the ACL.

I think I know what ACL is and does, I was referring more to the s-sticky bit in Linux, which grants the user process specific privileges to the relevant object carrying the s-bit even if the user account doesn't have them. No password is required.

VMS has the same permission feature where privileges are passed on to the user by the application while running and from within the application only. Those permissions are granted/restricted by a VMS ACL that the application has.

Just like sudo in Linux and "Run As" application in Windows. But these are interactive and have to enter a password.

A chroot or jail are global, just like "Run As" administrator shell in Windows.

Then again, It might have been wishful thinking on my part and misunderstood their features.

@Abe: Are you sure you aren't thinking of SUID/SGID?

Abe,

I can't speak to VMS. Don't know anything about it.

However, I'm certain that chroot and jail are not just like "Run As" in Windows.

Quoting:Are you sure you aren't thinking of SUID/SGID?

You are right Gus3, I confused the t & s bits. it was setuid & setgid that I was trying to point to.

T-bit - to pass on inheritance of directory permissions http://en.wikipedia.org/wiki/Sticky_bit

S-Bit - to assign elevated privileges http://en.wikipedia.org/wiki/Setuid

Quoting:S-Bit - to assign elevated privileges

Almost. It assigns alternate privileges, which may or may not be root. It can also be used to reduce privileges.

Quoting:Almost. It assigns alternate privileges, which may or may not be root. It can also be used to reduce privileges.

Gus3, Are we nitpicking here! :)

No seriously, although I appreciate your feedback, my first post was in response to the question whether Linux has a feature to set permissions that would secure/protect objects similar to BSD, not to write full documentation of it. Eventually, it reached to a combination of 3 different methods that you can use to accomplish what is needed. All three can be used and depends which supplements/overrides the others. That depends on the sequence the OS processes them and which overrides the others to create a composite security of all.

Disclaimer: For those who decide to use them, they should better make sure to know their details by referencing the documentation, test them to make sure they work the way they intend them to, before they implement them. They shouldn't take my general description as facts because I did create them and I am not an expert either.

If you notice in my previous posts, I do use amend, restrict, grant access in various places. In the last post, I was just using the same terminology used by Wikipedia, which I linked to.

To come to think about it, although you are technically correct, the S-Bit is mainly/primarily used to grant not restrict. If you were to restrict, you are better off to use G/UID permissions instead. It is simpler and more clear.