Random Sysadmin intrusion stuff and PECL

CS // 11 Apr 2009 // Comments Off on Random Sysadmin intrusion stuff and PECL

One of our clients was sending out spam unknowingly yesterday. I spent most of my afternoon cleaning it up, tracking down how the attackers were doing it.

In this clients case, they have their own server (which we maintain), and they mostly write their own code.
Most of the common garden variety vulnerability scans don’t work on their server, because they write their own code, although in this case it didn’t save them from being exploited.

In order to find out what was causing the spamming, I had to find out how the attackers got in.
Usually this means a check of the apache logs to check for anything untoward.

In this case, although the logs had plenty of vulnerability scans (which were to files that don’t exist on their server), I couldn’t see anything in the logs that immediately stuck out as being the cause.

Thinking that they might be sneaky, I decided to check if any files on the server had anything interesting in them. PHP has a few functions that are a little dangerous, most of these are system related – you can execute other software on the system using them.

As pretty much all the code on this server is php based, I went a little deeper, and did a filesystem scan of some common php system calls to see if anything popped out.

In unix, this is pretty easy

grep shell_exec * -R

A few seconds later, this popped up as a result.

/userpix/1216256595.jpg

Hmm, a jpg file containing shell_exec?
Seems strange.

A quick check of the file in a text editor showed that it was actually a shell script that contained the c99shell.
I soon found another script in that folder with a .jpg extension that contained the r57shell.

Both these scripts allowed for remote pseudo shell access to the server, as well as attempts to further hack the server using common vulnerabilities. They are both quite comprehensive in what they do, and are worthy of further dissection at a later date.

I removed the 2 files (after making copies for further analysis), and let the client know.
A check of the logs revealed that calls were indeed being made to those files.

Sneaky!

Turns out that the client was using a common upload library, and it appears that this had an upload vulnerability.
I advised them to do better error checking, update that library, and installed some additional features for them in their server.

PHP has an extension called FileInfo, which can check files to see if they’re actually what they say they are.
Its not perfect, but it does add a degree of safety for situations like this where things get abused.

Even so, you still need to avoid using FileInfo for checking gif files, as it pretty much assumes any file starting in GIF is a gif.

Sample code to safely check for gif below.

if (!$img = @imagecreatefromgif($uploadedfilename)) {
trigger_error('Not a GIF image!',E_USER_WARNING);
// do necessary stuff
}

Back to FileInfo. The typical way to install Pear extensions is

pecl install

In this case “pecl install FileInfo” wasn’t working, because we set /tmp as non executable for safety reasons.


pecl install Fileinfo
3 source files, building
running: phpize
Configuring for:
PHP Api Version: 20041225
Zend Module Api No: 20060613
Zend Extension Api No: 220060519
/usr/bin/phpize: /tmp/pear/cache/Fileinfo-1.0.4/build/shtool: /bin/sh: bad interpreter: Permission denied
Cannot find autoconf. Please check your autoconf installation and the $PHP_AUTOCONF
environment variable is set correctly and then rerun this script.

ERROR: `phpize' failed

So, we had to install it manually, sigh.

Manual installation of pecl files is simple, although undocumented.

Go to the download folder:
cd /tmp/pear/cache/

Move the file to a different folder with execute permissions
cp Fileinfo-1.0.4.tgz /downloads
cd /downloads

Extract, and install

tar -zxf Fileinfo-1.0.4.tgz
cd Fileinfo-1.0.4
phpize
./configure
make
make install

We also had to add extension=fileinfo.so to our /etc/php5/apache2/php.ini file.

Conclusions for us –

Luckily, they didn’t manage to do any damage except for send mass email.
Tripwire and Tiger (two IDS programs we use) showed that none of the system files on their server were compromised.

We’re planning to further tighten their system with a PHP Hardener called Suhosin, although this depends on their scripts being compatible. We’re currently trialing a Suhosin install with logging enabled to see what may break when its fully enabled.

We use Suhosin on a most of our servers already.
Ironically we had been talking about it with that client when they got hacked!