I have been attacked and don't know why.

[Gordon]

Nope. Those three(!) instances of /usr/bin/php-cgi running as root are started by the bubba-admin service. They serve the fastcgi method that allows you to control the B3 through the web interface.

[Puma]

Ubi,

Thanks for your fast reply.

After a restart of apache I have:

Code: Select all

root     28762  0.2  2.7  87568 14396 ?        S    00:43   1:20 /usr/bin/php5-cgi
root     28831  0.2  2.8  87568 14800 ?        S    00:43   1:19 /usr/bin/php5-cgi

Puma

[gif]

Grr. Only now I noticed this, and it seems that I've been infected too, ever since 31.Oct according to logs. And I've just been lazing on the sofa bitching about the bandwith of my ISP, not bothering to check the traffic .
Thanks to everyone for all the help and pointers in this thread, maybe now I'll find time to show more love to my firewall configs.

[jonte]

But I also searched for files owned by www-data to see if there was anything else besides the files in /tmp, and the only suspicious thing I can find is /var/lock/ttoy, a directory with a hidden subdir called ".m" and another subdir called "c". No files though, but maybe that's because whatever created the lock dir/files has been killed off.

Code: Select all

root@b3:/var/log# ls -laR /var/lock/ttoy/
/var/lock/ttoy/:
total 12
drwxr-xr-x 3 www-data www-data 4096 Nov 10 23:37 .
drwxrwxrwt 6 root     root     4096 Nov 12 23:19 ..
drwx------ 3 www-data www-data 4096 Nov 10 23:37 .m

/var/lock/ttoy/.m:
total 12
drwx------ 3 www-data www-data 4096 Nov 10 23:37 .
drwxr-xr-x 3 www-data www-data 4096 Nov 10 23:37 ..
drwxr-xr-x 2 www-data www-data 4096 Nov 10 23:37 c

/var/lock/ttoy/.m/c:
total 8
drwxr-xr-x 2 www-data www-data 4096 Nov 10 23:37 .
drwx------ 3 www-data www-data 4096 Nov 10 23:37 ..
root@b3:/var/log#

Anybody have a comment on /var/lock/ttoy? My google-fu gives me nothing at all on it.

I found a hidden /.m/ folder in /var/tmp/ which contained a floodbot shell script. Updated to 2.6.0.1 and deleted the folder.

//Jonte

[Gordon]

The methods differ. I have been monitoring attacks for about three weeks now and in most cases the attack script is deleted right after being loaded to memory. There does however appear to be a common factor and that is that they all open an IRC channel to some remote host (which is also a hacked machine).

Things to watch for:

1. High CPU utilization (your B3 will appear very slow) - this means that your machine is being used to attack others
2. An open connection between your B3 and some unknown other host - check established connections with netstat
3. Any port listening above 1023 that doesn't match the service it is bound to - verify with `netstat -tulpn` - this may mean that your machine is used to control other victims
4. Unknown files in your /home/web folder - check with `ls - sla` (one of those f*ckers puts his attack script in a folder named '...') - this means that your webserver is probably hosting the actual shellbot (common names are "a", "c", "lol", "unix", "robot.txt", "zap" and "zmuie")

[toukie]

There are still tons of "POST /cgi-bin/php?%…........." entries in Apache logs.
I have Nginx in front of Apache so Apache access log shows every entry as 127.0.0.1 and I can't use this script there: https://calomel.org/web_server_abuse_detection.html

The problem is that the needed libapache2-mod-rpaf-version from apt-get is for a later version of Apache. How do I get Apache to show real IP's under this set-up?

[Gordon]

That is weird... Are you bypassing the Excito repo? I have libapache2-mod-rpaf version 0.5-3+squeeze1 installed, which requires apache >= 2.2.16-6+squeeze7 and the installed version is 2.2.16-6+squeeze10

I created the conf file /etc/apache2/conf.d/nginx-remote-address:

Code: Select all

RPAFenable On
RPAFsethostname On
RPAFproxy_ips 127.0.0.1 192.168.1.254

(I have 1 B3 running Nginx and 1 B3 running Apache)

[toukie]

Well, the problem is that I know very little about computers, so I don't know for instance why I get this when I try to install mod_rpaf with apt-get:

"The following packages have unmet dependencies:
libapache2-mod-rpaf : Depends: apache2-api-20120211
E: Broken packages"

I have nginx version: nginx/1.4.4

It may be that I am bypassing the Excito repo but I don't know how I got there and why.

By the way, mod-rpaf has vulnerabilities, but I guess they have been taken care of by now: http://www.websecuritywatch.com/cve-2012-3526-mod_rpaf/

[Gordon]

Can you post the output of

Code: Select all

aptitude show libapache2-mod-rpaf

[toukie]

aptitude show libapache2-mod-rpaf
Package: libapache2-mod-rpaf
State: not installed
Version: 0.6-11
Priority: extra
Section: httpd
Maintainer: Sergey B Kirpichev <skirpichev@gmail.com>
Uncompressed Size: 68.6 k
Depends: libc6 (>= 2.4), apache2-api-20120211
Description: module for Apache2 which takes the last IP from the 'X-Forwarded-For' header
rpaf is short for reverse proxy add forward.

rpaf is for backend Apache servers what mod_proxy_add_forward is for frontend Apache servers. It does
exactly the opposite of mod_proxy_add_forward written by Ask Bjorn Hansen.

It changes the remote address of the client visible to other Apache modules when two conditions are
satisfied. First condition is that the remote client is actually a proxy that is defined in httpd.conf.
Secondly if there is an incoming X-Forwarded-For header and the proxy is in its list of known proxies it
takes the last IP from the incoming X-Forwarded-For header and changes the remote address of the client
in the request structure.
Homepage: http://stderr.net/apache/rpaf/

[Gordon]

Okay, so it seems you are using a non standard repo. I don't see that version anywhere on my B3

Try this to list available versions of libapache2-mod-rpaf and heir dependencies

Code: Select all

apt-cache showpkg libapache2-mod-rpaf

You can install a specific version by running (e.g. version 0.5-3+squeeze1 - the one I have)

Code: Select all

apt-get install libapache2-mod-rpaf=0.5-3+squeeze1

[toukie]

Thanks Gordon!

Now I got mod-rpaf installed. With your conf file it works and shows real IP's in Apache logs. Hopefully the version 0.5 has been fixed so that it is no security risk anymore. Now I'll see what the script can do for to tidy up my logs!

Is it right that the second IP in the config should be my LAN IP or is it the servers LAN IP?

[Gordon]

Hi Toukie,

The second IP is because I have Nginx running on a different B3 than the one that is running Apache. The server I have Apache on has IP 192.168.1.253

I don't really know which vulnerability you're referring to. According to the docs I could find the stable version 0.5-3+squeeze1 of mod_rpaf contains a fix for some kind of DDoS. The same docs state that the 0.6 versions of mod_rpaf are (test) versions for Wheezy BTW.

[toukie]

OK, now I have 127.0.0.1 and my server IP in /etc/apache2/conf.d/nginx-remote-address.
The vulnerability is said to have been fixed in both versions 0.5 and 0.6. The Web Server Abuse Detection Script from Calomel.org works fine. I use it instead of fail2ban.I never got fail2ban to work properly.

[samards]

Hi guys,

I'm new here although I have bubba 2 a long time, since 2009. I've got no any problems since, until a few weeks ago , when I've noticed significant fall in my internet speed... I've seen a lots cron entries from www-data but I thought that should be so... but... running netstat gave me a lots of strange connections, mostly from China and south Korean IPs. Some american as well.

So, I'm infected as well. I could not find the better source of information on internet than this! So I followed some of your instructions, like

Code: Select all

find -user www-data

, and found strange directory under /var/tmp. The directory name was .,, inside was a lots of scripts, and "update" as well which was run by cron job.

Also, some empty directories under /tmp, like ".ITP_unix" or similar... deleted those as well. Removed cron job from www-data.

But, I still have an old bubba release, I guess something like 2.5 or so. So, I guess my next step wouold be to upgrade to 2.6. I'm a little worried , because I've changed my apache configuration to suit my needs, put my wiki web site, and left only admin bubba application (not using anything else from bubba).

Would upgrade change my apache configuration and mess with my current web site? And is the newest upgrade enough to protect from this kind of attack?