I have been attacked and don't know why.

[Ubi]

You can delete all of these safely.

[Torsten]

Thank you, Ubi, will do it right away.

Will it make sense to gather all the various information from the previous posts into a brief how-to-guide on this, assuming that there will be more users discovering that they have been infected? I will be happy to take write the guide, provided that someone will take a look at it before it is released. I assume that most of the attacks have exploited the same vulnerability, and basically it seems that they tried to mine bitcoins and send phishing mails (at least that is what I have found on my box). Also they tried to crack the root pwd, but based on the files placed on my B3 they a) tried brute force instead of the standard pwd, and b) did not even include the default pwd in the password list they used as base. So I assume that this has not been a B23 specific attack.

/Torsten

[Ubi]

as Gordon mentioned before, there are multiple ways these criminals use exploited machines for their profit. A guide is therefore never complete and thus may give a false sense of security. But indeed there are some basic rules of thumb (like searching for cron jobs and files belonging to www-data) that may help for the majority of issues. The most important part is that users patch the flawed software though.

[mintz]

I just found out that my B3 also was under attack. Right away I took precaution and applied some of the quick fixes mentioned here in this thread. Since I don't really know much about server security I would really like to know if these are enough?

[*]Removed execution permission for all the cgi's in the folder.
[*]Disable crontab and shell for www-data
[*]Commented out this in 'default' virtualhost (I don't need any of the CGI so why not):

Code: Select all

   ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
   <Directory "/usr/lib/cgi-bin">
           AllowOverride None
           Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
           Order allow,deny
           Allow from all
   </Directory>

Here's the CGI attack that I just logged. Does this mean that it was returned with code 500 and rendered the attack unssuccesful?

Code: Select all

72.167.113.216 - - [30/Dec/2013:18:38:46 +0100] "POST //%63%67%69%2D%62%69%6E/%70%68%70?%2D%64+%61%6C%6C%6F%77%5F%75%72%6C%5F%69%6E%63%6C%75%64%65%3D%6F%6E+%2D%64+%73%61%66%65%5F%6D%6F%64%65%3D%6F%66%66+%2D%64+%73%75%68%6F%73%69%6E%2E%73%69%6D%75%6C%61%74%69%6F%6E%3D%6F%6E+%2D%64+%64%69%73%61%62%6C%65%5F%66%75%6E%63%74%69%6F%6E%73%3D%22%22+%2D%64+%6F%70%65%6E%5F%62%61%73%65%64%69%72%3D%6E%6F%6E%65+%2D%64+%61%75%74%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%70%68%70%3A%2F%2F%69%6E%70%75%74+%2D%64+%63%67%69%2E%66%6F%72%63%65%5F%72%65%64%69%72%65%63%74%3D%30+%2D%64+%63%67%69%2E%72%65%64%69%72%65%63%74%5F%73%74%61%74%75%73%5F%65%6E%76%3D%30+%2D%64+%61%75%74%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%70%68%70%3A%2F%2F%69%6E%70%75%74+%2D%6E HTTP/1.1" 500 822 "-" "-"

Did we find out why we were targeted in the first place?

[Cheeseboy]

Hi,

[*]Removed execution permission for all the cgi's in the folder.
[*]Disable crontab and shell for www-data
[*]Commented out this in 'default' virtualhost (I don't need any of the CGI so why not)

You also applied the patch right?
500 should mean "Internal Server Error" or similar, so it should have failed.
If you are unsure, look up the corresponding timestamp and IP address in the apache2 error.log.

You do not want to see things like these:

Code: Select all

... Connecting to nn.nn.nn.nn:21...
... connected.
... Logging in as ftp ...
... connected.
... HTTP request sent, awaiting response

What you want to see is something like this:

Code: Select all

<p>This PHP CGI binary was compiled with force-cgi-redirect enabled. ...

As both Gordon and Ubi has mentioned before though: this does by no means guarantee that you are safe.
There are a new exploits discovered all the time...
If you have any services exposed to the internet (or to other machines on your network that are), you will just have to keep an eye on them. Check the status every now and then.
Check for strange processes, unexpected CPU utilization, I/O, network access, etc.

Take the time to look through the logs every now and then (all of them - even the ones you don't know what they are for - if nothing else, you will learn stuff).
If you (like me) use denyhosts, logcheck or similar tools to keep you as safe as possible, do not trust them to catch everything.
Apache ignores hosts.deny for instance. So does proftpd unless you configure it otherwise. logcheck by default only looks at syslog and auth.log.
The built-in web interface doesn't even log failed login attempts...
Got an OpenVPN server? I was amazed at the number of attempts on it the first time I looked at the log file, and I'm not using the standard port or even protocol...

It took me until November 9 to realize something was wrong, even though I read my logcheck digests by email twice a day, even though this thread had been around for more than a week, and the first attacks seem to have started the 2nd. The only reason I noticed was because the darned thing was so slow I had to investigate...

[Gordon]

"500" indicates an "internal server error". In this case this error will be provoked by you marking the cgi as non-executable. It also means that your commenting out the cgi-bin section in one of the apache configs did not have any success at all, because that should have led to a 404, meaning you either did not reload apache or edited the wrong config file.

Yes anything other than 200 will mean that you stopped the attack.

We were not targeted. That is, we as B2|3 users were not specifically selected as such. We just happened to answer to a common vulnerability that hackers are searching for. It's an issue, but it does not call for panic.

[mintz]

Thanks for the very fast replies. You guys are as helpful as always
---
By "patch" did you mean the 2.6 update?
---
Looks like I manage to repel the attack this time.

Code: Select all

[Mon Dec 30 18:38:46 2013] [error] [client 72.167.113.216] (13)Permission denied: exec of '/usr/lib/cgi-bin/php' failed
[Mon Dec 30 18:38:46 2013] [error] [client 72.167.113.216] Premature end of script headers: php

---
I just took a quick look at my access.log again. I don't really understand the first two but from the latest ones it looks like somebody is trying to spool for wordpress.

Code: Select all

41.177.236.72 - - [30/Dec/2013:23:20:53 +0100] "\xd87\xa9\xfd\x8a\xf5\xe0\xc8\x1eC\xed\xd5|\xb4\xda\xfch\x91\xa8m\xf9\xa0^\x1c\xdd\x11\xce\xea\xd8\b\x8b\xb5F\xb9\xdeE\xa69\x03~"$
41.177.236.72 - - [30/Dec/2013:23:21:48 +0100] "\xe6\xf5W\xb5\x12\x03\x01\xdaT\x88" 501 297 "-" "-"
123.49.12.2 - - [30/Dec/2013:23:44:37 +0100] "GET //wp-login.php HTTP/1.1" 404 493 "-" "Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.9.0.15) Gecko/2009102815 Ubuntu/9.04 (jaunty$
123.49.12.2 - - [30/Dec/2013:23:44:37 +0100] "GET /blog//wp-login.php HTTP/1.1" 404 499 "-" "Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.9.0.15) Gecko/2009102815 Ubuntu/9.04 (j$
123.49.12.2 - - [30/Dec/2013:23:44:38 +0100] "GET /wordpress//wp-login.php HTTP/1.1" 404 504 "-" "Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.9.0.15) Gecko/2009102815 Ubuntu/9.$
123.49.12.2 - - [30/Dec/2013:23:44:39 +0100] "GET /wp//wp-login.php HTTP/1.1" 404 497 "-" "Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.9.0.15) Gecko/2009102815 Ubuntu/9.04 (jau$

---
Is there a way to determine if these requests are targeted at my domain or my ip address?

[Cheeseboy]

By "patch" did you mean the 2.6 update?

Check in the web interface, it should say: 2.6.0.1
http://forum.excito.net/viewtopic.php?f=1&t=4654

Yes, someone is looking for a poorly configured wordpress installation.
Happens all the time, and as you can see, it fails (404). If they'd found one installed, the next step would be to try to penetrate it.
You can stare yourself blind at the access log - people are trying all kinds of shit - all the time...

[Ubi]

It's not entirely harmless though, as these probing attacks can be so aggressive they can amount to a dos attack. But then again, so can bingbot...
If you notice bandwidth problems because of these probes, there are solutions like fail2ban that mitigate this problem partially

[Gordon]

Is there a way to determine if these requests are targeted at my domain or my ip address?

Yes and no. For each of your intended domains and vhosts you can set different log files. However that will still not tell you how someone got to your default host, although the probability that they used the IP address has grown significantly. If you're any good with php you can write your own 404 handler (and other errors as well) and grab the header of their requests, which will give you access to the full URI - i.e. method (http/https), server name and page.

fail2ban will work with a relatively small amount of attackers. Because fail2ban adds a rule to your firewall for each attacker, there will be some point where the firewall itself will cause your services to fail with timeouts. A better way of handling this is to use ipsets, which will use a single (incore) database lookup rather than process a bunch of individual iptables rules, but you will need the kernel modules for this and xtables-addons.

[samards]

Hi guys,

I've been attacked a while ago and I did everything that Gordon described (disabled cron, etc) and disabled port forwarding to temporarily put my web site down.

I had an old bubba release - 1.3.1, and tried to update to the new version. It did not go so smoothly - after change distribution like

Code: Select all

change_distribution elvin

I've got an error that "elvin" does not exist...

Anyway, after:

Code: Select all

apt-get dist-upgrade

It managed to update release to the version 2.0.5 That's the best that I can do so far, the system reports that there is no any newer version update...

I have Bubba 2 , and I guess that the newest version is 2.6.0.1. How can I get that version?

[Cheeseboy]

Hi samards,

When they brought the B2 code-line somewhat in-line with the B3 one, they had to do it in a manual install release (2.4.1).
So, in order to get your B2 up-to-date, you have to re-install it using an USB stick. There is no web install.
See: http://forum.excito.net/viewtopic.php?f=22&t=3758

Best regards,

Cheeseboy

[samards]

Thanx Cheeseboy,

I hope I will not have to reinstall everything, as I've made a lots of changes in apache configuration and deployed my web site, where I have the link only to bubba admin...

By the way, is it possible only to install the hotfixes for this php attack? Is there any benefit installing the whole newest bubba version, if I use only administration module?

Best regards,
Sasha

[Gordon]

@samards

Common logic says that if it ain't broken, you shouldn't need to fix it. While that may not be the smartest way of thinking in relation to interactive systems (i.e. a workstation, laptop etc) I think this is particularly true for non interactive type devices.

The thing of course here is that it is broken and that a fix was only released for the latest version. If you know your way around in the linux command line you will not need that fix though, even if that fix does target to keeping the cgi functionality where the solutions offered in this topic will destroy that (unneeded) functionality. In your case I would likely choose not to reinstall.

Disabling cron was actually not my idea. I do in fact not think this counts as a fix, since it only limits the hacker in installing a script that will automatically fire up his script after a reboot. While that may seem like a good idea, it will only keep you safe for maybe a day or so if you don't close the front door.

[johannes]

Ok, final fix released.

http://forum.excito.net/viewtopic.php?f=22&t=5090