Run a VPS? You Are Being Hacked Right Now

I get selected portions of my server logs emailed to me for certain servers I run. Normally there’s no need to check them real closely. A quick scan usually shows about one hundred or so break-in attempts along with much of the activities I had run on my own the previous day. Today was no different but for some reason I began to think about all of those server break-in attempts. They’re mundane and just par for the course in my eyes but what about all of those server newbies out there (like I once was) who think that their immune to hacking either because they think they’re too small to be discovered/cared about or they’re lulled into believing secure passwords are all one needs. Here’s a quick reality check to remind you how many people are trying to break into your server for one reason or another and what you can do to prevent it.

These people are trying to get in

First up, here’s a partial list of all the potential people who are trying to break into just one of my servers. The site(s) that run on this server are very specialized but not none of them have really launched yet. So how in the hell am I already getting break-in attempts on a 5 week old server with a non-publicized site that gets 95% of its visits from the people developing it?

The simple answer is that people are out there just scanning IPs and ports trying to get into servers for any number of reasons. It doesn’t matter why, exactly, they want to get in. What matters is that you probably don’t want anyone but those you have authorized getting into your server. So here’s that list of IPs that have tried to break into one of my servers in the past 24 hours:

1
2
3
4
5
6
7
8
9
10
11
12
Couldn't resolve these IPs:
    121.243.89.78.static-bangalore.vsnl.net.in [121.243.89.78]: 49 Time(s)
 
 Didn't receive an ident from these IPs:
    1.93.24.90: 1 Time(s)
    109.68.190.145 (vps.node71.doip.net): 1 Time(s)
    121.243.89.78 (121.243.89.78.static-bangalore.vsnl.net.in): 1 Time(s)
    130.88.148.85 (rtt20-pc.mt.umist.ac.uk): 1 Time(s)
    157.7.213.160 (v157-7-213-160.myvps.jp): 1 Time(s)
    193.107.16.206: 1 Time(s)
    200.54.114.147: 2 Time(s)
    23.100.116.48: 1 Time(s)

Okay, not bad. That’s just 12 lines of text. Whatever, right? Okay, fair enough. Regardless, that list is only of IPs that did not successfully identify themselves or resolve properly when they tried to connect to the server. They may have been spoofed or who knows what. The scary part isn’t who we know tried to break in, it’s the untold number of those we don’t. So here’s another list. This time of usernames attackers have used to break into, again, just one single server that’s relatively new online and barely publicized.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
undef: 64 times
test: 10 times
user: 7 times
oracle: 5 times
admin: 3 times
guest: 3 times
info: 3 times
postgres: 3 times
usuario: 3 times
testing: 2 times
username: 2 times
webadmin: 2 times
bash: 1 time
dasusr: 1 time
dasusr1: 1 time
fptuser: 1 time
ftp: 1 time
ftpuser: 1 time
henry: 1 time
linux: 1 time
maggie: 1 time
operator: 1 time
phpl: 1 time
roto: 1 time
server: 1 time
simon: 1 time
soporte: 1 time
trixbox1: 1 time
user1: 1 time
user2: 1 time
user3: 1 time
user4: 1 time
xbox: 1 time
130.88.148.85 (rtt20-pc.mt.umist.ac.uk): 43 times
user: 5 times
oracle: 3 times
test: 3 times
usuario: 3 times
testing: 2 times
username: 2 times
webadmin: 2 times
admin: 1 time
bash: 1 time
dasusr: 1 time
dasusr1: 1 time
ftp: 1 time
guest: 1 time
henry: 1 time
info: 1 time
linux: 1 time
maggie: 1 time
operator: 1 time
phpl: 1 time
postgres: 1 time
roto: 1 time
server: 1 time
simon: 1 time
soporte: 1 time
trixbox1: 1 time
user1: 1 time
user2: 1 time
user3: 1 time
user4: 1 time
xbox: 1 time
157.7.213.160 (v157-7-213-160.myvps.jp): 1 time
admin: 1 time
200.54.114.147: 14 times
guest: 2 times
info: 2 times
oracle: 2 times
postgres: 2 times
user: 2 times
admin: 1 time
fptuser: 1 time
ftpuser: 1 time
test: 1 time
222.219.187.9: 6 times
test: 6 times

80 different random people or bots have used those usernames to try to gain access to this one server. Remember again that this server has no websites indexed by Google yet and is brand new. I can show you logs with many hundreds of lines of these failed attempts to gain access to a server I run.

Take precaution

There are a handful of basic, well regarded measures you can take to secure your server from these attackers. If you have a server connected to the internet, you can’t stop randoms from trying to break in but you certainly can stop them from succeeding at it.

Disable root login

We’ll start off with the simplest thing you can do to minimize the damage an attacker can do on your system. Most VPSes will drop you straight into the root account when you spin up a server. It’s your responsibility to use that root user to create any other single user accounts, give them sudo privileges when necessary, then log out and log back in with your own user account and disable root login. Here’s how to do that on Ubuntu.

To add a new user

1
2
adduser you
usermod -a -G sudo you

To disable root login you’ll need to open up your sshd_config like this sudo nano /etc/ssh/sshd_config.

Now find the line that says PermitRootLogin and change the yes to no. Simple. Now no one can log (or attempt to log in) using the root user. The really dangerous thing about root is that everyone knows it’s a user and if you’re not throttling login attempts then an attacker has unlimited chances to try to brute force your root user’s password.

Disable password authentication

This is the single best thing you can do to protect yourself from brute force attacks. Disabling password authentication means that when an attacker tries to SSH into your server, regardless of whether or not they know a user’s name and/or password, the server won’t even prompt them for a password. Instead it’ll expect an SSH key to be sent with the request (which SSH does for you once set up). Using this method of authentication, you can only connect to your server if you have a public/private key-pair on the computer you’re using that the server knows about.

Here’s how to disable password authentication and use private key authentication instead.

  1. Generate a public/private key

If you’ve already done this for another site or service (like GitHub maybe?) and you want to use the same key, skip to the next step.

Run these commands on each computer you want to connect to your server from to generate a new key:

1
ssh-keygen

It’s going to ask you for some information including a password. If you include a password, you’ll be prompted for it each time you try to connect to the server. This password is connected to the key you are generating. Some people say this defeats the purpose of passwordless login. It doesn’t. The purpose to using key-pair authentication on a server is to ensure that only users who have the correct private key are able to connect to the server at all. Sure, you still have to enter a password but it’s still a huge extra layer of security. Anyone trying to connect without the private key will simply be denied access outright without the chance to even attempt a password. Key-pair authentication isn’t about convenience, it’s about security and once set up you’ll continue on entering a password like you’re used to.

On the other hand, if you choose to leave the password field blank, you still get a secure connection and many of the benefits of key-pair auth but if your private key is leaked, your computer is stolen, or even used by someone while you’re in another room, they can break into your server more easily than if password authentication were enabled.

Once the ssh-keygen utility runs you’ll have a public and a private key file stored in ~/.ssh. Now you need to upload the public key from each computer you generated a key on to your server. Do this:

1
scp ~/.ssh/id_rsa.pub you@yourserver.com:~/

This uploads the public key to the home directory of your user on the server. Now you’ll need to move it into place. To do this, SSH into your server again and run the following…

1
2
3
4
5
6
7
mkdir ssh # makes a new folder to store your public key
mv id_rsa.pub .ssh/authorized_keys

# Now make sure to give the files the proper permissions
chown -R youruser:youruser .ssh
chmod 700 .ssh
chmod 600 .ssh/authorized_keys

Now all you need to do is test it. Log out of your server and try to log back in. If your SSH key has no password you’ll be let right in, no questions asked. If your key does have a password attached, a system dialog box will pop up asking you to type it in. If the system dialog pops up, that means it’s working.

Once you verify this setup is working you’ll want to disable password authentication completely. Open up your sshd_config file again and look for the line that says PasswordAuthentication.

1
sudo nano /etc/ssh/sshd_config

Change the “yes” to a “no” and then restart the SSH daemon by entering sudo service ssh restart. You’re all set now.

Note: If you plan on connecting from more than one computer (like in cases where you’ll be working on your laptop, work maching, or home computer) you’ll want to upload your public key to the server before disabling password authentication.

Change the SSH port

You can do this if you want but I don’t recommend it. It’s debatable whether or not this is helpful for security. I call it security by obscurity. Sure, it’ll deter a lot of the less sophisticated attackers but those who are targeting you will just start scanning for open ports and try them all. I’m on the side that argues that changing the default SSH port is more trouble than it’s worth especially once you start using software that assumes it’s running on the default.

Install Fail2ban

Fail2ban is a little daemon that will monitor failed login attempts and block IPs after a specified number of them. Every server should have Fail2ban installed. Get it with sudo apt-get install fail2ban.

Create a Firewall

You need to make sure connections can only be made on ports that are meant to be connected to. That’s that five-year-old way of explaining why you need to implement a firewall on your server. Luckily, Ubuntu comes with everything you need to set it up from the start. All you need to do is a little configuration. Since this requires quite a bit of code (don’t worry, you can just copy and paste all of it) I’ll direct you to Linode’s security guide which does a great job of explaining how to get a basic firewall up and running in no time.

Do these things and you will be a step ahead of most people. You’d be surprised how many servers are sitting out there begging to be broken into while such simple solutions exist.

Server administration, Web development

« SSDs Fail. Hard Drives Fail. Back up your stuff now. Roll Your Own oAuth if You Dare »

Comments