Setting up Security Onion to Enhance Home Security

Hello everyone! I'm back after being busy for so damn long with some contract stuff. Now that I have some time off (for once), I should be able to post more actively. For my next few blogs I have planned I'll introduce Security Onion and Buffer Overflows. Look out for the Buffer Overflow tutorial tomorrow!

In this guide I'll detail setting up Security Onion in a typical home environment.

So what is Security Onion? It's a repo list for Ubuntu (or a standalone ISO of 14.04 LTS) that allows you to turn any 'ol Ubuntu VM into a badass network forensics tool, SIEM, and IDS. And yes, it will even send you alert notifications! It's pretty easymode to setup and if you're already rocking a home lab you're almost good-to-go.

An example of some fancy dashboards you can create with ELSA

An example of a fancy dashboard you can use for Bro

First and Foremost

You'll need a switch capable of port mirroring. If you do not have this, there's pretty cheap ways to obtain one. You can pick up used Cisco managed switches on Craigslist for next to nothing.

fig. 1: Case-and-point. SPAN capable switch, $50 dollary-doos

Here is a basic home network setup with a lab that is pretty common:

fig. 2: Your run of the mill home network

Simple enough, we have the EdgeRouter handling DHCP on the internal interface and NAT to the outside world on the external interface, that way we have 1 vlan (vlan1) and 1 address range (10.1.5.0/24) on the inside of the network. All traffic going into or out of your network will move over that switch. Every network is slightly different. But let's assume this is what the network for this tutorial looks like, since it's nice and simple.  

The physical connections on the hypervisor look like this:

fig. 3: Physical Interfaces on the XenServer

The virtual interfaces on the XenServer look like this:

fig. 4: Virtual Interfaces on the Guest OS and the XenBridges to physical NICs on the Host.

The Initial Setup

First, perform a fresh clean install of an Ubuntu 14.04 LTS Desktop on your Citrix XenServer (or other virtualization platform). If you're using dedicated hardware setup the box with Ubuntu 14.04 LTS Desktop if you want a GUI or Server if you want to do everything from the CLI - note you will need to have x11forwarding enabled in sshd_conf. A lot of the Security Onion setup scripts are GUI.

The ISO is available here:

http://releases.ubuntu.com/14.04/ 

Install the Guest OS

Once your server is installed and boots up for the first time., let's setup some basic configuration 

before we install Security Onion.

Insert the XenTools ISO disk to the drive of the VM, so we can install it.

Run the commands:

sudo mount /dev/cdrom /mnt

cd /mnt/Linux

sudo ./install.sh

And hit 'y' to confirm the install.

While waiting for that to install, update Ubuntu:

sudo apt-get update && sudo apt-get upgrade -y && sudo apt-get dist-upgrade

Install OpenSSH server to allow incoming SSH connections:

sudo apt-get install openssh-server

The default resolution in most XenCenter VMs are crap. Trying to work with 800x600 resolution is like looking through a submarine porthole. Let's increase the resolution of the VM:

sudo apt-get install xvfb xfonts-100dpi xfonts-75dpi xfstt

sudo nano /etc/default/grub

Press CTRL+W to search for the string below:

GRUB_GFXMODE

Set the line to the following. You can change the resolutions below to suit your preferred order. The first entry will be used on default. Don't forget  you can set it pretty high and then just click the "scale" option in the console window of XenCenter.

GRUB_GFXMODE=1280x960,1280x800,1280x720,1152x768,1152x700,1024x768,800x600

The add this underneath that line:

GRUB_PAYLOAD_LINUX=keep

Save the config file with CTRL+X and select "Y" to confirm changes.

Update Grub:

sudo update-grub

Now reboot once the updates are all finished installing. 

Installing Security Onion

Always consult the guide if you get lost:

https://github.com/Security-Onion-Solutions/security-onion/wiki/InstallingOnUbuntu

First, clean the apt repository:

sudo rm -rf /var/lib/apt/lists/* && sudo apt-get update

Next, install the Security Onion packages and repository:

sudo apt-get -y install software-properties-common

sudo add-apt-repository -y ppa:securityonion/stable && sudo apt-get update

You will be prompted to enter the MySQL root password. Make sure to select a secure one.

When that is finished you need to configure MySQL to not prompt for the root password:

echo "debconf debconf/frontend select noninteractive" | sudo debconf-set-selections

At this point Security Onion is installed but we need to do some configuration on the hypervisor.

Configuring Auto-Start on the Host

The point of having IDS is to monitor your network 24/7/365. IDS isn't doing you any good if the power goes out and comes back up and your VM is sitting there offline, now is it? Below I'll detail how to set your VM to start automatically as soon as the host loads XenCenter.

Get the UUID of your Storage Repository:

xe pool-list

Get the UUID of your Security Onion VM:

xe vm-list

If you're not sure which UUID to use, it's that top entry:

fig. 5: You want the top UUID

Set the pool to allow auto-poweron:

xe pool-param-set uuid=SRUUIDHERE other-config:auto_poweron=true

Set the VM to start when the Host OS starts:

xe vm-param-set uuid=VMUUIDHERE other-config:auto_poweron=true

Security Onion Setup - First Run

So now we're finally ready to setup Security Onion! Yay!

fig. 6: Let's break out the confetti

The Security Onion setup script, sosetup, has to be run twice. The first time will do some basic configuration and require a reboot. Then we will run it again after I demonstrate some quirks with XenServer and mirrored port traffic.

To run the script, simply type:

sudo sosetup

Follow the prompts to set the interface based on your particular configuration.

My sniffing interface is eth0 (detailed above in fig. 4). 

fig. 7: The first time sosetup runs, it will prompt to set your interfaces

My management traffic interface is eth1 (again, refer to fig. 3 and 4 for my configuration)

When the sosetup script finishes running, it will prompt you to restart.

DON'T run the sosetup script again on reboot yet.

You want to check and verify you can see all the traffic from your other devices on your monitor port.
When the OS loads back up login and open wireshark:

sudo wireshark

Set the interface to sniff on our monitoring interface (the interface that the switch is mirroring traffic to). 

Then, from your 1337 gaming rig, or your girlfriend's shitty malware infested laptop, ping your gateway.  

fig. 8: But nothing shows up on the monitoring interface! Oh noes!

Damn! No traffic. There's some reasons and things for this. I will detail them in the next section, and how to resolve them. If you're running dedicated hardware you should see the traffic that's on your internal vlan here.

Setting XenServer to Mirror Physical NIC Traffic

So we can't seem to get our Security Onion VM setup to receive packets that are sent to the physical interface from the port the switch configured to mirror on. 

No big deal, we just have to understand a bit about XenServer's internal networking. XenServer runs Open vSwitch. The main command to interact with this is below:

ovs-vsctl

There's a ton of syntax there, but first let's reference back to our setup:

fig. 9: The basic network architecture of our XenServer.

So we can see here that we will need to forward traffic from the physical NIC (eth1), to the XenBridge Open vSwitch port (xenbr1). Easy as flipping xenbr1 and eth1 into promisc mode and some fancy iptables, right? Well not so much. I wasn't able to get it working that way. The xenbr1 interface kept dropping the traffic. However, since it is a switch, we can set it to do port mirroring from the physical NIC straight to eth0 on our VM.

I have created a script specifically for this:

https://gist.github.com/BraveLittleRoaster/ec99043a55747e7ad64e7021a95f64eb

I'll walk you through it in a minute. For now copy it to this directory:

mkdir /etc/startup_scripts
cd /etc/startup_scripts
nano port_mirror.sh

Edit the script above with your interfaces and vm name-label (line 11), paste it into this file, and save.
Set the script to executable:

chmod +x port_mirror.sh

Set the script to run at startup:

nano /etc/rc.local

Add the following lines:

# Set the Security Onion VM to get mirrored port traffic
/etc/startup_scripts/port_mirror.sh

I'll explain a bit about how this script works now.
First, on line 6, we define the physical NIC that our managed switch is sending traffic to.
Then, on line 8, we define the Xen Bridge that this interface is mapped to.
If you're unsure you can check by running ifconfig and matching the MAC addresses.

fig 10. My eth1 interface.

ifconfig shows that eth1 matches the xenbr1 network:

On line 11, we query the server for the Security Onion name. You will have to edit this to match your name-label. If you don't know what it is, run the command:

xe vm-list

And find the name-label value there that you set for your VM when you built it.

Line 12 is more important than it looks. This will be the value passed to the ovs-vsctl command. It is the virtual interface label for your VM. The command 11 returns a numerical value that is then used to identify the domain ID of your virtual interface. If you have no idea what I'm talking about or what it is, I'll explain it real quick. On your Xen Server host, run ifconfig, and look for an interface named "vif"

fig. 11: My virtual itnerfaces.

These "vif", or virtual interfaces, are the actual interfaces of the VM. It might be easy to figure out at a glance which VIFs belong to which VM... But if you're running dozens of VMs, each with 2~4 interfaces, you're going to have a ridiculous amount of interfaces to comb through. Instead you can list the domain ID of each VM with this command:

xe vm-list params=dom-id,name-label

fig. 12: The current domain id of my Security Onion VM.

Here you can see the domain ID. But what about the ".number" you see tagged on the end of the vif?
Well that's the actual interface number of your VM! For example, my VM listens on eth0. So I need to be sending traffic to vif11.0
It's as easy as that!
So, you will wanted to edit line 12 of the script to represent the interface number you configured security onion to listen on For example: $DOM.0 for eth0, $DOM.1 for eth1

Lines 15 & 16 will set those interfaces to promiscuous mode, which will allow it to receive traffic that isn't destined to reach your MAC or IP without dropping it. Super important if you're cloning the traffic on your entire network and sending copies of it to 1 specific interface.

Line 19 is where the magic happens. It will first select the bridge, then create a mirror for the monitor interface. It will then set this interface to point the traffic to the domain ID of your VM. It will then instruct to mirror the traffic.

After making your changes to the script, run it, and let's test it out by pinging the gateway again from another machine.

Fig. 13: Now you can see your roommates or SOs porn browsing habits! Yay!

It works, but there's a problem! Those of us intimately familiar with XenServer will know it increments the domain ID every time that VM is restarted. For example, I'll restart my Security Onion VM.

Fig. 14: We can see the domain ID has incremented.

With these incrementing, that means we'd have to runt he port_mirror.sh script every time this restarts. So if the VM decides to restart itself for whatever reason, or if you trip over the power cable to your server rack while trying to get the Halloween decorations out, then you'll need to log in and rerun all these damn scripts. It won't hurt anything to have it run here and there. Set a crontab job to run it every 10 minutes. In the terminal at XenCenter run these commands:

sudo crontab -e

It opens a vi window, so press the INSERT key and paste this line:

*/10 * * * * /etc/startup_scripts/port_mirror.sh

Then press ESC and type !wq and hit enter to exit vi.

Verify everything turned out alright with:

crontab -l

Sweet. Now if the host restarts or the VM itself restarts, we will still be able to sniff traffic. See if you think of a better way to keep packets flowing to Security Onion. Whenever it's not monitoring, you're in a blind spot!

Setting up Security Onion - The Second sosetup run

Now that we have a consistent inflow of traffic on our listening interface @ eth0, we can move forward with finalizing our configuration of Security Onion. Rerun the sosetup scirpt:

sudo sosetup

Select Yes to skip the network configuration, and move on with the next phase of the install.

Select Production Mode:

Select standalone:

Select custom. 

Specify a username and password, then confirm the password.

Specify the amount of days you wish to retain logs. 30 is good 'nuff for me. 

Specify the number of days you want to rerun database repair. 7 is fine for me.

Select your preferred IDS. Suricata is my poison. 

AND STOP!

I'm guessing you didn't get an oinkcode yet :)?  

You: "What the fuck is an oinkcode?"

An oinkcode is an API for pulling down IDS rules. You need a continuous feed of threat intelligence for IDS to be any good.
So lets go get one!

Enroll here:

https://www.snort.org/users/sign_up

You'll need to confirm your email address. Don't forget to do that. Once you've done so, login to snort.org, and click your account name in the top right. 

Then, on the left click on "oinkcode"

Click that thing and it will appear to the right.

Now go back to the Security Onion VM and select "Snort VRT ruleset and Emerging Threats NoGPL Ruleset" and then paste in your oinkcode API key on the next window. It is up to you whether or not you want to maintain a paid or free subscription. I chose to use a free subscription. 

Fig. 15: Paste your oinkcode

Select the number of PF_RING slots

Your monitoring interface should already be selected. If not, select your monitoring interface.

Yes, enable the IDS engine.

Select the number of IDS instances that are recommended.

Set the subnet ranges of your internal network that the port mirroring is sending to your monitor interface. 

Yes, enable Bro

Yes, enable EXE extraction. (I will demo this feature in my next post)

I am going to be running primarily ELSA. To avoid duplication, I'm selecting NO.

I am also going to be running Bro for the session data. I'm choosing NO to disable Argus.

Again, disabling because I'll be using Bro.

Yes, enable full packet capture

Set your pcap file size. The default size of 150mb works for me.

Enable mmap I/O. You may chose not to do this if you don't have a lot of RAM dedicated.

Set PF_RING buffer size. I chose to set this to 512 mb.

Set the log purge threshold. This will purge your old logs once the disk reaches x% of utilization. If you find that you can't keep the full amount of log files you specified (in this case 30 days) before being purged, then increase disk storage or lower the number of days you wish to keep logs.  

Yes, enable ELSA.

Set how much storage ELSA will use. I chose to use the default of 50% of the allotted disk space. 

Confirm or discard changes, and fix anything you're not sure about.

Now wait for the setup to complete. If it appears hung, be sure to check on the terminal window you called the sosetup script from. It might be prompting for input for a password somewhere.

Be sure to read the next 9000 pop ups and record those directories in a text file so you don't forget.

Final Steps!

And that's all there is to it! Now you can really see what kind of dark stuff is lurking in your network.

If you noticed they locked down the ubuntu firewall to allow only port 22. You will want to open up your Security Onion instance so you can reach ELSA from a machine on your network. Use the so-allow script for this:

sudo so-allow

Press 'a' to specify an analyst machine, then type in the IP of your system and hit enter to confirm the changes.

Use so-allow to open the ufw to allow syslog devices and your remote machine to connect to view ELSA and Bro logs.

The desktop will now contain links to the ELSA page and other useful things. Be sure to add those as bookmarks to your analyst machine that you allowed with the so-allow script. 

You might want to login to your router, DOCSIS device (cable modem), managed switch and other servers and set them to forward their logs to your syslog service listening on your security onion VM. 

You can also do this with so-allow. Just select the 'L' option and specify the IP.

OSSEC

You also might want to install the OSSEC HIDS agent on all of your systems. I will detail this below.

First, grab the agent here:

https://ossec.github.io/downloads.html

They have it for damn near everything.

On Security Onion, run the so-allow script and use option 'o' to allow an OSSEC agent in through the firewall.

Then change to a root shell, and run the agent manger:

su

/var/ossec/bin/manage_agents

Select option "A"

Set a name for the agent, and the IP address it's connecting from.

Set an ID. I usually leave it blank and hit enter so it will automatically set the ID numbers for me.

Now, select option "E" and specify the ID. Be sure to set the full ID number (mine was 001 in this case). 

Copy that key, we'll need it later.

Download and install the OSSEC agent for your OS, then run the OSSEC Agent Manager as a privileged user.

Set the OSSEC server IP, and paste the key you got from the previous step.

Then click save.

Repeat for all your other devices that they make an agent for.

If you want to know more about the OSSEC agent, and adding or configuring them, check the documentation here: 

https://ossec-docs.readthedocs.io/en/latest/manual/agent/agent-management.html#managing-agents

THE END

Welp, you've made it this far. Take some time to really review security onion and learn the tools that are on it. It will make you that much more aware on what's going on in your network, and how often it's attacked. My personal home network usually gets several thousand attacks a day. To see traffic coming into your WAN, try setting a sensor in front of it - or running TCP Dump and pulling it into Bro every X minutes/hours (this is what I do). This is pretty typical traffic coming from a WAN interface. 99.999999% of this traffic is fairly harmless and will be dropped. They're mostly robo-scans and other malware traffic from elsewhere on the internet. Typical things like port scanning, banner grabbing, etc. Every once and a while though you'll see a real nasty try to hit you or a DoS. Now you'll be aware and can shut that down.

Tuning

There's a big 'now what' for those that aren't security experts who are used to dealing with this insight and network traffic. At first, your alarms will be going off fairly regularly. The default thresholds and rules will overtrigger on some events, and there will be plenty of false positivies as well.

Since every network is dramatically different in terms of traffic, there isn't much I can tell you on what to tune on your specific network (only you'll know the answer to that), but thankfully there are plenty of resources out there that detail better than I could on tuning. 

Check out some of these resources for more information on tuning Security Onion to meet your needs:

SECURITY ONION: SUPRESSING , DISABLING NOISY SNORT RULES

FINE-TUNING SNORT RULES IN SECURITY ONION

ManagingAlerts

You might need to add more cores or RAM to your VM. Mine stays pretty busy with 6 cores and 7gb of RAM. 

Try messing with things, and like always, RTFM:

https://github.com/Security-Onion-Solutions/security-onion/wiki/PostInstallation 

They have an excellent wiki, and I recommend checking out the above link to get started.

Happy Blue Teaming!!

Troubleshooting

I thought I'd do a quick edit. I was having trouble getting ELSA to start. Nothing I tried was working. To be honest, the fastest fix is to reformat the image with a clean install of Ubuntu and follow the first few instructions in the official guide here:

https://github.com/Security-Onion-Solutions/security-onion/wiki/ProductionDeployment 

What I did after is instead of selecting "custom" I opted for "best practices" and later disabled the things I wouldn't use. Remember, you can run sosetup as many times as you please. It will just overwrite all of the database data as soon as you confirm the information at the end. In my particular case ELSA had some kind of database authentication issue I wasn't able to easily resolve.