Category Archives: hardware

Homelab isolation with pfSense

If you are continuing on from my earlier homelab post, I mentioned that doing proper network isolation is very important when working with malware samples or doing any kind of reverse engineering work. There are other practical reasons you may want a separate network for your virtual machines, a good example is building your own Active Directory Domain and Domain Controller(s).

Like the first post, this contains some tips that I wish I had known starting out. I hope that it helps you out. This guide is very similar to pfSense’s virtualization guide, but with some additional commentary around InfoSec specific labs/virtualization.

Note: We are only going to discuss virtual IPv4 networking, IPv6 is disabled on my pfSense interfaces, but link-local addresses are still enabled for compatibility.

Moving from VMware Workstation/VirtualBox to Type 1 Hypervisor

When you move from VMWare Workstation / VirtualBox / Parallels (Type 2) to something like ESXi, you may notice that some key networking options have disappeared. The thing I neglected to take into account was NAT’d networking options

This is what you are probably used to

Notice when you create a new VM in ESXi, the default network connection for a VM is going to be bridged and your VM will grab a DHCP lease from your local network if available.

VMkernel / Virtual NICs.

So how do you get around this constraint and into something more familiar? The answer is Virtual Switches and Port Groups!

By default ESXi, has a vSwitch0 that is tied to your management network interface. You can create a second vSwitch that does not have an uplink to another network connection. Below is an example

Look at the vSwitch topology, no physical adapters or other uplinks!

You can see some sample options and that my virtual machines are not connected to a physical NIC or other interface that gives access to my local network. This vSwitch1 will become my isolated “Internal Lab Only” network. You can do this by associating this virtual switch with a “Port Group”.

This is all fine and well, but how am I going to get routing/DHCP/NAT (optional) and all of the other cool stuff that my Type 2 Hypervisor did for me?

Enter pfSense

You don’t have to use pfSense, but you need something that allows for proper routing and a firewall (the most hardcore will use iptables). pfSense is a really good option as it has a large community of users, is well documented, and is open source.

Since the intention of this post is isolating a homelab, I am not going to cover the installation process in detail. You can find that information via the Google or just use the official guide. I do suggest you double check your MAC addresses as you build your WAN and LAN interfaces to make sure they are correct.

Here is what I wanted to accomplish: (Internet) <-> (Home Network: 192.168.1.0/24) <-> (pfSense) – (Isolated Lab Only Network: 10.0.0.1/24).

For my WAN interface I made sure that my home network router provided my pfSense a reserved or static IP address. I defined the LAN as a different private IP space (Its recommended to make these distinct to aid in any troubleshooting).

Here we go!

Once you get pfSense up and running, you can then start creating Virtual Machines there are assigned to the “Internal Lab Only” port group. With my particular configuration I did want to allow my VMs to have internet access. At this point we are still not isolated!

Note: For the scenarios where I dont want internet access, I can turn off my WAN interface and/or use something like “inetsim” along with static routing in my VMs to make sure traffic goes only where I want to it to (I am sure there is a better way to do this).

Let’s take some steps to keep our 192.168.1.0/24 network from talking to our 10.0.0.1/24 network and vice versa. What do we need here?

  1. DNS – We want to make pfSense the default gateway and DNS server. We won’t be talking to 192.168.1.0/24 anymore
  2. DHCP – Same story as above, but you may want to have a small pool of IPs dedicated for management on other VMs. (Ex: 10.0.0.2 – 10.0.0.150 can be assigned, but 151-254 are reserved.)
  3. Firewall Rules! – Lets do some simple enforcement in both directions to make sure these networks can’t talk to each other.

Basic Services

Here are some sample settings for reference as of pfSense 2.4.5, as with anything you find on the internet. Do not just copy and paste. Use my blog as a reference but verify and test!

System -> General Setup -> DNS

You can use any public DNS provider here, I also choose not to let my pfSense box get its assigned DNS from the wan interface (DNS Server Override). I also want to use pfSense as a DNS resolver.

Also take a look under Services -> DNS Resolver and make sure you have the options you want selected. You may want to handle how DHCP hostnames are registered, or disable DNSSEC for example.

System -> Services -> DHCP Server

Here is where you can define where your range starts and begins. At the bottom of this page you can set your DHCP reservations/static mappings.

Firewall

Here is where we can enforce our network boundaries. If your WAN and LAN net are in private (RFC1918) IP space you want to make sure you change some of the pfSense defaults. Go to the interfaces page and check the following setting for your interfaces.

Yes, these networks reside in private IP space.

Now onto the firewalls page…

I have created the rules below and modified/removed some of the default rules to make this work for me. I am sure there are more elegant ways to do this, but it makes sense to me and my small lab. Note that my WAN and LAN nets are already predefined per their interfaces.

I recommend making sure you have a labeled config backup (Diagnostics -> Backup & Restore) incase you need to go back to a previous version of the firewall rules.

WAN Rules – I deleted all of the rules here. Traffic should be outbound only!

LAN Rules (and a word about NAT) – This is where you get most of your isolation. You can prevent LAN -> WAN communication in two ways. One is adjusting the default NAT rules so that any outbound traffic is not translated to 192.168.1.0/24. The second is to create a firewall rule, I prefer using both as an extra safety measure.

LAN Isolation #1: Manual NAT

For your LAN Firewall rules, make sure you have a rule to block any LAN -> WAN Traffic (Line #3).

LAN Isolation #2: A sample. Note the second rule is not necessary. Its for pfBlocker-ng

If you are new to reading firewall rules the bottom two lines can seem counter intuitive. pfSense like many other firewalls are “first match” meaning that anything on the LAN destined for the WAN will hit the block rule (3rd line) and not be allowed (4th line). If you need to troubleshoot, pfSense has great documentation on rule processing.

After hitting save and reloading your filter, you should not be able to access your firewall from the WAN side. Now it is time to test out our isolation!

Isolation: Verify!

Whatever your configuration or firewall rules are like, you need to verify them for yourself and use an automated means of doing so! This is great practice in general, don’t trust some random guy on the internet and copy settings blindly.

You can try using ping or SSH to hoists on either side of your network, but its best to use something like an nmap scan. Thankfully, there is a nmap package for pfSense! Go into pfSense’s package manager and search for nmap and then install it.

You can use the pfSense GUI (Diagnostics -> nmap) , but it is somewhat slow and I prefer the CLI version.

We want to try and scan the WAN IP space using the LAN interface and vice versa.

Here is how we go about it, go ahead and drop to a shell. See how vmx0 is our WAN interface. We can tell nmap to scan our lab network (LAN) from the WAN interface. If you did this right, you should get zero 0 hosts up.

You should see something like this!

You should also try this in the other direction and see similar output. “nmap -e vmx1 -A -v 192.168.1.0/24”.

Conclusion

Now you should have an isolated Lab network on your ESXi machine that you attach to your guest VMs and play around with all sorts of software that you wouldn’t necessarily want floating around your “trusted” home network.

As always, remember your OPSEC if playing around with malware.

If you wanted to get more into virtualization and InfoSec research/homelab, I hope this guide has helped you.

Until next time,

-Justin

If you have any comments or corrections, you can reach me at @dfir_justin or any of my contact methods.

InfoSec homelab for the frugal

Until recently, I had always run my Virtual Machine labs (SANS courses, CTFs, Hackthebox, Vulnhub, etc..) on a physical host with a general purpose operating system (Windows, macOS, Linux). However, the limitations of using VirtualBox, VMWare Fusion, Workstation Pro, etc.. become very clear when setting up a more complex lab with its own special networking requirements (especially a span or mirror port). I hope this post is useful for others like me who wanted to take the homelab jump, but did not know how to get started.

There is also the matter of Reverse Engineering Malware (REM) work that benefits immensely from running traffic across a real IDS (Disconnected from the internet of course!) Its a more mature environment, but one that was intimidating given the number of support posts related to getting ESXi working on “unsupported” hardware. Thankfully, after more hours of research and asking the kind folks on the SANS mailing list, I had a better idea of what to look for.

My criteria: Workstation form factor PC that is capable of running 4-5 Virtual Machines for light InfoSec work (IE: pfSense, Windows Server (as a DC), A Windows client or two).

My budget: Approximately $250, I figured this would at least get me a 4 Core/8 Thread processor, a decent amount of RAM, and if ESXi/vSphere Hypervisor did not work I could repurpose the machine for something else.

In looking at workstations, I found that Xeon based workstations with Intel NICs usually had the best compatibility. For these reasons, I narrowed my search down to HP Z Series Workstations and Dell PowerEdge towers. I also found this extremely useful presentation/slide deck by Jeff McJunkin called “Building a kick ass home lab”.

Now that I had my requirements and budget, I needed to find a place that sold used workstations. $250 was not going to get me the latest hardware and you need to expect a machine with several years of use and likely sold to a third party reseller after a company did a hardware refresh/end of life for their corporate workstations.

Note: Don’t be too concerned about old hardware, it should be plenty for this use case. Secondly improvements in CPU performance year-over-year have slowed down tremendously over the past decade. Intel has been stuck on 14nm chips for almost five years!

I found the best deals on eBay and the following subreddits (r/homelabsales and r/hardwareswap). I recommend doing an advanced search on Reddit and sorting by “New” to get an idea of prices on the gear you want. My experience was that Reddit had better deals, but you can get lucky on eBay with enough persistence.

What I ended up buying: An HP Z420 with a 4c/8t Xeon Processor (Sandy Bridge), 20GB of DDR3 ECC RAM, and 500GB of spinning rust. – $150

See the source image
HP Z420 Workstation

I took out the 500GB HDD and replaced it with a spare SSD I had lying around and purchased a 480GB Kingston SSD from Amazon. So approximately $200 for a solid performing workstation.

After seeing what was possible with ESXi, I quickly became greedy and realized that 20GB of RAM and a 4c/8t CPU was not going to cut it. Back to Reddit and eBay!

This is fun and addictive!

My next two acquisitions were an Xeon E5 2660 v1 /w 8c/16c for $30 shipped via Reddit and another pair of 8GB ECC DIMMs for $70. Now I am up to $300, but feeling comfortable about the amount of space I have to work with!

Here is what I have running so far and after some trial and error with learning ESXi, I feel comfortable with this setup. The screenshot below is with all VMs powered on and at idle.

I hope to expand on this post via a “lessons learned” and how I configured pfSense to isolate my VMs from the rest of my network and vice versa. Now get out there and learn and don’t be afraid to break things in your lab!

Synology RT2600ac Security Features Review

TLDR; For $200 you get 802.11ac, Suricata, VPN capabilities in a single box.

Disclosure: This review is not supported/endorsed by Synology. This is my perspective on some security specific features not covered by other reviews.

Home networking for the InfoSec curious or veteran is a topic that seems to come up regularly. Some people take great pride in their pfSense, Ubiquity switched/multi AP environment, with multiple VLANs (for IOT device network segmentation) setups. These are great for the more experienced Network Engineer and/or InfoSec guy or gal, but what about the more casual (SOHO) power user? Or even someone on a tight budget?

Enter the Synology RT2600ac (yes, the NAS company) which has plenty of networking and security prowess hiding in a $200 dual core ARM based appliance. This has been my go to recommended gateway for co-workers and family who are tech savy but not ready to make the deep dive and expensive into prosumer home networking. This review focuses exclusively on the security features. If you want some reviews that go over the NAT performance, wireless range/speed, go check out the folks over at SmallNetBuilder.

The RT2600ac will be two years old at the time of this writing (February 2019). An important question to ask is how often are security vulnerabilities being addressed and how often does the router receive updates?

Synology seems to be on top of this as an December 2018 update fixed an RCE vulnerability (no authentication required!) and the list of release notes is impressive. WPA3 support was also added a few months ago.

Basic Configuration: automatic Updates, default configuration, and authentication.

Before moving onto the more advanced features, lets get the essentials of out of the way. For your non-technical or non security user, having an automatic update mechanism is essential. Thankfully you can choose between security updates and/or operating system (SRM) feature updates. This is similar to the Windows feature update settings in Windows 10. I have found the SRM releases to be quite stable. However, if you are going to be using this router in a mission critical setting . I wouldn’t fault anyone for going with security updates only.

Gracefully, external access to the administrator page is only by “opt-in” (seriously though never do it!) and the router supports 2FA via the use of Time-based One Time Passwords (TOTP). This type of second factor is compatible with popular TOTP clients such as Google Authenticator, Authy, and others. This protection also extends to other apps in the SRM specifically Synology’s Web VPN.

Firewall and Geo-ip blocking

After you go through all of your configuration options and make sure everything works, you can start hardening your perimeter (also wise to turn off uPNP). Beginning with blocking several countries that you have no interest in exchanging traffic with and/or avoiding malicious infrastructure (botnets).

Each firewall rule listed above can have up to 15 countries in it. I just stick with 15, however you could go crazy and add 30, 45, 60 to cover even more territory. I wish that Synology would allow you to add as many countries as you want within one rule. The SRM UI is limited in the data it can display (more on that later).

Synology suricata

How in the heck does a $200 ARM based router run Suricata inline with little performance overhead? Well it took Synology almost two years to figure it out and after some tweaking it really does work! Prior to the November 2018 update, I was only pulling 30 Mbps from LAN <-> WAN on a family member’s 300 Mbps Comcast connection. Post update am seeing consistent 300 Mbps speeds. Really excellent for a 1.7 Ghz dual core ARM processor with only 512MB of RAM!

The downside is that managing Suricata from the GUI in its current state is less than ideal. The web server is fairly slow, there are several drill downs to find specific rules, and I see no place to write your own rules and/or download packet captures. I hacked away at it in the command line (Busybox shell) and could not get my custom rules to take effect in Synology’s version of Suricata (perhaps further testing will lead to another post).

I should also mention the “Safe Browsing” application that launched in November 2018 as well. Its a competent parental control style app that uses some data from Google’s safe browsing project and an unknown source of “Threat Intelligence”. I have had a few false positives, but the lists are updated daily and I have not noticed a performance hit.

You may be better off using an DNS provider that does filtering such as OpenDNS (family shield/security) or Quad9 (security).

VPN

Synology’s SRM supports many of the most popular tunneling protocols such as SSTP, OpenVPN, IPSec, and PPTP VPN in addition to Synology’s proprietary apps.

SRM does create some other private subnets that allow you to segment VPN users from your primary systems and/or the guest network. I briefly used OpenVPN, which was easy to set up and came with great instructions for getting the necessary certificates/configs exported.

If you want to fly a little lower under the radar, you could run Synology’s SSL VPN over a more common port (443) and your traffic would look like a normal HTTPs connection to the rest of the world. The Synology app also supports 2FA which is another bonus.

PROS and CONs

Pros

  • Extremely stable
  • Incredible value at $200
  • Regular security updates
  • Late 2018 update allowed for Suricata to run with no appreciable performance hit.
  • 2 Factor Authentication for admin access AND VPN
  • Easy to use GUI
  • Syslog support

Cons

  • Slow to reboot (10 minutes) for a consumer router
  • Odd UI bugs such LED on/off preferences not working or DNS settings greyed out for IPv6. (To their credit this was eventually fixed)
  • Hardware limited, I would love to see what an extra $100 would allow for in terms of CPU and Memory (more cores? higher clocks? x86 base 1-2GB of RAM?)
  • Easy to use GUI (Very few command line options)