Networking for SELF

In case you missed it, I have done an overview of everything we do for SELF here

For the SouthEast Linux Fest we take over the event space of the Sheraton Charlotte Airport Hotel to host our 3 day event. I have been helping with SELF for the past 5-6 years and in those years we would get to the hotel on the Thursday morning before the event and have all day to setup the network. We go live with the network Friday 8 am as that is the start of the event. With the permission of the hotel staff we turn off and power down the hotel’s wifi system in the event spaces were we will be setting up our own wifi. Doing this helps with reducing RF interference.

The logistics of getting there.

I live about a hour away from the hotel that we host SELF at as a result I really do not want to make two trips each way. I am fortunate to have a Yukon and large trailer so I am able to load everything in one trip. I have found those large Husky tool chests very nice. This was gearing up to SELF 2019.

In 2019 I took 9 displays

Internet connection

Over the years we had a few different forms of internet connection all around a coax connection from the local cable company. When I first started to help we started with 100×5 internet connection. Next we did 3 bonded 100×5 connections to gain more upload speed for videos. We progressed to what we currently use, a 940×35 coax connection. I wish we could have fiber but can’t afford it. We are trying to get a local Wireless ISP to get us a connection. If that happens then we should be able to get a 500×500 connection. We also run a full dual stack ipv4/ipv6 network.

Layer 1 the cabling

Over the years we run own network cabling both cat6 and fiber. We are able to leave the cabling in the ceiling year to year allowing to reuse and built up over time the cabling that we have in place. In 2019 we expand into a new part of the hotel so we needed to extend the network to new wing. This wing was connected to the our current wing with a 5 foot in diameter underground pipe that carried services between the two different parts of the hotel. In this tunnel we were able to run a 800 feet of fiber which allowed us to use 10gbit networking between our switches.

We deploy temporary cabling to run between the end devices and the switches. Those temporary cable runs are mostly made of cable trunks where we will group 2-8 ethernet cables together as a single run. We did this because we found that we were running multiple cables to the same location. Right now we have 67 cable runs which total over 7,000 feet of ethernet. Below you can see how we labeled each trunk cable. The 24 the cable ID and the 64ft is how long it is. All of the cable’s info is saved in a spreadsheet with the ID number as the reference point. Each port is then labeled started with A. So you get something like port 24A. Both ends of the cables are labeled like this.

Switching

Right now we use 5 Dell Powerconnect 5524 POE switches that we got from ebay for between $100-150 each. We use the 10gbit ports as trunk ports between switches. Currently the switches are setup by hand using the built in GUI and CLI, we are hopping to change that soon. Also in the future we are looking at some Brocade switch to get more 10gbit ports and some 40gbit ports.

Routing

At this time we only do inner-vlan routing on our PFsense firewall. The PFsense box has 4x 1gbit interfaces that get lagged together and linked to the switches. When we get more 10gbit ports we will be moving to 10gbit LAN link. The PFsense also has another 1gbit interface that we use as our uplink to the coax modem.

WIFI

Right now we use 20 Unifi APs bulk of them the AC-pro model but we also use nano-HD, AC-HD, AC-lite, AC mesh. All APs are powered from the POE switches and backhauled using ethernet. We host the Unifi controller onsite. We run 3 SSID at SELF, staff, public fast(5ghz only) and public slow(2.4ghz only). We use social engineering with the names, to encourage users to connect to the 5ghz only network.

Servers

We run all services that are needed for the event onsite.

namecpuramOSModelroles
Bigserver2x E5620 (8 core total)96gbUbuntu 18.04dns, nfs storage, mqtt, mysql
gameserver2x L5640 (12 cores total)24gb ramUbuntu 16.04lan cache, game servers
desktopw353016gbWindows 10camlytics
noderedi5-2400S4gbUbuntu 18.04node red, unifi-poller
NVRcore i7 4770s8gbWindows 10blue-iris

Lan party area

We host a lan party area at the Linux fest. In this area we offer tables with power and wired network drops. We host a few game servers onsite for games that support Linux well. Since the guest wireless and wired lan party network are open guests are allowed to run there own game servers if they wish. We built a power monitoring rig to allow us to track power usage to make sure that we are not about trip a breaker. From the hotel’s connection we get 2x 20amp breakers, we take one lead to each breaker. Inside of the gray box we have Shelly EM and two CT clamp. The CT clamp will give us, in real time, the number of amps running to each breaker. Each breaker will connect to one of the quad set of outlets.

Note that we did put outlet covers on after this picture was taken

NOC setup

For our NOC setup we have a shared space in the admin/command room. This is where we place the server and have all of the cables run back to. In the picture you can see the cabling coming down from the ceiling. For SELF 2021 we will be moving the server into a closet to reduce the noise and heat of the NOC area.