Build your own Ekahau ePerf & Speedtest.mini server!

One a recent forklift project, we decided to replace our aging 802.11a/g hardware and deploy new 802.11ac WLAN gear.

 

We designed the building with Ekahau’s ESS - our default WLAN survey and design tool.  After designing the WLAN to meet our healthcare requirement (in this case, Aeroscout tags, Vocera badges, 5GHz Voice and data) we installed the gear and then validated the WLAN.

 

For this initial 802.11ac deployment, we decided to do both passive and throughput validations.  With throughput surveys, we measure actual data, such as packet loss and jitter.   On a side note - for some time now, we have said to ourselves, “I wish we had a portable Ookla Speedtest server”.   Spoiler alert!

 

We needed a throughput server that would be both simple to use and portable.  After talking with our Ekahau team, we decided to use the Odroid C2, and configure it for two purposes.  (It turns out that Ekahau has done the homework for us, and a quick web search will unearth most of what we need to know.)  We decided we wanted it to be an iPerf3 server, and a speedtest.net “mini” server. 

 

Most of the time, we do passive surveys to validate our WLANs – however, every now and then we have the need to do an active survey, which is usually related to a WLAN that is experiencing issues.

 

I think we have all heard the usual complaints about “the Wi-Fi is horrible”, etc.  Users go to their favorite speed test site on the Internet from their mobile device and immediately blame the Wi-Fi when they don’t get the results they expected.  With our new throughput server, we can plug it into the LAN and have users browse to it and test their throughput, eliminating the Internet connection and all of the other unknowns!  Having a portable speedtest server and seeing the look on their faces is priceless!

 

Having a portable ePerf/Speedtest.mini server is awesome.  You may want to do spot checks to make sure everything is operating properly on your WLAN after setting up new access points and controllers, or your controller-less Wi-Fi gear.

 

I would like to mention that throughput surveys are not for everyone.  They take a lot of time, and your survey results may not look like what you expected.  If you have never done an active/throughput server, then I encourage you to do a small one and look at the results.  It is a great learning exercise, that’s for sure.

 

The team at Ekahau has done most of the homework for us, and created a shopping list for us.  Follow this link to see what is available.

 

https://docs.google.com/document/d/12SyzzzKbpJ_Xifr0I3xzu8HyMVZEpZcVePm0QIifT2o/edit#heading=h.arbwrv84jz4c

 

I chose the standard build from the document, and with a lot of help, loaded Diet Pi, Fruity WiFi, Ekahau iPerf server and Speedtest.mini.  I will warn you – you should know your way around the Linux command line, or have someone that can help you.  I could never have done it with without a lot of help.

 

Rumor has it that if you attend WLPC , you can build one yourself while you attend the conference.  That is a great excuse to go!  If you are like me and not a “Linux guy”, definitely go this route.

 

Here’s what I ended up purchasing:

 

 

 

 

 

After building the unit, we plugged it into our wired network.  It boots up, and we connect to it via Wi-Fi and then obtain the wired IP address.  We browse to the wired IP from another wired workstation and test the wired to wired throughput.  In this case, we get over 500 Mb/s.  That’s good enough for me – now I know how fast this can go.  We ignore the slow upload speed, as the Odroid can’t write to the flash that fast.  We’ll just be using download for testing purposes, anyway.

 

One thing I want to mention is that along the way, I ended up using another micro USB power cable and didn’t notice.  My Odroid was having issues powering up, and I asked around and someone mentioned that I might not be using the power brick that I purchased with the unit.  That was the case!   The next unit I build will have the wall transformer and power plug, so I don’t run into this again.

 

 

Here’s what my Speedtest.mini looks like, wired to wired.  Awesome!

 

 

 

 

If you have been pondering having your own Ekahau throughput/iperf server, this is the way to go!

 

Enjoy.

 

Proving "It's not the Wi-Fi network"

We’ve all been there – or at least most of us have, anyway.  The Wi-Fi network appears to be misbehaving and users are frustrated.

 

Your users will be working for several hours, and then, it looks to them as if someone shut the entire WLAN off.  Their workstation’s Wi-Fi icon, when hovered, states “no Wi-Fi connections are available”.

 

Now comes the fun part – well, to me it’s fun, anyway.  Let’s start out with what the normal operation of the WLAN client looks like.  This particular client is stationary.  It’s a laptop that is used like a stationary desktop, and is cabled to the desk via lock and key.  I wanted to clarify that because you won’t see any roaming in this packet capture, and you shouldn’t see any.

 

Since the client is associated to an AP on channel 36, I set my protocol analyzer to only look at that channel, and then set a filter to only look at the client adapter.  This makes is a lot easier to what’s going on.  I will use this as my baseline, since I want to know what it looks like when everything is working normally.  Here’s normal for this client:

 

Encrypted data from AP to client

Client Request to Send to the AP

AP responds with Clear to Send

Client sends data

AP acknowledges data with ACK packet

 

 

Now we will fast forward a few hours.  The user called you and told you “The Wi-Fi is down”.  Or is it?  To them, it is!

 

We start our same packet capture as before – same filter, same channel.

 

Now the picture is very clear.  The Client doesn’t see the beacon of the AP’s BSSID because it is sending probe request to ff:ff:ff:ff:ff:ff and we are seeing the unicast probe responses from the APs (only on channel 36 because of the filter) to the client.  I believe the client should send unicast probe request to the AP.

 

The c lient does not appear to see the beacon of the APs, and starts sending out probe requests.  The APs respond, but the client does not appear to see the probe response.  Almost as if they WLAN client has gone deaf.

 

  

Thankfully, the “quick fix” is to turn the WLAN adapter off and then back on.  Long-term fix will most likely entail downloading and installing new WLAN client drivers.

 

 

 

 

Using your autonomous AP as a Spectrum Analyzer

 

Most of us have all done an APoS (AP on a stick) survey, either active or passive, for a customer by now.  Many of us also take a snapshot of the spectrum while doing our WLAN surveys.  We either use an integrated Spectrum Analyzer, such as the DBx adapter coupled with Ekahau’s ESS software, or we use a spectrum analyzer adapter(hardware) and software to collect data for off-site analysis.

 

If you own the DBx adapter and use it with Ekahau, that doesn’t mean you own Chanalzyer, which is Metageeks Spectrum Analyzer software.  Which leads me to this post, as I do not own a copy of the software mentioned previously.

 

If you are on-site, using a Cisco 3602i series autonomous AP for your APoS active/passive survey, did you know that with just a little bit of effort you can use it to grab your Spectrum Analysis file?

 

I usually have my old Dell D630 workhorse with me, which has a PCM/CIA  slot, an old Cognio card, and Cisco Spectrum Expert loaded on it.  However, as most of you know, these machines weigh 4.6 metric tonnes after about three hours of carrying these monsters around, no matter what hand you carry it with.

 

After a brief conversation with @NoLANWiFi (giving credit where credit is due) I decided to lab it up.  I am assuming you have read @802Tophat’s blog on getting your site survey AP up and running – Thanks Richard for posting that for us!

 

I logged into my site survey AP running 15.3 code and changed both dot11 interfaces’ station roles to “station-role spectrum”.    Now you need the NSI key.   Type “show spectrum status” from the exec prompt and grab the NSI key.  It looks something like this:  NSI Key:  0FB30A960DA7F66952E30B59640563AC

 

There are two ways to connect to your “new” Spectrum Analyzer.  Connect to either one of the dot11 radio interfaces with your site survey laptop, like you normally do when active surveying, and use the other interface for spectrum collection.  Meaning, if you connect via the 2.4GHz radio, you will use the 5GHz radio for spectrum collection.  Or, connect via TCP/IP via the Ethernet interfaces.  This is how I am going to do it for this post.  I am going to use the Cisco POE injector, and one end is connected to my survey AP, the other to my laptop.  They’re both on the same /24, so I can simply connect via TCP/IP.

 

Next, you need to download Cisco Spectrum Expert.  If memory serves me, you need greater than 4.0 for remote spectrum analysis.   After downloading and installing it, go ahead and launch the app.

 

This is where you plug in that NSI key.

 

 

My laptop is 192.168.0.5, statically set on my Ethernet interface.  Notice how I am given a choice to go use either the 2.4GHz or 5GHz band.  This appears to be one of the downsides of using the AP for spectrum collection.  I cannot select both frequencies to collect data.

 

Here goes…  Let’s take a look.

 

 

As most of you probably already figured out, I have a known environment in my lab to test with, and sitting side by side is the old Dell D630 with the Cognio card.  After running some interference tests, it looks like the remote sensor and the Cognio are, for the most part, on the same page.

 

However…  something is missing!

 

When I look at the Channel Summary page, something looks awry.

 

 

I’m not seeing any Wi-Fi Present!  What’s the deal? (I know what the deal is, but wondering if you, the reader, can figure it out)

 

Now I am going to the Devices tab, and again, I don’t see anything!

 

 

What could be “wrong” with using the AP as a Spectrum Analyzer?   I left BIG hint for you in that last graphic…

 

All that said, I think I can use an AP for a spectrum analyzer, in a pinch, if I had to do it from a remote site.   Clearly, when using an AP as a remote spectrum analyzer, we don’t get all the functionality we would get out of our laptop/hardware based spectrum analyzers, such as Metageek’s Chanalyzer, Spectrum Expert (with Cognio card) or AirMagnet’s SpectrumXT.

 

Please use the comment section to chime in on why we are missing the data that we might want to see at a later date, if we were using the spectrum file as a baseline.

 

 

 

 

 

 

 

 

WLAN Surveying and Validating with Ekahau's integrated Spectrum Analzyer

 

When doing any WLAN Assessment or Remediation, we ALWAYS look at the spectrum.   In about 60% of the WLANs we assess and remediate, we find interference from a device the customer didn’t know they had, or knew they had but didn’t know it was sharing the same spectrum as their Wi-F.

 

The complaints vary from Customers that have interference issues.  We hear “I only have two bars”, “my wireless is slow” and “when I stand right here, my Wi-Fi doesn’t work”.

 

We’ll first start out with stating the obvious.  The 5GHz UNII bands are license free, which means it is a free-for-all when it comes to who is doing what.  Most of us all understand that.  A company, or their neighbors, can pretty much deploy anything they want, as long as they abide by the rules.

 

When we first start out on an Assessment, we do what we normally call a WLAN Validation.  We walk the entire facility with Ekahau’s ESS – or Site Survey Software.  Ekahau recently updated their software to include Spectrum Integration, so here is our first look using it in the real world. 

 

In this case, we have an area that we know we have WLAN Interference.  We have looked at it with other tools, however since Ekahau is our tool of choice at the moment, we want to compare what we see with our new spectrum integration to what we are used to seeing in our legacy equipment.  Here is a view of the area known to have an interferer on channel 40.  We looked at our Survey Inspector, and the Spectrum Channel Power view.  We clearly see something there, and can compare it with a quick glance to the other channels. 

 

 

 

And now we are going to take a look at our Spectrum Utilization view.  We scrolled up to another area of the survey/walkabout where we know we have another source of interference.  Again, we can clearly see that there is an issue in another part of the building.

 

 

 

During our Spectrum Integration analysis, we notice another feature called RTFM.  Forget what you know about this acronym, because it stands for Real Time Frequency Monitor.   This is the kind of tool I would use if I had my survey rig in my backpack, and someone told me of an area that was having Wi-Fi issues.  I don’t even need to build a project – I open ESS and hit the RTFM button, select the frequency I want to look at and give it a glance.  Here’s what I see below.  I must say, that’s a nice feature!  Thank you Metageek (for the SpecAn) and Ekahau!

 

 

Now that I have seen the interference, I, for whatever reason, want to see it in my Spectrum Analyzer software.  This is available from Metageek – where you would most likely purchase your SpecAn hardware.  This view is with other known interference devices, all by the same manufacturer, turned on – for our testing purposes.  As you can see, there is a lot of interference here, and some remediation and spectrum management needs to take place.

 

 

 

Now for a look from some of our legacy tools.  As you can see, the view is not of the same exact slice in time, however I promise you that what you are looking at is all caused by the same equipment.   This is a view of four devices energized, and one of them is changing channels.  No wonder these folks are complaining about their Wi-Fi not working well for them!

 

 

 

 

Here’s a view of channel 40 from another Spectrum Analzyer.  AS you can see, the numbers vary from tool to tool, but in each you can tell there is an issue.

 

 

Af first, it took us a while to track down the equipment, as we didn’t want to go into an operating room while they were doing their thing.  After several days of intermittent troubleshooting, we finally came to the conclusion that our source was mobile.  We tracked it down to Operating Room “towers”, which were mobile Endoscopy equipment.  The gear has a wireless transmitter and a remote monitor or two, and those remote monitors were connected via the 5GHz spectrum.  This equipment was moved around to whichever Operating Room need it.

 

 

 

 

Then we discovered something else.  When the equipment was turned on, it searched for a channel to use.  Not in a very Wi-Fi friendly way, though.  It seems as if it always starts on channel 36, then works its way around the first eight channels until it finds “home”.

 

If you are good with math and have a great imagination, multiply this one source of interference, and what is does to your WLAN, by four.  There were at least four of these devices being turned on and off during the course of any workday, each booting up and trying to find a channel to use.  Stomping on the Wi-Fi as it went.

 

 

 

Something about these tools is worth mentioning. They are somewhat complicated and expensive to own – unless you use them on a weekly basis.  They all do a great job of displaying to the operator what the electromagnetic spectrum looks like at a given frequency.  However, there is no “magic button” that you can click on that will tell you what is wrong with your network and how to fix it.  I highly encourage anyone interested in owning and operating these tools to first go to CWNP.com and purchase the CWNA curriculum and read it several times.  Of course you also have to read the manual of the spectrum analyzer you finally end up purchasing.  You can get the Spectrum Analyzer (and the software) that integrates with Ekahau ESS from http://www.metageek.com/

 

 

 

 

 

 

 

 

Is there a need for a Spectrum Policy within the Enterprise?

I recently came across this some equipment that interfered with Wi-Fi in the worst way – well, that’s my opinion, and I will let you be the judge.

 

We’ll first start out with stating the obvious.  The 5GHz UNII bands are license free, which means it is a free-for-all when it comes to who is doing what.  Most of us (the folks reading our blogs) all understand that.  However, when an Enterprise environment has millions and millions of square feet of office, retail, healthcare or manufacturing space, I think the Company’s Enterprise IT department owes it to their internal customers to have a Spectrum Policy to keep the spectrum in check.  Letting the company buy whatever they want, whenever they want, and deploying it randomly as they see fit, doesn’t work well, as you will see if you continue reading.

 

I recently came across a surgery center consisting of 8+ operating rooms, and the Wi-Fi was at the core of the complaints.  Nurses had to move their mobile workstations outside of the operating rooms in order to maintain a connection.

 

We went in with protocol analyzers at midnight and saw nothing.  We went in at 6pm, and saw nothing.  Then we went in during working hours with our Spectrum Analyzers, and that is when we saw a big difference!  We observed channel 40, in the 5GHz range, pegged at 90+ percent duty cycle, for well over an hour. 

 

Okay, so what, you think.  One channel – no big deal, right?  Most Enterprises use UNII-1,2 &3, so there is likely another channel readily available at a -67dBm. 

 

 

It took us a while to track down the equipment, as we didn’t want to go into an operating room while they were doing their business.  After several days of intermittent troubleshooting, we finally came to the conclusion that our source was mobile.  We tracked it down to Operating Room “towers”, which were mobile Endoscopy equipment.  The gear has a wireless transmitter and a remote monitor or two, and those remote monitors were connected via the 5GHz spectrum.  This equipment was moved around to whichever Operating Room need it.

 

 

 

 

Then we discovered something else.  When the equipment was turned on, it searched for a channel to use.  Not in a very Wi-Fi friendly way, though.  It seems as if it always starts on channel 36, then works its way around the first eight channels until it finds “home”.

 

 

 

 

What does this do to your Wi-Fi when this is happening?  If the RED color doesn’t spell it out for you, then nothing will!

 

Just to prove our point, we took our active survey access point (yes, we actually do “active” site surveys still) and put it on channel 36 and associated three clients with continuous pings to it.  We turned on the Stryker gear, and not only did the pings cease, the three clients no longer saw our site survey SSID, and then decided to join another WLAN on another channel.

 

In a nutshell, it obliterated the channel for however long it decided to rest there. As you can see from our swept spectrogram, there is no real pattern.

 

We also used two different spectrum analyzers that we happened to have in our kits that week.  Not saying that one is better than the other, and if we didn’t use one that isn’t here, it isn’t because we don’t like it.  We just happened to have both of these with us, and that’s what we used.

The screenshots below are from just one transmitter and one receiver pair.  Imagine a surgery center with ten operating rooms along with five of these units online, coupled with the different schedules and the appearance of random Wi-Fi issues.  When these units boot up, they search for the channel they want, obliterating the spectrum as they go.

 

 

 

 

 

 

 

If there was a Spectrum Policy, this kind of thing might be avoided.  The 5GHz bands can be divided up within an Enterprise, with UNII 1,2&3 reserved for Wi-Fi, and UNII-2e further divided up for cameras and other gear.

 

What are your thoughts?  Is it worth the investment to put together and enforce a Spectrum Policy?