How do you stage your access points?

 

This blogpost is about my quest to refine my access point staging process.

 

Back in the day, I used to stage the (autonomous) access points in my deployments one by one.  It took a long time to configure each one, and I’ve always been on a quest to perfect the perfect staging process. 

 

Fast forward to 2014, and I find myself surveying, designing, configuring and installing.   With most of my deployments nowadays, I hire Electricians and Telecommunications Technicians to install the access points.  However, I still find that staging the access points myself is usually worth the effort.  I’m a believer in powering up all access points, “burning them in”, configuring them, and labeling them – long before they ever end up in the Installer’s hands.  I guess it is personal preference.  I would rather know that an AP has been powered up, associated to a WLAN controller and has been configured, rather than stare up at a ceiling an wonder why if the AP is “bad” from the factory.  When I prime them, I know they’re ready, and can rule out AP configuration or DOA issues.

 

Not too long ago, Nigel Bowden (a WLAN Consultant) and I were conversing via email about the AP staging process.  I had a macro-enabled spreadsheet that I had used in the past with a Symbol scanner to make labels for new AP deployments.  He had a spreadsheet that he used to do CLI configuration of access points once they joined the controller.

 

This gave me an idea.  What if I combined my spreadsheet with his?  My idea was, if I labeled an AP right out of the box, created the same hostname “labels” in the spreadsheet and then scanned its barcodes (MAC and Serial Number), could I create a spreadsheet that would create the CLI without having to scrape the information from the WLC after plugging it in?

 

They answer is YES!  Now I will tell you how it was done, and it has dramatically decreased the time it takes me to stage and AP.  After all, staging 20 access points is “fun”, staging 2000 access points is “less fun”.

 

 

First, we’ll start with the AP numbering scheme.  Sometimes the designer can develop it, or the organization may already have one.  Let’s pretend we’re staging a few hundred access points for a hospital in Denver.  They already have a naming convention and they want you to follow it.

 

I use my new spreadsheet to create the number of access points per floor, per telco closet, per seating area, etc.  I’m sure you get the point.  In this example, we’ll go with 19 APs in this area.  Using the spreadsheet, create AP01, then drag down to AP19.

 

 

 

Next, I will create the “area” of the access point.  This could be the seating area of a baseball stadium, or a floor of a hospital.  The area stays the same, so I use Excel to make it all the same.

 

 

The spreadsheet adds the two columns together to make my hostname of the AP:

 

 

Then we use a few more columns to define the Primary and Secondary WLC, along with the IP addresses of them.  I never configure a tertiary WLC, as I want the AP to discover a tertiary WLC if the secondary is not available.

 

 

Next thing I do is take 20 APs and plug them into the PoE switch that I have configured with the WLC on the bench.  I want to have the APs boot up and find the WLC via option 43, download the version of code I will be using in production, and then configure them.

 

Here they are:

 

 

I want to make the labeling and configuring process as simple, accurate and fast as I possibly can.  When I see that all the APs have joined (not all do, and I usually have a few stubborn ones right out of the box) the WLC, I want to label, configure and box them up quickly.  Here’s how it works…

 

We know that when an AP boots up, it will show up on the WLC with a default name – one that is derived from the MAC address of the unit.  If the MAC address of the unit was de:ad:be:ef:ca:fe, the default hostname would be APdead.beef.cafe,right?  That MAC address (and the serial number) is also on the barcode on the underside of the AP.

 

I create the labels and affix them to the APs after I know all of them have joined the WLC.  I don’t care which label goes on what AP, however I usually start with AP01 and work my way through the pile in the same direction I plan on scanning. 

 

 

With the Ethernet cable still attached, I flip the AP over and scan the SN# and MAC address barcode of the matching line on my spreadsheet: (the apSerial and apMAC automatically populate and go down to the next row)

 

 

 

After I scan the APs on the bench, they automatically populate in the spreadsheet.  Notice the spreadsheet’s rows have the Primary and Secondary WLCs in there!

 

 

The spreadsheet then generates the CLI for the renaming of the AP from the default name to the name you want to use, along with the Primary and Secondary WLC.  All I do is copy/past these three columns into the WLC and this batch of APs is staged – accurately and efficiently.  And now I have a spreadsheet of the SN#, MAC, Hostname, and how each AP was configured.  I also have a column for APGroup, which I did not use in this example.

 

 

The best part is, I go from this:

 

 

To this in one simple copy/paste.

 

 

The command structure, for those who are interested, looks like this:

 

 

I am interested in hearing what other processes you may use to stage large quantities of APs.

 

How many APs can you stage in a day?  (this includes unboxing, code upgrades, labeling, configuring, and boxing back up)

 

What is your opinion for “must haves” in an AP naming convention?

 

I’m looking for an easier way to create AP labels.  Instead of using one of the handheld label makers, I would like to use something that creates them directly from the spreadsheet.  I’ve used sheets of labels in the past and not had the best of luck with some of them.  Problems included:

 

·         Labels peeling off resulting in a jammed printer

·         Reprinting one label meant wasting an entire sheet

·         Labels peeling off access points

 

What are your favorite labels?

 

Do you have a favorite labeling machine?  (possibly one with a USB interface that can print straight from your laptop)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

WLAN Site Survey Rig - Rev2

I recently decided to rebuild my site survey rig so that it would be able to broken down and put in the back seat of my small commuter car.  In all honesty, I started out trying to rebuild my rig, but ended up starting from scratch.

 

Searching eBay, I decided on this as my starting point: (it set me back about 90 bucks)

 

Foldable Table Laptop Cart Workstation Portable Roll Home Couch Office Desk Cart

 

               

 

Next, I decided I needed a round platform at the bottom of the rig to place a small UPS, orange cones, and anything else heavy enough to tip over my cart.  I purchased a piece of ¾ inch plywood and cut it into a round shape, cut a hole in the center, painted it orange (for safety) and screwed it to the legs that the wheels were fastened to.

 

I drilled a 1.5 inch hole in the upper platform for my painter’s pole.  I quickly discovered that I needed a pipe attached to the base because it was not stable when I attached my flexible painter’s handle with AP attached to the painters pole.  I also drilled and tapped a hole in the pipe so I could insert a thumb screw to tighten down the pole to keep it from spinning on me. 

 

On my flexible AP mounting arm, I fastened two Rare Earth Magnets that I found on eBay.  I wrapped them with electrical tape around the flexible arm, so when I slid the arm up against a ceiling t-bar, it would magnetically hold the pole steady.  It works beautifully.

 

After it was all said and done, I assembled the rig and gave it a test drive.  I placed my AP’s power supply in the shelf compartment after lining the shelf with sticky backed Velcro.  That keeps everything from falling out when pushing the rig down a hallway.

 

Here’s what it looks like:

 

   

 

The entire rig breaks down and fits into the back of my SUV nicely.  The shelf has a handle on it that allows the center support to raise up to desktop height.  The shelf is removable, making the entire rig portable.  Here’s a picture of it all broken down.

 

 

 

Next, I wanted to get the most of my AP’s power supply.  Many of you will recognize this power supply that is made for WLAN site surveying.  I use a Cisco 1140 series AP with the 2.4 GHz radio turned off, and the 5 GHz radio configured for 11 dBm for site surveying.  I wasn’t getting the “full days’ worth of site surveying” out of my power supply, so I decided to do a slight modification.

 

Here’s my power supply with the stock battery removed.  I decided to finally write down on the front of the power supply how to configure the switches for  charging, surveying, and storing the power supply.  I can’t count the times I have plugged it in to charge overnight and not configured the switches correctly!

 

 

I would like to also mention one more thing.  The manual mentions how to store your power supply.  The battery is mounted in the power supply in such a way that it should be stored on its back when not in use.

 

 

I wanted to purchase a new battery, and if possible, get a battery that would last a little while longer.  I found this battery on eBay for about 40 bucks.  The advertisement claimed the battery would last longer than conventional batteries.  When it arrived, I noticed the tabs were larger than the initial battery, so I had to use my Dremel tool to modify the tabs to fit onto the new battery.  Here’s what the battery looked like right out of the box.

 

 

After modifying the tabs of the new battery, it fit into the power supply perfectly.

 

 

I charged my new battery and drained it with my survey AP several times before measuring how long it would last.  After a week, I measured how long it power up my survey AP.  I checked it on the hour, and it lasted over six hours, but was not powered up on the seventh hour.  That said, I now know it will last about six hours of surveying.  I’ve hear that other Engineers get a full day out of surveying with their power supplies.  I would love to hear from you if you have any tips.

 

Last but not least, I broke the rig down and it fit inside my little commuter car.  Everything now fits in the trunk of my little car, with exception of the pole which fits up front.  That’s a pretty small trunk, and its all in there!

 

 

How to get your APs to leave your Cisco WLAN Controller in a pinch!

 

You’re probably wondering why on Earth I am writing a post on how to make your access points flee your WLAN Controller.

 

Here are a few reasons why you might want to do this: (I have seen all of these situations)

 

·         You have two WLAN controllers, each backing the other up and you want to upgrade them.

·         You just put a new image on your WLAN controller and it is corrupt, and your access points are hung up on downloading for hours.

·         You want to move your APs to the other controller quickly, but they are on the same subnet and you cannot use a VACL or other ACL.

·         You need to move the APs and don’t feel like scripting, and don’t have WCS/NCS/Prime Infrastructure Access Point Templates setup.

 

The one in the middle recently happened to me.   Here’s my scenario:   Two WiSM1 blades were half loaded with APs, each backing up the other.  I staged two new 5508s to replace the aging WiSMs.  Little did I know, there was a problem with the image on 5508 #1.  I moved a dozen APs manually from one of the WiSMs to the 5508, and they started downloading as expected.  I was migrating the APs from the WiSMs (on 7.0 code) to the 5508s running FUS 1.7.0.0 and 7.4.121.0 operating system, so as soon as they landed, they started to do the upgrade.  Problem was, the image was somehow corrupt and the access points would start downloading, then reboot, and then start downloading again.  If you’ve ever been in this situation before, you probably know that you cannot configure an AP while in the downloading state.  Obviously I didn’t know the image was correct, but knew I needed to back out of my change, and do it quickly.

 

Unfortunately it was 1 am in the morning, and there was about 75 miles of Interstate between me and the box.  I had three options at that moment:

 

·         Reboot the WLAN Controller.  When it comes back up, the APs will go back to the same situation

·         Shutdown the Port Channel at the switch level, leaving the controllers stranded.  (and leaving me locked out of them as well)

·         Somehow configure the WLAN controller so the APs leave and go find their configured Secondary Controller, but leave me access to the WLC.

 

I chose option three.

 

But how?  The WLC is on the same subnet as the others.  There’s no configuration check box that reads, “Do not respond to AP join requests”.  Hint hint. (feature request)

 

Here’s what I did.  I changed the hostname of the controller, because I had configured it on the APs that I moved to the WLC.

 

 

Then, I had to shut off the 5GHz and 2.4 GHz at the global level on the WLC:

 

 

 

And as soon as that is done, I changed my Country Code.  I removed the checkmark for US, a country that I have visited many times – France!

 

 

 

Viola!!  The access points fled quickly to their configured Secondary Controller, downgraded, and the network was back up and running.

 

This allowed me to go to bed, and leisurely wake up at 5am in the morning with several thoughts and ideas of what went wrong last evening.  I called Cisco TAC at 6am and got a response relatively quickly.  (note to self – call Cisco at 5am for faster response time)   During our troubleshooting I decided to see if another access point in a building that just happened to be closed all weekend would join the redundant controller I had configured.  It immediately joined and we came to the conclusion that the image on the WLC had somehow become corrupt.  I don’t know how, since I personally staged both WLCs with the same TFTP server and image while in the lab, and my test AP running that same image worked associated just fine.  It was definitely the “downloading” portion of the operating system that was faulty.

 

I downloaded the same exact image again from CCO, and then immediately transferred it via TFTP to the WLC and rebooted it.  I reversed the country code and other configurations and put it back to “normal”.  I then send an access point to the WLC and it joined, downloaded normally, and all is well now.

 

One thing I want to mention is during the heat of the moment, I did a web search of “APs stuck in downloading”, “Access Points stuck in downloading”, and “Cisco AP downloading” and did not get any hits.

 

I hope that this post can help other WLAN Engineers out that may encounter the same scenario and frantically Google those search words, only to come up with nothing. 

 

Cheers!

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Cisco power levels - Lightweight APs and channel and power plan (what you thought you knew)

I was recently called out to investigate why a wireless network was having voice over Wi-Fi issues.  It had been fine several months ago, and now they were having issues with dropped calls as nurses went into patient rooms.  I opened up the Prime Infrastructure floor plan and checked to see if any APs were offline, overloaded, and to see the power and channel plan.  At first glance, everything appeared normal.  Well, as normal as I would expect for a WLAN that had been designed for 802.11g data with APs in the hallways and now they’re running voice on the 5GHz radios.  As  you can see, most of the power levels are a power level of one and transmitting at 50mw.  Or are they…

 

 

When I visited the site, I had seen an 1130 series AP on the ceiling and incorrectly assumed that the entire hospital was most likely that model.  Turns out it was one of three, and the rest were 1242 series.

 

We’ve always been told that power level 1 is “the highest power”, and we assume that is either 100mw for 802.11b or 50mw for OFDM.  Turns out that a lot of digging will reveal that is not the truth.  Stay tuned.

 

I used AirMagnet Survey Pro to see what the 5GHz WLAN spectrum looked like on the floor.  I used a 15 foot guess range and used a Proxim 8494 adapter, which in my opinion, is most likely better than the chipset in the Cisco 7925 phone the nurses are using.  I dialed it down to -65dBm and this is what I saw (below).  I concur with what the nurses are complaining about.  When the user approaches the window, there are a lot of dropped calls.

 

 

 

After seeing the results, I Googled and found an old blog post from George Stefanick.  Here is his post on power levels:

 

http://www.my80211.com/cisco-wlc-labs/2009/10/30/what-you-need-to-know-about-tx-power-and-80211a-5ghz-on-a-ci.html

 

I used his blog post to remind myself of the debug commands and ran those commands while onsite and made a quick and dirty map on a whiteboard of what we were looking at.

 

For those of you who are just too tired to remember the channels and what band they’re in and don’t feel like looking it up, I’ll jot them down:

UNII-1 channels = 36,40,44 & 48
UNII-2 channels = 52,56,60 & 64
UNII-2e channels = 100,104,108,112,116,120,124,128,132,136 & 140 (a lot of organizations do not support these)
UNII-3 channels = 149, 153, 157 & 161

 

Now to use the debug commands to see what we’re looking at on these access points. (I changed the name for obvious reasons)

debug ap enable TEST-1
debug ap command "show controller do 1" TEST-1

<lots of output omitted>

TEST-1: -Channel Range- -------Rates------ Max Power Allowed

TEST-1: 36 to 48 by 4 6.0 to 54.0                    11
TEST-1: 52 to 60 by 4 6.0 to 54.0                  17 (THREE CHANNELS)
TEST-1: 64 to 64 by 4 6.0 to 54.0                    11
TEST-1: 100 to 116 by 4 6.0 to 54.0 17 UNII-2e, not supported by our clients
TEST-1: 132 to 140 by 4 6.0 to 54.0 17 UNII-2e, not supported by our clients
TEST-1: 149 to 153 by 4 6.0 to 54.0 17 (TWO CHANNELS)
TEST-1: 157 to 157 by 4 6.0 to 54.0    14
TEST-1: 161 to 161 by 4 6.0 to 54.0    11

Wow!  Did you catch that?  Most of the access points were on channels that the maximum power was 11dBm (12.5mw).  No wonder we’re not getting signal in some of those locations.  I can only guess that Cisco’s RRM decided to change some channels within the last few months (the channel plan was not frozen) and the result was decreased WLAN coverage.

A coworker and I whipped up a quick and dirty channel plan for the patient wing where the channels would support 17 dBm.  We decided to use channels 52, 56, 60, 149 & 153.  We assigned one channel twice, made the changes, and resurveyed.  It is my belief that every time you make a dramatic change like this, you NEED to resurvey using a WLAN Survey software package.  Here’s the result:

Remember that this is a hospital.  Rooms are available one minute, and two hours later they’re not.  Rooms open and close all the time, and we just have to accept that.

That room with the gray area is still a problem for us, even at an increased power level.  We’re going to do a complete redesign of the WLAN in the near future which will take care of the problem areas. 

After making the change and resurveying, we took two 7925 series phones to the floor and called each other.  I placed one phone on my laptop, situating the mouthpiece over the speaker of my laptop and played Beethoven while we walked through every room we could get into and never dropped the call.

If anyone has a link to where Cisco documents this in a comprehensive spreadsheet, please let us know!

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Adding a VLAN to a trunk for a WiSM1 WLAN controller

 

I came across something today I thought was a bit odd.  When adding a VLAN to a trunk in a core switch for the WiSM1 module, the command is slightly different than the usual “add” command.

The following two commands:

wism module 2 controller 1 allowed-vlan 2347

wism module 2 controller 2 allowed-vlan 2347

Result in the following configuration.  There’s no “add”…

wism module 2 controller 1 allowed-vlan 101,102,778,2347

wism module 2 controller 2 allowed-vlan 101,102,778,2347

 Here’s the lack of “add”…

My_Core_Switch(config)#wism module 2 controller 1 allowed-vlan ?

  WORD  vlan range 1-1001,1006-4094

 

What I found was the fact that issuing the command for wism module 2 controller 1 (to add VLAN 2347) on the switch resulted in a missed ping from my workstation  to the management IP of the controller.  Before making this change, I started a continuous ping from my wired workstation to the WLAN controller.  This has never been the case when using the normal “add” command when adding a VLAN to a trunk to a 4404 or 5508 WLAN controller.

The problem is when entering the command for the second module  "wism module 2 controller 2 allowed-vlan 2347".   The second module lost 25 of the continuous pings.  I was able to replicate this on the backup WiSM farm.  Same exact behavior.  My test workstation on Wi-Fi lost connectivity for almost two minutes.  I assume the trunk to the WLAN controller went down.

I didn’t see anything in the switch's logs about the interface going up and down in the switch.  However, I did look at the logs in the WLAN controller and it showed that it definitely took the trunk down for 90 seconds or so – depending how you look at the traps and interpret them.

 

 

 

 

 

WLAN Site Survey interference (check your noise floor)

I recently was tasked with doing a WLAN site survey in a healthcare facility.  There were two WLAN Engineers, me being one of them.  As we set up our gear down in the basement where we were staging all of our equipment, we noticed some rather odd results.

We started out with a Cisco 3502 with an autonomous image, both radios set to power level 11 as you might expect.  The 2.4 GHz radio set to channel one.

We loaded up our maps in the project and immediately noticed something strange:

We saw that we were sitting right next to our survey access point and seeing negative 59 or worse.  Much worse, actually.  We saw -76 as a signal strength, so we thought that there might be something wrong with the AP.  We pulled another AP out of the survey kit, also on channel one, and it produced the same result.  It had the same power and channel settings as the first one.

We started comparing laptops and coming up with other theories.  The only thing that was consistent was the signal level and both of us using the Proxim 8494 USB stick.

We decided to change the channel of the access point, and when we did, everything came back to “normal”.  If you happen to notice the dates are not in sync, that is because I had to go back and recreate some screenshots after we figured out what the problem was.

Since we were getting behind schedule, we decided to run with our survey on channel eleven and figure out the problem with the APs being on channel one later.

After we finished surveying,  I was able to break away and recreate the problem in our staging area predictably.  I also noticed that if I relocated to another wing of the facility, the problem did not exist.

After consulting a few friends (a shout out to JH & KP) I decided that I wasn’t as knowledgeable about as I thought I was in regards to how AirMagnet software worked.

I used my PCMCIA SpecAn card and set it to look at 2.4 GHz.  I was shocked when I saw the results when in our staging area:

Channel one was at nearly 100% duty cycle all the time.  Knowing this isn’t how the spectrum normally looks, I decided to walk the basement from end to end.  The diameter of the interference was about 300 feet, and only disappeared when I passed the mechanical room. 

I hopped on the elevator and went up each floor, walking the hallways with laptop in hand.

When I got to the fourth floor, this is what that signal looked like:

I came to the conclusion the interference was strongest in the immediate vicinity of our staging area.  After walking with my laptop hugging the walls, I was able to determine the source was in the clean laundry storage.  After walking through several times, I finally saw a small camera mounted above the door I entered through.  After consulting the proper employees, it was determined that it had been installed 6+ years ago by someone who no longer worked there.

This facility has been complaining about dropped VoWiFi calls for quite a long time.  Since the WLAN profile was set to 802.11a only, but a bug in the Cisco controller software allowed it to broadcast both frequencies, that explains the dropped calls.  The phones were set to either frequency, thinking that they would only be on 5GHz since the profile was configure that way.

There are several lessons learned from this task.

1.       Don’t assume your WLAN isn’t working due to a bad design.  Invest in the right tools and training so you can detect these anomalies and fix them.  This particular problem had gone on for years!

2.       If you are stumped, don’t be afraid to reach out to others in the industry.  The two people I consulted with are “theoretically” competitors, but they helped me out with their thoughts.

3.       If you don’t have any type of frequency coordination in your enterprise, it might not hurt to have a policy that gives some direction.

4.       Baselines.  What did it look like when it did work?  A routine WLAN validation with AirMagnet Survey Pro, along with some protocol and spectrum captures sure doesn’t hurt!

I still don’t understand why that the camera with 100% duty cycle affected my AirMagnet survey and analyzer results.  After cycling through our survey access points and finally wrapping my head around it, I noticed that our signal strength was -69 and the noise was -67.  Yes… the noise floor was higher than signal strength of the survey access point.

My access point was right next to the survey laptop.  The signal strength of the AP was really in the negative thirties, but it shows up as almost negative seventy.  I don’t understand why I don’t see -35 for Signal right next to -67 for noise.

Honestly, I should have paid attention to the noise floor as soon as I notice a result that I did not expect.  I also did not check to see what channels the existing access points were one.  Had I looked, I would have noticed that none of the APs were on channel one.  Hindsight is 20/20 I suppose. 

When we powered up the next survey AP and saw it was -75, I still didn’t notice the -70 noise floor.

Thinking about this experience, I do wonder how this will affect my WLAN Validations in the future.  What will my validations look like when I have a continuous wave transmitter hogging a channel?  Will it show a low coverage area?

Have you ever been down this road before and seen what I have seen?  What was the result?

Does anyone have any suggestions as to why I was seeing a -75 dBm right next to a survey access point?

 

 

 

 

 

 

High Density Wi-Fi on a Cisco Controller

This blog post applies to those of you who wish to deploy High Density Wi-Fi in your environment.  This is not meant to replace "professional services" – if you are a large sports venue with 75,000 seats, this blog post is not for you.  If you are a smaller environment – such as K-12, University or other college and wish to take a stab at it yourself, please continue reading.

Many of these ideas have been gleaned from the plethora of HD Wi-Fi deployment guides freely available to you on the Internet; Aerohive, Aruba and Cisco to name a few.  I'm going to be using Cisco gear for this post, however please be aware that almost every vendor I know of has an HD Wi-Fi guide and they all say about the same thing.  I'm not taking sides at all – I love all the competition out there!  I would also like to thank a little birdie for sending me a controller for the purpose of this post.

I am under the assumption you know something about how to build an enterprise network, and have deployed an enterprise class wireless network in the past, or had it done for you via professional services and you feel confident in your abilities and want to make your wireless network better.  You should probably have the skill level of a CCNA and CWNA.  I'm focusing on the wireless part – not the backend which will also need to be designed properly.  So, here goes…

The goal is to have smaller cell sizes in the 2.4 GHz range.  This enables faster communication due to higher data rates.  Higher gain directional antennas create small cells and reduce co-channel interference.  More cells equal more throughput, which means more capacity.  More capacity equals happy people. 

One thing to consider if you are designing for your public environment is how many SSIDs you might need.  You many need WLANs for the following groups of users:

a.      Ticketing (scanning tickets to enter the venue)

b.      Point of Sale (textbooks, t-shirts, hot dogs, etc)

c.       Administrative (employee access)

d.      Press (as annoying as they are, they still need access)

e.      Inventory

f.        Dual WLANs separated by frequency (a 2.4 GHz and a 5.x GHz WLAN for             the same purpose)

g.      VoWi-Fi

When designing your WLAN, it is important to have accurate floor plans.  They will help you visualize where your clients will be, where your RF will be, and where you don't want your RF to be.  Place close attention to that last sentence!

The 2.4 GHz spectrum is full.  There are so many devices in there, both 802.11 based and not, that it is difficult for these devices not to be stomping all over each other.  802.11 is a polite protocol – if an AP is on channel one and it hears another AP or client on that channel, the AP will wait until channel one is clear before transmitting.  Adding access points to a large auditorium with omnidirectional antennas does not necessarily increase throughput or the number of users that can be online if many of the access points are on the same channel and can hear each other.

Designing an HD WLAN can be difficult.  There are methodologies that are in place that most of the WLAN vendors all agree upon – so I intend to share them with you since I have read through most of the HD Wi-FI guides out there.  Here are a few that seem to be pretty common goals and methods:

a.      HD Wi-Fi is designed to minimize the cell size

b.      Antenna placement – don't avoid a difficult antenna location because of                  cost

c.       Design more APs than needed and turn off some 2.4 GHz radios if   
          necessary

d.      Use directional antennas in HD open environments – auditoriums, arenas,             conference centers

e.      Omnidirectional antennas may be used in regular, non-HD Wi-Fi areas – as          long as they are enclosed

f.        You're going to have to get creative with hiding APs and antennas in non-               conventional places

As example, let's use this seating chart of a 16,000 seat basketball arena.

This is a large, open air environment.  We'll aim for 250-500 seats per access point.

If you put three dozen dual band access points with omnidirectional antennas in there, you might find that the Wi-Fi doesn't work that well, even though "the math works out".  Your channel utilization will be excessive – I'll talk more about that later.

However, if you were able to mount APs in the ceiling structure (roughly 50 feet above the seats) with 30 degree beam width antennas aiming down at the seating areas (imagine each one of those colored seating areas as a cell) you might find that co-channel interference (CCI) was reduced drastically.  Notice I stated "from the ceiling structure above", and not "from behind the seating area".  The reasoning behind this is that when your venue is full of people, the signal is attenuated more when the antennas are behind the seating areas.  I've seen APs talk to clients at 93 feet when the antennas are behind the seating area – however it didn't work that well.

The other issue with CCI is that signals come from your non-bowl/arena access points.  You've changed out all those omnidirectional antennas for directional ones, but the signal still leaks in from the lobby and concourse areas.  The key here is to use patch antennas in those areas, aiming away from the seating area.  The more "directionally contained" your signals are, the less CCI you will have.

Keep in mind that there's other Wi-Fi out there.  You might have Wi-Fi in the parking lot outside your facility for the hot dog vendors, ticketing, etc.  The best thing to do here is to use directional antennas when possible to minimize CCI.  At least in those areas you won't have as many users as you might have inside the seating area of your basketball arena. 

Other Wi-Fi exists that you might overlook.  Moveable partitions slide to the side and open up your environment, changing the dynamics of your RF propagation.  Glass walls in arena suites slide open and let RF spill out into a seating area if you are using an omnidirectional antenna.  I cannot stress enough the need to properly survey your area and to use patch/directional antennas!

We're going to start to dive into some of the nuts and bolts of HD Wi-Fi shortly… starting with why we want to minimize the RF propagation of the cell.  By limiting the mandatory and supported data rates and by limiting the power output of the transmitter, we keep the cell size small.  When we do that, we can keep the CCI low and re-use the three channels in the 2.4 GHz spectrum.  The channels I speak of are 1,6, and 11.  If you use anything other than that, you risk overlapping.  If I had a TARDIS, I would go back in time and explain to the IEEE that we only need three channels and plead with them to make the in-between channels unavailable for configuring into your network.

In Wi-Fi, if a frame didn't get to the recipient (the transmitter didn't receive an ACK frame from the recipient) it will try and retransmit the frame and/or lower the data rate and try again.  This leads to an even busier channel!  Think of it as trying to talk to your best friend sitting right next to you at a Van Halen concert.  You have to repeat yourself three times, and then say it slower, then reduce yourself to charades in order to communicate.  Next thing you know, all clients are using charades and your Wi-Fi is unusable.  What do you do?  You go outside in the hallway or break room to have your conversation – you reduce your co-channel interference!

If you are planning an HD deployment for an existing facility, I highly recommend (at a minimum) getting a copy of AirMagnet Survey or equivalent product and learn how to use it.  It will give you the ability to test the coverage patterns of the antennas you would like to deploy.  You may go through several different antenna types before you are happy with your selection.  Keep in mind that surveying a venue full of people is challenging, and that your site survey's data will change dramatically when you compare an empty auditorium and a full one.  If budgets allow, purchase AirMagnet's bundle of Survey Pro, Wi-Fi Analyzer and Spectrum Analyzer.  All will prove themselves valuable over time.  I also highly recommend AirMagnet's training class for using their products.  Check out Keith Parsons' "Seven Rules for Accurate Site Surveys".   Check out all his other stuff, too.  http://www.youtube.com/watch?v=EdS2YYgHf3w

The Certified Wireless Network Administrator (CWNA) book from CWNP.com is also a great resource for aiming you in the right direction.

Don't forget to set your survey access point to the same settings (power, data rates) you want for your production HD WLAN environment.  I recommend surveying at 1.5mW, 3mW or 6mW with a higher minimum data rate of 18 Mbps.  That means that 1,2,5.5,6,9,11 & 12 are disabled.  Keep in mind you are going to be using higher gain, directional antennas which will increase the power in your small cell.

If you are designing Wi-Fi for an existing facility that may already have Wi-Fi installed for ticketing or Point of Sale, I highly recommend planning your new system to cover those areas for them and decommission the other system.  After all, you don't want competing WLANs in the same room!  I wouldn't mention this if I haven't come across it several times.

Let me take this subject to the next level.  Every SSID requires all the management overhead traffic associated with it.  This means the more SSIDs you have, the less airtime you have for your client devices.  If you have older, existing 802.11b ticketing and point of sale systems with 802.11b and other lower data rates enabled, the beacons will all go at the lowest data rate – using up valuable airtime.  One of the best practices in my opinion is to limit your high density environment (basketball stadium, auditorium, or library) to a single SSID.  Other areas in the facility can broadcast up to four, but by all means limit them to the best of your ability.  The next best practice is to disable 802.11b entirely, however some of those ticket/inventory/PoS devices are 802.11b only. 

Moving on – be sure that the gear you select can support the expected number of access points and concurrent client connections.  Don't skimp on redundancy – even though you might not have a basketball game twice a week, that's no reason to not have dual controllers, dual switches and batter backups in case of power failure.  Keep in mind you may see 30-40% of the seats with a client device – that means you may see 6,400 client devices in a 16,000 seat basketball stadium on a college campus.  Each one of those client devices will need an IP address – I highly recommend using a robust DHCP server to handle to the load.  Ignore temptations to use the DHCP service built into your controller.  I've had to dial down the lease times in heavily loaded college libraries to two hours to accommodate the masses flowing in and out on a classroom schedule.

Keep in mind this blog is not being written for you to use as a guideline to install Wi-Fi in a large public venue.  My intended audience is smaller venues where you are planning on doing the design and installation in-house.  If you are a 45,000 seat stadium and want to streaming media via multicast – seek a reputable firm with professional services.  Think CWNE.

Here are some Network Design and Configuration recommendations:

1.      Keep WLANs being broadcast down to one WLAN in HD areas, and four or                 less in your non-HD areas.  Cisco shops can use AP groups if you are sharing          a  controller.  Other vendors I'm sure have similar features.  *hint* *hint* (I don't            have any other gear to play around with)

2.      Examples of WLANs are:

a.      Employees

b.      Point of Sale

c.       Ticketing

d.      Press/Guest

e.      Students/fans

f.        VoWi-Fi

3.      WLC controller redundancy

4.      WLC sizing – make sure your controller can handle all those clients

5.      Aim for single digit Channel Utilization when your building is empty (it can be               done)

6.      Remove lower data rates on both 5GHz and 2.4 GHz (I will show you how)

7.      Use 5GHz for mission critical WLANs such as PoS, Ticketing & Press

8.      Cisco's RF Profiles in 7.2 and above will help out for HD vs non-HD WLAN                  coverage areas when using a single controller.

Network Operation recommendations:

1.      Use your spectrum analyzer to sweep the channels you are using to see if              there are any interference sources.  I once saw a     high definition camera            on a 5Ghz WLAN channel during a sporting event, so I quickly removed that          channel from our Dynamic Channel Allocation list so it would not interfere.

2.      If you are constantly changing your power and channels, you'll need to                    resurvey each time.

I mentioned Cisco's RF Profiles.  An RF Profile can be created for the 2.4 GHz radio or the 5GHz radio.  The profile is applied to a group of access points in an (Cisco) AP Group.  An AP group, for instance, is an area of your network that you want a specific group of WLANs broadcast – the lobby might require admin and press, but no guest WLAN.  If you were designing your college's basketball arena, you would probably create an AP group to limit the seating area to one SSID.  In that AP group, you would assign an RF Profile that removes all of those lower data rates and limits the amount of power via a threshold.

There is also a Cisco command that helps to deafen an access point.  What this means is that you can set your controller/access points to ignore everything starting at a level you determine, -78 dB, for example.  You won't want to do this for every AP in your network, so instead of using the global command you can set each AP individually.  The command is controller> config 802.11a|802.11b rx-sop threshold -78 ap <name of AP>

The threshold value is 0 for disabled, or -60 to -100 dB.  Use this command with caution!  With any luck, that will be built into the RF Profiles feature in the next version of code.

Now we're going to configure a controller for High Density Wi-Fi.   Let's pretend we're a college with a descent basketball team and a great library system - and both departments have dug deep into their pockets to fund this project.  Both departments have a high density of WLAN client devices and they're fed up with the really slow Wi-Fi.

Let's start with defining some of the SSIDs that have been requested on campus.  We have determined to put the Library and Basketball arena on the same controller along with a few small buildings, and we're going to size and configure the WLANs and VLANs as follows. 

  

1.      ChickenHawk

a.      5000 devices

b.      Secure Wi-Fi for students

c.       802.11a/g/n, WPA2,PEAPv0, MS-CHAPv2, AES, OKC, Silver QoS

d.      192.168.32.1-192.168.63.254

2.      GoHawksFast

a.      5000 devices

b.      Open access Wi-Fi for events

c.       802.11a/n/ac, Open auth, Bronze QoS

d.      192.168.64.1-192.168.95.254

3.      GoHawks

a.      5000 devices

b.      Open access Wi-Fi for events

c.       802.11g/n, Open auth, Bronze QoS

d.       192.168.64.1-192.168.95.254

4.      VoWiFi

a.      100 phones

b.      Non-broadcasted WLAN for Wi-Fi VOIP handsets

c.       802.11a/n, WPA2-PSK, AES, OKC, Platinum QoS

d.       192.168.100.1-192.168.100.254

5.      Barcode

a.      40 scan guns

b.      Non-broadcasted WLAN for ticketing

c.       802.11a/g/n, WPA2-PSK, AES, OKC, Platinum QoS

d.      192.168.101.1-192.168.101.254

6.      HawkPoS (point of sale)

a.      60 terminals

b.      Non-broadcasted WLAN for PoS terminals

c.       802.11a/g/n, WPA2-PSK, AES, OKC, Platinum QoS

d.      192.168.102.1-192.168.102.254

7.      PressBox

a.      60 press users

b.      Broadcasted WLAN for ease of use

c.       802.11a/g/n, WPA2-PSK, AES, OKC, Platinum QoS

d.      192.168.103.1-192.168.103.254

For security reasons, our ChickenHawk SSID will map to its own VLAN that will be secured on the backend, as will the VoWiFi and Barcode SSIDs.  Again, this blog is lightly touching on the basics of VLAN and WLAN controller configuration since I'm writing this more for how to configure HD Wi-Fi. 

The GoHawksFast (5GHz) and GoHawks (2.4 GHz) WLANs are for our HD areas – the basketball "bowl" and the large libraries where the students congregate and stream media while studying.  The thought behind the "fast" is to try to entice people to choose the 5GHz WLAN if they see it on their device.  Both WLANs are going to map to the same VLAN on the backend.

After a considerable amount off thought, I decided to lab up this fictitious HD environment in the lab on a Cisco controller running 7.6.x, however you'll only need 7.2 or greater to do the things I am writing about.  Stay tuned and I will show you show you what worked for us in our real environment.  Your environment may vary from ours, so please take that into consideration and don't copy this configuration in entirety and expect it to fit your environment perfectly.

I am also open for comments and suggestions and will revise this post as necessary.  If you see an error, or think I should add in more information, please feel free to comment and we can modify as needed.  After all, this post is to try to help others out as much as they help me!

I'll start off with screenshots of the 5GHz (802.11a) global parameters

                               

        

 

A number of Wi-Fi clients present in production environments do not actually support connections in the UNII-2 Extended Band. This allotment of channels ranging from 100 to 140 is essentially invisible to older Wi-Fi clients created before this band was made available for Wi-Fi by the FCC.  Since we have many channels to choose from, I'm going to exclude them.

   

 Now for the screenshots of the 2.4 GHz Global settings… (note 802.11b is turned on)

   

 

With the above defaults applied to all WLANs, I used AirMagnet Wi-Fi Analyzer to look at channel one's channel utilization.  This is a snapshot of the CU when five APs are currently set to channel one in a simulated HD Wi-Fi environment in an RF isolated lab.  The APs are in the default AP Group, and there is one client device - my laptop.  The CU is going to change drastically for the better and I'm going to show you how to do just that.

Now let's take a look at what this post is focusing on – the HD Wi-Fi WLAN Profile.  I'm going to focus on the GoHawks 2.4 GHz WLAN since that is the WLAN where we are seeing most of our wireless clients at sporting events.  Here's a screenshot of the most important WLAN when it comes to a basketball game, and a few "before and after" shots as well.  Below is the HD WLAN Profile for 2.4 GHz in the basketball arena.  It has no security, is mapped to a VLAN outside the network and is meant for fast Internet access.  It also has a high channel utilization!

 

It is tempting to turn on Application Visibility on this WLAN for statistics, but we chose not to.  You can read up on this nifty feature in a previous post of mine.

I'm going to point out that this is the only WLAN we are applying any ACLs to.  We're doing this to cut down on the Bonjour traffic since most of the clients devices in our HD Wi-Fi environment have that feature enabled.

So lets create two ACLs.   Here we go:

Apply those two ACLs to your HD WLAN.

Now here's the "secret sauce" for decreasing that channel untilization and making your HD environment a lot better for your clients.  We're going to use the Cisco's RF Profiles along with the AP Group feature to override those 2.4 GHz default settings.  I'm going to create an RF Profile then an AP group and call it Basketball_seating and lastly, apply it to all of the access points inside the arena.  Keep in mind when you do this, the arena APs all reboot.

Since this post is focusing on HD Wi-Fi, I'm going to concentrate on the basketball's 2.4GHz RF Profile.  Create an RF Profile.  Use a naming convention that makes sense for your environment.  Mine are "RFP" for RF Profile, <name of WLAN> <coverage area> <24=2.4GHz>.  You'll thank me later for this advice.

Here is where we override the 2.4 GHz defaults and disable 802.11b and make our lowest supported 802.11g rate 18 Mbps.

We're using the RRM tab to override the controller's defaults to limit the power thresholds in our basketball arena.

We didn't change anything on these last two tabs.

Now create an AP group for the HD Wi-Fi area.  I cannot stress enough to use a proper naming convention so you can figure out what it is for months from now.

Edit the AP group after it is created, and use a proper description so you can figure out what it is for a few months from now.

AP Groups allow you to only broadcast the WLANs you want.  For the basketball arena we only want to broadcast one WLAN.  If we used the default AP group, we would automatically broadcasts all WLANs.

Then we apply the RF profile we created in the earlier step, overriding the global 802.11b 2.4 GHz parameters.

Use the APs tab to add the arena's access points to the newly created AP Group.  They'll reboot.

After the APs reboot, they come up and I let my AirMagnet Wi-Fi Analyzer run for about ten minutes.  The CU is now down in the single digits!

So, to recap, this is what we did to make our HD Wi-Fi exponentially better.

1. Replace omnidirectional antennas with directional ones and created small cells
2. Create ACLs to block Bonjour
3. Create RF Profiles to limit data rates and transmit power
4. Create AP groups to limit number of WLANs being broadcasted

Here are a few screenshots from AirMagnet Survey Pro.  These are the coverage patterns from a recent walkabout through the bleachers with directional antennas mounted up high, aiming down at the seats:

This is the walkabout with all 2.4 GHz channels shown.  Notice how we're not getting all the way down to the front seats.  This is because those seats are the farthest from the antennas.

Here's channel one.  The light blue areas are where the signal is strongest, which is where the antennas are mounted.  This walkabout was done when the venue was empty.  This changes when full.

Here's channel six.

Channel eleven:

 

Here's channel 149.  I decided to show one 5GHz channel just so you could see the coverage pattern differences.  The cell is much smaller than the 2.4GHz channels - but that's fine because we have more 5GHz antennas online than 2.4GHz.  The APs are all dual band, however we shut some of the 2.4GHz radios off.




Please feel free to write to me with any feedback, comments, concerns, or what you did in your HD Wi-Fi environment.