How to get your APs to leave your Cisco WLAN Controller in a pinch!

 

You’re probably wondering why on Earth I am writing a post on how to make your access points flee your WLAN Controller.

 

Here are a few reasons why you might want to do this: (I have seen all of these situations)

 

·         You have two WLAN controllers, each backing the other up and you want to upgrade them.

·         You just put a new image on your WLAN controller and it is corrupt, and your access points are hung up on downloading for hours.

·         You want to move your APs to the other controller quickly, but they are on the same subnet and you cannot use a VACL or other ACL.

·         You need to move the APs and don’t feel like scripting, and don’t have WCS/NCS/Prime Infrastructure Access Point Templates setup.

 

The one in the middle recently happened to me.   Here’s my scenario:   Two WiSM1 blades were half loaded with APs, each backing up the other.  I staged two new 5508s to replace the aging WiSMs.  Little did I know, there was a problem with the image on 5508 #1.  I moved a dozen APs manually from one of the WiSMs to the 5508, and they started downloading as expected.  I was migrating the APs from the WiSMs (on 7.0 code) to the 5508s running FUS 1.7.0.0 and 7.4.121.0 operating system, so as soon as they landed, they started to do the upgrade.  Problem was, the image was somehow corrupt and the access points would start downloading, then reboot, and then start downloading again.  If you’ve ever been in this situation before, you probably know that you cannot configure an AP while in the downloading state.  Obviously I didn’t know the image was correct, but knew I needed to back out of my change, and do it quickly.

 

Unfortunately it was 1 am in the morning, and there was about 75 miles of Interstate between me and the box.  I had three options at that moment:

 

·         Reboot the WLAN Controller.  When it comes back up, the APs will go back to the same situation

·         Shutdown the Port Channel at the switch level, leaving the controllers stranded.  (and leaving me locked out of them as well)

·         Somehow configure the WLAN controller so the APs leave and go find their configured Secondary Controller, but leave me access to the WLC.

 

I chose option three.

 

But how?  The WLC is on the same subnet as the others.  There’s no configuration check box that reads, “Do not respond to AP join requests”.  Hint hint. (feature request)

 

Here’s what I did.  I changed the hostname of the controller, because I had configured it on the APs that I moved to the WLC.

 

 

Then, I had to shut off the 5GHz and 2.4 GHz at the global level on the WLC:

 

 

 

And as soon as that is done, I changed my Country Code.  I removed the checkmark for US, a country that I have visited many times – France!

 

 

 

Viola!!  The access points fled quickly to their configured Secondary Controller, downgraded, and the network was back up and running.

 

This allowed me to go to bed, and leisurely wake up at 5am in the morning with several thoughts and ideas of what went wrong last evening.  I called Cisco TAC at 6am and got a response relatively quickly.  (note to self – call Cisco at 5am for faster response time)   During our troubleshooting I decided to see if another access point in a building that just happened to be closed all weekend would join the redundant controller I had configured.  It immediately joined and we came to the conclusion that the image on the WLC had somehow become corrupt.  I don’t know how, since I personally staged both WLCs with the same TFTP server and image while in the lab, and my test AP running that same image worked associated just fine.  It was definitely the “downloading” portion of the operating system that was faulty.

 

I downloaded the same exact image again from CCO, and then immediately transferred it via TFTP to the WLC and rebooted it.  I reversed the country code and other configurations and put it back to “normal”.  I then send an access point to the WLC and it joined, downloaded normally, and all is well now.

 

One thing I want to mention is during the heat of the moment, I did a web search of “APs stuck in downloading”, “Access Points stuck in downloading”, and “Cisco AP downloading” and did not get any hits.

 

I hope that this post can help other WLAN Engineers out that may encounter the same scenario and frantically Google those search words, only to come up with nothing. 

 

Cheers!