On-site pre-staging of Cisco routers

From April 10th to the 16th, I was part of a team of engineers involved in pre-staging several Cisco 7200VXR and Cisco 7606 series routers for our client.

It has been quite a while since the last time that I dabbled with Cisco IOS release 12 or even performed hardware diagnostic tests on Cisco routers.

I learned a lot not only from the client’s team of senior engineers but from my own colleagues as well who are more experienced in this field than I am.

For the past decade, I’ve been involved with web development and systems administration while my network administration / infrastructure skills still require some more practice.

I learned some new IOS commands that wasn’t taught at CNAP (Cisco Network Academy Program) at DLSU back in 2003.

Although, the examples used back then were academic in nature and designed to help us pass the CCNA (Certified Cisco Network Associate) exam and not based on a real-world scenario.

The good thing back then, we were able to extensively play around with the Cisco 2500 series routers and had learned so much from it.

Pre-staging was a physically challenging task wherein it involved moving those heavy Cisco 7606 routers around the staging area from time to time.

Besides developing some muscle mass on both of my arms, I’ve gained a considerable amount of hands-on experience that no technical book or training can provide.

And this makes me one happy Cyberspace Janitor sailing along the vast seas of 0s and 1s, off to distant nodes beyond the blue nowhere to clean up someone else’s mess.

 

Case File 01: Who killed the PSU?

On the early morning of March 23, 2007 just minutes away to the break of dawn, my cousin Jan was playing Sims 2 on his home computer when he had been startled by a loud pop and electrical sparks coming from the power supply unit (PSU).

In a matter of split seconds, the electrical power on his computer was gone. I was informed about the incident a few of hours later while I was in the middle of a clean up in cyberspace but physically located at a client’s site.

In our exchange of SMS messages, he noted the absence of soot and scent of burned plastic insulation which supposed to be present based on the assumption that one of the PSU’s components blew up.

A couple of weeks earlier, he purchased a brand-new AVR (automatic voltage regulator) after his old one finally gave in.

After his home computer arrived at my makeshift lab, the PSU’s voltage switch was initially checked and verified that it had been set to 230v. This is to eliminate the possibility of foul play (well, you never know…), loose voltage switch or just plain user negligence (it happens to everyone… me included).

The PSU was then removed from the casing and checked for soot marks that Jan might have overlooked when he initially inspected it.

Next, I took a sniff from the PSU vents and hoped to get even a minute scent of burned or melted plastic insulation but to no avail.

Personal note:
Never take a whiff from one of the PSU’s vents caked with layers upon layers of dust because of obvious reasons which I (of course) stupidly forgot in my eagerness to solve this villainous mystery.

I then implanted the busted PSU into one of my test machines but it failed to power up its motherboard. Pressed for time, I tentatively abandoned the idea of opening up the PSU and study its internal components to figure out the source of the problem.

After that I inspected the motherboard and attached devices of Jan’s home computer for traces of burn marks and soot but again to no avail.

Rummaging from spare parts bin, I was able to retrieve a relatively old PSU and secured it on the motherboard and then hit the “on” switch, it powered up just fine.

Out of time, I rushed to the nearest reputable computer shop within the lab’s vicinity and purchased a brand-new PSU with the identical voltage and wattage.

Jan’s home computer was sent home after its new PSU had been implanted. Weeks had already passed since that incident and I still find myself wondering on what caused the blow-up.

Was it due to the PSU’s normal tear and wear? Was it due to the old AVR that might have affected the PSU’s internal components? Or could it be the new AVR?

I will definitely not cease wondering “who” killed Jan’s PSU until I set the time to perform an “autopsy” and find out.

 

Restarting the server’s OS can work wonders

Today, I’ve learned a very valuable lesson in debugging and testing a CMS (Content Management System) application running in Apache Tomcat specifically the version 5.0.28 for the Windows platform.

That is restarting the server’s operating system can work wonders!

After I’ve received the latest patch from the development team and implemented it on my test server then restarted Apache Tomcat’s service; the new enhancement doesn’t seem to work at all.

How could this be? I wonder.

My test server has the same directory structure and environment variables as with the development team’s server. I’ve checked and validated the checksum of each file to ensure that my copy hasn’t been corrupted in someway during the file transfer.

Before calling up the development team, I cleared the browser cache as well as Apache Tomcat’s Catalina folder which also functions as a cache; to be sure that none of the old files interferes with the operation of the new ones then I once again restarted Apache Tomcat service but still to no avail.

I called up the development team and tell them about the news; we spent more than an hour over the phone, figuring out what went wrong and why is it running flawlessly on their machine and not on mine.

We decided to call it for the day, take a hot shower, a meal and then some sleep and hope that tomorrow the answer to our problem will come up.

Just as I’m about to write an email to our client and inform them of our little predicament hoping to buy some time to figure out what is wrong with the patch, I restarted the server’s Windows Server 2003 OS out in a fit of frustration.

Then I made another test and to my surprise the new enhancement works!

It works!!!

All it needs is just a simple operating system restart!

Why didn’t I think of that in the first place?

Hmm, probably because I thought that restarting is too simple to solve the problem, that’s one valuable lesson to add up on my collection of career-related lessons.

A simple solution can save anyone from pouring countless man-hours as well as to gain more time in pursuing other important pursuits such as improving one’s quality of life and working smarter.