Wednesday, April 29, 2009

6" WATER MAIN - Do NOT assume that it is EMPTY!!!

So it started out as a nice peaceful day at work. I had several things on my schedule, but nothing too bad, just some server installs for a new system (that is what I do at work). Then during a conference call to support, we all of a sudden lost power. That happens quite a lot though in my offices, so I didn't think anything of it. Then we slowly started to realize that the power was out throughout the entire building.

I hear someone talking outside about a water main, and so I decided I better go investigate. Upon investigation, I walked into the back door of the warehouse that occupies the other end of our building, only to find about 4" of water being shoved out the door by crazed warehouse workers and secretaries.

I guess there was someone there working on the fire suppression system that had assumed that the lines were drained and the valves shut when he tried to remove a broken section of the water main, he was wrong! OOPS!!!

So there was a man made geyser in the warehouse for several minutes as he scrambled to figure out how to shut it off, hence the 4" of standing water on the warehouse floor.

So the fun part; the geyser (so powerful that it pealed the paint off the wall) was pointed directly at all of the power distribution panels for the building, which also included the power distribution panels that feed the Data Center/Server Room for over 70 buildings spread throughout the valley and 100,000 users).

Miraculously the data center was still functioning, while the rest of the building was down. Several minutes after the water stopped someone went to check on the breaker boxes, and found that they were full of water. The one that we use for the Data Center however only had about 6" of standing water in it and was still passing power through it???

The servers were up for the time being, but as the water soaked into the electronics, it started to freak out, and caused the power to fail, and then kicked on the generator. The generator feeds through that same box, so I don't know how it was working but everything, except for the cooling units, stayed on in the server room.

Without the cooling units, the server room started to get hot quick, so we were forced to shut down everything that was not marked as Critical. We left the main router up as well as the security devices that allow for Internet access to and from the site. Other than that we shut about everything down and went to help squeegee an estimated 5,000 gallons out of the warehouse, FUN!!!

We all walked around with wet shoes for the rest of the day, and are still waiting for repairs to be done on the power and AC units. In all though we never lost Internet connection throughout all of the pain and chaos that we faced all day, that was awesome as it was our priority to keep these services open for the thousands of users that depended on them.

Quite the day at work!!!

Monday, April 20, 2009

110 In the Shade - PGHS Spring Production

COME SEE IT! You won't be sorry!
Playing every night this week starting at 7:00 pm

See you there!

Le Rêve - Las Vegas

I Love Spring Break! Especially if I can get away and have a great time.

This year I ended up taking my sister down to Las Vegas for the long weekend to get away from the 6" of snow and cold. We succeeded and ended up finding the sun, which was wonderful.

We did several things, but I my favorite was the show, Le Rêve at the Wynn.

It was incredible. You walk into the theater, and there is seating all around what should have been a stage, but it was a huge pool of water. On the water was a bed, floating there in the middle of the pool. It was lit from all different angles and the light was slowly changing from blue, purple, pink, and several other colors. My sister and I just sat there and watched the bed and looked at all of the other things in the theatre in awe (I know, I am a tech geek).

I don't even know how to explain the show, but it incorporated about everything that I could think of. There were flying acrobatic moves, fun dancing, great costumes (they all got wet!) surprising props, techno effects, great sound, and everything appealing to an audience that could involve water, acting, dancing, and sound.

Here are a couple images that I found on-line, but it is late and I need to go to bed because if now I will become the monster that you can see in the third image (I think that it is the devil).


Fun show, I would recommend it!



While in Vegas my sister and I also spent some quality time watching the incredible Belagio fountains (I love them). We also had to spend time at the hotel pool soaking up as much sun as we possible could (which reminds me I need to apply aloe).


Hope you all had a great spring break, and if you maybe you could come with me next year! =0)