Are you running MAC OS X 10.7 and have a Celerra? It may be time for an upgrade!

As just about everyone on the internet knows, on July 20th Apple released the OS X 10.7 (aka Lion) to the public. $30 gets you a boat load of new features. One of these features is a completely rewritten CIFS client. For those of you who don’t know, CIFS is the protocol used for windows file sharing and is a big part of the EMC Celerra / VNX product. We have identified an incompatibility within our code. The good news there is a fix available for all DART code families (5.6, 6.0, and 7.0) and we are encouraging everyone to upgrade as soon as possible.


On July 14th, EMC has released ETA emc263721 (powerlink credentials required) to address this issue. An ETA (EMC Technical Advisory) is a way for EMC to notify customers proactively to address issues such as this before it happens in their environment. This details the problem and states the current fix. For this issue, we have put the fix into the following code levels:

• or higher

• or higher

• or higher

• or higher

• or higher

You can figure out your code version by running the following command from the CLI: “server_version ALL” (without the quotes). If your current version is the same or newer than the versions I listed above, then no action is required on your part and you are fine to deploy OS X 10.7 in your environment. If your code is below these levels I urge you to upgrade as soon as possible (especially if your environment contains a large number of Macintosh computers). To schedule an update, simply call EMC Support (800-782-4362), open a service request on powerlink, or speak with your local field resources.


Understanding the EMC VNX/Celerra AntiVirus Agent (CAVA): Part 1 – server_viruschk

CAVA is one of the few parts of the Celerra/VNX that cannot be configured and monitored from the GUI.  Most, if not all, of the information you need about cava can be found in the command line.  Over the course of a few posts, I will start with a fully working cava setup, and then work backwards to break it so you can see common implementation problems and possible performance bottlenecks.  In this first post of the series, I will go line by line through the output of server_viruschk so that you can understand just what the output is saying.  For reference, this is the output I will be working with:
[nasadmin@UberCS ~]$ server_viruschk server_2
server_2 :
 10 threads started.
 1 Checker IP Address(es):     ONLINE at Thu May 26 19:41:13 2011 (GMT-00:00)
                        MS-RPC over SMB, CAVA version:, ntStatus: SUCCESS
                       AV Engine: Symantec AV
                       Server Name: cava.thulin.local
                        Last time signature updated: Tue May 17 05:55:23 2011 (GMT-00:00)

 1 File Mask(s):
 5 Excluded File(s):
 ~$* >>>>>>>> *.PST *.TXT *.TMP
 RPC request timeout=25000 milliseconds.
 RPC retry timeout=5000 milliseconds.
 High water mark=200.
 Low water mark=50.
 Scan all virus checkers every 10 seconds.
 When all virus checkers are offline:
 Shutdown Virus Checking.
 Scan on read disable.
 Panic handler registered for 65 chunks.
 MS-RPC ClientName: ubercifs.THULIN.LOCAL


I will now go line by line starting with the first one.
  1. 10 threads started.
    • This is the number of threads for cava.  Each thread represents a file that can actively be scanned.  Cava will process up to 10 files at once to distribute across your available cava servers.  Any additional files will be put into a holding queue until cava can get to them.  This limit here is so that we don’t overwhelm the av software running on each cava server.  This limit is adjustable by the support lab if it is determined that this will solve a performance issue.
  2. 1 Checker IP Address(es):
    • This line tells you have many cava servers you have defined in your viruschecker.conf file.  In this example, I only have 1 server defined, but you should be running at least 2 servers at a minimum.
  3.                                  ONLINE at Thu May 26 19:41:13 2011 (GMT-00:00)
    • This line tells you the IP address of your cava server as well as the status and the last time we checked it.  If that line says anything other than ONLINE, there is a problem with the connection from the windows server to the celerra and that server will not be used for scanning.  More information on possible errors will be in a later post.
  4. MS-RPC over SMB, CAVA version:, ntStatus: SUCCESS
    • This has 3 pieces of useful information.  The first is the connection method we use to send commands to the cava agent.  In this case, we are using the MSRPC protocol.  Older clients may use the ONCRPC protocol, but this is not supported on 64 bit systems.  The next part tells you the version of cava you are running.  As of writing this, i am using the latest version (VNX Event Enabler 4.8.5).  Like above where we reported the connection from windows back to the celerra, the ntStatus section reports the status of our initial connection to the windows server.
  5. AV Engine: Symantec AV
    • This tells you the AV software we detected to use for CAVA.  This can be helpful if you have more than AV engine installed on the client.  In my case, I am using Symantec Endpoint.
  6. Last time signature updated: Tue May 17 05:55:23 2011 (GMT-00:00)
    • This is the last time you updated your AV definitions
  7. 1 File Mask(s):
    • The number of file masks you have set to scan for.  In this case, it’s just 1 mask.
  8. *.*
    • This is the file masks you have in place.  Any files that match the entries here will be processed for scanning.  In this case i have *.* (everything with a . in it), but you can cut down a lot of traffic if your only scanning for certain file types.
  9. 5 Excluded File(s):
    • This is how many file exclusion filters you have in place.  In this case i have 5.
  10. ~$* >>>>>>>> *.PST *.TXT *.TMP
    • These are the file filters i have in place.  There are a number of files that AV software just can’t scan (like database files).  I also have in place ~$* and >>>>>>>> to ignore Microsoft Office temporary files as they can become locked temporarily while being scanned and cause a loss of data in the office application.
  11. Share \\UBERCIFS\CHECK$.
    • This is the beginning of the UNC path that will be sent for file scan requests.  This is determined from the CIFSserver line in the viruschecker.conf and will change depending of if you defined it with the ip, netbios name, or FQDN.  The check$ folder is a hidden folder created just for CAVA.  The only account that can access this is the one granted the virus checking privilege.
  12. RPC request timeout=25000 milliseconds.
    • This is the amount of time we will wait for a file to be scanned before trying again.
  13. RPC retry timeout=5000 milliseconds.
    • This is the amount of time we wait for an acknowledgement of each RPC command.
  14. High water mark=200.
    • I spoke before about how we process 10 files at a time, and that addition files are put into a queue.  The high watermark is when we allocate additional resources to cava to process through AV files faster.  Hitting this high limit can cause a performance impact to your cifs servers, so try not to let the queue get this bad.  In my case, i have set the limit to the default of 200.
  15. Low water mark=50.
    • Just like the high watermark, this is a lower limit that starts to indicate that files are queuing up too fast.  This won’t cause a performance problem, but is an indicator of a possible problem to come.
  16. Scan all virus checkers every 10 seconds.
    • Every 10 seconds we will check the status of each cava server to make sure it’s still online and ready to take requests.
  17. When all virus checkers are offline:
    Shutdown Virus Checking.

    • This is the action we will take when all the cava servers are not marked as ONLINE.  This will shutdown cava so that files don’t continue to be queued and hit a high watermark.  The other options is to do nothing (a setting of ‘no’) or to shutdown cifs (what i like to call paranoia mode).
  18. Scan on read disable.
    • This means that scan on read is not enabled and that we are only processing scan on write.  If scan on read was enabled, the cutoff date and time would be listed in this place.
  19. Panic handler registered for 65 chunks.
    • This is mostly just for debug information and how many internal failures cava would survive before causing a panic.  Every process on the celerra has a panic handler and this information is of no use to basic cava troubleshooting.
  20. MS-RPC User: UBERCIFS$
    • Earlier i talked about how we use the MS-RPC protocol to connect to the cava agent servers.  This is the username we will use for the SMB connection.  In this case, we are using the compname of the cifs server for cava.
  21. MS-RPC ClientName: ubercifs.THULIN.LOCAL
    • This is the FQDN of the cifs server we are using for cava which is used as part of the MS-RPC process.
This concludes my line by line explanation of the cava output.  I hope you understand the output of cava a bit better.  In future posts on cava  Iwill talk about some of the different information you might see when there is an error as well as the output of the -audit option.  Please feel free to ask questions in the comment section below.

EMC World 2011 #nerdherd: Lights, Camera, Action!

So EMC World 2011 has come and gone.  Now is the time that we can look back and remember.  For those of you who were unable to make it live (or watch the webcast), a video of our #nerdherd has been posted on the EMC Community Network website.  I want to thank Alan Zenreich for filming and posting the video.  If you are one of the many people who prefer a more static image, EMC’s own David Elmes did most of the photography (including the photos of our meetup you see below).  To see the rest of his work and other’s, check out the EMC World 2011 flickr stream.  Once again, thanks to everyone that made this happen and enjoy the pictures and video.

EMC World 2011: Day 3 and Final Thoughts

First off, my apologies for not having this final EMC World post out sooner. Traveling and lack of sleep has pretty much kept me away from writing until now. Day 3 was the last full day of EMC World and it was just as packed as before.

Today was EMC Proven day at EMC World. Attendees that had received a proven certification were given an orange shirt and asked to wear it to show their support. After hearing news that Matt took 3 different exams yesterday, I decided it would be a good idea to take one as well. Being that it has been over a year since I started in NAS support, I should have retained enough knowledge to pass the NAS Specialist exam. Apparently I wasn’t the only one to have this idea because the line to register was out the door and it took over a half hour to get signed up for my exam. This also meant that all the timeslots were taken and it was standby only. The people at the proven center were well aware of this surge to pass exams, so a second testing room was opened up and my wait time was only a few minutes. This whole experience was well worth it as I passed my NAS Specialist exam.

We also had our #nerdherd today and it was a great experience. While several people joined us on camera, even more joined to watch, or participated via twitter and the live chat. All sorts of topics were discussed like the keynotes at EMC World, the vLabs, the VNXe, local #nerdherd meetings, and many other things. One of the comments that were brought up was the notion that “EMC eats its own dog food”. Taken out of context, this sounds like it has a negative connotation, but it really doesn’t. What was meant by this statement was that we use internally the same thing we are trying to sell to customers. We have a very large VDI deployment that works very well.

The day closed out with the final keynote, Chad’s World Live. Hopefully most of you already know the Chad’s World videos, but if you don’t, you should check them out. The venue was full of beer and laughs and it was a great time. The tech demos on stage were great.  They powered on 100 Wyse thin clients only to reveal that it was powered by the new Iomega PX6 (I want one of these). My only concern with this was the time. The performance was right after the sessions and show floor ended, so people were being pumped full of beer on an empty stomach. This did cause a lot of noise to be coming from the back of the room.

Las Vegas is a great place for a conference. Everything is within walking distance and there are plenty of things to do when you’re not on the show floor. You can take in an after party, a club, some great food, or a performance. When you are ready to unwind, the view can be incredible. I had an incredible time at EMC World. Len Devanna made a great place for a blogger like myself to get work done while at the show.  Thanks to everyone who made this an event to remember and I can’t wait to go to EMC World next year.

EMC World 2011: Day 2

So another day has wrapped up here at EMC World. Once again there are more keynotes, breakout sessions, and vendor booths to explore. The vLabs were in full swing and just about all of the kinks had been worked out. These are a great way to experience the products if you can’t make it to the break out sessions. If you attended multiple vLab sessions, you were able to win a special bobble head Chad and possibly get the real Chad to sign it.

The show floor was packed even more than yesterday. Vendors had all sorts of innovative products. One of my favorite products from the show floor was the battle ready VNXe.  We were told that this could withstand a bullet, but they wouldn’t let me live test it, so I’ll just have to take their word for it.

The bloggers lounge and Social City was also packed with great people. I had a nice talk with Chuck Hollis while he was working on a few blog posts. Chad dropped by the social city meet-up area to do a live webcast before rushing off to another customer meeting. If you haven’t seen all that’s going on at social city, you really need to check it out (and possibly win a pass to next year’s show). Don’t forget, tomorrow we are having the #nerdherd, so I expect you all to stop by social city on your way to lunch.



EMC World 2011: Day 1

In my previous post, I talked about the pre-conference festivities. Today was the official kick off of the show. After a quick breakfast, I started my day in the Blogger’s Lounge at Social City. If you have not seen this place, you should check it out. EMC’s Social Media team has provided an amazing space with plenty of couches, chairs, power strips, WIFI, and a Cappuccino Machine. They have made this a great place to sit down, have a great conversation, and do some blogging.

The morning kicked off with two great keynotes by Joe Tucci and Pat Gelsinger. I could go on and on about what was talked about, but everyone is covering that information, so you’ve probably already read about it. If not, feel free to check out the press releases on the EMC Website.

One of the first areas of EMC World that I checked out was the vSpecialist vLab. This is a great way to give customers a hands-on approach to trying out several of EMC’s different products. The lab setup has a large number of dual screen terminals powered by Wyse thin clients. This allowed the actual desktops and lab terminals to be thousands of miles away in Research Triangle Park. Leveraging this virtual setup, the lab can actually be deployed anywhere which will cut down on cost and setup time for future conventions.

Lunch was very good and efficient. The convention staff has really brought order to the chaos that is feeding 10,000+ people. I really have to give it up to them as there were no problems getting food and finding a place to eat.

After Lunch, since the Solutions Pavilion was not open yet, it was time for some breakout sessions. EMC World has a wide variety of presentations and hands on labs available to customers and it ranges from introductory level to advanced topics. Along with all the breakout sessions, the EMC Education Services staff are here and offering exams for half price. This is a great way to become proven and I plan on taking a specialist exam while I’m here.

Towards the end of the day, the solution pavilion opened and I was able to check out all that our partners had to offer. There is a lot going on in the area with demos of all the great products. Almost everyone is giving away an iPad 2 or a shirt. I didn’t spend too much time there at first, but I will be making another round before I leave Las Vegas.

EMC World 2011: Day 0

So I have been in Vegas for a little more than 24 hours. I have done a lot of walking around, I have won some money, I have met up with many of my twitter friends, and I also got my first taste of EMC World.  For those of you who may not know, this is my first tech conference, and I plan to get as much as possible out of it.

Today I learned my first valuable conference lesson: Wear comfortable walking shoes. Today I chose to wear a flat bottomed skater-style shoe. This was a bad choice and by lunch time I was already in the Walgreens buying a set of Dr. Scholl’s inserts. I may try a different kind of shoe tomorrow, but if my feet are hurting this much already, who knows what tomorrow will be like.

It was great to meet up with everyone. After having an amazing breakfast, we met up with Thomas Jones. He joined us on a trip around Las Vegas as we went all the way to Mandalay Bay to pick up our interop badges. After finding out the roller coaster at New York New York was closed, we made our way back to the Venetian because it was time to register for EMC World.  In a matter of minutes, I bumped into Chad Sakac, Len Devanna, David Hurst, and Jeramiah Dooley.

It was then finaly time to start the conference off right. For EMC employees, we were treated to a short presentation from Jeremy Burton on the company message and what to expect from EMC World. This was also where I mett Dave Henry and Mark Twomey for the first time. Then it was time for the pool. I didn’t realize just how big the pool area was, but I got lost for a while. Eventually, I met up with everyone, including Kelly Lavin, Denny Cherry, Julia Mak, Cody Bunch, and Edward Haletky. This great group of people will make this conference even better. I’m not even 1 day in and I’m loving every minute of it. See you on the show floor!

We’re having a #nerdherd at #emcworld, and you’re invited! [UPDATED]

So it’s official, Matt Brender, Luigi Danakos, and I will be hosting a #nerdherd as part of Social City on Wednesday, May 11th from 11:45 AM to 12:30 PM. We invite you to gather with your fellow technological enthusiasts and join us in an information discussion around Unified Storage, Social Media Crowdsourcing, and EMC Certifications. Use this time to meet other individuals who are striving in the world of Big Data. I’ll see all of you there!

UPDATE: So what if you can’t be there?  We’ve got you covered.  Social City will be live streaming the meetup (and many others) on the ECN website along with a live Q&A so you can share in the discussion.

Click the calendar icon to add our event to your personal calendar!

For more information on all the events that Social City has to offer, please see the following link:

The Three Musketeers are headed to EMC World!

If you couldn’t tell, I do almost everything technology related with Matt Brender and Luigi Danakos. People have dubbed us the 3 Musketeers and I think the name suites us well. I waited until now to talk about this because I am officially confirmed with flights, hotel, and passes to EMC World and Interop.

Our plan to get to EMC World started at the end of last year’s show. Both Matt and Luigi won a chance to be a part of the event. I however, being brand new to EMC, was not able to attend the show in Boston, but I did network at the after parties. Nevertheless, I still wanted to be a part of the fun and excitement next year. New York, Paris, or Milan, we had decided we were going to make our way to the show.

“You’re not pre sales or a customer, how did you get there?”. I’ve been asked this question more than once, and while the details are a bit classified, I can say this. The idea of sending us to EMC world has been talked around for several months now. Once we purchased our flights and hotel rooms, we ramped up our campaign. Well, we caught the attention of the right people who also agreed that we should be there and made it happen.

“So what are you going to do there?”. In short, everything I can. I plan to make multiple rounds of the show floor, checking out everything our departments and partners have to offer. I will also be a part of the bloggers lounge, observe some breakout sessions, and make my way to a few keynotes as well. Last year Luigi had a twitter shirt signing going on and if he’s going to do it again this year, I may join him. So look around for me on the show floor and make sure you say hello.