My VCP5 experience and a Workstation 9 giveaway!

VMW_10Q3_LGO_CERTIFIED_PRO_4For those of you who follow me on Twitter or Google+, you may have noticed that I recently passed my VCP5.  This exam was a long time in the making, but every time I would go to take it, something would come up.  Originally, I was scheduled to start the new year with my VCP5 exam, but a last minute trip to Seattle forced me to reschedule my exam for February.  When I rescheduled my exam for February, I had no idea I would be starting my new job, but as luck would have it, I scheduled it for the same week.  So once again, I rescheduled for March.  Hopefully this explains why I had been so quiet with my blog lately as I had spent a lot of my available time leading up to the exam studying.  Even though I switched into a marketing role, I’m still keeping my technical side.  I committed to passing this exam and even made it part of my goals with EMC.

 

The good news is that my studying paid off.  I passed my VCP5 with a better score than I got on the VCP4!  A few of my biggest takeaways from the exam were as follows:

  • Gone are the majority of the questions about minimums and maximums (I think I had only 1)
  • Even more questions were about real world examples
  • There was a heavy focus on performance troubleshooting and identification
  • Several questions focused on HA, DRS, and FT
  • Only a handful had images, so be sure to paint a mental picture from the descriptions

 

My exam prep consisted of many different resources.  I started off by reviewing the material found in 3 books:

I also reviewed the study guide and exam prep questions put together by Mike Preston.  This guide is very comprehensive and full of great information.  Simon Long also has practice exam questions on the VCP5 as well and was one of my primary resources when studying for my VCP4.

 

The final resource I used in my preparation was the recordings of the vBrownBag.  They went through every section of the VCP5 exam blueprint and is great to listen to.  I only wish that they did it again so that I could have asked my questions live.

 

So on to the giveaway.  One of the rewards for passing the VCP5 is a free VMware Workstation 9 license.  I want to give back to the community that has helped me so much, so I’m giving this key away to someone who needs it.  Wining this license key is simple:

  • Make sure you are following me on twitter
  • Leave a comment here telling me something about your VCP5 exam experience (or study plans if you haven’t taken the exam).  A helpful tip, a gotcha, anything will do and there is no wrong answer here!
  • Make sure that you also put your twitter handle in your comment (I will need this to DM you if you win)

The winner will be selected at random and you will earn an extra entry if you tweet about this blog post and mention my twitter handle in the tweet.  I’m going to to do this quick, so the contest will close at 12:01 AM Eastern on April 1st, 2013 and I will contact the winner shortly after that.

VNX INYO MR1–The Future is Now

EMC World 2012 - Day 1 150With a new year, comes a HUGE update to the VNX family.  As Chad Sakac reveled earlier in the year, INYO was the code name for the VNX FILE OE 7.1 and BLOCK OE 05.32 code release that surfaced last year.  Now, the time has come for a major update to the code, and with it some exciting new features.

 

VNX OE for FILE

On the FILE side, the biggest (and what I think is the most exciting) feature coming is support for SMB 3.0 and the VNX is the first array to support this.  Back in October of 2012, Microsoft released it’s latest versions of the Windows Operating system (Windows 8 and Server 2012).  With that came the latest enhancements to the SMB protocol (for more information, click here to read a great blog post by Microsoft).  With this upgrade (and the use of the SMB 3.0 protocol) you get a much less disruptive failover which includes keeping the open state of a file and file lock.  You will also notice enhanced throughput by being able to take advantage of the Multi Path IO over SMB 3.0 without the need to configure LACP or EtherChannel.

 

VNX OE for BLOCK

On the BLOCK side, the VNX gains support for ODX support and the ability to Offloaded copies to the array.  This cuts down on host CPU as well as SAN bandwidth as the transfers don’t leave the array.  This is done by breaking down the copy into a series of tokens and passing them between the hosts while the data is passed between luns (as demonstrated in the chart below)

image

A couple of things to note.  This does require an enabler, but you do not need to reboot the SP for that.  You will have to reboot the host for that (it’s a limitation of Microsoft, not EMC).  You will have to use Microsoft MPIO or the latest versions of PowerPath as well as an NTFS file system (with an allocation size of 8k or larger for better performance).

 

Also included with this release was several enhancements revolving around VAAI support on the VNX.  Most this included the XCOPY fix as described in solution emc313487 as well as a big performance improvement to VAAI Fastclones.  Chad has more on that subject here.

 

Unisphere Service Manager

Finally, there was also another enhancement that I wish I had when I was in tech support.  Starting with the new version of Unisphere Service Manager (1.2.26.1.0068) you will now find a 1-click health check available after the main login.  You may remember a previous blog post I did on how to run health checks on the VNX.  Now you can run a single check to verify the health of your array (BLOCK, FILE, or Unified).  Just click the health check link on the right hand side.  I have attached a screen shot below to show what the output of a healthy array looks like.

image

So what are you waiting for?  Get out there and enjoy these new enhancements.  Remember, you don’t have to wait for EMC to upgrade your array, you can do it yourself using USM.

VNX iSCSI and TCP Delayed Acknowledgement

vnx-5500I recently sat in on an internal VNX (and CLARiiON) performance crash course that was put together to help our new hires get up to speed.  Once of the things that stuck out to me was the subject of iSCSI and how it works with host TCP delayed acknowledgement (Delayed ACK).

 

Background Information

So what is delayed ACK?  As part of TCP, for every packet that is sent to a destination server, that server must send some sort of acknowledgement back to the source server.  This way the source server knows the information was successfully transmitted.  This adds a good amount of overhead, so in an effort to improve performance, TCP Delayed Acknowledgement (RFC 1122) was created which allows a destination server to respond to every 2nd packet instead.  This has become so popular, that support for delayed ACK is enabled by default in many popular client operating systems including Microsoft Windows and VMware ESX/i.

 

The problem with this is that many storage arrays do not support delayed ACK for one reason or another (usualy has to do with chipset drivers).  What happens in this case is that the array will send a packet, it will then wait for an acknowledgement before sending a 2nd packet.  Meanwhile, the host is waiting for a 2nd packet before sending an acknowledgement.  This standoff between the array and the host will last until the acknowledgement timeout (usually around 200ms) before continuing on.  This wreaks havoc on performance if every packet has to wait 200 milliseconds before sending another.  So if you’ve setup iSCSI and you are immediately seeing a performance issue, check your hosts to see if Delayed ACK is enabled, and turn it off to see if performance improves.

 

Disabling Delayed ACK in Microsoft Windows

 

In Microsoft Windows operating systems, you can simply set the TcpAckFrequency registry value to 1.  More information can be found Microsoft kb 328890.  On a side note, I found that if the registry value is missing, you can create it in the path specified in the kb and reboot the host.

 

Disabling Delayed ACK in VMware ESX and ESXiimage

VMware has created KB 1002598 to address this as well.  This adjustment is made per adapter instance and you can change this setting on a discovery address, a target, or (in my case) globally.  Once you’ve made your change, reboot the host and enjoy the performance boost.

 

I hope you’ve found this information useful.  It may not solve your iSCSI performance problem, but it is a good place to start.

A few initial thoughts about VMware vSphere 5.1

654375636VMware released vSphere 5.1 this past week, and while there is a large amount of people who have been holding off on upgrading, I have had the pleasure of upgrading both my home lab and the non production lab I use at the office.  Having played with it for a few days now, I wanted to share my first impressions.

 

The upgrade process

imagePrevious vCenter server upgrades simply consisted of just upgrading the server component (and the upgrade and away you go.  Now VMware has introduced two new components to the mix: vCenter Inventory Service & vCenter Single sign-on.  To take a lot of the guess work out of this, VMware provided the “simple installer” to the mix.  This runs through each of the setup applications in the proper order and makes the process very simple and I applaud VMware for this.  A couple things to note:

  • As with all vSphere upgrades, make sure you upgrade vCenter server BEFORE you upgrade any hosts.
  • If you plan on using LDAP authentication, make sure that you are logged in with an LDAP account when you install the single sign-on service.
  • If an upgrade fails during the “simple install”, you will need to continue installation by hand.
  • You won’t need to reboot the vCenter Server machine after the upgrade, the upgrade process will restart all the required services for you.
  • After upgrading the hosts, they did not automatically reconnect to vCenter, and you have to click reconnect on them.

Since I run my vCenter server with an internal database and on a virtual machine, I was able to simply snapshot the VM in the event that something went wrong.  Luckily nothing did and both my upgrades went smooth.

 

vCenter Server 5.1

imageOne of the things I noticed right away was the startup time of vCenter server from boot up.  It is at least 4x longer than vCenter server 5.0 and earlier.  A simple look at task manager during the start up process shows that the java process is eating up most of my CPU, so my guess is this delayed startup is due to the revamped web interface being used.  Needless to say, it threw me for a loop the first time and I thought my upgrade had been borked.  So just keep that in mind that it may take 20 minutes or so for vCenter to come up and be stable.

 

Now when using the standard vSphere client to connect, you will notice things are very similar to older versions.  This is be expected as I believe VMware is trying to make a shift to everyone using the Web Interface (more on this later).  Do keep in mind that the regular vCenter client does not give you access to all of the new features.  One of the big ones to note, is that the new enhanced vMotion can only be found in the web client (again more on this later).

 

vCenter Web Client

imageIt’s clear that VMware has spent a lot of time improving upon the web interface and it shows.  That’s good news for Mac users as they now have a fully functioning method for administering the environment.

 

Right away, you’ll notice a new icon that is not present in the standard vCenter client.  VMware has included vCenter Orchestrator right into the mix and I hope this is a sign of more plugins to be installed.  With that being said, I was disappointed to see that the update manager was not present in the web client, so you must still use the classic vCenter client to perform those updates.

 

One of the things I noticed (and was pointed out by Frank Denneman) was that the new Enhanced vMotion (the one that lets you change both hosts and datastores while the guest is powered on) can ONLY be found in the web client.  I’m not sure this was the best play by VMware, but my guess is that VMware’s long term plan is to make the web client the primary interface for management.

 

Final Thoughts

As with every upgrade, I’m always amazed by the refinement that is introduced.  More and more drawn out processes are being simplified and it seems like VMware really cares about the advanced users as well as those just getting started.  As with any major upgrade, first adopters have some challenges and I wanted to list out a few things that you should be aware of before you upgrade:

As with all my blog posts, I like to hear from the readers.  If you had a good or a bad upgrade experience, let me know in the comments.

Introducing EMC’s VNX Storage Analytics Suite (and Early Access Program)

imageWith VMworld in full swing, today EMC finalized the announcement of the new VNX Storage Analytics Suite (and you can try it out on the show floor).  You may remember that this was demoed first at EMC World.  As development was nearing completion, you may remember my call for beta testers (I hope you got in and got a chance to try out the technology).  Now we can start getting as excited as the software will be available for general consumption in Q4 of 2012.

 

imageThe VNX Storage Analytics Suite will offer you a an extensive platform to proactively identify bottlenecks, balance workloads and pinpoint the root cause of potential problems around health and performance.  The software is available as a stand alone product as well as integration for VMware’s vCOps Enterprise platform.  Pairing the two together allows you to have a full end-to-end visibility of your entire infrastructure from virtual machine to lun and every point in between.

 

imageFor those of you who are already familiar with vCOps, you will be presented with same interface just with extra options for looking at metrics.  The charts and display is the same.  Green is still good and Red is still bad.  This allows you to quickly visualize the health of your arrays (both block and file) using a simple “Performance-at-a-glance” tab.  Armed with this knowledge, you’ll be able to attach the problem directly instead of waiting for someone to tell you there is a slowdown in the environment.

 

imageBesides general health information, you can dig down into the specifics of utilization on the array.  In the picture to the right, you’ll be able to see the results (both historically and in real time) of enabling fast cache on your array and what it did to the overall health of your environment.  By focusing on the performance of the array and the pool disks, Fast Cache was able to increase the throughput of the SP while decreasing the strain on the spinning disk storage.

 

By this point you might be saying “Sean, this is so cool, I can’t wait until Q4!  How can I get my hands on this now?”.  Well for those of you who did not participate in the beta (and those of you who did and want to see the results of your feedback) there is good news.   EMC is going to be doing an Early Access program, it won’t cost you a penny, and I’ve got all the details right here!  The bad news is that it’s limited to 100 customers and it’s filling up fast.

 

If you may remember during the beta program I was asking people to email me if they wanted in.  This time I have no control over the process and you must go through your TC.  Before I tell you just how to get in, I need to make sure you have what is required in your environment.  You will need to meet the following prerequisites:

  • vCenter Operations Manager 5.0.1 Enterprise or higher (vApp distribution)
  • EMC VNX series system (Block: R31 OE version 05.31.000.5.720 and higher and VNX OE R32 version 05.32.000.5.006 and higher; File: VNX OE 7.0.35 and higher).

Seems simple enough, right?  Now on how to get in, as I said before you will need to contact your TC or Account Representative.  Have him or her place a Sales Evaluation order in Direct Express using Model # SASCONN-T90 and complete an RPQ.  Again, this is a no cost evaluation period and if you are selected, you will receive further instructions.  At the end of the preview period (which I believe is somewhere in Q4), you will be given the opportunity to purchase a license or uninstall the software.

 

If you are attending the show, please let me know your thoughts about the software in the comments section below.

Configuring VASA for use with a VNX

vnxWhen VMware introduced vSphere 5 to the world, one of the enhancements was a new API for storage arrays that provides vSphere with information as to the configuration and performance of your storage array.  For more information on VASA, please see this article from The Virtualization Practice.  VASA on a VNX (and other EMC arrays) historically used to be configured using an SMI-S provider.  This older configuration method has been covered very well by EMC vSpecialist Craig Stewart and can be found here.

 

Starting with VNX OE for FILE 7.1 and VNX OE for BLOCK 05.32, the VNX now has native VASA support.  This eliminates the need for the SMI-S provider and allows you to point vSphere directly to the control station and SP.  It really is a 1-step implementation and I will show you below.  And there is only 1 caviot to this, and it is VASA for the BLOCK and FILE are done separately.  if you are using, FC, FCoE, or iSCSI connections, you will want to use the BLOCK example, and if you are using NFS, you will want to use the FILE example.

 

You will want to start in vSphere by going to Home > Administration > Storage Providers.  From there you would click on “add…” to configure your connection.

 

VNX OE for FILE 7.1 VASA configuration example

You will start by naming this connection.  I chose VNX FILE to make it easy to distinguish between block and file connections.  You will then use the URL as follows: https://<ip.or.dns.of.control.station>:5989/vasa/services/vasaService.  The username/password would be one local to the control station (such as nasadmin or root).  The global accounts from the storage domain will not work here.  When it’s all said and done you should have something like the photo below:

VASA_VNX_FILE

You will probably be prompted to verify the SHA fingerprint, so just click yes and soon you’ll see your new provider listed with the following information:

VASA_VNX_FILE_2

 

VNX OE for BLOCK 05.32 configuration example

Just like the VNX OE for FILE example, you will start off by using a name.  This time the URL will be pointing to the SP.  The url will be as follows: https://<ip.or.dns.of.SP>/vasa/services/vasaService.  Please note the lack of a port specification as by default https uses port 443.  For the password you will want to use a storage domain account (such as sysadmin).  If you configured it correctly, it should look something like this:

VASA_VNX_BLOCK

 

Since I have a very basic configured array in the lab, I see provider information like this:

VASA_VNX_BLOCK_2

 

After you have successfully configured your providers, you can go and setup your storage profiles.  Go to Home > Management > VM Storage Profiles and add a new profile.  From there you can select from a multitude of options to pick the one that best matches the lun you are using for storage.

VASA_VNX_STORAGE_PROFILES

It really is that simple!  For more information on VASA on the VNX, read the Virtualization for EMC® VNX Release Notes (EMC Support credentials required).

Got a VNX and vCOps? We need your help!

imageIf you remember during EMC World 2012, the USD team showed off a preview of our new storage analytics package that would come as either a stand alone application or as a plugin for VMware vCOps.  As we move ever closer to a final product, we have entered the private beta phase.  This is where we need your help.

 

USD Engineering will be running a private beta between August 8th and August 29th and the test plan should take about 2 days to complete.  What you need is the following (UPDATED):

 

  • A VNX running recent code.  Both File 7.0 & 7.1 and Block 5.31 & 5.32 should be acceptable (this can be production or non production)
  • vCOps running the Enterprise or Enterprise+ (this must be a non production / test instance)
  • The name of your EMC contact (preferably a TC)
  • Please also indicate if you are a customer or a partner

You may need to sign an NDA, I’m not 100% on that but the team running the beta will be able to confirm.  Space is limited and we only have a few slots left so there is no guarantee that you will be accepted into the program.  If you are interested, please send an email to “sean.dot.thulin.at.emc.dot.com” with your contact information and what you have available as equipment (make / model of VNX and software version installed) and I will forward it on to the team.  Act fast as we are almost ready to start!

The NEVMUG Summer Slam is just 1 week away!

NEVMUG 025It was about 6 months ago that I was writing about the NEVMUG Winter Warmer.  It was my first VMUG and it was great.  For a free conference / user group, it was packed full.  The presentations were great, the sponsors had great break out sessions, and the conversations with attendees were inspiring.

 

Now it’s time to do it all over again, this time in Maine.  On Thursday, July 19th, the Virtualization Technology User Group will be presenting the New England VMware User Group Summer Slam in Brunswick, Maine.  The day will be filled with vast amounts of knowledge transfer via keynotes (including one by Chris Colotti of VMware), break out sessions (which will feature speakers such as Mike Foley, Ed Haletky, and Luigi Danakos), and the sponsors EXPO.  Once it’s all said and done, there will be a giant lobster bake over at Gritty’s.

 

I’ll be walking around, taking photos (and live tweeting a few for those of you who can’t make it), and taking in all that the NEVMUG has to offer.  I’ll also be heading up the night before, so if you want to grab a late drink and some food, hit me up on twitter.  For more information about the NEVMUG Summer Slam, go to the VTUG website.

The NEVMUG is only 2 days away!

NEVMUG

The New England VMware User Group Winter Warmer is an event I’ve been looking forward to for a while.  I’ve heard a lot of good things from this and the one during the summer over the years.  Unfortunately I’ve never been able to attend one in the past, so this will be my first VMUG ever.

 

So since this is my first VMUG, I’m not quire sure what to expect.  Greg Stuart did an excellent post on what the VMUG is and what to expect from it.  Just like it says in his blog post, I expect to be bombarded with a wealth of knowledge from people who are very passionate about virtualization.  I’m also looking forward to tech demos and hands on labs from the sponsoring vendors that make this event possible.  The event schedule is as follows:

 

  • 08:00 a.m. Registration and Networking
  • 09:00 a.m. What Oracle DBAs need to know about Virtualization
    • Cloud: vCloud Director Deep Dive: Paul Lembo/Chris Colotti
    • Healthcare: ISV Case Study : Scott Carpenter
  • 10:00 a.m. Building Your Cloud Infrastructure with VMware
  • 11:00 a.m. Scott Davis; VMW End User Computing Innovations for 2012
  • 12:00 p.m. Lunch
  • 01:30 p.m. Breakout Session 1
  • 02:00 p.m. Breakout Session 2
  • 03:00 p.m. Breakout Session 3
  • 04:30 p.m. Happy Hour / Giveaways
    • Sponsor Expo is open all day.

 

This looks like a very full day and I expect it to be very busy.  You can register for the event here and keep and eye out for Matt Brender and myself as we may be recording interviews for an “I Tech Therefore I Tool Around” podcast.  See you there!

I Tech Therefore I Tool Around – Episode 02 – The Home Lab

DSC_0022So our first episode of the “I Tech Therefore I Tool Around” was a smashing success!  Matthew Brender and I had a great time recording and we learned a lot about the process that goes into making a podcast.  The feedback from our listeners was great as well.

 

With that experience and knowledge in hand, we recorded our second podcast.  In this episode, we talk about the why and the how of building a home lab.  I touched upon my home lab during my blog post about the VCP4  Our special guest, Luigi Danakos, talks about his need for a home lab and how he acquired one without spending a dime.  We also touch on several blog posts that inspired our builds.  These are all great resources for building a home lab, so check them out:

 

A slight apology for Luigi as he was still getting over an illness and can be heard coughing every now and then during the recording (I cut out as much as I could).  Be sure to check out Matt’s post on this experience.

 

Click HERE to subscribe to the podcast on iTunes!

Or click HERE (or on the icon) to download the MP3 directly!