Welcome to Kraft Kennedy

For years organizations have relied on tape drives and changers for backup and recovery of their critical data. Despite many predictions to the contrary, tape is still alive as we begin 2010.

When virtualization became popular it presented a challenge to those looking to continue to use their tape drives in fully virtualized environments. If you were using VMware you could use SCSI pass-through to present a tape drive or changer directly to a virtual machine but that prevented you from using any advanced features like VMotion. It also tied your tape drive and VM to a single host containing a SCSI card, making things complicated if that host were to experience a hardware failure.

Continue reading…

With the release of VMware vSphere 4, VMware has released a very powerful management tool called Fault Tolerance (FT).  At a basic level, FT allows you to keep two virtual machines (a Primary VM and a Secondary VM) running in lockstep on two different physical ESX hosts.  If one of the ESX hosts were to experience a hardware failure, the VM protected with FT would remain running on the second host without any downtime.  This can greatly reduce downtime due to hardware failures and provide increased service levels for important applications.

FT is often compared to Microsoft Windows Failover Clusters, formerly Microsoft Cluster Server (MSCS), and in fact many have talked about how FT can replace Microsoft clustering altogether.  Rather than jump to conclusions like this, it is important to understand the use cases for both technologies.  In addition, there are several limitations to FT that need to be considered. Here are some important points to remember about FT:

Continue reading…

In a previous post, I discussed how to expand virtual machine boot volumes with no downtime using Dell’s ExtPart utility.  Using this method is useful if you are using Windows Server 2000/2003, but is no longer required when using Windows 2008.  With Windows 2008 becoming the preferred server operating system going forward, using the method described below will become more and more common.

Microsoft has made it possible to expand boot volumes easily and on the fly without downtime without the need for any additional tools.  In this example, we’ll expand the C: drive of a Windows 2008 virtual machine from 25GB to 30GB.

1. Expand the size of the virtual disk.

2. Launch Disk Management by right clicking on My Computer, selecting Manage, and then selecting Storage\Disk Management.  If you do not see the unallocated space, right click on Disk Management and select “Rescan Disks.”

Continue reading…

Tuesday was a exciting day at ILTA 2009 with lots of great sessions and discussions throughout the day.  About the only part of the day that wasn’t so great was the turkey bacon served at breakfast, which had the texture of construction paper and the flavor of it as well.

The morning saw a number of really good sessions, such as the session entitled “G100 Recap: Weathering the Storm and Cloud Computing.”  A number of key points were raised, such as the acknowledgment that technology spending was sacrificed during the economic downturn during the past couple of years.  The panelists believe that IT will actually help lead firms out of the recession and allow them to complete projects to help overall productivity.  On the subject of cloud computing, it was said that most law firms will likely begin to get exposure through telephony systems where there is less of the privacy concerns of other cloud based applications.

If you’re interested in exploring cloud computing in greater detail, Kraft Kennedy’s John Tsiofas and Dave Carlson will be speaking on data center relocation and cloud computing at 11:00AM on Wednesday in the Maryland C ballroom.

Also in the morning was a great session called “From the Trenches: Office 2007 Deployment Lessons.”  Here three different firms talked about their experiences, both good and bad, in upgrading to Office 2007.  The general consensus from the panelists was that the ribbon bar, the biggest visual change in the Office 2007 suite, actually turned out to be an insignificant issue overall.  The larger concerns were around application integration with document management systems and add-ins to Office applications that cause instability and performance issues.  In fact, one panelist described the project as “The document management system project featuring Office 2007,” highlighting the importance of the integration between Office 2007 and the DMS.

Office 2010 may have some of the same challenges.  Feel free to stop by Kraft Kennedy’s booth (721/723) to get a first hand look at Office 2010 running in our VDI environment.  The VDI desktops also feature Windows 7 and Microsoft Office Communications Server 2007 R2, so stop by and check them out!

VDI, or virtual desktop infrastructure, seems to be a hot topic at the conference this year.  Firms are starting to recognize that there are many benefits that can come with implementing a VDI solution, both in terms of financial savings as well as ease of administration and management.  We’ve spoken with many firms at the conference already this year that are starting to look at VDI as an alternative to traditional desktop deployment strategies.  We think the release of Windows 7 is also helping to drive VDI interest and adoption, as is the acceptance of virtualization technology in general.

With cost savings in mind, firms are also talking about how to contain costs or continue to reduce them.  Technologies like virtualization are helping to reduce costs as firms work to extend their hardware investments.  Developing a sound technology strategy has also been a reoccurring topic throughout the day as firms are preparing for what is coming next and want to ensure their strategy makes good business sense.

Finally, the day ended with the “Tut After Dark” party and casino night and everyone had a great time.  Even the party is a great place to learn new things, such as this blogger learning just how quickly you can lose $25,000 at the high rollers craps table.  But hey, it sure was a fun four minutes of my life!

Installing the VMware Tools package inside a VMware virtual machine improves overall performance and allows the use of advanced features and faster virtual hardware drivers.  The installation package also installs a tray icon that controls guest access to virtual hardware, time synchronization, etc.  Since most virtual machines are servers and end users don’t typically access the console of a server, worries about the security implications of leaving that tray application running have been fairly minimal.  However, as firms move towards solutions like virtualized XenApp servers or virtual desktops, this becomes more of a concern.

Removing administrator access to end users is unfortunately not enough.  For example, a user can open the VMware Tools tray icon and select the Devices tab, and from there can uncheck “NIC1″ and click Apply.  What happens?  You guessed it – the virtual NIC is disconnected and the user loses connection.  That’s bad in a virtual desktop environment since it will orphan the desktop and likely require a connection broker like XenDesktop to create another desktop but it is even worse on a XenApp server where the user potentially just disconnected dozens of other users as well.

This, and several other things found in the VMware Tools, can be dangerous to leave available to an end user even if they have no rights to the server itself.  To get around this, there are two approaches that make sense:

1) Remove access to the VMware Tools for end users.

2) Modify the VMX configuration file to prevent these actions.

I prefer the second method since it allows for more granular control over security, though if you’re interested in option one then you can read VMware’s KB article on the subject.  In order to prevent this at the VMX (virtual machine configuration file) level, simply add the following lines to the virtual machine(s) that you wish to protect (after powering it down):

isolation.device.connectable.disable = “true”
isolation.device.edit.disable = “true”

To see how to add one of these values to the VMX file via PowerShell and PowerCLI, it would look something like this:

$vm = Get-View (Get-VM NameofVM).ID
$vmConfigSpec = New-Object VMware.Vim.VirtualMachineConfigSpec
$vmConfigSpec.extraconfig += New-Object VMware.Vim.optionvalue
$vmConfigSpec.extraconfig[0].Key=”isolation.device.connectable.disable”
$vmConfigSpec.extraconfig[0].Value=”true”
$vm.ReconfigVM($vmConfigSpec)

There are many other security parameters that can be set in the VMX file that are covered in VMware’s Security Hardening document (PDF).  The document covers this and many other common security best practices for virtual machines.  As always, test any change you make (especially the script above) before putting anything into production.

Microsoft has done a nice job of late regarding supporting their operating systems and applications when they are run in a virtual machine. First they created the Server Virtualization Validation Program to validate their software running on hypervisors from various vendors, including their own Hyper-V.  They’ve taken the SVVP one step further by adding a new tool called the Support Policy Wizard that makes it fast and simple to verify Microsoft support.  It can be found at the following link:

http://www.windowsservercatalog.com/svvp.aspx?svvppage=svvpwizard.htm

This tool makes it easy to select a specific configuration, including operating system, hypervisor, and Microsoft application, and then verify it against the SVVP to ensure Microsoft supportability.  It also lists any specific application features that are or are not supported.  Validating your configuration against Microsoft’s Support Policy Wizard is an important step, especially when virtualizing mission critical applications like Exchange or SQL.

Try the wizard and see the results for yourself.  I tried it myself and the results are below.  I selected Exchange Server 2007 SP1 running on vSphere 4 with Windows Server 2008 x64 and received the following supportability statement back:

Summary Support Statement*
This configuration is Supported.

* Customers with Premier-level support agreements should contact their account manager for more information
* Additional information is available in the “Support policy for Microsoft software running in non-Microsoft hardware virtualization software” which can be viewed here

Support Statement Details
Product: Exchange Server 2007 Service Pack 1 on VMware vSphere with Windows Server 2008 (x64) Guest OS

Search the Knowledge Base for information related to this configuration

For Exchange Service 2007 Pack 1 and later, see here for specific configuration information. Note: The Exchange Server guest virtual machine must be deployed on the Windows Server 2008 operating system.

Specific third-party virtualization information is available at here

To get support on the virtualization solution, the customer also needs to have a support agreement with the third party vendor.

Supported features: Anti-Virus, Back-up Software, Virtual Machine Management Software, Cluster Continuous Replication (CCR), Virtual Processors

Unsupported features: Unified Messaging, Dynamically Expanding Virtual Disks, Virtual disks that use differencing or delta mechanisms, Hyper-V Quick Migration combined with Exchange Clustering, Virtual Machine Snap Shots

One of the great things about using virtual machines is how easy it is to quickly modify their hardware configuration.  One common change that administrators often make is to expand the size of the virtual machine’s hard disk.

Windows data volumes are easy to expand by simply growing the disk file and then using Microsoft’s Diskpart utility to make the extra space visible in the guest.  This can be done on the fly with no downtime to the virtual machine.

For boot volumes it is more complicated and is typically handled in one of two ways. First, the admin expands the boot drive of the virtual machine to the desired size. Then the VM is shut down, and one of the following two methods is used to expand the volume:

1. The expanded hard disk is attached to a second virtual machine, and then Diskpart is used to make the new space usable in the guest.  That second VM then has to be shut down, the disk removed from the configuration, and both servers booted back up.
2. The VM is booted with a GParted ISO which allows the volume to be expanded.  Windows will force a chkdsk run at next bootup but will then see the expanded size.

While these are acceptable solutions, both require downtime of one or more virtual machines.  To get around the downtime requirement, you can use a utility called ExtPart from Dell. It is used to expand partitions on Dell storage arrays but works just as well in a virtual machine. Note that it can only be used on Basic disks in Windows and does not work with Dynamic disks.

The following example shows expanding a 17GB C:\ drive to 20GB on a Windows 2003 VM with no downtime required.  Note that this process is not required on Windows Server 2008 since the ability to grow volumes is now a feature of Disk Management.  Simply right-click the volume you wish to expand and select Extend Volume.

1) Edit the virtual machine settings and increase the boot volume to the desired size.

2) Launch Disk Management and select “Rescan disks” to detect the new space.

3) Run ExtPart with the following syntax: extpart.exe <volume> <AdditionalSizeInMegabytes>

After completing steps 1-3, the new space should be recognized in Windows automatically.

When managing a VMware ESX host, most functions can be done via the VI Client. The VI Client offers an easy to use GUI interface for management and configuration of one or multiple ESX hosts. That said, there are times when connecting to the Service Console of the ESX host is required. Often times multiple administrators will login to the Service Console as the root (highest level access) user, making it difficult to know which administrator performed any task.

It is not recommended that all administrators connect to the ESX host as the root user. Further, as a security best practice connecting to the ESX host via SSH as the root user is restricted by default.

With these restrictions and best practices, how should Service Console access be managed on ESX hosts? In truth, ESX hosts are like all other servers and best practices for security and auditing should be followed – that is, all administrators that have to login to the ESX host should do so with their own account.

For administrators, managing multiple logins for different systems can be difficult and can lead to the use of weak or common passwords. To get around this issue with VMware ESX, administrators can enable Active Directory authentication which allows them to login with their domain credentials instead of a local Linux account.

The following steps must be completed on each ESX host in the environment in order to enable AD authentication:

Configure Active Directory Authentication on ESX

1. Connect to the ESX host as the root user. Issue the following command:

   esxcfg-auth –enablead –addmoain=ActiveDirectoryDomain –addc=FQDN.of.domain.controller
   

   For example, configuring AD authentication for Kraft Kennedy’s Research domain research.kraftkennedy.com with a domain controller named nyrdc01.research.kraftkennedy.com would look like the following:

   esxcfg-auth –enablead –addomain=research.kraftkennedy.com –addc=nyrdc01.research.kraftkennedy.com
   

   Additional domain controllers can be added via additional –addc command and should be done to provide some redundancy.

2. Create Linux accounts for each administrator that needs to connect to the ESX Service Console

   Useradd username
   

   To add the “admin.liebowitz” account to the ESX host, the command would be:

   useradd admin.liebowitz
   

   As administrators leave the organization, their accounts can be removed with the following command:

   userdel username
   

3. Once authenticated, if additional access is required the administrator can issue the following command to elevate to root level access:

   su -
   

Once the above steps have been completed, administrators can login to the ESX host via SSH using their AD credentials. This allows organizations to maintain best practices by restricting root level SSH access as well as makes it easier to see which administrators have logged into a particular server.

Yesterday VMware released Update 4 for ESX 3.5 and ESXi 3.5. It includes a few new features but no major enhancements. It also has some limitations to go with it that are important to note.

Here are some of the highlights:

• ESX 3.5 U4 does not work with all versions of vCenter 2.5. For example, you will need to have vCenter updated to at least Update 3 in order to be able to use it to manage ESX 3.5 Update 4. There is a compatibility matrix of which versions of ESX work with specific versions of vCenter here (this link opens a PDF).
• You can now enable the Enhanced vmxnet driver for 32-bit operating systems. Previously you had to set the guest OS to a supported type (like Server 2003 x64), enable the Enhanced vmxnet NIC, and then change the OS back. This is no longer required. This driver enables advanced networking features like TCP Segmentation Offload.
• VMware has included experimental support for PXE booting ESXi hosts. This would allow you to have a completely stateless and diskless server with ESXi installed on a flash drive able to PXE boot and be ready to host virtual machines in minutes.
• A large number of new hardware devices are supported, including NICs, SATA controllers, etc.

The full release notes can be found here for ESX 3.5 and here for ESXi 3.5.

As with any update to ESX, make sure you do it in the proper order. First upgrade vCenter and your database and then update each of your hosts.