Script: Gracefully Shut Down all VMs on a Given Set of Hosts (VMware/XenDesktop)

Cleanly shutting down all virtual machines on a given set of hosts is not as trivial as it might seem - especially if you want to be able to restore the original state once the planned maintenance you are doing this for is completed.

Maintenance Tasks

These are the things that need to be done in order to prepare VMware ESXi servers hosting XenDesktop VDI machines for maintenance:

1. Determine which VMs are present on the servers we need to shut down
2. Put the machines in XenDesktop maintenance mode to prevent user logons
3. Create a list of all VMs that are powered on (in order to be able to start only those when done)
4. Initiate a clean shutdown via VMware Tools
5. Only where the clean shutdown fails or VMware Tools are not installed: turn off the machine

The following is required to restore the original state when maintenance is finished:

1. Power up those VMs that were running before we started
2. Disable XenDesktop maintenance mode for all VMs

Automation

You do not have to worry about how to automate all those steps listed above. The script presented here does it all for you!

Usage

This is how you call the script:

a) Shutdown:

.\ShutdownVMsOnHost.ps1 shutdown servers.txt vCenterServer XenDesktopDDC_FQDN

b) Startup:

.\ShutdownVMsOnHost.ps1 startup servers.txt vCenterServer XenDesktopDDC_FQDN

servers.txt is a text file that you need to provide. It should contain the servers to be processed (one server name per line). vCenterServer obviously is the name of your vCenter server. XenDesktopDDC_FQDN is the fully qualified DNS name of your XenDesktop DDC (you will have more than one - one one will do).

Enjoy!

The Script ShutdownVMsOnHost.ps1

#
# Shuts down as gracefully as possible all VMs on a given set of hosts
#

#
# Script parameters
#
param
(
   [string] $action,
   [string] $serverFile,
   [string] $vCenter,
   [string] $DDC
)


#
# Sample usage:
#
# .\ShutdownVMsOnHost.ps1 shutdown servers.txt vCenterServer XenDesktopDDC_FQDN
#


#
# General options
#
#Requires -Version 2
Set-StrictMode -Version 2


#
# Global variables
#

$scriptDir = Split-Path $MyInvocation.MyCommand.Path
$logfile = $scriptDir + "\log.txt"
$runningVMs = $scriptDir + "\vms.txt"
$vmsPoweringDown = new-object system.collections.arraylist
$vmNameFilter = "*"                                         # Optionally filter the VMs to process


#
# Constants
#
# Return values
Set-Variable -Name RET_OK -Value 0 -Option ReadOnly -Force      # Successful execution
Set-Variable -Name RET_HELP -Value 1 -Option ReadOnly -Force    # The help page was printed
Set-Variable -Name RET_ERROR -Value 2 -Option ReadOnly -Force   # An error occured

# Log message severity
Set-Variable -Name SEV_INFO -Value 1 -Option ReadOnly -Force
Set-Variable -Name SEV_WARN -Value 2 -Option ReadOnly -Force
Set-Variable -Name SEV_ERR -Value 3 -Option ReadOnly -Force


#
# This is the real start of the script
#
function main
{
   try
   {
      if (-not (Test-Path $serverFile))
      {
         throw "File not found: $serverFile"
      }

      # Load snapins and modules
      LogMessage "Loading PowerShell snapins and connecting to vCenter (may take a while)..." $SEV_INFO
      LoadSnapins @("VMware.VimAutomation.Core")
      LoadSnapins @("Citrix.ADIdentity.Admin.V1")
      LoadSnapins @("Citrix.Broker.Admin.V1")
      LoadSnapins @("Citrix.Common.Commands")
      LoadSnapins @("Citrix.Configuration.Admin.V1")
      LoadSnapins @("Citrix.Host.Admin.V1")
      LoadSnapins @("Citrix.MachineCreation.Admin.V1")
      LoadSnapins @("Citrix.MachineIdentity.Admin.V1")

      # Connect to vCenter
      try
      {
         Disconnect-VIServer $vCenter -confirm:$false -ErrorAction SilentlyContinue
      }
      catch
      {
        # Do nothing
      }
      $script:viserver = Connect-VIServer $vCenter -NotDefault
      
      # Do it
      if ($action -eq "shutdown")
      {
         ShutdownVMs
      }
      elseif ($action -eq "startup")
      {
         StartupVMs
      }
      else
      {
         throw New-Object System.ArgumentNullException "Unknown action: $action"
      }
   }
   catch
   {
      LogMessage ("Error: " + $_.Exception.Message.ToString()) $SEV_ERR
      exit $RET_ERROR
   }
}

##############################################
#
# Shutdown VMs
#
##############################################

function ShutdownVMs ()
{
   LogMessage "========================================="
   LogMessage "`nInitiating shutdown...`n"
   
   # Delete the VM file
   if (Test-Path $runningVMs)
   {
      Remove-Item $runningVMs
   }

   # Process each server in the list
   get-content $serverFile | foreach {

      if ([string]::IsNullOrEmpty($_))
      {
         return;  # Next line
      }
   
      LogMessage "Server $_..."

      # Get all VMs on this server
      $vms = Get-VM -Location $_ -name $vmNameFilter -Server $viserver -ErrorAction stop
      
      # Process each VM on this server
      foreach ($vm in $vms)
      {
         LogMessage "   VM $($vm.Name)..."

         # Enable XenDesktop maintenance mode
         try
         {
            Get-BrokerPrivateDesktop -MachineName "*\$($vm.Name)" -AdminAddress $DDC | Set-BrokerPrivateDesktop -InMaintenanceMode $true
         }
         catch
         {
            LogMessage ("Error while trying to enable maintenance mode: " + $_.Exception.Message.ToString()) $SEV_ERR
            
            # Next item in the foreach loop
            return
         }

         # Further process only VMs that are powered on
         if ($vm.PowerState -eq "PoweredOn")
         {
            # Store the running VMs
            Add-Content -path $runningVMs $vm.Name

            # Try a clean shutdown, if not possible turn off
            $vmView = $vm | get-view
            $vmToolsStatus = $vmView.summary.guest.toolsRunningStatus
            if ($vmToolsStatus -eq "guestToolsRunning")
            {
               $result = Shutdown-VMGuest -VM $vm -confirm:$false
               $count = $vmsPoweringDown.add($vm)
            }
            else
            {
               stop-vm -vm $vm -confirm:$false -Server $viserver
            }
         }
      }
   }

   # Wait until all VMs are powered down (or we reach a timeout)
   $waitmax = 3600
   $startTime = (get-date).TimeofDay
   do
   {
      LogMessage "`nWaiting 1 Minute...`n"
      sleep 60

      LogMessage "Checking for still running machines...`n"

      for ($i = 0; $i -lt $vmsPoweringDown.count; $i++)
      {
         if ((Get-VM $vmsPoweringDown[$i] -Server $viserver).PowerState -eq "PoweredOn")
         {
            continue
         }
         else
         {
            $vmsPoweringDown.RemoveAt($i)
            $i--
         }
      }
   } while (($vmsPoweringDown.count -gt 0) -and (((get-date).TimeofDay - $startTime).seconds -lt $waitmax))

   # Shut down still running VMs
   if ($vmsPoweringDown.count -gt 0)
   {
      LogMessage "Powering down still running machines...`n"

      foreach ($vmName in $vmsPoweringDown)
      {
         $vm = Get-VM $vmName -Server $viserver
         if ($vm.PowerState -eq "PoweredOn") {
            Stop-VM -vm $vm -confirm:$false -Server $viserver
         }
      }
   }

   LogMessage "`nDone!`n"
}

##############################################
#
# Startup VMs
#
##############################################

function StartupVMs ()
{
   LogMessage "========================================="
   LogMessage "`nInitiating startup...`n"

   # Startup VMs that were previously running
   get-content $runningVMs | foreach {

      if ([string]::IsNullOrEmpty($_))
      {
         return;  # Next line
      }
   
      # Get the VM
      $vm = Get-VM -name $_ -Server $viserver
      
      # Start the VM
      Start-VM -vm $vm -confirm:$false -Server $viserver
   }
   
   # Disable XenDesktop maintenance mode for all VMs
   get-content $serverFile | foreach {

      # Get all VMs on this server
      $vms = Get-VM -Location $_ -name $vmNameFilter -Server $viserver
      
      # Process each VM on this server
      foreach ($vm in $vms)
      {
         # Disable XenDesktop maintenance mode
         try
         {
            Get-BrokerPrivateDesktop -MachineName "*\$($vm.Name)" -AdminAddress $DDC | Set-BrokerPrivateDesktop -InMaintenanceMode $false
         }
         catch
         {
            LogMessage ("Error while disabling maintenance mode: " + $_.Exception.Message.ToString()) $SEV_ERR
            
            # Next item in the foreach loop
            return
         }
      }
   }

   LogMessage "`nFertig!`n"
}

##############################################
#
# LogMessage
#
##############################################

function LogMessage ([String[]] $messages)
{
   
   $timestamp = $([DateTime]::Now).ToString()
   
   foreach ($message in $messages)
   {
      if ([string]::IsNullOrEmpty($message))
      {
         continue
      }
      
      Write-Host "$message"

      $message = $message.Replace("`r`n", " ")
      $message = $message.Replace("`n", " ")
      Add-Content $logFile "$timestamp $message"
   }
}

##############################################
#
# LoadSnapins
#
# Load one or more PowerShell-Snapins
#
##############################################

function LoadSnapins([string[]] $snapins)
{
   $loaded = Get-PSSnapin -Name $snapins -ErrorAction SilentlyContinue | % {$_.Name}
   $registered = Get-pssnapin -Name $snapins -Registered -ErrorAction SilentlyContinue  | % {$_.Name}
   $notLoaded = $registered | ? {$loaded -notcontains $_}
   
   if ($notLoaded -ne $null)
   {
      foreach ($newlyLoaded in $notLoaded)
	  {
         Add-PSSnapin $newlyLoaded
      }
   }
}


##############################################
#
# Start the script by calling main
#
##############################################

main