We had an ESXi host with an issue when powering any VM.  When users click the start button, always have an error “Insufficient Resources.”

Was a strange error, since we know that this ESXi host has enough resources to run more VMs. So we need to troubleshoot the issue and check what is triggering this problem.

First, we check the ESXi resources, and as we can see, there are more enough resources to run a VM.

Looking at the VM reservations to see if any reservations values can trigger this problem, but again no reservations are in the this particularly VM.

If is not ESXi resources or VM reservations we check if the Datastore where the VM is allocated to have enough free space.

Again,  as we can see the Datastore have 1.4Tb Free space, so the issue is Datastore related(at least the one where VM is allocated). So the next step is to check VMkernel logs.

In the log we can see this error:

2016-05-17T12:39:05.199Z| vmx| I120: numa: VCPU 1: VPD 0 (PPD 0)
2016-05-17T12:39:05.199Z| vmx| I120: numaHost: 1 virtual nodes, 1 virtual sockets, 1 physical domains
2016-05-17T12:39:05.199Z| vmx| I120: CreateVM: Swap: generating normal swap file name.
2016-05-17T12:39:05.199Z| vmx| I120: FTCpt: (0 unk) vmk overheads: 0 pages (0 MB)
2016-05-17T12:39:05.200Z| vmx| I120: Swap file path: ‘/vmfs/volumes/5253f39e-5cc0b637-0a9f-d89d67180e96/VMFolder07.182/VMName07.182-ccbc2e32.vswp’
2016-05-17T12:39:05.200Z| vmx| I120: Disk_GetDigestDiskCount: numDigestDisks = 0
2016-05-17T12:39:06.134Z| vmx| I120: Msg_Post: Error
2016-05-17T12:39:06.134Z| vmx| I120: [vob.swap.extend.failure.status] Failed to extend swap file from 0 KB to 2097152 KB.
2016-05-17T12:39:06.134Z| vmx| I120: [vob.swap.extend.failure.moreinfo] Current swap file size is 0 KB.
2016-05-17T12:39:06.134Z| vmx| I120: [msg.vmmonVMK.creatVMFailed] Could not power on virtual machine: No space left on device.
2016-05-17T12:39:06.134Z| vmx| I120: [msg.monitorLoop.createVMFailed.vmk] Failed to power on VM.
2016-05-17T12:39:06.134Z| vmx| I120: —————————————-
2016-05-17T12:39:06.137Z| vmx| I120: Vigor_MessageRevoke: message ‘msg.monitorLoop.createVMFailed.vmk’ (seq 4276552) is revoked
2016-05-17T12:39:06.137Z| vmx| I120: Module MonitorLoop power on failed.

So ESXi is not allowing the VM to start because of lack of space to save the swap files, but as we have seen in the previous image, there is more than enough space to allocate swap or any VM.

So we need to identify the VMFS volume /vmfs/volumes/5253f39e-5cc0b637-0a9f-d89d67180e96/ but also why is full.

There is a similar VMware KB for this logs errors: KB1030719

In the image above we check all Datastores that are in this ESXi, and we can see that one local Datastore is full and there is no free space. But by default swap files should stick together in the same VM folder/Datastore. So we need to check the ESXi host configuration regarding the Virtual Machine Swap files location.

That information we get in ESXi hosts – Configuration Tab – in the Software area – “Virtual Machines Swapfile Location.”

As we can see in the image above, is saving the Swapfiles in a different location of the VM folder. Not following the recommended option “Store files in the same directory as the virtual machine.” We can use a different Datastore to allocate all our Swapfiles, but we do not use that here, and someone did a wrong configuration of the host. Also, this option is not recommended in vMotion tasks(will degrade the performance while trying to perform a Storage vMotion).

Note: The ESXi host was working before, as long there was space in the Local Datastore there were no issues, we only notice the problem and the bad configuration when the Local Datastore was full and trigger the error.

In the same configuration tab, click edit that is in the right upper corner and then changes the option to the right one.

As you see in the image above, we need to select the first option and then click ok.

After this changes, we start the VMs again, and there are no issues. All VMs in this ESXi hosts were started without any problems.

Hope this can help you not only troubleshooting the issue(the steps that we need to do) but also fix the issue.

Note: Share this article, if you think it is worth sharing.

©2017 ProVirtualzone. All Rights Reserved