May 14, 2016 However with the low disk space, you should run the backup with either the VM powered off, or out of business hours. Veeam - like other backup applications - will create an additional temporary snapshot for the backup which will consume disk space too. Oct 17, 2016 It was pointing to vmdks which contain “diskname-000002.vmdk (snapshot delta vmdks)” for each and every disks, there were no actual VMDKs as it was pointing in the.vmx file. Then I had to manually edit the.vmx file as I had no other choice, we were not able to restore this Virtual Machine at that time.
VMware snapshots are not deleted after a VM backup occurs, ANS2718W is seen when the next backup runs
Snapshot consolidation is failing due to a VMware timeout.
An ANS2718W message may be observed in the client error log:
ANS2718W The virtual machine 'TESTVM1234WV' requires snapshot consolidation
The ANS2718W message indicates that snapshot consolidation may be failing.
TSM does not perform the snapshot consolidation, this is strictly a VMware function.
The hostd.log and vmware.log should be reviewed for the following errors:
hostd.log:
2015-12-03T11:25:42.782Z [363C2B70 info 'Vmsvc.vm:/vmfs/volumes/55846daf-992a61d8-e034-3440b5c5c2e0/TESTVM1234WV/TESTVM1234WV.vmx'] Consolidate Disks failed: vim.fault.GenericVmConfigFault
2015-12-03T11:25:42.782Z [363C2B70 verbose 'Vmsvc.vm:/vmfs/volumes/55846daf-992a61d8-e034-3440b5c5c2e0/TESTVM1234WV/TESTVM1234WV.vmx'] Consolidate Disks message: The virtual machine has exceeded the maximum downtime of 12 seconds for disk consolidation.
--> Synchronous consolidate failed for scsi0:1: Operation was canceled.
--> An error occurred while consolidating disks: The operation failed.
2015-12-03T11:25:42.782Z [35081B70 info 'Vimsvc.ha-eventmgr'] Event 9132 : Virtual machine TESTVM1234WV disks consolidation failed on LGS2ESX1201XVP.ampf.com in cluster LGS2ESX1201XVP.ampf.com in ha-datacenter.
vmware log:
2015-12-03T11:25:22.680Z| vcpu-0| I120: SNAPSHOT: SnapshotCombineDisks: Consolidating from '/vmfs/volumes/55846daf-992a61d8-e034-3440b5c5c2e0/TESTVM1234WV/TESTVM1234WV_1-000004.vmdk' to '/vmfs/volumes/55846daf-992a61d8-e034-3440b5c5c2e0/TESTVM1234WV/TESTVM1234WV_1.vmdk'.
2015-12-03T11:25:22.681Z| vcpu-0| I120: DISKLIB-VMFS : '/vmfs/volumes/55846daf-992a61d8-e034-3440b5c5c2e0/TESTVM1234WV/TESTVM1234WV_1-flat.vmdk' : open successful (24) size = 139586437120, hd = 261200877. Type 3
...
2015-12-03T11:25:42.739Z| vcpu-0| I120: SnapshotVMXConsolidateHelperProgress: Stunned for 20 secs (max = 12 secs). Aborting consolidate.
...
2015-12-03T11:25:42.749Z| vcpu-0| I120: SNAPSHOT: Snapshot_ConsolidateWorkItem failed: Operation was canceled (5)
2015-12-03T11:25:42.749Z| vcpu-0| I120: SnapshotVMXConsolidateOnlineCB: Synchronous consolidate failed for disk node: scsi0:1. Adding it to skip list.
The key elements are the messages regarding exceeding 12 seconds.
The consolidation problem is completely external to Tivoli Storage Manager and is a VMware issue. The following VMware document discusses the root cause of the problem and how to resolve it.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2082886
The recommendation is to implement the fix in the document, if this does not resolve the consolidation failure, VMware support should be engaged to investigate the problem.
TSM
So I'm currently running into an issue with snapshots that has got me stumped thus far.
A few days ago we had an issue where the storage for a number of VMs went down shortly after nightly backups were finishing for one of the VMs. In fact, it failed while it was removing the snapshot the backup system took for this particular VM.
The backup system tried to remove the snapshot a few more times and finally succeeded after HA restarted the VM elsewhere. The next night, it ran it's backups again, but this time after trying to delete the snapshots disk consolidation failed.
At this point, manual disk consolidation is failing in my vSphere Client consistently with: 'An error occurred while consolidating disks: 5 (input/output error)'
Reviewing the vmware.log for the VM, I see this thrown at the same time as the above failure:
2015-06-11T18:20:46.349Z| SnapshotVMXCombiner| I120: OBJLIB-FILEBE : FileBEIoctl: ioctl operation failed on '/vmfs/devices/deltadisks/3555fd86-VMNAME_1-000002-delta.vmdk' : Input/output error (327682) 2015-06-11T18:20:46.350Z| SnapshotVMXCombiner| I120: DISKLIB-VMFS_SPARSE : VmfsSparseExtentCombine: failed: for 1 level and start 164132 Input/output error. 2015-06-11T18:20:46.350Z| SnapshotVMXCombiner| I120: DISKLIB-CTK : End Combine 2015-06-11T18:20:46.353Z| SnapshotVMXCombiner| I120: DISKLIB-CTK : Attempting unlink of (null) 2015-06-11T18:20:46.353Z| SnapshotVMXCombiner| I120: SnapshotVMXCombineFinalCb: Done with combine of 2 links, starting from 1 in 2362133383 usec with error 0x50009: Input/output error
I've seen a few posts on the VMware communities talk about solving this by shutting down the VM, adding an empty snapshot, then consolidating again, but I'm nervous to try that as others have said when they are in this state if they shut down they can't start the VM again.
Anyone ever run into an issue like this with consolidation?