I've started to see this issue on random VM's on our companys ESX cluster (3 hosts). I've had to stop working on this cluster until we can resolve the issue
I've run into a similar issue before that related to snapshots, however this problem is occuring on machines that have never been snapshotted, and in the past, had been working flawlessly.
Here is a snippet from the VMware log file of the VM:
2013-07-09T00:41:32.918Z| Worker#0| I120: DISK: OPEN scsi0:0 '/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager.vmdk' persistent R[]
2013-07-09T00:41:32.918Z| Worker#1| I120: DISK: OPEN scsi0:1 '/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager_1.vmdk' persistent R[]
2013-07-09T00:41:32.919Z| Worker#1| I120: FileIOErrno2Result: Unexpected errno=12, Cannot allocate memory
2013-07-09T00:41:32.919Z| Worker#1| I120: AIOGNRC: Failed to open '/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager_1-flat.vmdk' : Cannot allocate memory (c0002) (0x2013).
2013-07-09T00:41:32.919Z| Worker#1| I120: OBJLIB-FILEBE : FileBEOpen: can't open '/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager_1-flat.vmdk' : Cannot allocate memory (786435).
2013-07-09T00:41:32.920Z| Worker#1| I120: DISKLIB-VMFS : "/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager_1-flat.vmdk" : failed to open (Cannot allocate memory): ObjLib_Open failed. Type 3
2013-07-09T00:41:32.920Z| Worker#1| I120: DISKLIB-LINK : "/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager_1.vmdk" : failed to open (Cannot allocate memory).
2013-07-09T00:41:32.920Z| Worker#1| I120: DISKLIB-CHAIN : "/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager_1.vmdk" : failed to open (Cannot allocate memory).
2013-07-09T00:41:32.920Z| Worker#1| I120: DISKLIB-LIB : Failed to open '/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager_1.vmdk' with flags 0xa Cannot allocate memory (786441).
2013-07-09T00:41:32.920Z| Worker#1| I120: DISK: Cannot open disk "/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager_1.vmdk": Cannot allocate memory (786441).
2013-07-09T00:41:32.920Z| Worker#0| I120: DISKLIB-VMFS : "/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager-flat.vmdk" : open successful (10) size = 53687091200, hd = 3565896. Type 3
2013-07-09T00:41:32.920Z| Worker#0| I120: DISKLIB-DSCPTR: Opened [0]: "JD-5-manager-flat.vmdk" (0xa)
2013-07-09T00:41:32.920Z| Worker#1| I120: Msg_Post: Error
2013-07-09T00:41:32.920Z| Worker#1| I120: [msg.literal] Cannot allocate memory
2013-07-09T00:41:32.920Z| Worker#0| I120: DISKLIB-LINK : Opened '/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager.vmdk' (0xa): vmfs, 104857600 sectors / 50 GB.
2013-07-09T00:41:32.920Z| Worker#1| I120: [msg.disk.noBackEnd] Cannot open the disk '/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager_1.vmdk' or one of the snapshot disks it depends on.
2013-07-09T00:41:32.920Z| Worker#1| I120: ----------------------------------------
2013-07-09T00:41:32.920Z| Worker#0| I120: DISKLIB-LIB : Opened "/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager.vmdk" (flags 0xa, type vmfs).
2013-07-09T00:41:32.920Z| Worker#0| I120: DISK: Disk '/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager.vmdk' has UUID '60 00 c2 9c cf e7 7a d0-c9 a4 29 f2 3f 42 e7 e9'
2013-07-09T00:41:32.920Z| Worker#0| I120: DISK: OPEN '/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager.vmdk' Geo (6527/255/63) BIOS Geo (0/0/0)
2013-07-09T00:41:32.933Z| vmx| I120: DISK: Opening disks took 16 ms.
2013-07-09T00:41:32.933Z| vmx| I120: Module DiskEarly power on failed.
2013-07-09T00:41:32.933Z| vmx| I120: VMX_PowerOn: ModuleTable_PowerOn = 0
2013-07-09T00:41:32.933Z| vmx| I120: scsi0:0: numIOs = 0 numMergedIOs = 0 numSplitIOs = 0 ( 0.0%)
2013-07-09T00:41:32.933Z| vmx| I120: Closing disk scsi0:0
2013-07-09T00:41:32.933Z| vmx| I120: DISKLIB-VMFS : "/vmfs/volumes/51afe5a5-0bf2a06a-2bd9-842b2bfe9c89/JD-5-manager/JD-5-manager-flat.vmdk" : closed.
2013-07-09T00:41:32.937Z| vmx| I120: WORKER: asyncOps=3 maxActiveOps=2 maxPending=0 maxCompleted=2
And the vmdk file of the affected disk:
# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=4ed52c7e
parentCID=ffffffff
isNativeSnapshot="no"
createType="vmfs"
# Extent description
RW 629145600 VMFS "JD-5-manager_1-flat.vmdk"
# The Disk Data Base
#DDB
ddb.deletable = "true"
ddb.virtualHWVersion = "8"
ddb.longContentID = "8264138edeb9f6fb2afaf3be4ed52c7e"
ddb.uuid = "60 00 C2 98 c9 03 10 f9-19 1a c3 34 a1 ba e3 1c"
ddb.geometry.cylinders = "39162"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.thinProvisioned = "1"
ddb.adapterType = "lsilogic"
As you can see the parentCID is the correct setting for a vmdk with no snapshots.
Any tips for troubleshooting this problem?
Thanks kindly.