ESX Host:
IBM x3650 M3 – 2 X 6 Core CPU (Hyperthreading enabled)
ESXi 5 Update 2
144GB RAM
VM:
Windows 2008 R2 Datacentre running SQL 2012 Enterprise with CAL licensing model SP1 CU4.
24 vCPUs
140GB RAM
Hot swappable memory and CPU feature turned on. (If relevant, might be)
Issue being experienced:
When setting the max memory settings for SQL to 100GB, the server becomes completely unresponsive, SQL uses 100% CPU, and we are experiencing “insufficient memory in resource pool default” errors with in SQL. The only way to get the server back to normal is to stop the SQL Server service and set the SQL memory back to max 70GB, then everything is back to normal.
After allot of research on this it seems like NUMA could be causing this, SQL 2012 works with NUMA directly and its almost as if ESXi is not exposing the NUMA functionality to the guest OS. Even though VMWare states that this has been vastly improved in ESXi5.
Also SQL Server 2012 Enterprise on CAL licensing model limits SQL to only use a maximum of 20 cores. Licensing restrictions. Now it also looks like this could be causing NUMA node imbalances after reading the below article:
http://www.sqlperformance.com/2012/11/system-configuration/2012-cal-problems
My next step is to disable hyper threading in the BIOS on the server. Assign the VM, 12 Cores, set CPU affinity to 0-11 assign 136 GB of memory to the VM and disabled CPU & Memory hot swop feature, as I have also read that this disables NUMA?? Then assign SQL 100GB of ram.
Has anyone had this similar issues with SQL 2012 Enterprise in a VM environment?