Henk's tech blog

a weblog about IT optimization

With the introduction of Processor affinity groups in WIndows 2008 R2 Datacenter Edition, there are also some new rules to play with to set affinity when a system runs with more than 64 cores:

On a 96 core Unisys ES7000 model 7600R there are , by default, 2 processor affinity groups , with 48 cores each. The cores are also divided by Node numbers; Node 0-3, CPU0-23.

(Depending on the BIOS settings you will find in Group 0, Numa Node 0, with CPU 0 .. 23 , Node 1 with also CPU0 ..23, and  in Group 1, Numa Node 2 there is CPU 0 ..23 and in finally, in Node 3 CPU 0 ..23).

This is achieved by setting the Partition BIOS like this:

The current Windows API calls to handle affinity allow you to set a preferred NUMA node to run on, however if the OS decides to schedule it on a different node, there is not much you can do, unless…

My colleague Jeroen Kooiman wrote a utility “RunOnGroupNodeCore.exe” which defines the entry point for the console application and will check if the preferred node is actually picked.

If not it will simply try again, overruling the round-robin and default "preferred NUMA node" API calls the Hara-kiri way, in other words, simple terminate and try again.

As a result we have full control over all the 96 cores and we can pick on what core an application is allowed to run.

As an example, we “wrote” the text “Tech Ed 2009” in Windows Taskmanager ;-) (see the 3 pictures below)  or  look at the SQLBits Promo on Youtube

or: 

check out Mark Russinovich’ PDC session on Win7 / Windows2008 R2 Kernel changes; http://microsoftpdc.com/Sessions/CL29 , after 54 minutes.;-)

 

 

Code snippet

// Loop create process until it runs on requested processor group

int numTries = 2000;

do {

// Create

if (CreateProcess(program, NULL, NULL, NULL, FALSE, EXTENDED_STARTUPINFO_PRESENT | CREATE_SUSPENDED | CREATE_NEW_CONSOLE, NULL, NULL, (LPSTARTUPINFOW)&startupInfoEx, &processInfo) == 0)

{

DWORD err = GetLastError();

printf("Error running program. Error code: %i (0x%x)", err, err);

return 1;

}

// Request group affinity

GROUP_AFFINITY groupAffinity;

GetThreadGroupAffinity(processInfo.hThread, &groupAffinity);

printf("%i) Created thread running on group %i, affinity mask=0x%p\n", 2001-numTries, groupAffinity.Group, groupAffinity.Mask);

// Does the program run in the desired processor group?

if (groupAffinity.Group == groupNum)

{

break; // yes, end loop

}

// No, terminate process

TerminateProcess(processInfo.hProcess, 99);

Sleep(1);

numTries–;

} while (numTries > 0);

// Failed

if (numTries==0)

{

printf("Error starting program in group %i", groupNum);

return 1;

}

Setting Processor Affinity in Windows2008R2, 5.0 out of 5 based on 2 ratings