K-Jam and ToolHelp

My good friend Cristiano Severini has just published his first Windows Mobile application: CSB (standing for Call and SMS Blocker). One of the chores CSB must accomplish is to shut down tmail.exe during setup in order to install MAPI filters. The setup process finishes with a soft-reset, restoring all processes.

To shut down a process, you must get a handle to it and this is usually done through the ToolHelp API. This API creates a static snapshot of all running processes and allows the application to enumerate them and kill these processes at will. Interestingly, this technique worked quite well until Cristiano tested it in a K-Jam. Being the fortunate owner of one, I was volunteered to help (forced, actually) and found out that the CreateToolhelp32Snapshot funtion would fail every time when called with the TH32CS_SNAPPROCESS flag.

Thanks to one very clever fellow MVP (Paul Tobey) and Microsoft employee (Ilya Tumanov), I now know that this function was failing because it was generating way too much data in the snapshot. To limit the amount of data it generates, one needs to add another flag: TH32CS_SNAPNOHEAPS. Interestingly, this flag is not defined in tlhelp32.h file, where it was supposed to be. Here it is, with my thanks to Paul and Ilya:

// optimization for text shell to not snapshot heaps
#define TH32CS_SNAPNOHEAPS 0x40000000