DBPROP_SSCE_MAXBUFFERSIZE

Looking for ways to make batch insert work faster under OLE DB, I tried to fiddle a bit with the SQL CE 3.1 database properties. My first approach was to radically increase the maximum buffer size to 2 MB using the DBPROP_SSCE_MAXBUFFERSIZE property. The early experiments look promising with speed performance increases in the 50% range (yes, that much). I promise to come back to you with a better case in the form of an article where I will compare insertion performance under different scenarios and property settings.

For now, keep this in your mind: increasing the value of DBPROP_SSCE_MAXBUFFERSIZE may actually help your database perform better.

MFC Revisited

A few weeks ago, Mike Hall posted about "Why use MFC on Windows CE?" This was a bit of a surprise to me because you hardly see Microsoft discussing MFC these days. When the Compact Framework was launched, native code got such a bad press that I thought Microsoft was about to kill its C++ developer base. Fortunately I was wrong.

I (re)started developing for mobile devices in MFC and that was a very interesting experience. In those days the best PDA around was a Compaq iPAQ 3870 with Bluetooth and your development environment of chioce was eVC3. MFC seemed to be an obvious choice due to tool support but there were a few shortcomings. The frameworks imposes a document-view model that sometimes you have to fight against. Documentation was scarce (yes, the online help was bad) and your best resource was to read the MFC source files. This was how I got to understand how the framework was designed and why the tap-and-hold always made the dots circle twice. Interestingly you can see this behavior in a lot of the most recent devices on the Settings applets meaning that these are written in MFC (and the double circling bug is still there - sigh).

There is one good reason why you should not use MFC: it is slow. I understood this when I was working on the SqlCeSpy code and found that MFC would refresh a virtual list view noticeably slower than other applications, especially the ones written in ATL. MFC's message handling mechanism is a bit complex and convoluted meaning that when you need to handle a reflected message it gets bounced back and forth before it actually gets to where it is needed. The result for me was a slow refresh of the virtual list view. So slow in fact that it prompted me to write a grid-like control using virtual methods instead of reflected notification messages. Come to think of it, I never published this code...

So what is the best choice? As far as I can tell, when handling a user interface in native code, your best choice is to use WTL, ATL's extension now freely available from sourceforge. The thunk mechanism for handling the message maps make it very fast and efficient to process windows messages and applications tend to be both faster and smaller (compared to the statically linked MFC applications). Also, you feel like having much better control of what is going on and the templated approach is very elegant, although it requires conquering a steeper learning curve. But once you're there, you're hooked.

Installing the WTL 7.5 Application Wizards in VS 2005 under Vista

Don't even try to use the install script: it will not work. Vista's security mechanisms prevent the script from doing anythig on your VS 2005 install directories. I just went through this situation this morning and realized there was just one way to go about it: manually.

It's not that difficult, but took some time to reverse-engineer what the script was doing. The first thing you need to do is to locate the VS application wizards directory. In my case it's in:

C:\Program Files\Microsoft Visual Studio 8\VC\vcprojects

Under this directory you must create a new one named WTL (what else?). Now go to the directory where you installed the WTL 7.5 files and copy the following files:

WTLAppWiz.ico
WTLAppWiz.vsz

to the vcprojects directory. You will need administrator clearance to do this. Now change the security settings of WTLAppWiz.vsz to allow editing. Change it so it looks like this:

VSWIZARD 7.0
Wizard=VsWizard.VsWizardEngine.8.0
Param="WIZARD_NAME = WTLAppWiz"
Param="WIZARD_VERSION = 8.0"
Param="ABSOLUTE_PATH = D:\src\WTL75\AppWiz\Files"
Param="FALLBACK_LCID = 1033"

Replace the D: path with your own WTL 7.5 directory. Finally, copy the WTLAppWiz.vsdir file to the WTL directory you created before and, going through the same security checks you did for the vsz file, change it so it reads:

..\WTLAppWiz.vsz ATL/WTL Application Wizard1An application that uses the Windows Template Library. 67774096#1154

You are done.

Data Port Console 1.0 Beta 1 available for download

I have just uploaded the Data Port Console 1.0 Beta 1 setup file. This is the first public Beta of the product and it already has the license mechanism in place, limiting the use to 30 days. For the people I promised to offer a free license, now is the time to collect it: please send me an email with your desired license name so I can issue the key.

There are some new features wothy of notice:

• Connection breakups are detected and handled. This means that if you inadvertently break the USB connection both the desktop and the device will gracefully recover (the device may take a few seconds before it becomes responsive, but this is a RAPI issue).
• The device component is now manually installed from a specific menu. I have included a very comprehensive set of device cabs: Pocket PC 2003, Windows Mobile 5 (both Pocket PC and Smartphone), Windows CE 4.2 (all CPUs) and Windows CE 5.0 (all CPUs). Hopefully the WM5 cab will install correctly on WM6 devices.
• The new version of DesktopSqlCe (1.8) is included (the console is built on top of DesktopSqlCe 1.8) and the Gold will ship with a single user license of this component. This means that if you just want to use the component in one PC (the same where the Console is installed) you will not have to pay the extra DesktopSqlCe license.

I'm really interested in reading your opinions on this product and what features you would add / remove / change.

Diagnosing cab installations

This is an old time favorite but I seem to keep forgetting it. So for my future reference, here are the registry keys for enabling cab installation diagnostics on the PC:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows CE Services\AppMgr\ReportErrors = (DWORD)1

The dangers of not terminating your strings

After a bit of a long break I'm here to tell you about my latest wild bug hunt. A few months ago I was contacted by a French company to develop a high-speed text to SQL CE converter. This component allows the user to send plain or compressed text files to a device and locally process inserts, updates and deletes. Due to its very nature, the code was supposed to accept either UTF8 or UTF16 character encodings.

My first approach was to use the text conversion functions (wcstombs and mbstowcs). This way I had only one set of conversion functions (I mean converting from text to a GUID, to a DBTIMESTAMP and so forth). After the first tests the customer complained about stability: the data would be written to the database but the component DLL crashed almost consistently.

Back to the eVC4 debugger (now running on my Vista box in a Parallels virtual machine) I was confronted with an ugly beast. After successfully running the code, the whole thing would blast into hyperspace when closing the data source object (yes, that's the last thing you close). Hunting the bug forced me to look everywhere (even in the tested ATL OLE DB code). The thing was nowhere to be found. Often times I had to hard reset the debug device in order to get it back to its feet. Not a nice thing...

Bugs like these "smell" like memory issues: you write to a wild pointer and you may destroy some very fundamental data structures in your application. Needless to say that you are in for some unexpected outcomes.

I did not kill the bug directly. Instead, I rewrote the code so as to avoid text conversions. After running the new code for the first time the whole thing just behaved correctly. So should we avoid the text conversion functions altogether? Absolutely not! Just be wiser than me when you use them and make sure the converted strings are terminated. An unterminated string may cause this kind of problems in your application. So beware: you can either add one to the source string length or you can do that to the target string and make sure the buffer is zeroed before the conversion. Or you can do both.