CDataSource and CSession

With this post, I'm presenting the first incarnation of the CDataSource and CSession classes on the OLE DB client library. These still bear some similarity to the ATL OLE DB Consumer Templates counterparts (I did stole some variable names, sorry), but you will see some differences. For instance, there is a CDataSource::Create method that can optionally open a CSession object. This means that besides creating the database, you can immediately open a session with it. As always, I'm using the CreateSDF sample to implement this code.

From now on things will start to get a bit more interesting and useful: besides creating or opening a database using OLE DB, we want to access and edit both data and structure. We can do this either through SQL commands or through the exposed interfaces. My next challenge will be to tackle base table cursors and SQL commands. These require the introduction of two other objects that are central to OLE DB programming: the rowset and the accessor. While a rowset represents a set of rows that you can navigate on, an accessor defines how the individual columns are accessed and mapped to your application data. Rowsets are not only used to handle cursor data (both from a table and from a SQL query) but they are also internally used by the OLE DB provider to report other data to the consumer (like schema information). Accessors are required to map data between your application's memory and the provider's data, for three specific purposes:

• Table or query data;
• SQL command parameters;
• Index column data.

Here we will meet some very interesting challenges, like handling the BLOB peculiarities of SQL Compact, but this will be an interesting ride. I promise.

Connecting to a SQL Compact database via OLE DB

The generic principles of connecting to a database via an OLE DB provider are described in this MSDN page: First you create a Data Source object and then you use it to create a Session object. While the Data Source represents the target database, the Session object represents an individual connection to that database. You use the Session object to (among other things) create and execute SQL commands, open base table cursors for fast data insertion, manage transactions and make changes to the database schema.

Both the Data Source and the Session objects are COM objects and thus can be exposed thrugh a number of different interfaces (see the "CoType" definitions for both objects). Some of the interfaces are marked as mandatory while others as optional. SQL Compact implements a subset of these interfaces:

DataSource

• IDBCreateSession
• IDBDataSourceAdmin
• IDBInitialize
• IDBProperties
• IDBInfo
• ISupportErrorInfo

Interestingly, the mandatory IPersist interface is not implemented in SQL Compact.

Session

• IGetDataSource
• IOpenRowset
• ISessionProperties
• IAlterIndex
• IAlterTable
• IDBCreateCommand
• IDBSchemaRowset
• IIndexDefinition
• ISupportErrorInfo
• ITableCreation
• ITableDefinition
• ITableDefinitionWithConstraints
• ITransactionLocal

Now, we can start writing high-level code to connect to a SQL Compact database. On my next post I will publish a minimalistic approach to both these objects.

Detecting the installed SQL Compact OLE DB providers

Here is a minor (but important) improvement to the CreateSDF project: installed SQL Compact engine detection. Why should the little application allow the user to select all versions of the SQL Compact engine when the device has only one installed? We are opening the door for a bad user experience and some unexplained errors. So what can we do to detect the installed SQL Compact engines?

A word of caution is required here: this method detects only the installed OLE DB providers, not the SQL Compact engines. From version 3.0 onwards, Microsoft split the engine interfaces into two different stacks (managed and native) that are separately installable. This means that you can install the managed SQL Compact 3.5 SP1 stack without installing the native one. This methods detects only the installed OLE DB stacks.

The process of detecting the installed OLE DB providers is quite simple: just try to instantiate the IDBInitialize interface using the engine's CLSID:

bool IsInstalled(const CLSID& clsid)
{
CComPtr<IDBInitialize> spInitialize;
HRESULT hr;

hr = CoCreateInstance(clsid, NULL, CLSCTX_INPROC_SERVER, IID_IDBInitialize, (void**)&spInitialize);

return SUCCEEDED(hr);
}

You simply use this code like this:

if(IsInstalled(CLSID_SQLSERVERCE_3_5)) ...

In the revised sample, this test is used to filter out the nonexistent OLE DB providers from the engine combo box.

CDbException

After my last post's explanation on how to retrieve OLE DB error information, here's a first implementation approach (sample code here). To retrieve the error information you just need to create an instance of the CDbException class, using the HRESULT value that you used to detect the error. Right now this class provides a minimal interface to retrieveing error information and will be updated in future releases. Nevertheless, you can now get a textual description that you can associate to the HRESULT value when reporting back the error to the user.

These errors tend to be very descriptive about what cause them, but there are some exceptions. For instance, try to create a SQL CE 2.0 or 3.0 database without a password and specify that you want to encrypt it. Database creation will fail with a very terse "Errors occurred." message...

When such an error occurs, you should also look at the "basic error information" (ERRORINFO structure) for the native error code. You must look this value up on the provider's published error messages where you should get more detailed info about the error.

Handling OLE DB Errors

Just by looking at the MSDN documentation you will have a bit of a hard time figuring out how to report error information from OLE DB. The first thing you have to do is to test the HRESULT value that most methods return by using either the SUCCEEDED or FAILED macros. Typically an error is negative and a success status is positive or zero. Please note that some methods may return S_FALSE which is not an error (SUCCEDED(S_FALSE) is true). Now that you know that an error was generated, you can retrieve more information about it.

The first thing you need to do is call the GetErrorInfo API function. If you call any other OLE DB method the error information will be reset and lost. The GetErrorInfo function returns an IErrorInfo interface pointer that does not contain any valuable information. Instead, you must use it to call QueryInterface and request an IErrorRecords interface pointer. This interface implements an error collection (typically contais one error only) where each error object is exposed as an IErrorInfo interface. Confused? It gets a bit better...

Getting the underlying error information requires some work because it is split between the IErrorRecords and each contained IErrorInfo. For instance, to retrieve the error parameters (the six parameter values you might have seen in a SqlCeError object), you must call the IErrorRecords::GetErrorParameters method. It takes as parameters the requested zero-based error index and a pointer to a DISPPARAMS structure. You can also get some basic error information by calling IErrorRecords::GetBasicErrorInfo with an ERRORINFO structure pointer as the second argument. For each IErrorInfo in the error collection, you can get information such as description, the source (reports the database engine and version) and the interface GUID that generated the error.

Oh, and when you are done make sure everything is cleaned up. Yes, we need another abstraction to manage error reporting. Hopefully this will be the theme for the next post and from then on we can move on to opening a database connection, executing SQL statements and more.

CDbProp and CDbPropSet

On my last post I promised to implement a a better way to handle property values - using the raw DBPROP and DBPROPSET structures is a pain, adds unnecessary code and reduces code readability. My approach to solve this problem is very similar to what you can find in the ATL OLE DB Consumer Templates: I defined two classes, each publicly deriving from the OLE DB strucrures and then added some conveninence code.

CDbProp
This class is a very simple wrapper around DBPROP that provides only safe initialization and disposal. It also knos how to make copies of itself. You will see this code in action when a property is added to a property set.

CDbPropSet
Besides wrapping the DBPROPSET structure, this class adds a few methods to ease the addition of properties to a property set. For instance, setting the file name is now simply:

propSetProvider.AddProperty(DBPROP_INIT_DATASOURCE, szFileName);

This makes for less code and especially for more readable code. The AddProperty method is overloaded for booleans, integers and strings (the most common property types) and simply sets all the DBPROP structure values according to the data type (see how the vVariant value is set), and then adds this structure to the existing property set. Property values are set through an instance of CDbProp class because it clears the variant data when it goes out of scope. Also it knows how to make a copy of itself into a DBPROP pointer (the CopyTo method).

Property set memory is managed through the CoTaskMemRealloc and CoTaskMemFree functions. Why didn't I use the new and delete operators? Some OLE DB interface methods will return this type of data back to the client and the client needs to correctly delete it. The provider allocates all its data using the CoTaskMem functions, so our code needs to comply in order to avoid memory leaks. The downside of this design is that we lose the delete [] semantics and must implement it ourselves (see CDbPropSet::Clear)...

The second version of the sample code can be downloaded from here.

Creating SDF files from OLE DB

Following up on my last post I wrote a very simple Windows Mobile application that will create SDF files (versions 2.0, 3.0 and 3.5) using straight OLE DB calls. This sample application illustrates why it is so hard to develop against the OLE DB interfaces without a proper abstraction. Included in the ZIP file you will find the sqlce_ex.h header file that complements the original sqlce_oledb.h by adding the GUID declarations for the older databases and also by modifying some of the constants.

When Microsoft upgraded to SQL CE 3.0, some of the property ID constants were changed while retaining their names. This meant that the same property ID would have different values in 2.0 and 3.0. To overcome this, the extension header file undefines the conflicting properties and redefines them with new names that refer to the version:

#undef DBPROP_SSCE_COL_ROWGUID
#undef DBPROP_SSCE_MAXBUFFERSIZE
#undef DBPROP_SSCE_DBPASSWORD
#undef DBPROP_SSCE_ENCRYPTDATABASE
#undef DBPROP_SSCE_TEMPFILE_DIRECTORY

These are redefined as:


#define DBPROP_SSCE3_COL_ROWGUID 0x1F9L
#define DBPROP_SSCE3_MAXBUFFERSIZE 0x1FAL
#define DBPROP_SSCE3_DBPASSWORD 0x1FBL
#define DBPROP_SSCE3_ENCRYPTDATABASE 0x1FCL
#define DBPROP_SSCE3_TEMPFILE_DIRECTORY 0x1FEL

And:

#define DBPROP_SSCE2_COL_ROWGUID 0x69
#define DBPROP_SSCE2_MAXBUFFERSIZE 0x70
#define DBPROP_SSCE2_DBPASSWORD 0x71
#define DBPROP_SSCE2_ENCRYPTDATABASE 0x72
#define DBPROP_SSCE2_TEMPFILE_DIRECTORY 0x73

The aplication itself is a very simple WTL 8.0 dialog with just a few simple properties:

• File name
• Password (optional)
• Locale (optional)
• Encryption (behavior depends on engine version)
• Engine version

When the user presses the "Create" menu option the dialog calls the OnCreateSdf message handler where all the action takes place. As you can see, this is a piece of ugly code. Most of it involves managing the DBPROPSET array and the contained DBPROP arrays, something that you will see in a large number of OLE DB method calls.

A DBPROPSET array contains an array of DBPROP arrays. Each DBPROP structure contains the value of a single OLE DB property (like file name, locale, encryption mode...) and these are grouped in property sets. Each property set has its own unique ID, like DBPROPSET_DBINIT (generic OLE DB database initialization properties) and DBPROPSET_SSCE_DBINIT (SQL CE-specific initialization properties). This scheme only works correctly if you put each property in its own specific set and you must read the documentation to learn which goes where. In our case, we have:

DBPROPSET_DBINIT:

• DBPROP_INIT_DATASOURCE - database file name
• DBPROP_INIT_LCID - database locale ID

DBPROPSET_SSCE_DBINIT:

• DBPROP_SSCE_ENCRYPTIONMODE - database encryption mode (for 3.5 only)
• DBPROP_SSCE2_ENCRYPTDATABASE - database encryption status (for 2.0 only)
• DBPROP_SSCE3_ENCRYPTDATABASE - database encryption status (for 3.0 only - deprecated in 3.5)
• DBPROP_SSCE2_DBPASSWORD - database password (for 2.0 only)
• DBPROP_SSCE3_DBPASSWORD - database password (for 3.0 / 3.5 only)

After filling up the property arrays, we can call the IDBDataSourceAdmin::CreateDataSource method and, if all properties are correctly set, we sould get a brand new SDF file.

At the end, don't forget to cleanup all the resources you have allocated. This includes the input properties and any output data. Also, don't forget to release the used OLE DB interfaces!

Before you make this code work on a Windows Mobile device, you must install at least one of the SQL Compact engines. For versions 3.0 and 3.5 make sure you also install the replication cab file because this is how the OLE DB provider is installed. Needless to say, you can have all three engines installed on the same device.

What's next? First, we need a better abstraction for handling all the property values. Second, we need a better way of retrieving error information whenever one pops up. I will start the OLE DB client library by implementing these two abstractions.

SQL Compact 3.5 SP1 and OLE DB

Yesterday the SP1 of SQL Compact 3.5 was announced with some interesting additions like 64 bit support. I have been using SQL CE since version 1.0 (since 2002?) and always from native code. My early applications were written with ADOCE which was later deprecated when the Pocket PC 2003 platform was introduced. The only alternative left for native applications was to use straight OLE DB interfaces. Interestingly, the ATL OLE DB Consumer Templates were available on the original Pocket PC 2003 SDK, but did not compile correctly. My early work with this code (two header files, essentially) was to adapt it in order to make it work with eVC3 and eVC4. Finally, Microsoft completely dropped these files from the Windows CE and Windows Mobile SDKs and this sent a very clear signal (or so I think): don't use this code.

What can we do now? Using the OLE DB interfaces directly is a big pain because these are very low-level abstractions. We need something that elevates the abstraction to something of the ATL OLE DB Consumer Templates level. And this is just what I'm proposing to do (yes, I'm crazy).

The Consumer Templates are a very interesting and informative piece of code. The usual ATL templated classes are there and the classes model very closely all sorts of consumer objects that you may use. But they have one very interesting drawback. As I found out, you have to tweak the code under some very specific circumstances, when you don't know the exact outcome of a command execution, for example. These are not insurmountable and reflect the purpose of the class library: use a class template instantiation that is customized for your exact needs. I will use some of the Consumer Templates' concepts (not the code) in this new library which will be prmarily targeted to embedded and mobile devices. As a side effect, it should also be able to work in a desktop application.

Given this scope, the primary target of this new library is to develop applications for all versions of SQL Compact (SQL CE), namely 3.5, 3.0 and 2.0. Instead of writing SQL Compact-specific code only, this will be built in two layers: a generic OLE DB layer and a SQL Compact-specific layer built on top of the OLE DB layer.

So what do you need to get started? First of all you need the SQL Compact header file. For SQL Compact 3.5 you need the sqlce_oledb.h, while for SQL Compact 3.0 and 3.1 you need ssceoledb.h. These files contain both the OLE DB Interface, structure and constant declarations as well as the SQL Compact-specific GUIDs and constants. This means that, without tweaking, you musy have a header file for each version of SQL Compact. Fortunately this is not required and we may use just one header file to compile an application that will consume all versions of the database engine - a very nice feature indeed.

On my next post I will provide an add-on header file that will allow you to use sqlce_oledb.h to target all SQL Compact versions.

ClearType on a memory DC

I recently developed a small information browser application for Windows CE 5.0 devices. This small application uses a touch list as an item selector and displays an in-memory bitmap with textual information related to the selected item.

The touch list uses a ClearType-rendered font (see the latest Touch List sample) painted to a memory DC. I also used a ClearType font to paint the text on the memory bitmap containing the detailed information about the main list item. Unfortunately the font was not being rendered in ClearType mode. Why was this? Maybe a Windows CE issue?

After quickly recompiling the code to target a Windows Mobile 6.0 device, I got exactly the same result. So I turned to the rendering code looking for a bug, but found none. Instead I found an interesting difference between the touch list rendering and the bitmap rendering.

If you look at the touch list samples, you will see that the memory DC is created from a CPaintDC. On the bitmap rendering code, I was using a memory DC created from a CClientDC (got the same result by using NULL as the HDC value).

So apparently you can only render ClearType fonts to a memory DC when this is created as compatible with the DC you get on BeginPaint... Has anyone else experienced this?

Windows Mobile API Usage Tool

Microsoft has just released a new tool to determine the Windowm Mobile API usage in your application. This is especially useful for deprecated functions! Go take a look here.

The Touch List - II

I finally have some time to come back and write about the Touch List window. It's been quite some time since I wrote the first post about this code, and it has changed a lot (especially because I had to use it for a customer project), and I have also learned a lot about how to make this code work on both Windows Mobile and Windows CE devices.

Code Changes
The first challenge I had to face was to decouple the "kinetics" from the list and implement it in a more general WTL base class. This was a project requirement for one of the data windows where a bitmap is displayed and the same auto scrolling behavior was sought.

The second challenge was to implement some sort of scroll bar into the window. I could use the native Windows scroll bars but I decided to use something nicer and possibly "cooler".

All the "touch" code was put into one single header file pompously named atltouch.h (see the sample code here). This header file contains the following classes:

• CScrollBarData - Contains scroll bar data (either horizontal or vertical) and associated calculations.
• CTouchWindow - The "touch" base class, where you will find all generic "kinetics" functions.
• CTouchListItem - Base class for touch list items. Implement one for your items (see sample code).
• CTouchList - The touch list class template.

Please understand that this code is still very much under construction so you will see some redundant concepts and the occasional bug.

Finger Use
Most of the current devices are not ready for finger input, only for stylus input. Sure you can use your finger but I have found that on some devices, when you put your index finger to the screen, the application window receives the expected WM_LBUTTONDOWN, but may also receive a lot of WM_MOUSEMOVE messages... Why is this? Your finger is way larger than the expected stylus tip so you are effectively touching more than one screen point. What I have experienced is that most devices will produce the mouse move messages and your list will start scrolling just by touching it with your finger. This is less likely to happen when using the stylus.

The solution for this was to implement a "sensitivity" factor that will ignore movements within a given range (see the SetSensitivity method). If you use this with any value larger than 1 (in either direction) your list will become less sensitive, but you will have to adjust this value to your device...

Scroll Bars
Scroll bars have an unusual implementation, but I hope you find them useful (if not, suggestions
are mostly welcome). To use the vertical scroll bar, just drag your finger (or stylus) up or down and the list will scroll accordingly.

On Windows Mobile 5 and 6 devices, these are implemented as a transparent layer on top of your list so you can use the full screen size to display data.