CDataSource and CSession

With this post, I'm presenting the first incarnation of the CDataSource and CSession classes on the OLE DB client library. These still bear some similarity to the ATL OLE DB Consumer Templates counterparts (I did stole some variable names, sorry), but you will see some differences. For instance, there is a CDataSource::Create method that can optionally open a CSession object. This means that besides creating the database, you can immediately open a session with it. As always, I'm using the CreateSDF sample to implement this code.

From now on things will start to get a bit more interesting and useful: besides creating or opening a database using OLE DB, we want to access and edit both data and structure. We can do this either through SQL commands or through the exposed interfaces. My next challenge will be to tackle base table cursors and SQL commands. These require the introduction of two other objects that are central to OLE DB programming: the rowset and the accessor. While a rowset represents a set of rows that you can navigate on, an accessor defines how the individual columns are accessed and mapped to your application data. Rowsets are not only used to handle cursor data (both from a table and from a SQL query) but they are also internally used by the OLE DB provider to report other data to the consumer (like schema information). Accessors are required to map data between your application's memory and the provider's data, for three specific purposes:

• Table or query data;
• SQL command parameters;
• Index column data.

Here we will meet some very interesting challenges, like handling the BLOB peculiarities of SQL Compact, but this will be an interesting ride. I promise.

Connecting to a SQL Compact database via OLE DB

The generic principles of connecting to a database via an OLE DB provider are described in this MSDN page: First you create a Data Source object and then you use it to create a Session object. While the Data Source represents the target database, the Session object represents an individual connection to that database. You use the Session object to (among other things) create and execute SQL commands, open base table cursors for fast data insertion, manage transactions and make changes to the database schema.

Both the Data Source and the Session objects are COM objects and thus can be exposed thrugh a number of different interfaces (see the "CoType" definitions for both objects). Some of the interfaces are marked as mandatory while others as optional. SQL Compact implements a subset of these interfaces:

DataSource

• IDBCreateSession
• IDBDataSourceAdmin
• IDBInitialize
• IDBProperties
• IDBInfo
• ISupportErrorInfo

Interestingly, the mandatory IPersist interface is not implemented in SQL Compact.

Session

• IGetDataSource
• IOpenRowset
• ISessionProperties
• IAlterIndex
• IAlterTable
• IDBCreateCommand
• IDBSchemaRowset
• IIndexDefinition
• ISupportErrorInfo
• ITableCreation
• ITableDefinition
• ITableDefinitionWithConstraints
• ITransactionLocal

Now, we can start writing high-level code to connect to a SQL Compact database. On my next post I will publish a minimalistic approach to both these objects.

Detecting the installed SQL Compact OLE DB providers

Here is a minor (but important) improvement to the CreateSDF project: installed SQL Compact engine detection. Why should the little application allow the user to select all versions of the SQL Compact engine when the device has only one installed? We are opening the door for a bad user experience and some unexplained errors. So what can we do to detect the installed SQL Compact engines?

A word of caution is required here: this method detects only the installed OLE DB providers, not the SQL Compact engines. From version 3.0 onwards, Microsoft split the engine interfaces into two different stacks (managed and native) that are separately installable. This means that you can install the managed SQL Compact 3.5 SP1 stack without installing the native one. This methods detects only the installed OLE DB stacks.

The process of detecting the installed OLE DB providers is quite simple: just try to instantiate the IDBInitialize interface using the engine's CLSID:

bool IsInstalled(const CLSID& clsid)
{
CComPtr<IDBInitialize> spInitialize;
HRESULT hr;

hr = CoCreateInstance(clsid, NULL, CLSCTX_INPROC_SERVER, IID_IDBInitialize, (void**)&spInitialize);

return SUCCEEDED(hr);
}

You simply use this code like this:

if(IsInstalled(CLSID_SQLSERVERCE_3_5)) ...

In the revised sample, this test is used to filter out the nonexistent OLE DB providers from the engine combo box.

CDbException

After my last post's explanation on how to retrieve OLE DB error information, here's a first implementation approach (sample code here). To retrieve the error information you just need to create an instance of the CDbException class, using the HRESULT value that you used to detect the error. Right now this class provides a minimal interface to retrieveing error information and will be updated in future releases. Nevertheless, you can now get a textual description that you can associate to the HRESULT value when reporting back the error to the user.

These errors tend to be very descriptive about what cause them, but there are some exceptions. For instance, try to create a SQL CE 2.0 or 3.0 database without a password and specify that you want to encrypt it. Database creation will fail with a very terse "Errors occurred." message...

When such an error occurs, you should also look at the "basic error information" (ERRORINFO structure) for the native error code. You must look this value up on the provider's published error messages where you should get more detailed info about the error.

Handling OLE DB Errors

Just by looking at the MSDN documentation you will have a bit of a hard time figuring out how to report error information from OLE DB. The first thing you have to do is to test the HRESULT value that most methods return by using either the SUCCEEDED or FAILED macros. Typically an error is negative and a success status is positive or zero. Please note that some methods may return S_FALSE which is not an error (SUCCEDED(S_FALSE) is true). Now that you know that an error was generated, you can retrieve more information about it.

The first thing you need to do is call the GetErrorInfo API function. If you call any other OLE DB method the error information will be reset and lost. The GetErrorInfo function returns an IErrorInfo interface pointer that does not contain any valuable information. Instead, you must use it to call QueryInterface and request an IErrorRecords interface pointer. This interface implements an error collection (typically contais one error only) where each error object is exposed as an IErrorInfo interface. Confused? It gets a bit better...

Getting the underlying error information requires some work because it is split between the IErrorRecords and each contained IErrorInfo. For instance, to retrieve the error parameters (the six parameter values you might have seen in a SqlCeError object), you must call the IErrorRecords::GetErrorParameters method. It takes as parameters the requested zero-based error index and a pointer to a DISPPARAMS structure. You can also get some basic error information by calling IErrorRecords::GetBasicErrorInfo with an ERRORINFO structure pointer as the second argument. For each IErrorInfo in the error collection, you can get information such as description, the source (reports the database engine and version) and the interface GUID that generated the error.

Oh, and when you are done make sure everything is cleaned up. Yes, we need another abstraction to manage error reporting. Hopefully this will be the theme for the next post and from then on we can move on to opening a database connection, executing SQL statements and more.

CDbProp and CDbPropSet

On my last post I promised to implement a a better way to handle property values - using the raw DBPROP and DBPROPSET structures is a pain, adds unnecessary code and reduces code readability. My approach to solve this problem is very similar to what you can find in the ATL OLE DB Consumer Templates: I defined two classes, each publicly deriving from the OLE DB strucrures and then added some conveninence code.

CDbProp
This class is a very simple wrapper around DBPROP that provides only safe initialization and disposal. It also knos how to make copies of itself. You will see this code in action when a property is added to a property set.

CDbPropSet
Besides wrapping the DBPROPSET structure, this class adds a few methods to ease the addition of properties to a property set. For instance, setting the file name is now simply:

propSetProvider.AddProperty(DBPROP_INIT_DATASOURCE, szFileName);

This makes for less code and especially for more readable code. The AddProperty method is overloaded for booleans, integers and strings (the most common property types) and simply sets all the DBPROP structure values according to the data type (see how the vVariant value is set), and then adds this structure to the existing property set. Property values are set through an instance of CDbProp class because it clears the variant data when it goes out of scope. Also it knows how to make a copy of itself into a DBPROP pointer (the CopyTo method).

Property set memory is managed through the CoTaskMemRealloc and CoTaskMemFree functions. Why didn't I use the new and delete operators? Some OLE DB interface methods will return this type of data back to the client and the client needs to correctly delete it. The provider allocates all its data using the CoTaskMem functions, so our code needs to comply in order to avoid memory leaks. The downside of this design is that we lose the delete [] semantics and must implement it ourselves (see CDbPropSet::Clear)...

The second version of the sample code can be downloaded from here.

Creating SDF files from OLE DB

Following up on my last post I wrote a very simple Windows Mobile application that will create SDF files (versions 2.0, 3.0 and 3.5) using straight OLE DB calls. This sample application illustrates why it is so hard to develop against the OLE DB interfaces without a proper abstraction. Included in the ZIP file you will find the sqlce_ex.h header file that complements the original sqlce_oledb.h by adding the GUID declarations for the older databases and also by modifying some of the constants.

When Microsoft upgraded to SQL CE 3.0, some of the property ID constants were changed while retaining their names. This meant that the same property ID would have different values in 2.0 and 3.0. To overcome this, the extension header file undefines the conflicting properties and redefines them with new names that refer to the version:

#undef DBPROP_SSCE_COL_ROWGUID
#undef DBPROP_SSCE_MAXBUFFERSIZE
#undef DBPROP_SSCE_DBPASSWORD
#undef DBPROP_SSCE_ENCRYPTDATABASE
#undef DBPROP_SSCE_TEMPFILE_DIRECTORY

These are redefined as:


#define DBPROP_SSCE3_COL_ROWGUID 0x1F9L
#define DBPROP_SSCE3_MAXBUFFERSIZE 0x1FAL
#define DBPROP_SSCE3_DBPASSWORD 0x1FBL
#define DBPROP_SSCE3_ENCRYPTDATABASE 0x1FCL
#define DBPROP_SSCE3_TEMPFILE_DIRECTORY 0x1FEL

And:

#define DBPROP_SSCE2_COL_ROWGUID 0x69
#define DBPROP_SSCE2_MAXBUFFERSIZE 0x70
#define DBPROP_SSCE2_DBPASSWORD 0x71
#define DBPROP_SSCE2_ENCRYPTDATABASE 0x72
#define DBPROP_SSCE2_TEMPFILE_DIRECTORY 0x73

The aplication itself is a very simple WTL 8.0 dialog with just a few simple properties:

• File name
• Password (optional)
• Locale (optional)
• Encryption (behavior depends on engine version)
• Engine version

When the user presses the "Create" menu option the dialog calls the OnCreateSdf message handler where all the action takes place. As you can see, this is a piece of ugly code. Most of it involves managing the DBPROPSET array and the contained DBPROP arrays, something that you will see in a large number of OLE DB method calls.

A DBPROPSET array contains an array of DBPROP arrays. Each DBPROP structure contains the value of a single OLE DB property (like file name, locale, encryption mode...) and these are grouped in property sets. Each property set has its own unique ID, like DBPROPSET_DBINIT (generic OLE DB database initialization properties) and DBPROPSET_SSCE_DBINIT (SQL CE-specific initialization properties). This scheme only works correctly if you put each property in its own specific set and you must read the documentation to learn which goes where. In our case, we have:

DBPROPSET_DBINIT:

• DBPROP_INIT_DATASOURCE - database file name
• DBPROP_INIT_LCID - database locale ID

DBPROPSET_SSCE_DBINIT:

• DBPROP_SSCE_ENCRYPTIONMODE - database encryption mode (for 3.5 only)
• DBPROP_SSCE2_ENCRYPTDATABASE - database encryption status (for 2.0 only)
• DBPROP_SSCE3_ENCRYPTDATABASE - database encryption status (for 3.0 only - deprecated in 3.5)
• DBPROP_SSCE2_DBPASSWORD - database password (for 2.0 only)
• DBPROP_SSCE3_DBPASSWORD - database password (for 3.0 / 3.5 only)

After filling up the property arrays, we can call the IDBDataSourceAdmin::CreateDataSource method and, if all properties are correctly set, we sould get a brand new SDF file.

At the end, don't forget to cleanup all the resources you have allocated. This includes the input properties and any output data. Also, don't forget to release the used OLE DB interfaces!

Before you make this code work on a Windows Mobile device, you must install at least one of the SQL Compact engines. For versions 3.0 and 3.5 make sure you also install the replication cab file because this is how the OLE DB provider is installed. Needless to say, you can have all three engines installed on the same device.

What's next? First, we need a better abstraction for handling all the property values. Second, we need a better way of retrieving error information whenever one pops up. I will start the OLE DB client library by implementing these two abstractions.

SQL Compact 3.5 SP1 and OLE DB

Yesterday the SP1 of SQL Compact 3.5 was announced with some interesting additions like 64 bit support. I have been using SQL CE since version 1.0 (since 2002?) and always from native code. My early applications were written with ADOCE which was later deprecated when the Pocket PC 2003 platform was introduced. The only alternative left for native applications was to use straight OLE DB interfaces. Interestingly, the ATL OLE DB Consumer Templates were available on the original Pocket PC 2003 SDK, but did not compile correctly. My early work with this code (two header files, essentially) was to adapt it in order to make it work with eVC3 and eVC4. Finally, Microsoft completely dropped these files from the Windows CE and Windows Mobile SDKs and this sent a very clear signal (or so I think): don't use this code.

What can we do now? Using the OLE DB interfaces directly is a big pain because these are very low-level abstractions. We need something that elevates the abstraction to something of the ATL OLE DB Consumer Templates level. And this is just what I'm proposing to do (yes, I'm crazy).

The Consumer Templates are a very interesting and informative piece of code. The usual ATL templated classes are there and the classes model very closely all sorts of consumer objects that you may use. But they have one very interesting drawback. As I found out, you have to tweak the code under some very specific circumstances, when you don't know the exact outcome of a command execution, for example. These are not insurmountable and reflect the purpose of the class library: use a class template instantiation that is customized for your exact needs. I will use some of the Consumer Templates' concepts (not the code) in this new library which will be prmarily targeted to embedded and mobile devices. As a side effect, it should also be able to work in a desktop application.

Given this scope, the primary target of this new library is to develop applications for all versions of SQL Compact (SQL CE), namely 3.5, 3.0 and 2.0. Instead of writing SQL Compact-specific code only, this will be built in two layers: a generic OLE DB layer and a SQL Compact-specific layer built on top of the OLE DB layer.

So what do you need to get started? First of all you need the SQL Compact header file. For SQL Compact 3.5 you need the sqlce_oledb.h, while for SQL Compact 3.0 and 3.1 you need ssceoledb.h. These files contain both the OLE DB Interface, structure and constant declarations as well as the SQL Compact-specific GUIDs and constants. This means that, without tweaking, you musy have a header file for each version of SQL Compact. Fortunately this is not required and we may use just one header file to compile an application that will consume all versions of the database engine - a very nice feature indeed.

On my next post I will provide an add-on header file that will allow you to use sqlce_oledb.h to target all SQL Compact versions.

ClearType on a memory DC

I recently developed a small information browser application for Windows CE 5.0 devices. This small application uses a touch list as an item selector and displays an in-memory bitmap with textual information related to the selected item.

The touch list uses a ClearType-rendered font (see the latest Touch List sample) painted to a memory DC. I also used a ClearType font to paint the text on the memory bitmap containing the detailed information about the main list item. Unfortunately the font was not being rendered in ClearType mode. Why was this? Maybe a Windows CE issue?

After quickly recompiling the code to target a Windows Mobile 6.0 device, I got exactly the same result. So I turned to the rendering code looking for a bug, but found none. Instead I found an interesting difference between the touch list rendering and the bitmap rendering.

If you look at the touch list samples, you will see that the memory DC is created from a CPaintDC. On the bitmap rendering code, I was using a memory DC created from a CClientDC (got the same result by using NULL as the HDC value).

So apparently you can only render ClearType fonts to a memory DC when this is created as compatible with the DC you get on BeginPaint... Has anyone else experienced this?

Windows Mobile API Usage Tool

Microsoft has just released a new tool to determine the Windowm Mobile API usage in your application. This is especially useful for deprecated functions! Go take a look here.

The Touch List - II

I finally have some time to come back and write about the Touch List window. It's been quite some time since I wrote the first post about this code, and it has changed a lot (especially because I had to use it for a customer project), and I have also learned a lot about how to make this code work on both Windows Mobile and Windows CE devices.

Code Changes
The first challenge I had to face was to decouple the "kinetics" from the list and implement it in a more general WTL base class. This was a project requirement for one of the data windows where a bitmap is displayed and the same auto scrolling behavior was sought.

The second challenge was to implement some sort of scroll bar into the window. I could use the native Windows scroll bars but I decided to use something nicer and possibly "cooler".

All the "touch" code was put into one single header file pompously named atltouch.h (see the sample code here). This header file contains the following classes:

• CScrollBarData - Contains scroll bar data (either horizontal or vertical) and associated calculations.
• CTouchWindow - The "touch" base class, where you will find all generic "kinetics" functions.
• CTouchListItem - Base class for touch list items. Implement one for your items (see sample code).
• CTouchList - The touch list class template.

Please understand that this code is still very much under construction so you will see some redundant concepts and the occasional bug.

Finger Use
Most of the current devices are not ready for finger input, only for stylus input. Sure you can use your finger but I have found that on some devices, when you put your index finger to the screen, the application window receives the expected WM_LBUTTONDOWN, but may also receive a lot of WM_MOUSEMOVE messages... Why is this? Your finger is way larger than the expected stylus tip so you are effectively touching more than one screen point. What I have experienced is that most devices will produce the mouse move messages and your list will start scrolling just by touching it with your finger. This is less likely to happen when using the stylus.

The solution for this was to implement a "sensitivity" factor that will ignore movements within a given range (see the SetSensitivity method). If you use this with any value larger than 1 (in either direction) your list will become less sensitive, but you will have to adjust this value to your device...

Scroll Bars
Scroll bars have an unusual implementation, but I hope you find them useful (if not, suggestions
are mostly welcome). To use the vertical scroll bar, just drag your finger (or stylus) up or down and the list will scroll accordingly.

On Windows Mobile 5 and 6 devices, these are implemented as a transparent layer on top of your list so you can use the full screen size to display data.

How to draw gradient buttons

A few years ago, I wrote an MFC PocketPC Numeric Key Pad control (a dialog, actually) to help with numeric input on the Pocket PC. Recently I had to convert this code to WTL for a Windows CE 5.0 application and I decided to try and give it a modern look (see picture). This was something I was keen on doing because it was a bit of a mystery to me how that effect was achieved.

It's no big deal after all. Each button is painted with two gradients of gray and the digits are painted with a ClearType-rendered bold Tahoma font. The code to create the button effect (what do you call it: chiseled?) is very simple:

TRIVERTEX vertex[4] = { 0 };
GRADIENT_RECT grRect[2] = { 0 };
int nHeight2 = m_rc.Height() / 2;

This declares the variables we need to draw the button, where m_rc contains the button rectangle coordinates. Now we need to give the gradient renderer the information on how to render the button. Each button is split into two gradient-filled rectangles that must be described by their top left and bottom right points and color information. The top rectangle is:

vertex[0].x = m_rc.left;
vertex[0].y = m_rc.top;
vertex[0].Red = 0x0000;
vertex[0].Green = 0x0000;
vertex[0].Blue = 0xb000;

vertex[1].x = m_rc.right;
vertex[1].y = m_rc.top + nHeight2;
vertex[1].Red = 0x0000;
vertex[1].Green = 0x0000;
vertex[1].Blue = 0x7000;
vertex[1].Alpha = 0x0000;

Note how the color components are encoded. Now, we describe the bottom rectangle:

vertex[2].x = m_rc.left;
vertex[2].y = m_rc.top + nHeight2;
vertex[2].Red = 0x0000;
vertex[2].Green = 0x0000;
vertex[2].Blue = 0x2000;
vertex[2].Alpha = 0x0000;

vertex[3].x = m_rc.right;
vertex[3].y = m_rc.bottom;
vertex[3].Red = 0x0000;
vertex[3].Green = 0x0000;
vertex[3].Blue = 0x7000;
vertex[3].Alpha = 0x0000;

Finally, we must group these together for the API call:

grRect[0].UpperLeft = 0;
grRect[0].LowerRight = 1;
grRect[1].UpperLeft = 2;
grRect[1].LowerRight = 3;

GradientFill(dc, vertex, 4, (PVOID)grRect, 2, GRADIENT_FILL_RECT_V);

You may want to try different colors and shades to fit your taste.

HRESULT Values

Here's a great place to find all the HRESULT Values.

Installing WTL Helper in VS 2008

Last April my good friend Cristiano Severini managed to recompile Sergey Solozhentsev's WTL Helper for VS 2008. He wrote a few instructions about how to do it, but they are a bit incomplete. I have just reviewed the whole process with him and managed to successfully install his version of the WTL Helper DLL on a VS 2008 under Vista. Here's how to do it:

• Install the original WTL Helper package;
• Download WtlHelper9.dll and copy it to the same install directory;
• Download WtlHelper9.reg and import it into your own registry using Regedit (you may have to edit the path);
• Open a command line (use Admin rights under Vista), go to the install directory and run the regsvr32 WtlHelper9.dll command;
• Start VS 2008 - the WTL helper must be there.

Enjoy!

The Touch List

This has been a busy month for me, as the number of posts clearly shows. After publishing the article on animating WTL child view transitions on Code Project, I decided to investigate how to implement a "touch" list on Windows Mobile. This kind of controls are all the rage right now, thanks to devices such as the Apple iPhone, or the HTC Touch Diamond. Many other device manufacturers are also implementing their own versions of these lists because they are easy to use and are "finger-friendly". Nowadays the trend seems to be towards using your own fingers and not the stylus.

So what does a "touch" list have to do? I would say it has to:

• Scroll smoothly;
• Follow your finger when you drag, scrolling the list;
• Automatically scroll the list when you finish dragging it.

Let's look at each one of these requirements.

Smooth Scrolling
The native Windows CE list controls don't scroll very smoothly. In fact, they usually scroll taking the item height as the scrolling quantum. On some ocasions, you will even see a full list refresh when you request a new page. This is not an acceptable behavior for a touch list.

A touch list scrolls one pixel at a time if needed be, clipping the items that are partially visible.

Drag Scrolling
This feature is also not implemented in none of the native list controls. If you click an item and drag the stylus, the list will not scroll to follow it. The underlying item will be selected, or some other custom behavior will occur, but the list will not scroll.

A touch list follows your finger or stylus and scrolls to keep the selected item in sync.

Autoscrolling
This is the coolest feature of a touch list that makes it behave like a flywheel: you nudge it and it keeps turning until the spindle friction dissipates all energy and the wheel stops. I have yet to see if the standard implementations support some kind of list "inertia" where successive quick drags accelerate the scrolling speed. Of course, this is not implemented in any of the native lists.

My Proposal
After considering all of the above, I wrote a sample implementation of a touch list (download here). The sample displays your contacts list (the FileAs property) and allows you to scroll the list up and down. It follows your finger and autoscrolls if you drag it quickly.

The implementation is far from complete and you will see lots of things changing in the next few days as I implement item states, mutiple columns, horizontal scrolling and some other cool features. Ah, this is WTL 8.0, of course!

Animating Child View Transitions - The article

I have just posted a new article on Code Project with the last version the the child view transition animation code. You can read the article here.

An alternative to the SHMENUBAR resource

The SHMENUBAR resource is a pain. A real pain. Why is there no support for this on the VS 2005 / 2008 resource editor? This resource is actually quite powerful when it comes to defining the WM application menu and fine-tuning its appearance. But when you need to change something, you need to remember the string table IDs, the toolbar button options and all that.

Most WM applications I have seen so far use a popup menu attached to each of the menu bar buttons. If this is your case, you can forget about the SHMENUBAR resource and use the menu editor. Create a menu with two items and their popups on the resource editor. If you are using WTL, go to your CMainFrame::OnCreate method and instead of the simple

CreateSimpleCEMenuBar();

write:

CreateSimpleCEMenuBar(IDR_MAINFRAME, SHCMBF_HMENU);

The IDR_MAINFRAME is your main menu ID. And that's all you need.

If you are using straight API calls, when calling the SHCreateMenuBar API, make sure that the dwFlags member of the SHMENUBARINFO contains the SHCMBF_HMENU flag and that nToolbarId contains your menu ID.

The tamed tree view

On my last post I described a strange behavior of the tree view control when it is used as a WTL child view and when it is scrolled into view using the SlideView sample application that I have been developing to illustrate child view animations. The solution for this problem was, as I suspected, to host the tree view control in another window and making this window the frame's child view. All animations work correctly now when the tree view control is showed up.

In this latest version of the sample, I extended the use of the CSelectionBar control (some call it the header bar control) in order to select which animation to perform (using the left menu). Now the code also supports switching views without any animation (select 'None').

This control will be updated to allow userd to display arbitrary windows instead of a standard popup menu and also will allow the inclusion of other controls in the toolbar (like an edit control or a combobox).

The misbehaved tree view

Following up on the WTL child view animation code I have been working on, I have noticed that when the tree view child window is selected into view (see the sample application), the animation never works as expected. In fact, the tree does not seem to correctly repaint while it is being moved, contrary to the form and list view. After selecting the tree view it also fails to correctly repaint when you open another application and return to the sample (use the Memory applet).

This problem became more apparent when I decided to adapt a very old piece of code to WTL 8.0: the selection tool bar (what you see on top of the PPC File Explorer). Being a toolbar, it must coexist with the child view within the same frame, so the scrolling code had to be adapted.

I will post the solution for this problem when I find it. Most likely I will have to host the tree in another window...

Animating Child View Transitions - II

Now that I undestood how to animate child views in a Windows Mobile WTL 8.0 applicaction (see Animating Child View Transitions), it's time to remove all the animation code from the sample's main frame and move it into specialized classes. Child view animations are performed by the CChildViewAnimate mix-in class (see the sample project). To use it, you merely add it to your main frame's inheritance list:

class CMainFrame :
  public CFrameWindowImpl<CMainFrame>,
  public CUpdateUI<CMainFrame>,
  public CChildViewAnimate<CMainFrame>,
  public CMessageFilter,
  public CIdleHandler

Next, change the main frame's PreTranslateMessage method to this:

if(CFrameWindowImpl<CMainFrame>::PreTranslateMessage(pMsg))
    return TRUE;

return ChildPreTranslateMessage(pMsg);

The call to ChildPreTranslateMessage ensures that the active child view class has a chance to do some custom message translation work. To enable this, you must derive all you child view classes from CChildView, like this:

class CSlideFormView :
  public CDialogImpl<CSlideFormView>,
  public CChildView<CSlideFormView>

If you need to do custom message translation, you must override the PreTranslateMessage:

virtual BOOL PreTranslateMessage(MSG* pMsg)
{
    if(!::IsWindow(m_hWnd))
        return FALSE;
    return CWindow::IsDialogMessage(pMsg);
}

Note the IsWindow test: it is there to avoid having this method called when the child window is already destroyed.

ATL ASSERT on exit

If you tried the sample code I posted yesterday, you might have seen an elusive ATLASSERT dialog box (in Debug mode, of course). When the assertion fires, it happens when the application is closing and the dialog box is very briefly displayed and then closes. You never get to see what caused the assertion in the first place nor do you get a chance to break into the debugger.

To handle this situation, I changed the ATLASSERT macro definition the following way:

extern void AssertBreakpoint();
#define ATLASSERT(expr) { if(!(expr)) AssertBreakpoint(); \
_ASSERTE(expr); }

When the assertion is triggered, the code first calls the AssertBreakpoint function that is there just for you to place a debugger breakpoint. I implemented it with the following rocket-science code:

void AssertBreakpoint() { int a = 3; }

Now you can run the application and you will be sure that you know where the assertion fired. Why? When the debugger stops on your breakpoint you can immediately check the stack and look at the offending code. In my case the assertion was being fired in the CSlideFormView::PreTranslateMessage method. The form view HWND was already destroyed when the

return CWindow::IsDialogMessage(pMsg);

line was executed and this function validates m_hWnd in debug mode.

So if you find yourself in such a situation, use this technique because by defining your own version of ATLASSERT you essentially "infect" all the ATL and WTL code with your custom implementation. All your ATLASSERT are belong to me.

Animating Child View Transitions

On my last post I promised to start writing the OLE DB client code to replace the ATL Consumer Templates on mobile and embedded devices. Whenever I write a library, I always look for an interesting application to use the library. In this case there is an obvious candidate: a replacement for the SQL Compact "Query Analyzer".

Instead of writing a dull imitation of the existing tabbed interface, I looked for ways to make a WTL SDI application dynamically change the frame's child view and there was no shortage of ideas, especially on CodeProject: Switch Views in a WTL SDI Application. The code works very well, but there is no provision for animating the child view transition. So I decided to do a little research into this issue and produced an early prototype.

The sample application flips between two views (Form and List) by horizontally scrolling the changing views. The idea is that the form view is on the left and the list view is on the right so when you switch views they animate intuitively: the form hides and the list shows by scrolling to the left and the reverse happens by scrolling to the right. This concept can be extended to vertical scrolling so the user can associate the process of navigating through the application with a map.

There were some challenges I had to overcome to implement this. To make animations smoother, I thought about rendering both views (the old that is going out and the new that is coming in) in a big bitmap and then just paint the bitmap in the frame's client area. Interestingly, there is no documented way to render a hidden window to a memory DC so there was no way to render the new view in the bitmap. You can overcome this if you have full control of the new window's paint cycle, but if you use Windows controls like the list view you are out of luck. I even tried forcing a WM_PAINT message with the memory HDC on the WPARAM, but the list would not render the header... So now I only render to a bitmap the old view and scroll in the new view using MoveWindow.

Another issue I had to take care of is the scrolling speed. The speed of ScrollDC increases as the area to scroll diminishes, so you get a scrolling procedure that gets faster and faster. To make it work at the same apparent speed, I had to time the first scroll and then use the system timer to compensate for faster execution times when the scrolled bitmap gets smaller.

I still need to implement the vertical scrolling and to encapsulate all of the code in a mixin class for the WTL main frame. Stay tuned.

SQL Compact Command Parameter Sizes

This is not a native issue, but one of the things I learned from the SQL Compact sessions from the last MVP Summit (and I think is not under my NDA, so I can talk about it) is the fact that you can speed up your SqlCeCommand (yes, I'm talking managed code here) by explicitly setting the parameter sizes. This was a bit of a surprise for me because with OLE DB you have no such niceties - when you create a command parameter, you must correctly specify the command parameter size, otherwise you just may see your accessor writing in very strange places...

Coming up: I'm starting to write the native alternative to the ATL OLE DB Consumer Templates for devices. This is going to be a lot of fun...

MFC: The new Phoenix?

So if you think MFC was dead, think again.

Jessica Liu posted on the Visual C++ Team Blog about the new MFC samples for everything from Office 2007 applications to Visual Studio-like user interfaces. So, is native code dead?

Nope.

Windows XP SP3

If you are an MSDN subscriber, go get XP SP3. It's now available on MSDN subscriptions!

Off to Seattle!

Tomorrow I'm off in a crazy trip to Seattle for the 2008 MVP Summit. If you want to meet me, I'll (try to) be here:



Hopefully, I will try to blog about what I can disclose about the Summit and any info I get there.

WTL Helper and VS 2008

My good friend Cristiano Severini has figured out a way to adapt Sergey Solezhentsev's WTL Helper to VS 2008 with a little help from me (I helped him match the VS 2005 typelibs to 2008 because I have both installed side by side). Here's how he did it: WTL Helper and VS 2008.

Crino ROCKS!

Smartphone keyboard handling with GAPI

Here's the scenario: you are working on a DirectDraw full screen application and you require full control of the device keyboard. Things have worked nicely with the Pocket PC and even with the WM5 Smartphone, but when you move to WM6 your keyboard is gone. You do get the WM_KEYDOWN and WM_KEYUP messages, but most of them only report the dreaded VK_PROCESSKEY and ImmGetVirtualKey() does not seem to help at all. What can you do?

I found a solution for this problem that solves most of the issues above, but also uncovers some other issues. The solution is to call ImmAssociateContext() on your main window passing NULL as the HIMC parameter. This will remove the input context association from the window, and you get the keyboard back. But...

... now you have to handle the keyboard mappings yourself. That's the price you pay. On some devices this means having a mixed keyboard (numbers and alpha keys) so you must internally implement a keyboard map in order to make it work as the user expects. To make matters worse, the keyboard map is device dependent so you are in for a bit of work. Finally, I have found that on some devices you don't get all the keys, especially the special keys for mail and contacts. These devices seem to have these keys wired to launch the matching applications and not even GAPI seems to release them...

Smartphone display timeout

How do you prevent a Smartphone (sorry, a Windows Mobile 6 Standard) device from falling asleep? I tried everything, including sending null keys:

keybd_event(VK_NONAME, 0, KEYEVENTF_SILENT, 0);

This nasty little trick works but also forces your backlight to the maximum and you may not want it. What I really want to do is mimick going to the Power Management applet and setting the "Display time out" to "Never". With this setting, your Smartphone device will never go to sleep. So how do you achieve this? It's actually quite easy.

The display time out value is stored on the device registry under:

HKEY_CURRENT_USER\ControlPanel\Power

All you have to do is set the Display value (DWORD) to -1 and make the system know that the value changed. How do you break the news? Simple again:

PostMessage(HWND_BROADCAST, WM_WININICHANGE, 0, 0);

If you use this in your application to avoid the device from falling asleep (I also call SystemIdleTimerReset and SHIdleTimerReset every few seconds), please remember to set the value back to the original value.

Insomnia rules!

Windows Mobile Remote Control - Published

You can now get the latest version of the Windows Mobile Remote Controller application here. This code adds compression to the screen data transport so it becomes a bit faster. Enjoy!

My first hours with VS 2008

My first few hours of work with VS 2008 have been a pure joy. This new beast performs like a champ and has no apparent quarrels with Vista. Upgrading the Data Port Console solution to the 2008 format was a breeze and I was even rewarded with a C# compilation error that had gone unnoticed in VS 2005. The whole product feels much more robust and reliable under Vista and the visuals are also nicer. So far, it has been a great experience. Highly recommended!

Don't worry - if I have anything to gripe about, you will know it.

Installing VS 2008

I finally decided to install VS 2008 this morning. Why so late? I have been very busy and my customers use mainly VS 2005 so I had no urgency to install VS 2008. Now that it has been officially launched I will surely get more and more requests to migrate the products to .NET 3.5 so VS 2008 is a requirement.

Installation went as smothly as possible except for a mysterious shutdown at the end. I was not present, so I cannot tell you what happened but I suspect that the Vodafone 3G card is playing tricks on me. Again.

After installing, I looked at the C drive to check for free space and was appalled to see that it was below 50%. After installing Vista SP1, my C drive used space magically shrunk and has been growing very fast ever since. The culprit? System restore points! After clearing them I got back to post SP1 install conditions. Cool, my disk is back.

Raffael's Blog

Now, here is an ultra-cool blog: Mobile Development! Thank you Chris for the heads-up.

WMDC woes

It's almost a year now since I started using Vista Ultimate. The new OS came on the ASUS VX2 I bought last year and I have found that the overall experience has been very positive. There are a few minor snags like the apparent sluggishness in the couple of minutes after Windows boots and WMDC. This is my major gripe with Vista - I never know when one of my WM5 or WM6 devices will connect or not. One of the quick and dirty solutions I have found is to uninstall the device network adapter, disconnect the device and reconnect. To do this, go to the Control Panel, open the System applet and select the Device Manager link. Under the "Network adapters" tree node, locate the Windows Mobile device and uninstall it. Remove the device and connect it again. WMDC should recognize the device again.

There is an alternative that I have not tested yet and was pointed to me by Mark Arteaga on the PocketPC FAQ site. Anyway this should be an easier experience, especially for people like me who need to connect to everything, from Pocket PC 2003 devices to the latest WM6 devices throwing in a few bizzare Windows CE 5 GPS devices in between...

Windows Mobile Remote Control

Here is the first version of the Windows Mobile Remote Control application. It basically extends the concepts I developed for the Code Project article "A Remote Windows Mobile Screen Grabber". Instead of a blocking RAPI call, this project uses a streamed call so that a live connection persists between the device and the desktop. Mouse and keyboard events are now sent over to the device so you can remotely control your device from the desktop. Please note that this is still a bit crude and will fail under some circumstances, like when you launch a Direct Draw application on the device. For regular GDI applications, it should work just fine (if you can live with 4-5 fps, that is).

Now I must add some way to send the special device keys, like the phone and left / right selection buttons.

Windows Mobile Developer Center

MSDN has revamped the Windows Mobile Developer Center. Go take a look (yes, you will find native code there). I am especially interested in the DirectX samples.

On another note, I will be working throughout this weekend on a Windows Mobile remote control server (something like Pocket Controller). After writing the screen grabber tool, this is an extension I really want to have a go at. Will post the results here as they become available.

Remote Screen Grabber

As promised: A Remote Windows Mobile Screen Grabber

One of the things I tried out during the development of this code to send mouse events to the device. When the user clicked the desktop screen with the device screen capture, a mouse message would be sent to the device via a different remote call. Although a bit crude, it was possible to remotely control the device (with some notable exceptions...). So this set my mind in motion (again) and now I'm writing a streamed version of the RAPI DLL so that it accepts commands from the desktop (such as mouse messages and even keyboard presses) and returns device screens to the desktop. I will post the code when it is ready (and working).

Interview

If you are interested to know a little bit more abut me, take a look at Christopher Fairbairn's blog. Thank you for this, Chris!

Windows Mobile Screen Capture

While waiting for Vista SP1 to hit the MSDN downloads section, I decided to investigate a little bit on an issue that has caught my curiosity time and again: remote capturing of the device screen. There are lots of products on the market that will do this for you, but the question of how it is done kept my curiosity alive.

Searching the net for source code samples turned nothing (blame Live and Google). The best approach I got was from a CodeProject article but the proposed GDI method does not work on Windows Mobile because the CAPTUREBLT flag is not supported. This means that I needed another solution.

GAPI seemed to be a very good candidate. It has a very simple API and I had used it in the past, but it is now declared deprecated by Microsoft and I was prompted to use Direct Draw instead. Willing to learn a bit more about this technology, I decided to give it a go and spent a couple of hours reading through MSDN docs to figure out how this can be done. As it turned out, it's very simple.

To capture the device screen using Direct Draw you need to follow these simple steps (I will write an article on this illustrating a desktop tool to capture the device screen via RAPI):

1. Initialize Direct Draw using the DirectDrawCreate function using NULL as the first parameter.
2. Set the cooperative level to DDSCL_NORMAL. This is enough for accessing the primary surface in read-only mode.
3. Create the primary surface using this code.
4. Get the surface's HDC and use it to create a compatible DC.
5. Create a compatible bitmap using the screen dimensions and select it to the compatible DC.
6. Blit from the surface DC to the compatible DC using BitBlt.
7. Done! The compatible bitmap now contains the device's screen capture.

I wrote this using a DIBSection in order to marshal it via a RAPI call and was able to deserialize it on the desktop and display the device screen capture. I'm planning to use this same marshalling technique to implement a remote file browser using the device's own system image list.

Stay tuned for the article!

Vista SP1 around the corner

According to this, Vista SP1 goes RTM today. I really want to install the SP1 and check the improvements it has in store for me. I have to tell you that Vista was a bit of a let-down when it refused to run eVC3 and eVC4 (I don't care about VS 2003). Microsoft's Virtual PC was also disappointing because it does not support USB, so I had to resort to a third-party VM (Parallels) to get a USB 1.1 emulation. Note that I do not expect that SP1 will add support for the eVC compilers, but I do expect to see some improvements in overall speed and especially when getting out of sleep mode. My Asus VX2 does not always wake up correctly and some drivers may not start correctly. The worst situation happened when the temperature control died after sleeping and the system overheated. I knew what happened because the whole thing shut down abruptly. So I'm really interested in seeing some improvements.

Remote File Viewer

A few days ago a Pocket PC Developer Network forum reader asked for a RAPI remote file viewer (like the one that ships with Visual Studio). I know that OpenNETCF has a free library to support this, but I decided to reinvent the wheel just for the fun of it. The (still crude) result is here. There are a few interesting points to note (especially for the beginner) like how the returned array of CE_FILE_DATA structures is read into a list. Adding file upload and download features should be a snap, now.

WTL on MSDN Magazine

This helps explain why I'm a WTL addict. Shame you don't see articles like these for Windows Mobile...

The collapsible toolbar

Here is an interesting challenge: implement the behavior of this collapsible toolbar. I'm wondering if this can be done by reusing the shell's toolbar control or if the whole thing needs a custom implementation. I will investigate an post back my findings.

After a bit of a rush trying to amend my own lack of testing before publishing new product releases (I should know better at 42...) I will also start looking at what it takes to implement the "Consumer Templates-less" OLE DB library for Windows CE (and Win32, why not?).

DBPROP_SSCE_TEMPFILE_MAX_SIZE

Never set this property to zero - SQL Compact Edition will not open your database... Yes, I just got my hands really dirty with this one.

New year, new life

This is what we say in Portugal: "Ano novo, vida nova" - "New year, new life" meaning that the turn of the year is a great time to ponder about last year and make the changes for the new one. Focusing deeper on less things is definitely something I will do this year. I just created a new blog for all the issues that pertain to my work at Primeworks and will keep this for all the other stuff, especially the native code stuff.

The community project that I chose to focus on is to create a replacement for the ATL OLE DB Consumer Templates. More and more I see native code developers looking for an easier way to access SQL Compact Edition databases from C++, and my solution for tweaking the PPC 2003 SDK header files does not feel right. Besides, Microsoft does not ship the Consumer Templates header files anymore so this must mean that this code is not supported (but is it licensed?). Developing a new library for OLE DB on devices will surely be an interesting challenge and will keep me busy writing code and publishing articles throughout 2008.

Once again, thank you Microsoft for the fifth MVP award!

Detecting the SDF file version

SDF files don't have an explicit way of letting you know their versions. What can you do to tell the version of a given SDF file? In my products I detect the SDF file version as a side effect of opening it: if a given engine can open that SDF file then I can safely assume it has the same version as the engine. This is not a very straightforward approach because it requires that you have all engines installed on the device (not a typical scenario).

The alternative way to detect the SDF file version is by reading a few signature bytes on the file. Just open the file, set the read location at offset 16 and read the 32 bit int there. The following values will tell you the file version:
2.0: 0x73616261
3.0: 0x002dd714
3.5: 0x00357b9d

I wrote a very simple .NET CF 2.0 application to illustrate this. The download link is this. If you find any SDF file that fails this test, please let me know.

WM6 GPSID problem workaround

John Spaith from Microsoft has blogged about a workaround solution to the HTC GPS problems on WM6. Apparently the size of the GPS_POSITION structure must be tweaked in order for the call to GPSGetPosition to work, but this can only be applied on devices where the first call returns 87... I actually worked around this by falling back to the COM port.

COM Port Enumeration

If you want to enumerate the available COM ports on your WM device, there is a very simple solution: look under the HKEY_LOCAL_MACHINE\Drivers\Active for all keys that have the Name value starting with the "COM" string. With some luck, you can even find a FriendlyName value under the key specified by the Key value.

The only problem with this very simple method is that it does not enumerate the Microsoft Bluetooth stack emulated COM ports. If you create a partnership with a device that has the serial port profile, you will be able to assign a free COM port to it. This COM port works like any regular serial port but does not show up in the list of active drivers on the registry. Microsoft decided to enumerate all Bluetooth virtual COM ports under HKEY_LOCAL_MACHINE\
Software\Microsoft\Bluetooth\Serial\Ports. To find all available COM ports you must enumerate all the child keys and look for the Port value under each key.

Interestingly, the Widcomm stack does put the virtual COM ports where they should be (or at least where I expected them to be).

VS2008 Install Heads Up

Before you install Visual studio 2008, make sure you read the readme file. There are two important implications for native and mobile developers: SQL CE 3.5 must be uninstalled prior to installinv VS2008 and the Windows Mobile 6 SDK will have to be uninstalled and reinstalled after installing VS2008.

VS2008 is out!

Check out the subscriber downloads in MSDN!

Windows Live

Off-topic: Today I noticed that for the first time in the last few years Google is not on the top of the list of search engine referrals on my comany's site web log data. Out of the blue (or maybe not) Windows Live is leading the searches! This is intersting news for a chart that used to show a huge Google bar and essentially nothing else.

Performance Improvements in VS 2005

Jim Springfield has just posted in the Visual C++ Team Blog a very interesting piece of information: there is a new performance improvement QFE for Visual Studio 2005. This download addresses a number of performance issues and its features will roll over to VS 2008. I'm getting it right now to give it a spin.

Carousel Demo

Now that the carousel code is working correctly across a range of devices, I decided to upload it as a preview (download here). The code illustrates the carousel animation and user control as a strip on the child window of the WTL main frame. I'm still wondering how this "control" should be implemented, either as a "widget" on the main frame or as a toolbar-like control.

The main frame approach would implement the carousel code in the main frame allowing it to correctly resize the child window and even allowing the application to change the child view (like in a property sheet). With this approach it might also be easier to implement a means to change the location of the bar whan the screen is rotated. In fact, the carousel looks much nicer in a vertical position when you rotate your screen to landscape (right or left would be up to you).

Implementing this as a toolbar control would have the advantage of allowing the developer to reuse the code outside the main window, especially on dialogs and modified property sheets...

While I think about what to do, please do take a look at the code. The icons were "created" with Axialis IconWorkshop, one of the best icon editors around. Please note that these icons can only be used by registered users of the product and I'm not sure that you can reuse them on your applications.

GradientFill and a memory DC with a viewport

I’m working on a “carousel” control similar to the one you can find in the Yahoo Go! 2.0 application. This thing showed up in my HTC S620 WM6 upgrade and the control is very cool, so I decided to write my own implementation of it in C++ using WTL 8.0. The scrolling animation is achieved by painting to a memory DC and then blitting it to the screen. All went smoothly until I decided that the carousel background would look so much nicer with a gradient, making the center item stand out in a lighter background.

WTL 8.0 does a lot for you and I found myself having more work creating the icons for the sample than actually implementing the code... I started by using the CMemoryDC class which worked beautifully until I used GradientFill. All other GDI functions work correctly, but the gradient refused to appear correctly or at all. Interestingly, when the carousel strip was set at the top of the client window it would paint correctly, but would fail at the bottom.

Looking at CMemoryDC's source code I found that the constructor sets a viewport origin in order to cope with any rectangle. When setting the carousel strip at the top, the gradient would show up because the viewport origin would match the child window's (0,0), but fail when set to the bottom because the viewport origin would be different.

I ended up by using a slightly modified version of CMemoryDC, by removing the SetViewportOrg call on the constructor and zeroing the source bitmap coordinates at the destructor's BitBlt call. The carousel now works correctly on all four window edges (top, bottom, left and right).

Presently the carousel is implemented as a child view window but I'm looking at ways to make it work as a toolbar.

HTC P3300 GPS problems (again)

After posting in January about the failed implementation of COM events on the GPS port of the HTC P3300, I have another sad story to tell. After upgrading the device to Windows Mobile 6, the GPSGetPosition function always returns 87 (ERROR_INVALID_PARAMETER). There seems to be no appropriate set of parameters that this thing will take, you will always get back an error 87. Great! The GPSID on the P3300 is useless. The solution is to fall back to the COM port code...

Access 2007 woes

I don't have any complaints about using the Access 2007 application whatsoever. After the not so brief period that took me to learn the new user interface (some stuff on the older UI was quicker to use), the experience is quite smooth. Not so on the nuts-and-bolts side of the question.

As you already know I'm an OLEDB freak, and I use it for all native code database access I write both on the device and on the desktop. Last year I did write a sizeable amount of code for my data synchrnization component (Data Port Sync) and this targeted the JET 4.0 database engine. The code I wrote implemented both the Access database preparation and also change tracking. Database preparation involves creating a few extra tables and adding two columns to every tracked table. This looks pretty simple to do using straight SQL, but when I consistently failed to create self generating "replication id" columns (the SQL equivalent of the uniqueidentifier type with the rowguidcol property) I turned to the low-level
ITableDefinitionWithConstraints OLEDB interface to do all the table creation work. The code is not nice to see, but it is effective and gave me an added bonus: I can hide the tables from the user. To add the extra columns to the tracked tables, I used the
ITableDefinition interface.

Recently I started to get requests from my customers to support the new Access 2007 file format on my data transfer products. Changing the provider from Microsoft.Jet.OLEDB.4.0 to Microsoft.ACE.OLEDB.12.0 did the trick quite nicely because Microsoft kept all the other connection string properties untouched. BTW: where can I find a reference to all the properties and most importantly, where are the new header files for OLEDB?

The first implementation of the Access 2007 database engine was shipped on the Data Port Console release. Although it does not create the mdb or accdb files (yet), it successfully imports and exports data to and from the various SDF formats.

Going back to the desktop synchronization code (which will eventually see its way into the Console) I recently began to write a small desktop agent for simple desktop scenarios. Putting the new code to work was quite easy until I decided to test the code against an Access 2007 database. I picked up the old Northwind sample and converted it to the new engine only to find that my preparation code would fail miserably. There were no issues when the tracking columns were created, but it proved next to impossible to create the tracking tables: the engine would throw an E_FAIL error with a very strange OLEDB error description: "Could not find field." Period. I changed every possible factor that could be causing this: column names, the table name, absence or presence of delimiter brackets. Nothing would work. Out of desperation, I replaced the code with a straight SQL DDL command only to be greeted with a similar error: "Could not find field 'SID'." I beg your pardon? There is no 'SID' in my command! Has Access gone mad?

No, Access is working perfectly but the accdb file seems to be corrupt. After reverting all the code back to what it was, I decided to create a new accdb from scratch and then import new data from an SDF using the Console. After running it through the preparation code there was no error! Is the database conversion code broken in Access 2007?

Bottom line: don't convert mdb to accdb. Create a new accdb and then import the old data into it. Sigh.

The edit control under Windows Mobile 2003 and 5.0

After publishing my last article, I found out an interesting thing about the edit control on a Windows Mobile 2003 Pocket PC device: the up and down arrow keys behave differently from the edit control in Windows Mobile 5. In WM5, the single line edit control responds to the up and down navigation keys by setting the focus to the parent window. When this happens, the parent window receives the corresponding WM_KEYUP messages and thus is able to set up a navigation mechanism. Not so with the Windows Mobile 2003 edit control. This version of the control seems to "eat" these messages so the parent has no clue about the requested user navigation. This became quite apparent on my implementation of the CFormListCtrl: the navigation would not work.

This problem was quickly solved by adding the OnKeyDown message handler to the CCeEdit class and forcing the control to set the focus to the parent window whenever an up or down navigation keys are detected. Voilá!

A list-based form for Windows Mobile

I have just published an article on CodeProject that better describes the list-based form control. As you can see from the article, I added a few other bells and whistles to the code.

Next two projects: port this thing to WTL and start writing an OLE DB class library for Windows Mobile. This second project is necessary because Microsoft no longer ships the atloledbcli.h file on the SDKs and this must mean something...

Bug in the FormListSample

There is a bug in the FormListSample that causes a stack fault exception if you close the dialog with a created item control (edit, combo or date time). I'm looking at this right now.

Solution: On the CFormListCtrl::OnDestroy method, set the m_iEdit member variable to -1 after calling ShowEditor.

Meanwhile, I added some code to handle edit control resizing when the screen is rotated. The code can be found on the same location.

The list-based form control

Back in 2003 when the Smart Device Extensions were still in Beta and about to be renamed .NET Compact Framework, some people had to use C++ to develop database applications. I was one of these developers struggling with a crude set of tools: ADOCE for database access and the eVC3 dialog editor to design the user interfaces.

Soon we learned that ADOCE would be deprecated and so I started to look at the ATL OLE DB consumer templates. This is a great technology and I still fail to understand why it is not present on the WM5 and WM6 SDKs. In my code, I use an adapted version of the header file you get on the Pocket PC 2003 SDK but to be honest I don't like this approach (I will come back to this issue in a future post).

On the user interface side, we already had a very nice alternative to dialogs: the list-based form control used by the Pocket Outlook Contacts application. Not only this is an intuitive control for the user, it also promised to be easy for a developer to use: no designer needed to be involved, you just specified the list of items to edit and fired the object.

The big problem with this control was that there was no API for it so I had to write one. Back then I was using MFC big time due to the compiler support for it on eVC3 and eVC4. Now I am not so sure that MFC is a good option for native user interfaces and I'm leaning towards WTL. You now have great tool support in VS 2005 with the WTL Helper tool which I am using on a daily basis and with great results.

So I eventually wrote an MFC control to implement a list-based form and in the process I threw in couple of goodies such as grouping, context menus for managing the items, control notifications and a few other bells and whistles. After a recent request of fellow MVP Christian Forsberg for a similar control, I decided to go back to my 4 year old code, adapt it to VS 2005 and publish it. I had to do more work than expected because nowadays there are lots of devices that have a keyboard, display in landscape and even have VGA screens! Back then I was used to portrait QVGA screens and no keyboard, so some changes were in order.

Before I publish this code as an article, I decided to preview it here so you get a chance to look at it and let me know what you think. Download FormListSample.zip

Supporting SQL Compact 3.5

Native code is so cool! As of today I can write applications that target SQL CE 2.0, 3.0 and the new 3.5 Beta 2. The trick? Extend this technique to the SQL Compact 3.5 header files. Microsoft made our life really easy by making very small changes to the header file, so you only need to copy the definition of CLSID_SQLSERVERCE_3_5 and all the property definitions of
DBPROP_SSCE_DBENCRYPTIONMODE. Now your header file can be used for 2.0, 3.0 and 3.5!

What this means for me is that SQL Compact 3.5 will be supported by the release version of Data Port Console and I will also issue a last version of DesktopSqlCe that will also support the new database engine. This will be the only game in town for .NET 2.0 desktop applications that must support both the 3.0 and the 3.5 database formats (without code changes, of course).

Cool!

SQL Compact 3.5 Beta 2 and VS2005

Ready to move forward and add SQL Compact 3.5 support to my products, I downloaded the Beta 2 bits in the hope to find the required header files. Unfortunately they were nowhere to be found. Does that mean that you cannot use VS2005 with SQL Compact 3.5 Beta 2? No. Fortunately there is an indirect way to get the much-needed header files: download and install either Orcas Beta 2 (not yet for me) or Microsoft Visual C# 2008 Express Edition Beta 2. Apparently the header files are also available on the Visual Basic edition.

Now back to work.

Database Diagrams for Data Port Console

The second release candidate of Data Port Console 1.0 is almost out and while I was removing some bugs, I failed to resist the urge to bring back some code that had been sleeping for amost one year: database diagrams. Yes, coding should also be fun so I decided to add this to the 1.0 and in this post I'm offering my readers the chance to try this new feature. You can create multiple diagrams per database and now all information is persisted as an XML file instead of using the registry. You will also notice that there is a new folder for tables and that you can group them by their first letter. Please let me know what you think about these new features while RC2 is not uploaded to the site. Grab your copy of this interim build here.

ZIP and RAPI - The article

My latest musings on this subject have been published on CodeProject. Read the article here.

ZIP and RAPI - Yes, it's faster

I really was not pleased with the performance results I got from the RAPI Unzip extension I published in my last post. My reasoning was that if you send a compressed data stream to a device and decompress it you should get a better performance than decompressing the data on the PC and sending it to the device. The results I got were not very satisfying: the RAPI extension DLL was not clearly faster than the PC decompression, especially for ZIP files with small compression rates.

So I decided to take a harder look at the code and found the culprit: WriteFile latency. The function was being called way too often and sometimes with small sets of data. If you look at the code, you will see that WriteFile is called when either the decompression input or output buffers are emptied. This is wrong. The call must only be made when the output buffer is empty and not when the input buffer is. When the input buffer is emptied it must be refilled from the RAPI stream and the output buffer should not be flushed, otherwise we risk flushing a small amout of data expending one WriteFile call.

The final result is now ready and is consistently faster than the PC decompression approach. Try it!

ZIP and RAPI - Is it worth it?

After a few more days working on Lucian Wischik's Zip Utils code I finally managed to build a working RAPI ZIP decompression server. The architecture of this RAPI extension is quite simple: it exports a single method (ZipServer) that implements a streamed RAPI call. Streaming (non-blocking) RAPI calls are very nice because they allow you to keep an open connection with the desktop which is especially useful when expanding very large compressed files: you can slice the compressed stream and have it decompress on the device. For more implementation details please see the source VS 2005 project in the above link.

To make this work I changed the CeUnzip project in order to support calling a RAPI extension and added an extra mandatory command line argument: -i for expanding on the PC from a ZIP file on the device and -e for the exact opposite operation. It's here that I make the remote call to the RAPI extension DLL. Please note that the two projects use different versions of the unzip.cpp file. I will correct this before publishing an article that describes in greater detail the inner workings of the code.

How about results? Is this really faster than using an on-the-fly decompression algorithm on the PC and using CeWriteFile to write the decompressed data on the device?

My first results were very disappointing. When compared to an installer application I wrote for a customer that decompresses on the PC and writes to the device, I consistently got worse timings. The only remedy that made the device DLL run as fast as the PC was to increase the IRAPIStream read buffer size to 128 or 256 KB. As a matter of fact, RAPI read latency is a big issue here. Before giving up I looked at the particular ZIP file I was using to test: it was quite large and had a relatively small compression ratio. Maybe this explains something?

My next test was to use a very high compression ratio ZIP file and run the same tests (expanding it to the device using the two different approaces). This ZIP file was built from a very large SDF file which almost disappeared when compressed: 193 KB to only 3 KB. Now the results were a bit different...

Depending on the device, the DLL expansion was from 2 to 4.5 times faster than the PC's. This means that for compression ratios closer to 1 it is (almost) indifferent which approach to use, but when the ZIP compression ratios get larger, it's far better to use the server DLL.

ZIP and RAPI

For quite sometime I have used Lucian Wischik's Zip Utils, a set of zipping and unzipping functions that work very well both under Win32 and Windows CE. This code has allowed me to do a lot of fun stuff both on the desktop (specialized installers) and on devices (an HTML browser for ZIP files).

Recently I had to use this code to write a specialized device installer (no cabs) for a consumer application that runs on all sorts of devices. This installer was designed to extract files from a ZIP on the desktop and copy the extracted files to the device over an ActiveSync connection. I used Zip Utils to extract the data and RAPI to copy the expanded files to the device. This works but can be slow on some devices. Wouldn't it be great if I could expand the files on the device while having the source ZIP file on the PC? I would surely make the whole thing faster because the RAPI transport times are a big bottleneck in this whole process.

Due to time constraints, the installer is now shipping like this: data is expanded on the PC and copied to the device. Now I'm starting to write an application that will update the device code and data. Interestingly most of the static data is stored on ZIP files on the device and the updater must extract specific files on the ZIPs in order to know some details. The problem is that some of these ZIP files are over 2 MB and copying them to the PC just to extract one tiny file is out of the question due to slow transfer times.

No options here: I had to roll up my sleeves and started to read Lucien's code in order to see what could be done. This was no walk in the park because he uses a very packed code indentation, and understanding the code was not easy at first. Then I noted that all the ZIP file access methods were neatly encapsulated in specific functions (like lufopen, lufread and so on) so it would be quite easy to replace the desktop file access functions with RAPI's.

My approach was to expand the LUFILE structure and include a set of function pointers that would be initialized with the regular API when the ZIP file is on the PC and with RAPI's when the ZIP file is on the device. After 20 minutes the code was working and I was able to write a very simple command line unzipper (get the code here).

Now I want to solve the first problem I had: efficiently unzipping a file to a CE device. This will require a RAPI DLL on the device that will have all the decompression code. When this is ready I will publish the results in an article. Stay tuned.

Bitten by the desktop GC

If you downloaded the Console Beta 4 I published yesterday, you might have noticed how badly it bombed when exporting an SDF database to Access. The error message you see usually means that some native code DLL is writing to NULL pointers or doing some other equally responsible stuff. In this case what you see is a NULL pointer dereference (yes, that bad).

While running perfectly in Debug mode, the error almost always showed up in Release mode. When exporting very short databases or single tables it might not show up. But when I tried to export the Northwind sample the Console would bomb exporting one of the largest tables (Order Details or Orders). The keyword here is random because as it turned out, the error was being caused by the garbage collector (well, my bad code was causing it - the GC just made it apparent).

The Access database code was implemented in C++ with a similar approach as DesktopSqlCe's low-level classes: there are separate classes for the table structure and the table cursor. In order to open a base table cursor and insert data on the table (no SQL is used for data manipulation) the code creates a table object and "opens" it returning a "rowset" object. My mistake was to have the table class include an ATL CTable-derived object, not a pointer to it. The CTable-derived class contains all the accessor and rowset logic that are used by the rowset object. Making it static is a bad idea because the desktop .NET GC did not know about this and kills the table object because it thinks it is no longer needed. After all the only active object is the rowset... Now you see the problem: by killing the table, the rowset is implicitly killed as well because it shares an object that is statically owned by the table. Closing the rowset means that a few internal pointers will be deleted and set to NULL. Bang! Am I smart or what? (The answer is "No").

I uploaded a revised Beta 4 that solves this issue, so if you installed the one I uploaded yesterday please replace it with the new one.

Data Port Console 1.0 Beta 4 available for download

I have just uploaded the new Data Port Console Beta 4. As previously promised, import and export to Microsoft Access (2002-2003 and 2007) is now supported and this looks like it will be the last Beta. Now I will focus on cleaning a few bugs and will try to have the final version 1.0 by mid-August.

What's next? Primeworks.Data - the merger between DesktopSqlCe and Data Port Sync with a few "benefits" such as the ability to make P2P connections via Bluetooth or WiFi between two devices. Also I will add support for the JET (ACE) engines on the desktop server and will publish the services via a muti-threaded Windows Sockets server (I don't anticipate the need for IOCP anytime soon because there will be few concurrent connections).

The new Beta 4 can be downloaded from here.

Bluetooth Service Publishing

The code for my forthcoming article on Microsoft Bluetooth Stack inquiry and discovery (remember, inquiry is for devices and discovery is for services) is ready. Apart from inquiring the available devices, it also discovers the existing services and lists the SDP record contents.

After being able to find a device with the service you want to consume, you can connect to that device using Windows Sockets. The reverse process involves publishing a service for other devices to consume and to serve requests using WS. Unfortunately the SDK has no means (nor explanation) on how to create an SDP record. Platform Builder users have the BTHNSCREATE sample to play with, but what about us, mere SDK users? Looks like Microsoft thinks this is something that should only concern people designing new Windows CE or Windows Mobile devices. I sincerely hope not and I'd rather think this is a design oversight. Why would I need such a thing?

Well, one of my plans is to provide P2P support for my products. If you look around the newsgroups you will see lots of people asking for an SDF-only synchronization package. This will require that two SDF files can "talk" to one another. I already support having a PC talk to an SDF file on a device, but what about having one device talk to another? A year ago I actually achieved this using WiFi, but this is not the correct protocol for ad-hoc P2P scenarios - Bluetooth is better. So if I want to support this scenario, I must publish a new Bluetooth service. Yes, of course I could use emulated COM ports and the code already supports serial communications. But I want the real thing: a dedicated service with its own GUID.

So how am I going to circumvent this? Most likely I will use Doug Boling's solution: use a templated SDP record and fill in the gaps (not nice, I know). What really bugs me is that the Widcomm stack makes it so much easier to publish a new service...

SHMENUBAR and MF_GRAYED

My WTL 8.0 experience with Windows Mobile development is going just great. The application wizard worked out correctly with only a very minor issue: the WTL include directory is not set up either for C++ compilation nor for resource compilation. As I said, this is a minor issue that you overcome quite easily.

While working for the new Microsoft Bluetooth device inquiry and service discovery (these seem to be the appropriate Bluetooth terms) I found out that Sergey Solozhentsev's WTL Helper tool works like a charm in Windows Mobile projects under VS 2005. All of the message handling functions for the main frame window were generated with this tool and it makes it very easy to write all the UI code - this tool just rocks.

Today I found out about something that does not seem to be documented anywhere: when using the SHMENUBAR resource, never specify a menu resource with a grayed (MF_GRAYED) menu item - the menu will not show up. The WTL code is very straight forward and uses the SHCreateMenuBar API, so this was not where the problem lied. After removing the grayed style from the resource editor the menu magically showed up! So is this a bug on the shell or on the compiler?

Back to Bluetooth the WTL way

I'm writing this post from (today not so) sunny Algarve using my brand-new Vodafone Mobile connect card. This thing really works but the Vista version of the software seems to be a bit slow when booting. Also, if I boot the machine with the card plugged in the driver will not find it. Apart from this it's a treat - I get internet access pretty much wherever I want to (coverage permitting).

I'm coming back to writing about Bluetooth on Windows Mobile devices and this time I'm exploring the Microsoft stack. Using MFC code I had written almost a year ago, I'm now turning to WTL to rewrite the sample for the article. So far I can tell you that the Windows Mobile WTL wizard looks quite competent at generating the basic application code. More impressions on this experience will be published here. Stay tuned.

CeGetDiskFreeSpaceEx - II

This is not an ActiveSync issue: my iPAQ 2210 does not support this API nor do some Windows CE 5.0-based GPS devices I've been working with. At the end of the day, I had to write a RAPI DLL to support this. The code is quite simple:

typedef struct _DISKSPACE
{
    ULARGE_INTEGER uliFreeUser,
                   uliTotalSize,
                   uliTotalFree;
} DISKSPACE;

__declspec(dllexport)
int DevGetDiskFreeSpace(DWORD cbInput, BYTE* pInput,
                        DWORD* pcbOutput, BYTE** ppOutput,
                        LPVOID pReserved)
{
    DISKSPACE ds;
    BOOL      bOk;

    bOk = GetDiskFreeSpaceEx((LPCTSTR)pInput, &ds.uliFreeUser,
                              &ds.uliTotalSize, &ds.uliTotalFree);
    if(bOk)
    {
        *ppOutput = (BYTE*)LocalAlloc(LPTR, sizeof(ds));
        memcpy(*ppOutput, &ds, sizeof(ds));
        *pcbOutput = sizeof(ds);
    }

    return 0;
}

Of course, you need to put this in a device DLL and consume it as a blocking RAPI call.