This is not as straightforward as I thought it would be. After copying and expanding the recommended cab files for the platform (in my case I installed it on both a Pocket PC 2003 SE and on the Windows Mobile 5 Pocket PC emulator) you will not be able to run Query Analyzer. Simply put, this application like my device component are OLE DB consumers and the OLE DB provider is not automatically installed. Microsoft designed SQL Mobile in a very componentized way and the OLE DB provider is one such component. To make this work, I had to manually copy the sqlceoledb30.dll and sqlceca30.dll to the device and register them. Everything now works as expected.
Note: If you are developing against SQL Mobile, you might want to install the error messages DLL - sqlceer30EN.dll.