I had made a test application in [HKLM\Init and database status] for trying to figure out the database status in boot up sequence. And as I said in that post, the application run fail when I try to put it earlier than services.exe (launch60).
And I found a solution about the issue: signature.
I still cannot find any document mention about this limitation, but after I sign my test application with my develop certificate, the application can be run at anywhere I wanted.
I already describe the way to create, install and sign application use test certification in [Create and use test certificate]. And I found that VS2008 already provides the ability in project property.
You still need to create your test certificate using makecert.exe first. You can follow the steps to create it. If you follow the sample, the certificate should be installed in user store already, and it’s name is “Tim’s Dev Authority”.
I am keep looking at how to stop service, drivers and applications to release resources (file, database) to make backup & restore process more smoothly.
This post is a note of the testing result, it will keep updating when anything was found.
Notes of study “Implement a Network Service on Windows CE”. I think this article is a good start point of learning mobile service development.
When develop a service on window mobile, the service dll file need to be signed for services.exe to run it normally. Otherwise, your service cannot be run be services.exe when system start up. So, here is the topic of this post.
How to create a certificate for testing and development use?
Before I list the procedure, I need to declare first: There are still lot of things I don’t know in windows certification, this procedure it’s just a way I found I can finish my target only.
Ok, here we go…
The original post is on MSDN develop forums, cause the post there sometimes get lost, and the coding style is totally crash by HTML. So, I made one copy here for backup and clear the coding style for easy read.
Below link it the original post:
This post share source code of two functions: BackupMain() and RestoreMain() to backup and restore cemail.vol, The idea is very simple.
Again, I made a diagram for hive-base registry.
Key points of this diagram:
突然想到，解釋這麼落落長，那不就代表這張圖沒有用…
I made a diagram for my study report.
Key points of this diagram:
Except the strange terminology on Windows CE database, I finally got another key point that why I am struggle on study CEDB document. That is:
There is no schema on CEDB database.
What a key point of CEDB feature, but I did not notice that! Cause I already used to other database engines on the world. Without this realization, it’s hard to understand the CEDB APIs.
Each record in CEDB refers to a set of data. Each data has a unique property ID and data type. CEDB allow records in same database has different properties (but I really don’t know who need this).
With this cognition, everything is clear. Why CeCreateDatabaseEx() API does not have parameter for properties, why CEDB using CeWriteRecordPropertiresEX() API to create properties, …, etc.
I think, I really waste a lot of time on study CEDB with wrong assumption… 
Although MSDN mentions that EDB is designed to replace CEDB, but I did not find documents descript about how to detect the database in my device is EDB or CEDB. And more important is, I do not how to use EDB APIs in my develop machine (When using VS2008 with SDK 6.0, it using CEDB APIs by default, and I don’t know how to switch it).
Finally, I got some post that really help me to solve my problem.
This post is a case study of backup and restore one application called XCamera which is a camera application with it’s own driver included.
After analysis, I found the components I think it should be backup for restoring.
| Files in \Program Files\XCamera | The application install folder. There are lot of exe, dll and ini files in this folder. |
| Two driver files in \Windows folder | File camera.dll and backlight.dll. There are drivers run by device.exe. |
| A link file in \Windows\開始功能表\程式集 | The shortcut LNK file. |
| Files in \Windows\AppMgr\[XCamera ….] | Looks like some files created by AppMgr. I am not sure what they are, but they should be backup. |
| Registry | XCamera does not user hive, we only need to backup system hive. |
So, here is the procedure to backup system for XCamera.
I don’t get any problem on backup process (
). So, let’s clean boot the device and start our restore procedure.
There are two things cause you need to reboot the device: First, the de-actived devices will be active again after reboot (Of course, you can active them by yourself). Second, the RegReplaceKey() API will un-mount system hive, you need to reboot the device and OS will use your hive to replace system hive.
After reboot, I can see XCamera in 程式集, and it can be run normally.
But, I found another problem on this procedure. After restore, I cannot find the application XCamera in application manager, that means I cannot uninstall and upgrade the application. 
Looks like application manager using other resource on disk. Maybe database??? I think I found my next topic…
In my case study, the target application will replace some drivers under \Windows folder. Those drivers are run by device.exe when boot up.
In my backup and restore study, although I can copy the driver file into backup image, but I cannot simply write it back to device.
My first plan is develop a program run before device.exe and replace those files. The program only needs file system APIs, and the APIs are ready in early of filesys.exe, so I guess it should work.
But as my test result on [HKLM\init and database status], I go problem to run any application before device.exe, even service.exe. And I cannot find solution for it until now.
So, here is the second solution I got:
First, find the sub-key(s) in “HKLM\Drivers\Active” which using the driver file. In my case: camera.dll, I found it at “HKLM\Drivers\Active\38”. Here is the entries in that registry key:
| Hnd | 3490960 |
| Name | CAM1: |
| Key | Drivers\BuiltIn\CameraDriver |
| BusParent | 206896 |
| InterfaceType | 0 |
| BusName | BuiltIn_0_13_0 |
| ClientInfo | 0 |
Three keys you need to pay attention:
| Hnd | HANDLE of running device. You will need it to deactive the device for release driver file. |
| Key | The registry location which store the driver settings for device.exe. You need it for device active API. |
| ClientInfo | Driver’s init parameter. If device.exe active driver with parameter, it will put it in this registry. You will need it when you want to active the device again. If the registry key not exists, ignore it. |
After found that registry, all information needs for my testing application to replace driver are ready. Here is the procedure of camera.dll:
I think most of drivers can use this procedure to replace, but I still think it’s too complex.
Cause that means after I scan all files does not in ROM image ([Windows Mobile File system Study]), I need to differentiate which files are used by device.exe and using different way to perform backup & restore function for them.
And here is anther issue about this solution: When I de-active the device, something device will popup “device.exe failure” error screen. Although that does not cause any problem for our backup & restore operation, but the user experience is bad for user to see that screen.
On the post Enumerate Database Volumes, we saw a lot of database volumes has been mounted and locked by some applications or services. I am trying to find a position in boot up sequence to get less database locked as I can.
This is the volumes mounted when boot up finished:
| \Documents and Settings\Default.vol \ReplStrorVol \mxip_notifi.vol \mxip_swmgmt.vol \cemail.vol \mxip_system.vol \mxip_lang.vol |
I modified my enumeration application to output the enumeration to a file. Cause this application should only use database and file system API, I hope it can be executed very early in boot sequence.
And I find the first place to try: HKLM\Init. The applications registered in this registry will be run in the lastest step of filesys.exe. At that moment, the database and file system APIs should be ready. So I start my testing.
I put my execution file in \Windows folder, and add a registry in HKLM\ini as “LunchXX” with my application file name as value.
First, let me list my KHLM\Init:
| Launch4 | sdpready.exe |
| Launch19 | initvmmap.exe |
| Launch20 | device.exe |
| Launch30 | gwes,exe |
| Launch50 | shell32.exe |
| Launch60 | services.exe |
| Launch70 | connmgr.exe |
| Launch82 | ShellNotification.exe |
This is not a full list, but enough for my description.
First try, I think device.exe loads drivers, the system state should be ready to execute my application. So I made “Lunch35” as my key name, and them, the application run fail, I don’t know why, but just cannot see the output file.
Second try, I want to make sure my application can be run in HKLM\init. So I modify the key name to “Launch135”. And great, it can be run, and the output is:
| \Documents and Settings\Default.vol \ReplStrorVol \mxip_notifi.vol \mxip_swmgmt.vol \cemail.vol |
Another great news, some volumes missed. Then I try to put my test application on “Launch65”, and the cmail.vol missed, too.
| \Documents and Settings\Default.vol \ReplStrorVol \mxip_notifi.vol \mxip_swmgmt.vol |
And then when I try to run it earlier at “Launch55”, it fail to run again.
According my HKLM\init list, the item between 55 and 65 is “Launch60: services.exe”. Seems my application need to wait for all service ready. I am depressed about that. Cause I think my application only use database and file system APIs, I hope it can be run earlier then that.
So, until now, I have four database volumes need to study about how to release them for backup and restore operation.
As I said in [Windows Mobile (CE) Database Study], the terminology of CEDB and EDB is almost killing me. The topic is enumerate tables in database, if using all other common database engines.
I cannot find any document descripts CEDB and EDB which one used in Windows Mobile 6.1 (my test device). But I cannot use EDB only APIs on my develop environment (VS2008 + WM 6.1 SDK) , so the sample code here is using CEDB.
Here is the code piece, follow the sample code in [Enumerate DB Volumes], we got the DB volume’s CEGUID (in variable ‘guid’) and then try to enumerate all database (tables) in that volume.
| HANDLE hFind = CeFindFirstDatabaseEX(&guid, 0); CEOID oid = 0; while ((oid = CeFindNextDatabaseEx(hFind, NULL)) != 0) { CEOIDINFOEX oidinfo; memset(&oidinfo, 0, sizeof(CEOIDINFOEX)); oidinfo.wVersion = CEOIDINFO_VERSION; if ((CeOidGetInfoEx2(&guid, oid, &oidinfo)) { /// TODO: print out oidinfo.infDatabase.szDbaseName } } CloseHandle(hFind); |
Back to enumeration, you can use CREATE_INVALIDGUID to initialize variable 'guid’, then the code will enumerate all database (tables) in all mounted volume (I am not sure the order)..
And you can specified a PCEGUID in the second parameter of CeFindNextDatabaseEx() function to get the volume’s GUID which the database belongs to.
Below is the enumeration of my device, just for reference.
| VOL: \Documents and Settings\default.vol |
Original document:
http://msdn.microsoft.com/en-us/library/ee489764.aspx
I only list items I may needed
1. Mount boot hive (Boot.hv) from ROM
2. Loads and initializes Storage Manager (HKLM\System\StorageManager)
3. Boot phase zero
4. Initialize system locale
5. Boot phase one
6. Device.exe (if needed ??)
7. Wait for file system contains system hive ready (MountAsBoot?)
8. Mounts ROM portion of system registry (\Windows\Default.hv)
9. Check IOCTL_HAL_GET_HIVE_CLEAN_FLAG for clean system hive
10. Check HIVECLEANFLAGS_SYSTEM for clean system hive
11. Load system.hv from file system
12. Check system hive signature for clean system hive
13. Swap out boot registry hive
14. Check IOCTL_HAL_GET_HIVE_CLEAN_FLAG for delete user.hv
15. Check HIVECLEANFLAGS_USERS for delete user.hv
16. Mounts ROM portion of user hive (\Windows\User.hv)
17. Load user.hv from file system
18. Check user hive signature for clean user hive
Phase 0: system locale not ready
Phase 1: system locale ready, but total registry no loaded
Phase 2: system locale ready and total registry has been loaded
I wrote a test application to enumerates all EDB volumes on my WM6.1 devices. Here is the code piece:
| CEGUID guid; CREATE_INVALIDGUID(&guid); while (CeEnumDBVolumes(&guid, wszName, 255)) if (GetLastError != ERROR_NO_MORE_ITEMS) |
According the MSDN, the size of wszName should be enough to CEDB_MAXDBASENAMELEN. But actually I got ERROR_INSUFFICIENT_BUFFER error when using that size.
After increase the size to 255, the enumeration finish normally, and here is the output in my device.
| VOL: \Documents and Settings\default.vol |
It’s more than I expected….
And notice the first one, the length is absolutely larger then CEDB_MAXDBASENAMELEN (32), that means the reference guide of CeEnumDBVolumes() was wrong.
Back to the list I got, there are more than that when I searching the *.vol files on root file system. Cause CeEnumDBVolumes() only list the mounted database volumes.
And of course, the volumes listed has been locked after my mobile device boot up, there is a long road for me to understand how to release them for backup/restore purpose.
Notes about study CEDB and EDB on Windows Mobile (CE).
When study MSDN about CEDB and EDB, some wording keep confusing me. Cause CEDE and EDB using different terminology with other common database engine.
| EDB and CEDB term | SQL Server Compact term |
| Sort order | Index |
| Data type | Data type |
| Property | Column |
| Record | Row |
| Database | Table |
| Volume | Database |
Especially the Database mapping to Table one keep confusing me on reading the document…
We can see CEDB and EDB as a file-base database like Sqlite. A volume is a file including multi database (table, you see why I am confusing).
Before using the database, you need to mount (‘open’ in normal terminology) the volume. If the volume is not exists, you need to create (create a new file using CEDB API) first. When you don’t need the database anymore, you need to un-mount (‘close’) the volume.
There is a default database volume in CE (Mobile) system used by object store. It will be mounted when system start up and released when system shutdown (that means the file will be locked in whole running cycle).
And applications can create volume in any location they wanted, including external storage (like SD card).
The EDB and CEDB APIs using CEGUID as identity to access a volume, database or even a record. For the detail usage, need to refer to each method’s document for detail.
Obviously, we need different procedures to handle the default volume (object store) and others.
For extra created database volume, we just need to stop all process which using these database volume. And then we can just follow the root file system backup & restore procedure. It should be a easy one.
For object store database volume….
If I can find the way to un-mount the object store, it should be easier to handle it like other database volume. But seems it’s not so easy to find the way.
Otherwise, we can enumerate all object store database and dump them into our format. And then push them back when restore. I am not sure if there is some database has write-protect, but I think it should be not….
The same reason as the previous one for Windows mobile registry. This post is the notes of Windows mobile file system study.
Theoretically, we can scan all files and folders in root system, and then using GetFileAttributes to get the attribute. And of course we can use the attribute to differentiate the file is stored in ROM image, writable root file system or external media storage (like SD card).
| FILE_ATTRIBUTE_INROM | Indicate the file store in ROM image. It’s read-only and cannot be modified. And we don’t need to worry about it on backup function. |
| FILE_ATTRIBUTE_SYSTEM | The file is used by the OS. |
| FILE_ATTRIBUTE_HIDDEN | The file or directory is hidden. I am not sure if we can find hidden file using FindFirstFile API. |
| FILE_ATTRIBUTE_ARCHIVE | MSDN says application use this attribute to mark files for backup or removal. I am not really understand what he means. I need more study on this, maybe that relative with backup and restore function we wanted. |
| FILE_ATTRIBUTE_TEMPORARY | Temp files, we can ignore them for backup function. |
There are more attributes list in MSDN, I don’t know if I miss anything important for backup application. I need more testing on real device for detail.
I had write a testing code to scan all storages on my mobile device follow the sample here.
And I get output like this:
Store: SMFLASH
Part: Name(Part00) Vol() FS(BOOT) ATT(0)
Part: Name(Part01) Vol() FS(RAWFS) ATT(0)
Part: Name(Part02) Vol() FS(imgfs.dll) ATT(16)
Part: Name(Part03) Vol() FS(fatfsd.dll) ATT(16)
Store: DSK1:
Part: Name(Part00) Vol(\Storage Card) FS(fatfsd.dll) ATT(16)
I can see I have one SD card using FAT file system and mounted on “\Storage Card”. And the internal storage is SMFLASH with 4 partitions.
I can understand that we don’t need to care about partition 0 and 1, but I still cannot know which is the root file system we are using. (The partition attribute 16 means READONLY, AUTOFORMAT and MOUNTED.)
Ps. The sample code using library storeapi.lib, this library does not exists in Windows Mobile SDK. I got it from WM616 adapter kit.
Theoretically, if we can list all files which are not store in ROM image, then we can create the file backup image for later restore.
I had wrote a test application which scan all files in root file system to find out the files and directories does not store in ROM image using API FindFirstFile and FindNextFile.
But we still have issues need to be solve:
1. When mobile OS running, lot of files were been locked for read (for example: driver dll in windows folder). I cannot copy those files for creating backup image.
2. Like previous issue, of course the running (locked) files cannot be write for restoring.
According to MSDN, here are the APIs initialized by filesys.dll in sequence:
I had not read the detail yet, but I am surprising the database API initialed before file system. The database does not need to refer to file system??
I meet this issue when develop a window mobile application which needs huge file IO operations on our company's handheld device. Until now, I still don't know this issue cause by official windows mobile STD library or our company's OS image only.
The target application randomly read binary files like a RO-database. I using STD ofstream API to read data from file and found the performance was badly. I try to debug this performance issue on that application and found that the root cause is fstream API.
I made a 100MB dummy file and one simple application to read the file sequentially (a for-loop), the total ready time using fstream is almost ten times of another one using fopen/fread interface.
I am lazy to test the same behavior on other company's device (actually, I don't have other device….), so I just re-write the code to use fread() API and the problem solved.
PS. I got the same behavior on two models of our company's handheld device, and they are use Window Mobile 6.1 system.
