kSyncML: New Enhydra Open Source
Project
SyncML in Wireless Devices
By Alexandr Koloskov, alexk@reaxion.com,
Oleg Oksyuk, olego@reaxion.com
Today it’s not enough to have your personal information such as e-mail,
contacts, calendar entries and appointments stored inside your laptop.
I’m not even mentioning desktop computers. People need to access information
everywhere, regardless of device, network and location. What’s even more
important is not only viewing the information, but ability to modify and
synchronize changes back to centralized storage. Not only that, such functionality
should be available from devices such as PDA or better yet mobile phone.
Mentioned requirements led to creating standard way to exchange and synchronize
information between various devices. The independent organization SyncML
consortium (www.syncml.org) was formed
by industry leaders such as Ericsson, IBM, Nokia and now accounts of several
hundred members to promote and oversee development of SyncML - XML-based
common mobile data synchronization protocol. As stated in SyncML whitepaper
(http://www.syncml.org/technology.html):
“…A data synchronization protocol defines the workflow for
communication during a data synchronization session when the mobile device
is connected to the network. The protocol must support naming and identification
of records, common protocol commands to synchronize local and network
data, and it can support identification and resolution of synchronization
conflicts…”.
SyncML is transport protocol agnostic. So far standard addresses two
transport mechanisms: SyncML over HTTP and SyncML over OBEX, but developers
are free to implement SyncML over any transport. SyncML is also agnostic
to data format itself. Standard describes synchronizing of common personal
data formats such as vCard and vCalendar, but the only restriction applied
to document format is ability to name and locate specific record inside
document, so SyncML can be used for synchronizing text files, RDBMS tables,
XML documents and almost any other existing document formats.
SyncML future looks bright. GSM Association considers SyncML along with
Java and MMS as one of the key technologies for handset manufactures to
support (see press release, http://www.gsmworld.com/news/press_2001/press_releases_24.html).
Several phones already contain SyncML implementations. Just to name a
few: Nokia 6310, 7650, Nokia Communicator 9210 and 9290, Ericsson T39m,
R520m . It seems that every new mobile phone that includes some sort of
personal organizer software uses SyncML to synchronize this data.
Developing custom SyncML engine
Even though SyncML is being widely adopted, developers who want to incorporate
SyncML into their mobile application will face the necessity to write
their own implementation of SyncML, since most existing solutions are
incorporated on mobile phone firmware and do not expose API to developers.
Also there aren’t any commercially available SyncML solutions for wireless
devices, except maybe SyncML client from Symbian OS. Due to heavy constraints
which applied by limited device capabilities, writing custom solution
maybe the only choice. In this article we would like to highlight several
problems that developers will face implementing custom SyncML engine.
Also we will describe our own solution, and our solutions to common problems.
Common problems
XML parser. First of all, SyncML is based on XML, so you need
an XML parser to read/write SyncML packets. It’s not a problem on desktop
or server where variety of XML parsers are available, but in mobile development
the choice is poor. Neither J2ME, BREW, PalmOS or Symbian OS have built-in
XML API, so you should write your own one or use third-party library.
There are few compact parsers available for J2ME such as kXML (kxml.enhydra.org), MinML (http://www.wilson.co.uk/xml/minml.htm)
and NanoXML (nanoxml.sourceforge.net).
But for BREW, PalmOS or SymbianOS the only choice we know about is porting
James Clark’s expat parser (www.jclark.com/xml/expat.html),
which is quite compact and portable. Reaxion
will also release XML parser for BREW and other platforms soon. The second
problem with XML parser is that DOM parser is too heavy for mobile environment,
so developers should stick with SAX-based parser, which is harder to understand,
but much thinner. Some parsers like kXML even sacrifice SAX compatibility
and implement own lightweight API to decrease library size.
Network problems. Despite all efforts of wireless network operators
networks are still very unstable and suffer from problems such as:
- High network latency
- Limited bandwidth
- Low reliability of both data and connectivity.
Keeping all this restrictions in mind developer should choose a right
strategy to avoid them. First of all, it probably makes sense to make
an asynchronous messages exchange to avoid problems with connectivity.
Next, we should assure that no changes are lost and no duplicate changes
are made. We can store each change in persistent storage and re-send it
every synchronization session until we receive confirmation from other
side that change is applied and confirmed. But this highlights another
problem – a need to have an unique identifier for each change, SyncML
supplies one, but it’s unique only in session scope and we need uniqueness
in wider scope to support asynchronous messaging. In end of this article
we present our way to avoid this problem, but it’s still an open question.
And the third problem is that all those XML languages such as SyncML are
quite large, which leads to increase in network traffic. The native solution
is usage of WBXML – standard for presenting any XML documents in binary
form. It’s already used to pack WML pages in WAP, but it seems that there
aren’t any built-in APIs to work with WBXML. One possible solution is
using already mentioned kXML library which has WBXML parser, but it adds
additional kilobytes to your application, which can be critical in many
cases.
Binding to document formats. As we already mentioned, SyncML
is document-agnostic. But you need some document-specific code to apply
changes to document. Such code maybe implemented with SyncML engine (at
least for some common document formats), but in mobile development every
feature which is potentially useless in some cases should be avoided,
since it increases size of application which is as we mentioned again
and again is critical. So, developer should choose to either incorporate
document-specific code for some formats into SyncML engine or define common
set of interfaces, which every document should implement to be synchronizable.
Our own solution – Tequila kSync client.
Here at Reaxion Corp. (www.reaxion.com) we have faced the necessity
of writing SyncML engine for our product. So we developed our own solution
code-named Tequila kSync. The initial version of tequila kSync is already
available as open-source at ksync.enhydra.org,
though it lacks some important features, described above. Next version
2.0 solves many of mentioned problems and will be available as open-source
soon. Below, we describe key elements of kSync 2.0 architecture and hope
that this will be helpful for other developers implementing their own
SyncML engines.
What is Tequila kSync Client 2.0
- A J2ME library that implements SyncML HTTP Transport Adapter, SyncML
Client Agent and Changelog Storage for mobile devices.
- Doesn’t enforce any restrictions on data format.
- Supports 3 SyncML scenarios (http://www.syncml.org/technology.html):
2-Way Sync, Refresh from Server and Slow Sync.
- Is optimized on binary file size. Obfuscated binary file size is 17k
without MinML XML parser (which adds another 8k).
- Does not contain any Sync Engine functionality. All logic that could
be moved on server is not present on client.
- Facilitates conflicts resolution on a server side where all conflict
resolution logic is implemented.
- Guaranties data integrity in event of communication failure.
Sync Server
kSync Client based application is supposed to interact with corresponding
kSync Server. The usual approach in Java is to implement such server as
servlet-based Java Application. Enhydra application (www.enhydra.org)
server could be used as servlet engine for it. Our kSync Server 1.0 (ksync.enhydra.org) supports valid XML documents.
Server is implemented as a servlet-based Java Application and runs under
Enhydra Application Server.
Architecture
Tequila kSync Client 2.0 could synchronize only one document in one synchronization
session.
This is done in order to make the client as THIN as possible. However
this limitation does not prohibit synchronizing another document in the
next synchronization session. We don’t make any restrictions on number
of documents that could be synchronized.
Tequila kSync Client 2.0 has 3 open interfaces that should be implemented
in for all documents that wish to be synchronized:
IDocumentSerializer – has only
one method that serializes the document to String
IDocumentParser – effrectively a corresponding deserializer
interface and has only one method that retrives the original document
from String
IDocumentChanger – has 3 methods: add, delete, refresh for
performing appropriate operations on the document.
When any changes are committed on the client, application should call
appropriate methods in kSync Client,
Here is the code example of working the Application with kSync Client
// Create an instance of ISyncClient.
ISyncClient syncClient = new SyncClient(“http://sync.server.com”, “IMEI:001004FF1234567”,
“John”);
// Make the document “testDocumentId” available for being synchronized.
// Set instances of IDocumentSerializer, IDocumentParser,
// IDocumentChanger interfaces
IDocumentChanger changerFromClient = syncClient.setDocument(“testDocumentId”,
clParser, clSerializer, changerFromServer);
//Refresh the document from Server
changerFromClient.refreshFromServer(“testDocumentId”);
//Add item to the doc
changerFromClient.add(“itemId1”, null, “itemValue2”);
//Replace item’s value
changerFromClient.replace(“itemId1”, null, “itemValue2”);
//Delete the item
changerFromClient.delete(“itemId1”, null, null);
//Synchronize the document with server
syncClient.sync2Way(“testDocumentId”);
Implementation Issues
XML Parser
Tequila kSync Client uses MinML XML parser (http://www.wilson.co.uk/xml/minml.htm)
for parsing SyncML packets. If application that uses kSync need to parse
XML it is recommended for the application also use MinML XML parser to
minimize application size
Stability
Sending HTTP request to server doesn’t guarantee that message will reach
the server. Same is true for the HTTP response. This is especially relevant
for mobile devices, since network connections are highly unstable. We
need to insure that every change reaches its final reciepient, Let’s look
at few possible scenarios:
1. sending message once doesn’t not guarantee anything, since message
may not reach the server and even if it did, we may not get a response.
2. Another possibility is to keep sending the change until server replies
with change status response. In this scenario the change could be applied
several time by Server, because server won’t have any knowledge if this
change was received earlier.
To avoid both of these issues we identified all client and server changes
with unique IDs. This ID is unique not only for current SyncML message,
as SyncML specification demands, but it is unique within session. This
design decision doesn’t conflict with SyncML specification, because unique
change ID is automatically unique per SyncML message.
The change ID is sent to or received from Server in <CmdID> SyncML
element. On each change received from server client sends back a status
change and stores it in RMS until the synchronization session is finished
and server sent back a change status (confiramation). This opearion is
not limited to single request/response pair, therefore ensuring the data
integrity in an event of communication failure. If the connection fails
during sending the change request to the server and no response was received
from server, client will send change request in the next synchronization
session. If client receives the status change more than once it won’t
apply it the next time, because change status was already stored in RMS
|