Classes | |
class | ClientTestFactory |
generates tests on demand based on what the client supports More... | |
class | TestFileSource |
This code uses the ClientTest and TestFileSyncSource to test real synchronization against a server. More... | |
Functions | |
void | simplifyFilename (string &filename) |
const string & | getCurrentTest () |
static void | printTests (CppUnit::Test *test, int indention) |
int | main (int argc, char *argv[]) |
static USE_NAMESPACE std::list < std::string > | listAnyItems (SyncSource *source, SyncItem *(SyncSource::*first)(), SyncItem *(SyncSource::*next)()) |
utility function to iterate over different kinds of items in a sync source | |
static std::list< std::string > | listNewItems (SyncSource *source) |
static std::list< std::string > | listUpdatedItems (SyncSource *source) |
static std::list< std::string > | listDeletedItems (SyncSource *source) |
static std::list< std::string > | listItems (SyncSource *source) |
std::list< std::string > | listItemsOfType (SyncSource *source, itemType type) |
generates list of UIDs in the specified kind of items | |
static int | countNewItems (SyncSource *source) |
static int | countUpdatedItems (SyncSource *source) |
static int | countDeletedItems (SyncSource *source) |
static int | countItems (SyncSource *source) |
int | countItemsOfType (SyncSource *source, itemType type) |
utility function which counts items of a certain kind known to the sync source | |
static void | importItem (SyncSource *source, std::string &data) |
static std::string | updateItem (CreateSource createSource, const std::string &uid, const char *data) |
deletes specific item locally via sync source | |
static void | deleteItem (CreateSource createSource, const std::string &uid) |
deletes specific item locally via sync source | |
template<class T , class V > | |
int | countEqual (const T &container, const V &value) |
static USE_NAMESPACE SyncSourceConfig * | createBriefcaseDefaultConfig () |
static void | setDefaults (SyncManagerConfig *config, const char *testName, const bool setClientDefaults, ArrayList *defaultSources=NULL) |
Sets the default values (from DefaultConfigFactory) in the passed SyncManagerConfig. | |
const StringBuffer | initAdapter (const char *testName) |
Initializes the PlatformAdapter for a given test. | |
SyncManagerConfig * | getNewSyncManagerConfig (const char *testName, const bool setClientDefaults, ArrayList *defaultSources) |
Generates and returns a NEW ALLOCATED default configuration (SyncManagerConfig). | |
DMTClientConfig * | getNewDMTClientConfig (const char *testName, const bool setClientDefaults, ArrayList *defaultSources) |
Generates and returns a NEW ALLOCATED default configuration (DMTClientConfig). | |
char * | loadTestFile (const char *testName, const char *fileName, bool binary) |
Loads and returns (NEW ALLOCATED) the content of a test file. | |
StringBuffer | getTestFileFullPath (const char *testName, const char *fileName) |
Returns the full path of a given test file (concat "testcases/testName/fileName"). | |
StringBuffer | getTestDirFullPath (const char *testName) |
Returns the full path of a given test file (concat "testcases/testName/") with the final slash at the end. | |
virtual void | LocalTests::addTests () |
adds the supported tests to the instance itself; this is the function that a derived class can override to add additional tests | |
virtual std::string | LocalTests::insert (CreateSource createSource, const char *data, bool relaxed=false) |
opens source and inserts the given item; can be called regardless whether the data source already contains items or not | |
virtual void | LocalTests::update (CreateSource createSource, const char *data, bool check=true) |
assumes that exactly one element is currently inserted and updates it with the given item | |
virtual void | LocalTests::deleteAll (CreateSource createSource) |
deletes all items locally via sync source | |
virtual void | LocalTests::compareDatabases (const char *refFile, SyncSource ©, bool raiseAssert=true) |
takes two databases, exports them, then compares them using synccompare | |
virtual int | LocalTests::insertManyItems (CreateSource createSource, int startIndex=1, int numItems=0, int size=-1) |
insert artificial items, number of them determined by TEST_EVOLUTION_NUM_ITEMS unless passed explicitly | |
virtual void | LocalTests::testOpen () |
virtual void | LocalTests::testIterateTwice () |
virtual void | LocalTests::testSimpleInsert () |
virtual void | LocalTests::testLocalDeleteAll () |
virtual void | LocalTests::testComplexInsert () |
virtual void | LocalTests::testLocalUpdate () |
virtual void | LocalTests::testChanges () |
virtual void | LocalTests::testImport () |
virtual void | LocalTests::testImportDelete () |
virtual void | LocalTests::testManyChanges () |
virtual void | LocalTests::testLinkedItemsParent () |
virtual void | LocalTests::testLinkedItemsChild () |
virtual void | LocalTests::testLinkedItemsParentChild () |
virtual void | LocalTests::testLinkedItemsChildParent () |
virtual void | LocalTests::testLinkedItemsChildChangesParent () |
virtual void | LocalTests::testLinkedItemsRemoveParentFirst () |
virtual void | LocalTests::testLinkedItemsRemoveNormal () |
virtual void | LocalTests::testLinkedItemsInsertParentTwice () |
virtual void | LocalTests::testLinkedItemsInsertChildTwice () |
virtual void | LocalTests::testLinkedItemsParentUpdate () |
virtual void | LocalTests::testLinkedItemsUpdateChild () |
virtual void | LocalTests::testLinkedItemsInsertBothUpdateChild () |
virtual void | LocalTests::testLinkedItemsInsertBothUpdateParent () |
SyncTests::SyncTests (const std::string &name, ClientTest &cl, std::vector< int > sourceIndices, bool isClientA=true) | |
virtual void | SyncTests::addTests () |
adds the supported tests to the instance itself | |
virtual void | SyncTests::compareDatabases () |
compare databases of first and second client | |
virtual void | SyncTests::deleteAll (DeleteAllMode mode=DELETE_ALL_SYNC) |
deletes all items locally and on server | |
virtual void | SyncTests::doCopy () |
get both clients in sync with empty server, then copy one item from client A to B | |
virtual void | SyncTests::refreshClient () |
replicate server database locally: same as SYNC_REFRESH_FROM_SERVER, but done with explicit local delete and then a SYNC_SLOW because some servers do no support SYNC_REFRESH_FROM_SERVER | |
virtual void | SyncTests::testDeleteAllRefresh () |
virtual void | SyncTests::testRefreshSemantic () |
virtual void | SyncTests::testRefreshStatus () |
virtual void | SyncTests::testUpdate () |
virtual void | SyncTests::testComplexUpdate () |
virtual void | SyncTests::testDelete () |
virtual void | SyncTests::testMerge () |
virtual void | SyncTests::testTwinning () |
virtual void | SyncTests::testOneWayFromServer () |
virtual void | SyncTests::testOneWayFromClient () |
virtual void | SyncTests::testItems () |
virtual void | SyncTests::testAddUpdate () |
virtual void | SyncTests::testManyItems () |
virtual void | SyncTests::doVarSizes (bool withMaxMsgSize, bool withLargeObject, const char *encoding) |
implements testMaxMsg(), testLargeObject(), testLargeObjectEncoded() using a sequence of items with varying sizes | |
virtual void | SyncTests::sync (SyncMode syncMode, const std::string &logprefix="", CheckSyncReport checkReport=CheckSyncReport(), long maxMsgSize=0, long maxObjSize=0, bool loSupport=false, const char *encoding="") |
executes a sync with the given options, checks the result and (optionally) the sync report | |
virtual void | ClientTest::registerTests () |
This function registers tests using this instance of ClientTest for later use during a test run. | |
ClientTest::ClientTest (int serverSleepSec=0, const std::string &serverLog="") | |
virtual LocalTests * | ClientTest::createLocalTests (const std::string &name, int sourceParam, ClientTest::Config &co) |
Creates an instance of LocalTests (default implementation) or a class derived from it. | |
virtual SyncTests * | ClientTest::createSyncTests (const std::string &name, std::vector< int > sourceIndices, bool isClientA=true) |
Creates an instance of SyncTests (default) or a class derived from it. | |
static int | ClientTest::dump (ClientTest &client, SyncSource &source, const char *file) |
utility function for dumping items which are C strings with blank lines as separator | |
static int | ClientTest::import (ClientTest &client, SyncSource &source, const char *file) |
utility function for importing items with blank lines as separator | |
static bool | ClientTest::compare (ClientTest &client, const char *fileA, const char *fileB) |
utility function for comparing vCard and iCal files with the external synccompare.pl Perl script | |
virtual void | ClientTest::postSync (int res, const std::string &logname) |
This is called after successful sync() calls (res == 0) as well as after unsuccessful ones (res != 1). | |
static void | ClientTest::getTestData (const char *type, Config &config) |
A derived class can use this call to get default test cases, but still has to add callbacks which create sources and execute a sync session. | |
virtual void | CheckSyncReport::check (int res, SyncReport &report) const |
checks that the sync completed as expected and throws CPPUnit exceptions if something is wrong | |
TestFileSource::TestFileSource (const std::string &id) | |
virtual void | TestFileSource::getSourceConfig (int source, Config &config) |
Called to fill the given test source config with information about a sync source identified by its index. | |
virtual int | TestFileSource::sync (const int *activeSources, SyncMode syncMode, const CheckSyncReport &checkReport, long maxMsgSize, long maxObjSize, bool loSupport, const char *encoding=0) |
Execute a synchronization with the selected sync sources and the selected synchronization options. | |
SyncSource * | TestFileSource::createSource (int source, const char *trackingSuffix) |
Variables | |
ClientListener | syncListener |
It fulfills several different goals:
Some of the design goals were:
The test runner "client-test" is implemented in client-test-main.cpp. It automatically assembles all CPPUnit tests it was linked against and runs them all if started without parameters. Output is ASCII. The return code indicates success or failure of any test. "-h|--help" print some usage information and a full list of all available tests. Tests or test groups as printed in that list can be given as command line parameters to run just these tests. Because tests do not communicate directly with the command line front-end, they must be passed parameters via environment variables. See the description of TestFileSource for further information about running those end-to-end synchronization tests.
The test runner itself also understands some environment variables: CLIENT_TEST_FAILURES can be set to a comma separated list of tests which are allowed to fail without affecting the return code of the test runner.
On Linux the test runner supports setting a timeout which is triggered if a test runs for more than the number of seconds set in the environment variable CLIENT_TEST_ALARM. This relies on signals and therefore is disabled in compilations on Windows with an ifdef HAVE_SIGNALS_H.
Because of the dependency on CPPUnit the testing needs to be enabled explicitly. This can be done separately for tests embedded in the object files of the library (unit tests) and the end-to-end synchronization (integration tests) which can be used with release versions of the library.
On POSIX systems the configure --enable-integration-test and --enable-unit-tests switches enable the testing. "make check" automatically executes the unit tests.
On Windows the Visual Studio solution file build/win32/client-test.sln was prepared to build the "client-test" executable. The cppunit project is required when loading the client-test solution, so you need to retrieve it and put it under the "tools/cppunit" directory. The cppunit project can be checkout from: https://core.forge.funambol.org/svn/core/trunk/tools/cppunit
Unit tests are only found if they are pulled into the executable. On Linux this is done by searching for variables with a special name, created by the FUNAMBOL_TEST_SUITE_REGISTRATION() macro in test.h, and setting link flags on the fly. On Windows an additional C file with references to these variables is used instead. This is done by a Python script, thus python.exe must be installed and found in the search path of Visual Studio. Any suggestion how to avoid the dependency on external scripting tools without having to manually register tests in a central file is welcome...
The integration tests depend on auxiliary files (test/synccompare.pl and test/testcases) which must be found in the directory where the test is run. On Linux the makefile rules automatically copy these, but the Visual Studio project file does not. Note that test/synccompare.pl is written in Perl (and thus creates another external dependency) because a) C/C++ do not have a standard library to implement the complex regular pattern matching/replace done in that script and b) the script is also meant to be shipped with clients (SyncEvolution does that) and thus needs to run on as many systems as possible. Perl is still the most universally available scripting language for that purpose.
Adding new unit tests is best done by following the example in src/c++/common/base/util/BasicTime.cpp.
Regular testing of a server can be done by using the reference data provided together with ClientTest and synchronizing the server against simple files on the client side via a TestFileSource. This is what enabling the integration tests adds to the "client-test" binary. By keeping the client side fixed regressions in the server can be detected. Regular testing of the client library can be done by testing the current client library source against stable servers.
For testing a client based on the C++ library the client developer needs to implement the ClientTest interface. There are some additional requirements for the client's SyncSource implementations:
The TestFileSource is an example how the ClientTest interface could be implemented.
void LocalTests::addTests | ( | ) | [virtual, inherited] |
adds the supported tests to the instance itself; this is the function that a derived class can override to add additional tests
adds the supported tests to the instance itself
References ClientTest::Config::childItem, ClientTest::Config::compare, LocalTests::config, ClientTest::Config::createSourceA, ClientTest::Config::createSourceB, ClientTest::Config::dump, ClientTest::Config::import, ClientTest::Config::insertItem, ClientTest::Config::parentItem, ClientTest::Config::templateItem, ClientTest::Config::testcases, ClientTest::Config::uniqueProperties, and ClientTest::Config::updateItem.
void CheckSyncReport::check | ( | int | res, | |
SyncReport & | report | |||
) | const [virtual, inherited] |
checks that the sync completed as expected and throws CPPUnit exceptions if something is wrong
res | return code from SyncClient::sync() | |
report | the sync report stored in the SyncClient |
References CLIENT, SyncSourceReport::getItemReportFailedCount(), SyncSourceReport::getItemReportSuccessfulCount(), SyncSourceReport::getSourceName(), SyncSourceReport::getState(), SyncReport::getSyncSourceReport(), SOURCE_INACTIVE, and SyncReport::toString().
Referenced by TestFileSource::sync().
void LocalTests::compareDatabases | ( | const char * | refFile, | |
SyncSource & | copy, | |||
bool | raiseAssert = true | |||
) | [virtual, inherited] |
takes two databases, exports them, then compares them using synccompare
refFile | existing file with source reference items, NULL uses a dump of sync source A instead | |
copy | a sync source which contains the copied items, begin/endSync will be called | |
raiseAssert | raise assertion if comparison yields differences (defaults to true) |
References SyncSource::beginSync(), LocalTests::client, ClientTest::Config::compare, LocalTests::config, LocalTests::createSourceA, ClientTest::Config::dump, and SyncSource::endSync().
int countItemsOfType | ( | SyncSource * | source, | |
itemType | type | |||
) |
utility function which counts items of a certain kind known to the sync source
source | valid source ready to iterate; NULL triggers an assert | |
itemType | determines which iterator functions are used |
References listItemsOfType().
LocalTests * ClientTest::createLocalTests | ( | const std::string & | name, | |
int | sourceParam, | |||
ClientTest::Config & | co | |||
) | [virtual, inherited] |
Creates an instance of LocalTests (default implementation) or a class derived from it.
LocalTests provides tests which cover the SyncSource interface and can be executed without a SyncML server. It also contains utility functions for working with SyncSources.
A ClientTest implementation can, but doesn't have to extend these tests by instantiating a derived class here.
SyncTests * ClientTest::createSyncTests | ( | const std::string & | name, | |
std::vector< int > | sourceIndices, | |||
bool | isClientA = true | |||
) | [virtual, inherited] |
Creates an instance of SyncTests (default) or a class derived from it.
SyncTests provides tests which cover the actual interaction with a SyncML server.
A ClientTest implementation can, but doesn't have to extend these tests by instantiating a derived class here.
DMTClientConfig* getNewDMTClientConfig | ( | const char * | testName, | |
const bool | setClientDefaults, | |||
ArrayList * | defaultSources = NULL | |||
) |
Generates and returns a NEW ALLOCATED default configuration (DMTClientConfig).
testName | the name of the test which called this method (for deviceID) | |
setClientDefaults | if true, will set the default deviceConfig and accessConfig | |
defaultSources | ArrayList of source names (StringBuffer), to create default SyncSourceConfig. If not specified, non is created. Default is NULL. |
References initAdapter(), and setDefaults().
SyncManagerConfig* getNewSyncManagerConfig | ( | const char * | testName, | |
const bool | setClientDefaults, | |||
ArrayList * | defaultSources = NULL | |||
) |
Generates and returns a NEW ALLOCATED default configuration (SyncManagerConfig).
testName | the name of the test which called this method (for deviceID) | |
setClientDefaults | if true, will set the default deviceConfig and accessConfig | |
defaultSources | ArrayList of source names (StringBuffer), to create default SyncSourceConfig. If not specified, non is created. Default is NULL. |
References initAdapter(), and setDefaults().
void TestFileSource::getSourceConfig | ( | int | source, | |
Config & | config | |||
) | [virtual, inherited] |
Called to fill the given test source config with information about a sync source identified by its index.
It's okay to only fill in the available pieces of information and set everything else to zero.
Implements ClientTest.
References ClientTest::Config::createSourceA, ClientTest::Config::createSourceB, getTestData(), and TestFileSource::sources.
Referenced by TestFileSource::TestFileSource().
void ClientTest::getTestData | ( | const char * | type, | |
Config & | config | |||
) | [static, inherited] |
A derived class can use this call to get default test cases, but still has to add callbacks which create sources and execute a sync session.
Some of the test cases are compiled into the library, other depend on the auxiliary files from the "test" directory. Currently supported types:
References ClientTest::Config::childItem, compare(), ClientTest::Config::compare, ClientTest::Config::complexUpdateItem, dump(), ClientTest::Config::dump, ClientTest::Config::import, ClientTest::Config::insertItem, ClientTest::Config::mergeItem1, ClientTest::Config::mergeItem2, ClientTest::Config::numItems, ClientTest::Config::parentItem, ClientTest::Config::sizeProperty, ClientTest::Config::sourceName, ClientTest::Config::templateItem, ClientTest::Config::testcases, ClientTest::Config::type, ClientTest::Config::uniqueProperties, ClientTest::Config::updateItem, and ClientTest::Config::uri.
Referenced by TestFileSource::getSourceConfig().
StringBuffer getTestDirFullPath | ( | const char * | testName | ) |
Returns the full path of a given test file (concat "testcases/testName/") with the final slash at the end.
testName | the test name |
StringBuffer getTestFileFullPath | ( | const char * | testName, | |
const char * | fileName | |||
) |
Returns the full path of a given test file (concat "testcases/testName/fileName").
testName | the test name | |
fileName | the file name |
Referenced by utilsTest::getFileModTimeTest(), and loadTestFile().
const StringBuffer initAdapter | ( | const char * | testName | ) |
Initializes the PlatformAdapter for a given test.
To avoid spamming the config folder, the root context is composed so that all the tests will work under "Funambol/client-test", then the test name.
Referenced by getNewDMTClientConfig(), getNewSyncManagerConfig(), and TestFileSource::TestFileSource().
std::string LocalTests::insert | ( | CreateSource | createSource, | |
const char * | data, | |||
bool | relaxed = false | |||
) | [virtual, inherited] |
opens source and inserts the given item; can be called regardless whether the data source already contains items or not
The type of the item is unset; it is assumed that the source can handle that.
relaxed | if true, then disable some of the additional checks after adding the item |
References LocalTests::source.
int LocalTests::insertManyItems | ( | CreateSource | createSource, | |
int | startIndex = 1 , |
|||
int | numItems = 0 , |
|||
int | size = -1 | |||
) | [virtual, inherited] |
insert artificial items, number of them determined by TEST_EVOLUTION_NUM_ITEMS unless passed explicitly
insert artificial items, number of them determined by config.numItems unless passed explicitly
createSource | a factory for the sync source that is to be used | |
startIndex | IDs are generated starting with this value | |
numItems | number of items to be inserted if non-null, otherwise TEST_EVOLUTION_NUM_ITEMS is used | |
size | minimum size for new items |
createSource | a factory for the sync source that is to be used | |
startIndex | IDs are generated starting with this value | |
numItems | number of items to be inserted if non-null, otherwise config.numItems is used | |
size | minimum size for new items |
References LocalTests::config, LocalTests::createSourceA, ClientTest::Config::numItems, ClientTest::Config::sizeProperty, ClientTest::Config::templateItem, and ClientTest::Config::uniqueProperties.
char* loadTestFile | ( | const char * | testName, | |
const char * | fileName, | |||
bool | binary = false | |||
) |
Loads and returns (NEW ALLOCATED) the content of a test file.
The test files are located under the dir "testcases/<test name>".
testName | the test name | |
fileName | the file name | |
binary | if true the file has to be opened in binary mode |
References getTestFileFullPath().
Referenced by utilsTest::getFileModTimeTest().
USE_NAMESPACE void ClientTest::postSync | ( | int | res, | |
const std::string & | logname | |||
) | [virtual, inherited] |
This is called after successful sync() calls (res == 0) as well as after unsuccessful ones (res != 1).
The default implementation sleeps for the number of seconds specified when constructing this instance and copies the server log if one was named.
res | result of sync() | |
logname | base name of the current sync log (without ".client.[AB].log" suffix) |
References serverLogFileName, and serverSleepSeconds.
Referenced by SyncTests::sync().
void ClientTest::registerTests | ( | ) | [virtual, inherited] |
This function registers tests using this instance of ClientTest for later use during a test run.
The instance must remain valid until after the tests were run. To run them use a separate test runner, like the one from client-test-main.cpp.
References factory.
static void setDefaults | ( | SyncManagerConfig * | config, | |
const char * | testName, | |||
const bool | setClientDefaults, | |||
ArrayList * | defaultSources = NULL | |||
) | [static] |
Sets the default values (from DefaultConfigFactory) in the passed SyncManagerConfig.
config | the SyncManagerConfig to fill | |
testName | the name of the test which called this method (for deviceID) | |
setClientDefaults | if true, will set the default deviceConfig and accessConfig | |
defaultSources | ArrayList of source names (StringBuffer), to create default SyncSourceConfig. If not specified, non is created. Default is NULL. |
References SyncManagerConfig::getAccessConfig(), SyncManagerConfig::getDeviceConfig(), SyncManagerConfig::setClientDefaults(), SyncManagerConfig::setSourceDefaults(), and SyncManagerConfig::setSyncSourceConfig().
Referenced by getNewDMTClientConfig(), and getNewSyncManagerConfig().
int TestFileSource::sync | ( | const int * | activeSources, | |
SyncMode | syncMode, | |||
const CheckSyncReport & | checkReport, | |||
long | maxMsgSize, | |||
long | maxObjSize, | |||
bool | loSupport, | |||
const char * | encoding = 0 | |||
) | [virtual, inherited] |
Execute a synchronization with the selected sync sources and the selected synchronization options.
The log file in LOG has been set up already for the synchronization run and should not be changed by the client.
activeSources | a -1 terminated array of sync source indices | |
syncMode | the synchronization mode to be used | |
checkReport | has to be called after a successful or unsuccessful sync, will dump the report and (optionally) check the result; beware, the later may throw exceptions inside CPPUNIT macros | |
maxMsgSize | >0: enable the maximum message size, else disable it | |
maxObjSize | same as maxMsgSize for maximum object size | |
loSupport | if true, then the sync is expected to enable Large Object support | |
encoding | if non-empty, then let client library transform all items into this format (guaranteed to be not NULL) |
Implements ClientTest.
References CheckSyncReport::check(), TestFileSource::config, SyncClient::getSyncReport(), SyncSourceConfig::setEncoding(), SyncSourceConfig::setSync(), TestFileSource::sources, and SyncClient::sync().
USE_NAMESPACE TestFileSource::TestFileSource | ( | const std::string & | id | ) | [inherited] |
void LocalTests::update | ( | CreateSource | createSource, | |
const char * | data, | |||
bool | check = true | |||
) | [virtual, inherited] |
assumes that exactly one element is currently inserted and updates it with the given item
The type of the item is cleared, as in insert() above.
check | if true, then reopen the source and verify that the reported items are as expected |
References CreateSource::createSource, and LocalTests::source.