When transaction's commit() starts, there is a certain point in time when other transactions can start to see changes in the database (if database does not allow dirty reads, otherwise they could see changes before that). So, exactly at this point in time the DataStruct and Query caches would be out of synchronization with the database.
Following the principle that the caches always have to show the same data as if it was accessed the database, it should be avoided that another transaction can read or modify (add entries to the DataStruct cache) until database commit is completed and DataStruct cache and Query caches re-synchronized with the database (DataStructs of modified DOs replaced, simple queries re-evaluated, complex and multi-join queries removed).
The cache re-synchronization can only happen after the successful commit, because there could be errors in the database during the commit and there should not be inconsistent cache after such a failure. And the commit() can take some time...
This problem with transaction's commit() is solved with the Global Cache (Wrapper) and the Negative lists in the DataStruct caches.
The Global cache is a Singleton. It contains and synchronizes all applications DataStruct caches. It has the following methods:
getInstance() - static method
Returns Wrapper object if exists, otherwise creates it and returns it.
registerCache(DataStructCache dc)
Registers (adds) QueryCacheImpl cache (implementation of all caches per table) to the Wrapper.
lock() - synchronized method
Returns 0 if the Wrapper is already locked. If not, locks all DataStruct caches (so that they can not be used) and query caches if needed, and returns time (in ms) when this lock expires. This way nobody can lock Wrapper indefinitely.
unlock() - synchronized method
Unlocks DataStruct caches (they can be used again) and query caches if needed.
The negative list is contained in every DataStruct cache. It blocks access (read and modify) of just some parts of the caches. It contains DataStructs that are in the cache, but at the moment can not be read from (and modified in) the cache because that DataStructs may not be consistent with the database. So, the DataStructs that are in the negative list are not visible for read and modify methods. Since more than one transaction can add DataStructs to the negative list, the list counts the number of times (for every DataStruct) the DataStruct was made invisible. When the counter becomes zero, the DataStruct object is removed from the negative list.
The transaction's commit() method uses the Wrapper and the negative lists of DataStructs caches in the following way:
makeQueryCachesInvisible()
Locks all query caches (simple, complex and multi-join) for all classes (tables) whose DOs are modified in transaction. QueryCaches are locked before commit to database until cache re-evaluation.
makeQueryCachesVisible()
Unlocks Query caches.
The negative list is also used for locking QueryCaches similarly to DataStructCache.
After the successful executed inserts, updates and deletes of DOs, the Wrapper must be locked, so that used DOs would be hidden for use (put in the negative lists in DataStructs caches). Due to a possible changes of DOs, the query caches are also hidden for use.
If the Wrapper is already locked, the method waits for CacheLockTimeout time, CacheLockRetries number of times (two parameters in the application's configuration file). If even after that number of tries with that amount of time the Wrapper stays locked, the method throws SQL Exception with the message suggesting that the method could not wait any more and the rollback() is performed.
If the method managed to lock the Wrapper, it makes invisible DOs that were changed in the DataStructs cache (it puts them in the DataStruct cache negative list) and makes invisible query caches. After that, the Wrapper is unlocked and the DataStruct caches can be used again (except some cache parts - invisible DataStructs that are in the negative lists can't be used).
Then, the transaction is committed.
If an exception occurred during the commit, the rollback() is performed and the used DOs are reloaded from the database.
If the database commit was successful, all objects are notified that the transaction succeeded and the changed DOs (DataStructs) and changed cached queries are written back to the global cache (changes are re-evaluated in queries).
No matter the commit was successful or not, the update of negative lists must be performed. For this, the locking of the Wrapper is again needed (DataStruct caches synchronization is needed).
If the Wrapper had been locked before, the method must wait until it becomes unlocked, and then locks it again, updates the negative lists (the DataStruct objects that were put to negative lists by this method are removed from there) and makes query caches again visible. After the update of the negative lists, the Wrapper is unlocked again.
When a transaction is committed, the DataStruct cache is re-synchronized with the database, in all DOs is attribute dirty set back to false, DataStruct objects are moved from data to originalData pointers, newly created, rows/DOs are set to "existing" (can be derived because originalData was null before).
New method DO.doLock() is added: a DO can get locked (even if no data is changed). This way a row that is not updated at all can still be ensured that will not be changed in the database till the commit (pessimistic locking). It gets executed against database immediately, with no regard for AutoWrite parameter.
New method DO.doTouch() is added: a DO can get locked (even if no data is changed). This way a row that is not updated at all can still be ensured that won't be changed in the database till the commit (pessimistic locking). It gets executed against database immediately, with no regard for AutoWrite parameter, and increments version.
New method DO.doCheck() is added: it marks a DO for locking just before the commit. This provides that this row will not be changed during the commit (optimistic locking). This type of locking is executed in commit() method and locks DOs which were marked (for locking) and modified in this transaction.