issue2550839: Xapian, DatabaseLockError: Unable to get write lock on

 db/text-index: already locked

I put in a retry loop that will attempt to get the lock. 10 tries with
a delaythatvaries from .02 seconds to .64 seconds. Total delay over 10
cycles approx. 4.5 seconds.

I can't figure out how to make pytest run two parallel processes to
test this code. I did try running:

./run_tests.py -k Xapian test/test_indexer.py &
./run_tests.py -k Xapian test/test_indexer.py

and confirmed that one of the processes seemed to hang on a test and
then threw a lock failure error. So at least the code path is being
exercised.

If anybody knows how to correctly test this I would love to do a real
test.

Author: rouilj

CHANGES.txt

@@ -277,6 +277,9 @@ Fixed:
   module added to ``doc/installation.txt``.
 - issue2550776: imapServer.py problem. Fixed a missing initialization of the
   logging level if no logging level option is supplied. (John Rouillard)
+- issue2550839: Xapian, DatabaseLockError: Unable to get write lock on
+  db/text-index: already locked. Put in a retry loop that will attempt
+  to get the lock. Total delay approx 4.5 seconds. (John Rouillard)
 
 2016-01-11: 1.5.1
 

roundup/backends/indexer_xapian.py

@@ -1,6 +1,6 @@
 ''' This implements the full-text indexer using the Xapian indexer.
 '''
-import re, os
+import re, os, time
 
 import xapian
 
@@ -17,7 +17,18 @@ def __init__(self, db):
 
     def _get_database(self):
         index = os.path.join(self.db_path, 'text-index')
-        return xapian.WritableDatabase(index, xapian.DB_CREATE_OR_OPEN)
+        for n in range(10):
+            try:
+                # if successful return
+                return xapian.WritableDatabase(index, xapian.DB_CREATE_OR_OPEN)
+            except xapian.DatabaseLockError:
+                # adaptive sleep. Get longer as count increases.
+                time_to_sleep = 0.01 * (2 << min(5, n))
+                time.sleep(time_to_sleep)
+                # we are back to the for loop
+
+        # Get here only if we dropped out of the for loop.
+        raise xapian.DatabaseLockError("Unable to get lock after 10 retries on %s."%index)
 
     def save_index(self):
         '''Save the changes to the index.'''