Skip to content

Commit 554c110

Browse files
author
Richard Jones
committed
fixed export/import of retired nodes [SF#685273]
1 parent eee2592 commit 554c110

File tree

6 files changed

+132
-49
lines changed

6 files changed

+132
-49
lines changed

CHANGES.txt

Lines changed: 29 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -2,34 +2,15 @@ This file contains the changes to the Roundup system over time. The entries
22
are given with the most recent entry first.
33

44
2003-??-?? 0.6.0
5+
Fixed:
56
- better hyperlinking in web message texts (sf bug 669777)
6-
- support setting of properties on message and file through web and
7-
email interface (thanks John Rouillard)
8-
- allow additional control over the roundupdb email sending (explicit
9-
cc addresses, different from address and different nosy list property)
10-
(thanks John Rouillard)
11-
- applied patch for nicer history display (sf feature 638280)
12-
- cleaning old unused sessions only once per hour, not on every cgi
13-
request. It is greatly improves web interface performance, especially
14-
on trackers under high load
15-
- added mysql backend (see doc/mysql.txt for details)
16-
- fixes to CGI form handling
17-
- switch metakit to use "compressed" multilink journal change representation
187
- fixed bug in metakit unlink journalling
19-
- metakit now handles "unset" for most types (not Number and Boolean)
20-
- fixed bug in metakit search-by-ID
218
- applied unicode patch. All data is stored in utf-8. Incoming messages
229
converted from any encoding to utf-8, outgoing messages are encoded
2310
according to rfc2822 (sf bug 568873)
2411
- fixed cookie path to use TRACKER_WEB (sf bug 667020) (thanks Nathaniel Smith
2512
for helping chase it down and Luke Opperman for confirming fix)
26-
- added ability to display localized dates in web interface. User input is
27-
convered to GMT (see doc/upgrading.txt).
28-
- added a form to show a specific issue
2913
- fixed layout issues with forms in sidebar
30-
- more proper sorting/grouping on mulitilink properties. Sorting is performed
31-
not only by number of links, but also by links itself. This makes usable
32-
grouping e.g. by topic multilink
3314
- fixed templating filter function arguments (sf bug 678911)
3415
- fixed multiselect in searching (sf bug 676874)
3516
- fixed parsing of content-disposition filenames (sf bug 675116)
@@ -42,12 +23,36 @@ are given with the most recent entry first.
4223
- added warning filter for "FutureWarning: hex/oct constants > sys.maxint will
4324
return positive values..." (literal 0xffff0000 in portalocker.py)
4425
- fixed ZPT code generating SyntaxWarning for assignment to None
45-
- add "ago" to intervals in the past (sf bug 679232)
46-
- clarified licensing
47-
- another attempt to fix cookie misbehaviour - customise cookie name using
48-
tracker name
4926
- fixed error in indexargs_url (thanks Patrick Ohly)
5027
- fixed getnode (sf bug 684531)
28+
- open static files using binary mode (sf bug 693208)
29+
- fixed deja-vu bug 692910
30+
- don't display "Editing" on read-only pages (sf bug 651967)
31+
- re-worked detectors initialisation - woohoo, no more cross-importing!
32+
- fixed export/import of retired nodes (sf bug 685273)
33+
34+
Feature:
35+
- support setting of properties on message and file through web and
36+
email interface (thanks John Rouillard)
37+
- allow additional control over the roundupdb email sending (explicit
38+
cc addresses, different from address and different nosy list property)
39+
(thanks John Rouillard)
40+
- applied patch for nicer history display (sf feature 638280)
41+
- cleaning old unused sessions only once per hour, not on every cgi
42+
request. It is greatly improves web interface performance, especially
43+
on trackers under high load
44+
- added mysql backend (see doc/mysql.txt for details)
45+
- switch metakit to use "compressed" multilink journal change representation
46+
- metakit now handles "unset" for most types (not Number and Boolean)
47+
- fixed bug in metakit search-by-ID
48+
- added ability to display localized dates in web interface. User input is
49+
convered to GMT (see doc/upgrading.txt).
50+
- added a form to show a specific issue
51+
- more proper sorting/grouping on mulitilink properties. Sorting is performed
52+
not only by number of links, but also by links itself. This makes usable
53+
grouping e.g. by topic multilink
54+
- add "ago" to intervals in the past (sf bug 679232)
55+
- clarified licensing
5156
- included UN*X manual pages from Bastian Kleineidam
5257
- implemented extension to form parsing to allow editing of multiple items
5358
and creation of multiple items (but only one per class)
@@ -56,17 +61,13 @@ are given with the most recent entry first.
5661
(e.g. images). They are accessible naturally: _file/images/img.gif
5762
- altered Class.create() and FileClass.create() methods to make "content"
5863
property available in auditors
59-
- re-worked detectors initialisation - woohoo, no more cross-importing!
6064
- can now configure CC to author only for messages creating issues (sf
6165
feature 625808)
6266
- registration is now a two-step process, with confirmation from the email
6367
address supplied in the registration form
6468
- added support for last-modified and if-modified-since headers for static
6569
file serving
6670
- added Node.get() method
67-
- open static files using binary mode (sf bug 693208)
68-
- fixed deja-vu bug 692910
69-
- don't display "Editing" on read-only pages (sf bug 651967)
7071

7172

7273
2003-??-?? 0.5.6

TODO.txt

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,17 +7,12 @@ done, it's moved to the CHANGES file.
77
State Component Description
88
======= ========= ============================================================
99
pending example meta/parent bug implementation (feature request #506815)
10-
pending example replace the "extended" example with a "help desk" one, and
11-
rename "classic" to "bug tracker"
1210
pending example script for retrieval of "mbox" archive of all messages
1311
pending hyperdb range searching of values (dates in particular).
1412
Filter specifies {property: (comparison function, value)}
1513
comparison functions: lt, le, eq, ge, gt. eq and
1614
[value, value, ...] implies "in"
1715
pending hyperdb migrate "id" property to be Number type
18-
pending hyperdb multilink sorting by length is dumb
19-
pending hyperdb lastchangedby auto-property giving last user to change an
20-
item
2116
pending tracker split instance.open() into open() and login()
2217
pending mailgw allow commands (feature request #556996)
2318
like "help", "dump issue123" (would send all info about

roundup/admin.py

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616
# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
1717
# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
1818
#
19-
# $Id: admin.py,v 1.38 2003-02-25 10:19:31 richard Exp $
19+
# $Id: admin.py,v 1.39 2003-02-26 23:42:49 richard Exp $
2020

2121
'''Administration commands for maintaining Roundup trackers.
2222
'''
@@ -922,8 +922,11 @@ def do_export(self, args):
922922
propnames.sort()
923923
print >> f, p.join(propnames)
924924

925-
# all nodes for this class
926-
for nodeid in cl.list():
925+
# all nodes for this class (not using list() 'cos it doesn't
926+
# include retired nodes)
927+
928+
for nodeid in self.db.getnodeids(classname):
929+
# get the regular props
927930
print >>f, p.join(cl.export_list(propnames, nodeid))
928931
return 0
929932

roundup/backends/back_anydbm.py

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
1616
# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
1717
#
18-
#$Id: back_anydbm.py,v 1.105 2003-02-25 10:19:31 richard Exp $
18+
#$Id: back_anydbm.py,v 1.106 2003-02-26 23:42:50 richard Exp $
1919
'''
2020
This module defines a backend that saves the hyperdatabase in a database
2121
chosen by anydbm. It is guaranteed to always be available in python
@@ -922,6 +922,10 @@ def export_list(self, propnames, nodeid):
922922
elif isinstance(proptype, hyperdb.Password):
923923
value = str(value)
924924
l.append(repr(value))
925+
926+
# append retired flag
927+
l.append(self.is_retired(nodeid))
928+
925929
return l
926930

927931
def import_list(self, propnames, proplist):
@@ -963,6 +967,10 @@ def import_list(self, propnames, proplist):
963967
value = pwd
964968
d[propname] = value
965969

970+
# check retired flag
971+
if int(proplist[-1]):
972+
d[self.db.RETIRED_FLAG] = 1
973+
966974
# add the node and journal
967975
self.db.addnode(self.classname, newid, d)
968976

@@ -1325,10 +1333,10 @@ def retire(self, nodeid):
13251333

13261334
self.fireReactors('retire', nodeid, None)
13271335

1328-
def is_retired(self, nodeid):
1336+
def is_retired(self, nodeid, cldb=None):
13291337
'''Return true if the node is retired.
13301338
'''
1331-
node = self.db.getnode(cn, nodeid, cldb)
1339+
node = self.db.getnode(self.classname, nodeid, cldb)
13321340
if node.has_key(self.db.RETIRED_FLAG):
13331341
return 1
13341342
return 0

roundup/backends/back_metakit.py

Lines changed: 74 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,8 @@
3838
_dbs = {}
3939

4040
def Database(config, journaltag=None):
41+
''' Only have a single instance of the Database class for each instance
42+
'''
4143
db = _dbs.get(config.DATABASE, None)
4244
if db is None or db._db is None:
4345
db = _Database(config, journaltag)
@@ -81,8 +83,10 @@ def __getattr__(self, classname):
8183
if self.journaltag is None:
8284
return None
8385

86+
# try to set the curuserid from the journaltag
8487
try:
85-
self.curuserid = x = int(self.classes['user'].lookup(self.journaltag))
88+
x = int(self.classes['user'].lookup(self.journaltag))
89+
self.curuserid = x
8690
except KeyError:
8791
if self.journaltag == 'admin':
8892
self.curuserid = x = 1
@@ -91,6 +95,7 @@ def __getattr__(self, classname):
9195
return x
9296
elif classname == 'transactions':
9397
return self.dirty
98+
# fall back on the classes
9499
return self.getclass(classname)
95100
def getclass(self, classname):
96101
try:
@@ -100,6 +105,7 @@ def getclass(self, classname):
100105
def getclasses(self):
101106
return self.classes.keys()
102107
# --- end of ping's spec
108+
103109
# --- exposed methods
104110
def commit(self):
105111
if self.dirty:
@@ -182,9 +188,10 @@ def getjournal(self, tablenm, nodeid):
182188
#usernm = userclass.get(str(row.user), 'username')
183189
dt = date.Date(time.gmtime(row.date))
184190
#rslt.append((nodeid, dt, usernm, _actionnames[row.action], params))
185-
rslt.append((nodeid, dt, str(row.user), _actionnames[row.action], params))
191+
rslt.append((nodeid, dt, str(row.user), _actionnames[row.action],
192+
params))
186193
return rslt
187-
194+
188195
def destroyjournal(self, tablenm, nodeid):
189196
nodeid = int(nodeid)
190197
tblid = self.tables.find(name=tablenm)
@@ -215,13 +222,22 @@ def close(self):
215222

216223
# --- internal
217224
def __open(self):
225+
''' Open the metakit database
226+
'''
227+
# make the database dir if it doesn't exist
218228
if not os.path.exists(self.config.DATABASE):
219229
os.makedirs(self.config.DATABASE)
230+
231+
# figure the file names
220232
self.dbnm = db = os.path.join(self.config.DATABASE, 'tracker.mk4')
221233
lockfilenm = db[:-3]+'lck'
234+
235+
# get the database lock
222236
self.lockfile = locking.acquire_lock(lockfilenm)
223237
self.lockfile.write(str(os.getpid()))
224238
self.lockfile.flush()
239+
240+
# see if the schema has changed since last db access
225241
self.fastopen = 0
226242
if os.path.exists(db):
227243
dbtm = os.path.getmtime(db)
@@ -236,18 +252,28 @@ def __open(self):
236252
else:
237253
# can't find schemamod - must be frozen
238254
self.fastopen = 1
255+
256+
# open the db
239257
db = metakit.storage(db, 1)
240258
hist = db.view('history')
241259
tables = db.view('tables')
242260
if not self.fastopen:
261+
# create the database if it's brand new
243262
if not hist.structure():
244263
hist = db.getas('history[tableid:I,nodeid:I,date:I,user:I,action:I,params:B]')
245264
if not tables.structure():
246265
tables = db.getas('tables[name:S]')
247266
db.commit()
267+
268+
# we now have an open, initialised database
248269
self.tables = tables
249270
self.hist = hist
250271
return db
272+
273+
def setid(self, classname, maxid):
274+
''' No-op in metakit
275+
'''
276+
pass
251277

252278
_STRINGTYPE = type('')
253279
_LISTTYPE = type([])
@@ -632,10 +658,12 @@ def retire(self, nodeid):
632658
ndx = view.find(id=int(nodeid))
633659
if ndx < 0:
634660
raise KeyError, "nodeid %s not found" % nodeid
661+
635662
row = view[ndx]
636663
oldvalues = self.uncommitted.setdefault(row.id, {})
637664
oldval = oldvalues['_isdel'] = row._isdel
638665
row._isdel = 1
666+
639667
if self.do_journal:
640668
self.db.addjournal(self.classname, nodeid, _RETIRE, {})
641669
if self.keyname:
@@ -646,6 +674,16 @@ def retire(self, nodeid):
646674
self.db.dirty = 1
647675
self.fireReactors('retire', nodeid, None)
648676

677+
def is_retired(self, nodeid):
678+
view = self.getview(1)
679+
# node must exist & not be retired
680+
id = int(nodeid)
681+
ndx = view.find(id=id)
682+
if ndx < 0:
683+
raise IndexError, "%s has no node %s" % (self.classname, nodeid)
684+
row = view[ndx]
685+
return row._isdel
686+
649687
def history(self, nodeid):
650688
if not self.do_journal:
651689
raise ValueError, 'Journalling is disabled for this class'
@@ -796,6 +834,7 @@ def addprop(self, **properties):
796834
raise ValueError, "%s is already a property of %s"%(key,
797835
self.classname)
798836
self.ruprops.update(properties)
837+
# Class structure has changed
799838
self.db.fastopen = 0
800839
view = self.__getview()
801840
self.db.commit()
@@ -1037,6 +1076,10 @@ def export_list(self, propnames, nodeid):
10371076
elif isinstance(proptype, hyperdb.Password):
10381077
value = str(value)
10391078
l.append(repr(value))
1079+
1080+
# append retired flag
1081+
l.append(self.is_retired(nodeid))
1082+
10401083
return l
10411084

10421085
def import_list(self, propnames, proplist):
@@ -1064,11 +1107,24 @@ def import_list(self, propnames, proplist):
10641107
value = int(calendar.timegm(value))
10651108
elif isinstance(prop, hyperdb.Interval):
10661109
value = str(date.Interval(value))
1110+
elif isinstance(prop, hyperdb.Number):
1111+
value = int(value)
1112+
elif isinstance(prop, hyperdb.Boolean):
1113+
value = int(value)
1114+
elif isinstance(prop, hyperdb.Link):
1115+
value = int(value)
1116+
elif isinstance(prop, hyperdb.Multilink):
1117+
value = map(int, value)
10671118
d[propname] = value
1068-
view.append(d)
1069-
creator = d.get('creator', None)
1070-
creation = d.get('creation', None)
1071-
self.db.addjournal(self.classname, newid, 'create', {}, creator,
1119+
# is the item retired?
1120+
if int(proplist[-1]):
1121+
d['_isdel'] = 1
1122+
# XXX this is BROKEN for reasons I don't understand!
1123+
ndx = view.append(d)
1124+
1125+
creator = d.get('creator', 0)
1126+
creation = d.get('creation', 0)
1127+
self.db.addjournal(self.classname, newid, _CREATE, {}, creator,
10721128
creation)
10731129
return newid
10741130

@@ -1101,11 +1157,20 @@ def rollbackaction(self, action):
11011157
self.rbactions.append(action)
11021158
# --- internal
11031159
def __getview(self):
1160+
''' Find the interface for a specific Class in the hyperdb.
1161+
1162+
This method checks to see whether the schema has changed and
1163+
re-works the underlying metakit structure if it has.
1164+
'''
11041165
db = self.db._db
11051166
view = db.view(self.classname)
11061167
mkprops = view.structure()
1168+
1169+
# if we have structure in the database, and the structure hasn't
1170+
# changed
11071171
if mkprops and self.db.fastopen:
11081172
return view.ordered(1)
1173+
11091174
# is the definition the same?
11101175
for nm, rutyp in self.ruprops.items():
11111176
for mkprop in mkprops:
@@ -1136,7 +1201,7 @@ def getview(self, RW=0):
11361201
return self.db._db.view(self.classname).ordered(1)
11371202
def getindexview(self, RW=0):
11381203
return self.db._db.view("_%s" % self.classname).ordered(1)
1139-
1204+
11401205
def _fetchML(sv):
11411206
l = []
11421207
for row in sv:
@@ -1155,7 +1220,7 @@ def _fetchPW(s):
11551220
return p
11561221

11571222
def _fetchLink(n):
1158-
''' Return None if the string is empty ?otherwise ensure it's a string?
1223+
''' Return None if the link is 0 - otherwise strify it.
11591224
'''
11601225
return n and str(n) or None
11611226

0 commit comments

Comments
 (0)