Wrote most of the upgrading documentation (please read!)

Removed the hard coded template path
Removed unused config vars from instance_config

Author: Richard Jones

doc/upgrading.txt

@@ -10,18 +10,340 @@ accordingly.
 Migrating from 0.4.x to 0.5.0
 =============================
 
+This has been a fairly major revision of Roundup:
+
+1. Brand new, much more powerful, flexible, tasty and nutritious templating.
+   Unfortunately, this means all your current templates are useless. Please
+   don't hesitate to ask on roundup-users for help (or complete conversions
+   if you're completely stuck)!
+2. The database backed got a lot more flexible, allowing Metakit and SQL
+   databases! The only SQL database implemented at present is gadfly, but
+   others shouldn't be a whole lot more work.
+3. A brand new, highly flexible and much more robust security system including
+   a system of Permissions, Roles and Role assignments to users. You may now
+   define your own Permissions that may be checked in CGI transactions.
+4. Journalling has been made less storage-hungry, so has been turned on
+   by default *except* for author, recipient and nosy link/unlink events. You
+   are advised to turn it off in your instances too.
+5. Because of the above changes, the instance configuration has seen some
+   major changes. See below for the details.
+
+Please, *back up your database* before you start the migration process. This
+is as simple as copying the "db" directory and all its contents from your
+instance to somewhere safe.
+
+
 0.5.0 Configuration
 -------------------
 
-TODO: mention stuff about indexing
-TODO: mention that the dbinit needs the db.post_init() method call for
-reindexing
-TODO: dbinit now imports classes from selct_db
-TODO: select_db needs fixing to include Class, FileClass and IssueClass
-TODO: migration of security settings
-TODO: nosy reactor has been updated
-TODO: user.item template html needs updating for new security
-TODO: maybe something about export/import?
+Due to the new templating having a top-level ``page`` that defines links for
+searching, indexes, adding items etc, the following variables are no longer
+used:
+
+- HEADER_INDEX_LINKS
+- HEADER_ADD_LINKS
+- HEADER_SEARCH_LINKS
+- SEARCH_FILTERS
+- DEFAULT_INDEX
+- UNASSIGNED_INDEX
+- USER_INDEX
+- ISSUE_FILTER
+
+The new security implementation will require additions to the dbinit module,
+but also removes the need for the following instance config variables:
+
+- ANONYMOUS_ACCESS
+- ANONYMOUS_REGISTER
+
+but requires two new variables which define the Roles assigned to users who
+register through the web and e-mail interfaces:
+
+- NEW_WEB_USER_ROLES
+- NEW_EMAIL_USER_ROLES
+
+in both cases, 'User' is a good initial setting. To emulate
+``ANONYMOUS_ACCESS='deny'``, remove all "View" Permissions from the
+"Anonymous" Role. To emulate ``ANONYMOUS_REGISTER='deny'``, remove the "Web
+Registration" and/or the "Email Registration" Permission from the "Anonymous"
+Role. See the section on customising security in the `customisation
+documentation`_ for more information.
+
+
+0.5.0 Schema Specification
+--------------------------
+
+0.5.0 Database backend changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Your select_db module in your instance has changed a fair bit. Where it used
+to contain::
+
+ # WARNING: DO NOT EDIT THIS FILE!!!
+ from roundup.backends.back_anydbm import Database
+
+it must now contain::
+
+ # WARNING: DO NOT EDIT THIS FILE!!!
+ from roundup.backends.back_anydbm import Database, Class, FileClass, IssueClass
+
+Note the addition of the Class, FileClass, IssueClass imports. These are very
+important, as they're going to make the next change work too. You now need to
+modify the top of the dbinit module in your instance from::
+
+ import instance_config
+ from roundup import roundupdb
+ from select_db import Database
+
+ from roundup.roundupdb import Class, FileClass
+
+ class Database(roundupdb.Database, select_db.Database):
+     ''' Creates a hybrid database from:
+          . the selected database back-end from select_db
+          . the roundup extensions from roundupdb
+     '''
+     pass
+
+ class IssueClass(roundupdb.IssueClass):
+     ''' issues need the email information
+     '''
+     pass
+
+
+
+to just::
+
+ import instance_config
+ from select_db import Database, Class, FileClass, IssueClass
+
+Yes, remove the Database and IssueClass definitions and those other imports.
+They're not needed any more!
+
+
+0.5.0 Journalling changes
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Journalling has been optimised for storage. Journalling of links has been
+turned back on by default. If your tracker has a large user base, you may wish
+to turn off journalling of nosy list, message author and message recipient
+link and unlink events. You do this by adding ``do_journal='no'`` to the Class
+initialisation in your dbinit. For example, your *msg* class initialisation
+probably looks like this::
+
+    msg = FileClass(db, "msg",
+                    author=Link("user"), recipients=Multilink("user"),
+                    date=Date(),         summary=String(),
+                    files=Multilink("file"),
+                    messageid=String(),  inreplyto=String())
+
+to turn off journalling of author and recipient link events, add
+``do_journal='no'`` to the ``author=Link("user")`` part of the statement,
+like so::
+
+    msg = FileClass(db, "msg",
+                    author=Link("user", do_journal='no'),
+                    recipients=Multilink("user", do_journal='no'),
+                    date=Date(),         summary=String(),
+                    files=Multilink("file"),
+                    messageid=String(),  inreplyto=String())
+
+Nosy list link event journalling is actually turned off by default now. If you
+want to turn it onn, change to your issue class' nosy list, change its
+definition from::
+
+    issue = IssueClass(db, "issue",
+                    assignedto=Link("user"), topic=Multilink("keyword"),
+                    priority=Link("priority"), status=Link("status"))
+
+to::
+
+    issue = IssueClass(db, "issue", nosy=Multilink("user", do_journal='yes'),
+                    assignedto=Link("user"), topic=Multilink("keyword"),
+                    priority=Link("priority"), status=Link("status"))
+
+noting that your definition of the nosy Multilink will override the normal one.
+
+0.5.0 User schema changes
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Users have two more properties, "queries" and "roles". You'll have something
+like this in your dbinit module now::
+
+    user = Class(db, "user",
+                    username=String(),   password=Password(),
+                    address=String(),    realname=String(),
+                    phone=String(),      organisation=String(),
+                    alternate_addresses=String())
+    user.setkey("username")
+
+and you'll need to add the new properties and the new "query" class to it
+like so::
+
+    query = Class(db, "query",
+                    klass=String(),     name=String(),
+                    url=String())
+    query.setkey("name")
+
+    # Note: roles is a comma-separated string of Role names
+    user = Class(db, "user",
+                    username=String(),   password=Password(),
+                    address=String(),    realname=String(),
+                    phone=String(),      organisation=String(),
+                    alternate_addresses=String(),
+                    queries=Multilink('query'), roles=String())
+    user.setkey("username")
+
+The "queries" property is used to store off the user's favourite database
+queries. The "roles" property is explained below in `0.5.0 Security
+Settings`_.
+
+
+0.5.0 Security Settings
+~~~~~~~~~~~~~~~~~~~~~~~
+
+See the `security documentation`_ for an explanation of how the new security
+system works. In a nutshell though, the security is handled as a four step
+process:
+
+1. Permissions are defined as having a name and optionally a hyperdb class
+   they're specific to,
+2. Roles are defined that have one or more Permissions,
+3. Users are assigned Roles in their "roles" property, and finally
+4. Roundup checks that users have appropriate Permissions at appropriate times
+   (like editing issues).
+
+Your instance dbinit module's *open* function now has to define any
+Permissions that are specific to your instance, and also the assignment
+of Permissions to Roles. At the moment, your open function
+ends with::
+
+    import detectors
+    detectors.init(db)
+
+    return db
+
+and what we need to do is insert some commands that will set up the security
+parameters. Right above the ``import detectors`` line, you'll want to insert
+these lines::
+
+    #
+    # SECURITY SETTINGS
+    #
+    # new permissions for this schema
+    for cl in 'issue', 'file', 'msg', 'user':
+        db.security.addPermission(name="Edit", klass=cl,
+            description="User is allowed to edit "+cl)
+        db.security.addPermission(name="View", klass=cl,
+            description="User is allowed to access "+cl)
+
+    # Assign the access and edit permissions for issue, file and message
+    # to regular users now
+    for cl in 'issue', 'file', 'msg':
+        p = db.security.getPermission('View', cl)
+        db.security.addPermissionToRole('User', p)
+        p = db.security.getPermission('Edit', cl)
+        db.security.addPermissionToRole('User', p)
+    # and give the regular users access to the web and email interface
+    p = db.security.getPermission('Web Access')
+    db.security.addPermissionToRole('User', p)
+    p = db.security.getPermission('Email Access')
+    db.security.addPermissionToRole('User', p)
+
+    # May users view other user information? Comment these lines out
+    # if you don't want them to
+    p = db.security.getPermission('View', 'user')
+    db.security.addPermissionToRole('User', p)
+
+    # Assign the appropriate permissions to the anonymous user's Anonymous
+    # Role. Choices here are:
+    # - Allow anonymous users to register through the web
+    p = db.security.getPermission('Web Registration')
+    db.security.addPermissionToRole('Anonymous', p)
+    # - Allow anonymous (new) users to register through the email gateway
+    p = db.security.getPermission('Email Registration')
+    db.security.addPermissionToRole('Anonymous', p)
+    # - Allow anonymous users access to the "issue" class of data
+    #   Note: this also grants access to related information like files,
+    #         messages, statuses etc that are linked to issues
+    #p = db.security.getPermission('View', 'issue')
+    #db.security.addPermissionToRole('Anonymous', p)
+    # - Allow anonymous users access to edit the "issue" class of data
+    #   Note: this also grants access to create related information like
+    #         files and messages etc that are linked to issues
+    #p = db.security.getPermission('Edit', 'issue')
+    #db.security.addPermissionToRole('Anonymous', p)
+
+    # oh, g'wan, let anonymous access the web interface too
+    p = db.security.getPermission('Web Access')
+    db.security.addPermissionToRole('Anonymous', p)
+
+Note in the comments there the places where you might change the permissions
+to restrict users or grant users more access. If you've created additional
+classes that users should be able to edit and view, then you should add them
+to the "new permissions for this schema" section at the start of the security
+block. Then add them to the "Assign the access and edit permissions" section
+too, so people actually have the new Permission you've created.
+
+One final change is needed that finishes off the security system's
+initialisation. We need to add a call to ``db.post_init()`` at the end of the
+dbinit open() function. Add it like this::
+
+    import detectors
+    detectors.init(db)
+
+    # schema is set up - run any post-initialisation
+    db.post_init()
+    return db
+
+You may verify the setup of Permissions and Roles using the new
+"``roundup-admin security``" command.
+
+
+0.5.0 CGI interface changes
+---------------------------
+
+The CGI interface code was completely reorganised and largely rewritten. The
+end result is that this section of your instance interfaces module will need
+changing from::
+
+ from roundup import mailgw
+ from roundup.cgi import client
+ 
+ class Client(client.Client): 
+     ''' derives basic CGI implementation from the standard module,
+         with any specific extensions
+     '''
+     pass
+
+to::
+
+ from roundup import cgi_client, mailgw
+ from roundup.i18n import _
+ 
+ class Client(cgi_client.Client):
+     ''' derives basic CGI implementation from the standard module,
+         with any specific extensions
+     '''
+     pass
+
+0.5.0 HTML templating
+---------------------
+
+You'll want to make a backup of your current instance html directory. You
+should then copy the html directory from the Roundup source template that you
+used to create your instance, and modify it according to your local schema
+changes.
+
+If you need help with the new templating system, please ask questions on the
+roundup-users mailing list (available through the roundup project page on
+sourceforge, http://roundup.sf.net/)
+
+
+0.5.0 Detectors
+---------------
+
+The nosy reactor has been updated to handle the instance not having an
+"assignedto" property on issues. You may want to copy it into your instance's
+detectors directory. Chances are you've already fixed it though :)
 
 
 Migrating from 0.4.1 to 0.4.2
@@ -48,7 +370,7 @@ and::
 
  <roundup source>/roundup/templates/extended/instance_config.py
 
-and the documentation in customizing_ for information on how they're used.
+and the `customisation documentation`_ for information on how they're used.
 
 
 0.4.2 Changes to detectors
@@ -345,4 +667,5 @@ distribution. Make sure you update the ROUNDUP_INSTANCE_HOMES after the
 copy.
 
 
-.. _customizing: customizing.html
+.. _`customisation documentation`: customizing.html
+.. _`security documentation`: security.html

roundup/cgi/client.py

@@ -1,10 +1,10 @@
-# $Id: client.py,v 1.5 2002-09-01 23:57:53 richard Exp $
+# $Id: client.py,v 1.6 2002-09-02 07:46:55 richard Exp $
 
 __doc__ = """
 WWW request handler (also used in the stand-alone server).
 """
 
-import os, cgi, StringIO, urlparse, re, traceback, mimetypes, urllib
+import os, os.path, cgi, StringIO, urlparse, re, traceback, mimetypes, urllib
 import binascii, Cookie, time, random
 
 from roundup import roundupdb, date, hyperdb, password
@@ -275,15 +275,15 @@ def serve_static_file(self, file):
         # we just want to serve up the file named
         mt = mimetypes.guess_type(str(file))[0]
         self.header({'Content-Type': mt})
-        self.write(open('/tmp/test/html/%s'%file).read())
+        self.write(open(os.path.join(self.instance.TEMPLATES, file)).read())
 
     def template(self, name, **kwargs):
         ''' Return a PageTemplate for the named page
         '''
         pt = RoundupPageTemplate(self)
         # make errors nicer
         pt.id = name
-        pt.write(open('/tmp/test/html/%s'%name).read())
+        pt.write(open(os.path.join(self.instance.TEMPLATES, name)).read())
         # XXX handle PT rendering errors here nicely
         try:
             return pt.render(**kwargs)