role checking for PGP mail and docs

Erik's suggestion to allow the admin to specify a set of roles to
perform PGP processing on seemed like a reasonable one I implemented
it. There is a new config option to control it.

I also realized that the signature verification had a slight problem:
it was simply checking for a valid, known signature before continuing
on. If another user in the keyring forged mail it was pass the PGP
check and then modify the db as the forged user. I changed the logic
to make sure that the author of the email matches the key of the
verifying signature.

As I was adding the documentation for the PGP processing I noticed
that there were several other new-ish options that didn't appear in
customizing.txt so I added them as well.

Author: Justus Pendleton

doc/customizing.txt

@@ -2,7 +2,7 @@
 Customising Roundup
 ===================
 
-:Version: $Revision: 1.221 $
+:Version: $Revision: 1.222 $
 
 .. This document borrows from the ZopeBook section on ZPT. The original is at:
    http://www.zope.org/Documentation/Books/ZopeBook/current/ZPT.stx
@@ -95,6 +95,14 @@ Section **main**
   Path to the HTML templates directory. The path may be either absolute
   or relative to the directory containig this config file.
 
+ static_files -- default *blank*
+  Path to directory holding additional static files available via Web
+  UI.  This directory may contain sitewide images, CSS stylesheets etc.
+  and is searched for these files prior to the TEMPLATES directory
+  specified above.  If this option is not set, all static files are
+  taken from the TEMPLATES directory The path may be either absolute or
+  relative to the directory containig this config file.
+
  admin_email -- ``roundup-admin``
   Email address that roundup will complain to if it runs into trouble. If
   the email address doesn't contain an ``@`` part, the MAIL_DOMAIN defined
@@ -150,6 +158,9 @@ Section **main**
   your tracker. See the indexer source for the default list of
   stop-words (e.g. ``A,AND,ARE,AS,AT,BE,BUT,BY, ...``).
 
+ umask -- ``02``
+  Defines the file creation mode mask.
+
 Section **tracker**
  name -- ``Roundup issue tracker``
   A descriptive name for your roundup instance.
@@ -164,6 +175,11 @@ Section **tracker**
  email -- ``issue_tracker``
   Email address that mail to roundup should go to.
 
+ language -- default *blank*
+  Default locale name for this tracker. If this option is not set, the
+  language is determined by the environment variable LANGUAGE, LC_ALL,
+  LC_MESSAGES, or LANG, in that order of preference.
+
 Section **web**
  http_auth -- ``yes``
   Whether to use HTTP Basic Authentication, if present.
@@ -204,6 +220,13 @@ Section **rdbms**
  password -- ``roundup``
   Database user password.
 
+ read_default_file -- ``~/.my.cnf``
+  Name of the MySQL defaults file. Only used in MySQL connections.
+
+ read_default_group -- ``roundup``
+  Name of the group to use in the MySQL defaults file. Only used in
+  MySQL connections.
+
 Section **logging**
  config -- default *blank*
   Path to configuration file for standard Python logging module. If this
@@ -277,6 +300,16 @@ Section **mail**
   precedence. The path may be either absolute or relative to the directory
   containig this config file.
 
+ add_authorinfo -- ``yes``
+  Add a line with author information at top of all messages send by
+  roundup.
+
+ add_authoremail -- ``yes``
+  Add the mail address of the author to the author information at the
+  top of all messages.  If this is false but add_authorinfo is true,
+  only the name of the actor is added which protects the mail address
+  of the actor from being exposed at mail archives, etc.
+
 Section **mailgw**
  Roundup Mail Gateway options
 
@@ -294,6 +327,10 @@ Section **mailgw**
   Default class to use in the mailgw if one isn't supplied in email subjects.
   To disable, leave the value blank.
 
+ language -- default *blank*
+  Default locale name for the tracker mail gateway.  If this option is
+  not set, mail gateway will use the language of the tracker instance.
+
  subject_prefix_parsing -- ``strict``
   Controls the parsing of the [prefix] on subject lines in incoming emails.
   ``strict`` will return an error to the sender if the [prefix] is not
@@ -319,6 +356,42 @@ Section **mailgw**
   an issue for the interval after the issue's creation or last activity.
   The interval is a standard Roundup interval.
 
+ refwd_re -- ``(\s*\W?\s*(fw|fwd|re|aw|sv|ang)\W)+``
+  Regular expression matching a single reply or forward prefix
+  prepended by the mailer. This is explicitly stripped from the
+  subject during parsing.  Value is Python Regular Expression
+  (UTF8-encoded).
+
+ origmsg_re -- `` ^[>|\s]*-----\s?Original Message\s?-----$``
+  Regular expression matching start of an original message if quoted
+  the in body.  Value is Python Regular Expression (UTF8-encoded).
+
+ sign_re -- ``^[>|\s]*-- ?$``
+  Regular expression matching the start of a signature in the message
+  body.  Value is Python Regular Expression (UTF8-encoded).
+
+ eol_re -- ``[\r\n]+``
+  Regular expression matching end of line.  Value is Python Regular
+  Expression (UTF8-encoded).
+
+ blankline_re -- ``[\r\n]+\s*[\r\n]+``
+  Regular expression matching a blank line.  Value is Python Regular
+  Expression (UTF8-encoded).
+
+Section **pgp**
+ OpenPGP mail processing options
+
+ enable -- ``no``
+  Enable PGP processing. Requires pyme.
+
+ roles -- default *blank*
+  If specified, a comma-separated list of roles to perform PGP
+  processing on. If not specified, it happens for all users.
+
+ homedir -- default *blank*
+  Location of PGP directory. Defaults to $HOME/.gnupg if not
+  specified.
+
 Section **nosy**
  Nosy messages sending
 
@@ -349,6 +422,12 @@ Section **nosy**
   a separate email is sent to each recipient. If ``single`` then a single
   email is sent with each recipient as a CC address.
 
+ max_attachment_size -- ``2147483647``
+  Attachments larger than the given number of bytes won't be attached
+  to nosy mails. They will be replaced by a link to the tracker's
+  download page for the file.
+
+
 You may generate a new default config file using the ``roundup-admin
 genconfig`` command.
 

doc/installation.txt

@@ -2,7 +2,7 @@
 Installing Roundup
 ==================
 
-:Version: $Revision: 1.126 $
+:Version: $Revision: 1.127 $
 
 .. contents::
    :depth: 2
@@ -81,10 +81,17 @@ pyopenssl
   proxy through a server with SSL support (e.g. apache) then this is
   unnecessary.
 
+pyme
+  If pyme_ is installed you can configure the mail gateway to perform
+  verification or decryption of incoming OpenPGP MIME messages. When
+  configured, you can require email to be cryptographically signed
+  before roundup will allow it to make modifications to issues.
+
 .. _Xapian: http://www.xapian.org/
 .. _pytz: http://www.python.org/pypi/pytz
 .. _Olson tz database: http://www.twinsun.com/tz/tz-link.htm
 .. _pyopenssl: http://pyopenssl.sourceforge.net
+.. _pyme: http://pyme.sourceforge.net
 
 
 Getting Roundup

roundup/configuration.py

@@ -1,6 +1,6 @@
 # Roundup Issue Tracker configuration support
 #
-# $Id: configuration.py,v 1.48 2007-09-22 07:25:34 jpend Exp $
+# $Id: configuration.py,v 1.49 2007-09-26 03:20:21 jpend Exp $
 #
 __docformat__ = "restructuredtext"
 
@@ -720,6 +720,10 @@ def str2value(self, value):
     ("pgp", (
         (BooleanOption, "enable", "no",
             "Enable PGP processing. Requires pyme."),
+        (NullableOption, "roles", "",
+            "If specified, a comma-separated list of roles to perform\n"
+            "PGP processing on. If not specified, it happens for all\n"
+            "users."),
         (NullableOption, "homedir", "",
             "Location of PGP directory. Defaults to $HOME/.gnupg if\n"
             "not specified."),

roundup/mailgw.py

@@ -73,7 +73,7 @@ class node. Any parts of other types are each stored in separate files
 an exception, the original message is bounced back to the sender with the
 explanatory message given in the exception.
 
-$Id: mailgw.py,v 1.191 2007-09-24 09:52:18 a1s Exp $
+$Id: mailgw.py,v 1.192 2007-09-26 03:20:21 jpend Exp $
 """
 __docformat__ = 'restructuredtext'
 
@@ -146,29 +146,70 @@ def getparam(str, param):
                 return rfc822.unquote(f[i+1:].strip())
     return None
 
-def check_pgp_sigs(sig):
-    ''' Theoretically a PGP message can have several signatures. GPGME returns
-        status on all signatures in a linked list. Walk that linked list making
-        sure all signatures are valid.
+def gpgh_key_getall(key, attr):
+    ''' return list of given attribute for all uids in
+        a key
     '''
-    while sig != None:
-        if not sig.summary & pyme.gpgme.GPGME_SIGSUM_VALID:
-            # try to narrow down the actual problem to give a more useful
-            # message in our bounce
-            if sig.summary & pyme.gpgme.GPGME_SIGSUM_KEY_MISSING:
-                raise MailUsageError, \
-                    _("Message signed with unknown key: %s") % sig.fpr
-            elif sig.summary & pyme.gpgme.GPGME_SIGSUM_KEY_EXPIRED:
-                raise MailUsageError, \
-                    _("Message signed with an expired key: %s") % sig.fpr
-            elif sig.summary & pyme.gpgme.GPGME_SIGSUM_KEY_REVOKED:
-                raise MailUsageError, \
-                    _("Message signed with a revoked key: %s") % sig.fpr
-            else:
-                raise MailUsageError, \
-                    _("Invalid PGP signature detected.")
+    u = key.uids
+    while u:
+        yield getattr(u, attr)
+        u = u.next
+
+def gpgh_sigs(sig):
+    ''' more pythonic iteration over GPG signatures '''
+    while sig:
+        yield sig
         sig = sig.next
 
+
+def iter_roles(roles):
+    ''' handle the text processing of turning the roles list
+        into something python can use more easily
+    '''
+    for role in [x.lower().strip() for x in roles.split(',')]:
+        yield role
+
+def user_has_role(db, userid, role_list):
+    ''' see if the given user has any roles that appear
+        in the role_list
+    '''
+    for role in iter_roles(db.user.get(userid, 'roles')):
+        if role in iter_roles(role_list):
+            return True
+    return False
+
+
+def check_pgp_sigs(sig, gpgctx, author):
+    ''' Theoretically a PGP message can have several signatures. GPGME
+        returns status on all signatures in a linked list. Walk that
+        linked list looking for the author's signature
+    '''
+    for sig in gpgh_sigs(sig):
+        key = gpgctx.get_key(sig.fpr, False)
+        # we really only care about the signature of the user who
+        # submitted the email
+        if key and (author in gpgh_key_getall(key, 'email')):
+            if sig.summary & pyme.gpgme.GPGME_SIGSUM_VALID:
+                return True
+            else:
+                # try to narrow down the actual problem to give a more useful
+                # message in our bounce
+                if sig.summary & pyme.gpgme.GPGME_SIGSUM_KEY_MISSING:
+                    raise MailUsageError, \
+                        _("Message signed with unknown key: %s") % sig.fpr
+                elif sig.summary & pyme.gpgme.GPGME_SIGSUM_KEY_EXPIRED:
+                    raise MailUsageError, \
+                        _("Message signed with an expired key: %s") % sig.fpr
+                elif sig.summary & pyme.gpgme.GPGME_SIGSUM_KEY_REVOKED:
+                    raise MailUsageError, \
+                        _("Message signed with a revoked key: %s") % sig.fpr
+                else:
+                    raise MailUsageError, \
+                        _("Invalid PGP signature detected.")
+
+    # we couldn't find a key belonging to the author of the email
+    raise MailUsageError, _("Message signed with unknown key: %s") % sig.fpr
+
 class Message(mimetools.Message):
     ''' subclass mimetools.Message so we can retrieve the parts of the
         message...
@@ -349,7 +390,7 @@ def pgp_encrypted(self):
         return self.gettype() == 'multipart/encrypted' \
             and self.typeheader.find('protocol="application/pgp-encrypted"') != -1
 
-    def decrypt(self):
+    def decrypt(self, author):
         ''' decrypt an OpenPGP MIME message
             This message must be signed as well as encrypted using the "combined"
             method. The decrypted contents are returned as a new message.
@@ -375,7 +416,7 @@ def decrypt(self):
         # key to send it to us. now check the signatures to see if it
         # was signed by someone we trust
         result = context.op_verify_result()
-        check_pgp_sigs(result.signatures)
+        check_pgp_sigs(result.signatures, context, author)
 
         plaintext.seek(0,0)
         # pyme.core.Data implements a seek method with a different signature
@@ -386,7 +427,7 @@ def decrypt(self):
         c.seek(0)
         return Message(c)
 
-    def verify_signature(self):
+    def verify_signature(self, author):
         ''' verify the signature of an OpenPGP MIME message
             This only handles detached signatures. Old style
             PGP mail (i.e. '-----BEGIN PGP SIGNED MESSAGE----')
@@ -419,7 +460,7 @@ def verify_signature(self):
 
         # check all signatures for validity
         result = context.op_verify_result()
-        check_pgp_sigs(result.signatures)
+        check_pgp_sigs(result.signatures, context, author)
 
 class MailGW:
 
@@ -1133,19 +1174,31 @@ def handle_message(self, message):
 
         # if they've enabled PGP processing then verify the signature
         # or decrypt the message
-        if self.instance.config.PGP_ENABLE:
+
+        # if PGP_ROLES is specified the user must have a Role in the list
+        # or we will skip PGP processing
+        def pgp_role():
+            if self.instance.config.PGP_ROLES:
+                return user_has_role(self.db, author,
+                    self.instance.config.PGP_ROLES)
+            else:
+                return True
+
+        if self.instance.config.PGP_ENABLE and pgp_role():
             assert pyme, 'pyme is not installed'
+            # signed/encrypted mail must come from the primary address
+            author_address = self.db.user.get(author, 'address')
             if self.instance.config.PGP_HOMEDIR:
                 os.environ['GNUPGHOME'] = self.instance.config.PGP_HOMEDIR
             if message.pgp_signed():
-                message.verify_signature()
+                message.verify_signature(author_address)
             elif message.pgp_encrypted():
                 # replace message with the contents of the decrypted
                 # message for content extraction
                 # TODO: encrypted message handling is far from perfect
                 # bounces probably include the decrypted message, for
                 # instance :(
-                message = message.decrypt()
+                message = message.decrypt(author_address)
             else:
                 raise MailUsageError, _("""
 This tracker has been configured to require all email be PGP signed or