Branch in order to continue working, fixing problems

 - Legacy-Id: 6329

Author: mcr

r6328/.gitignore

@@ -0,0 +1,8 @@
+/.project
+/.pydevproject
+/.settings
+/unix.tag
+/testresult
+/mergelog
+/mergequeue
+/INSTALL.pdf

r6328/INSTALL

@@ -0,0 +1,271 @@
+=====================================
+Datatracker Installation Instructions
+=====================================
+
+
+General Instructions for Deployment of a New Release
+====================================================
+
+
+ 1. In order to fetch a new release of the django datatracker code, simply
+    check out the appropriate tag from svn::
+
+      svn co http://svn.tools.ietf.org/svn/tools/ietfdb/tags/$releasenumber
+
+ 2. Don't forget to copy $releasenumber/ietf/settings_local.py from the
+    old release to the new one; otherwise things won't work!
+    ::
+
+      cp $oldreleasenumber/ietf/settings_local.py $releasenumber/ietf/
+
+ 3. Change into the directory of the new release and run migrations::
+
+      cd $releasenumber
+      PYTHONPATH=$PWD ietf/manage.py migrate
+      cd ..
+
+ 4. Then re-point the 'web' symlink::
+
+      rm ./web; ln -s $releasenumber web
+
+ 5. Restart apache::
+
+      sudo /etc/init.d/apache2 restart
+
+ 6. It's now also a good idea to go to the datatracker front page:
+
+      http://datatracker.ietf.org/
+
+    to check that it's alive and kicking, and displaying the new release
+    number at the bottom of the left-side menubar :-)
+
+ 7. If things **aren't** cool, revert the symlink step, re-pointing the
+    symlink to the release that was running before the new release, and restart
+    apache again to roll back to that.
+
+
+Installing from Scratch
+=======================
+
+In addition to the new release deployment instructions above, the
+settings_local.py file has to be set up properly, and Apache has to be
+configured.  Since the IETF datatracker is only intended to be deployed from
+scratch once, these instructions don't cover this scenario in any more detail.
+The general Django depoloyment instructions are relevant, however.
+
+
+Patching a Production Release
+=============================
+
+Sometimes it can prove necessary to patch an existing release.  The following
+process should be used:
+
+ 1. Code and test the patch on a development system copy of the production
+    release which has no other code changes (or on trunk, with no uncommitted
+    code changes, if it's sufficiently close).
+
+ 2. Produce a patch file, named with date and subject::
+
+      ~/src/ietfdb/working $ svn diff > 2013-03-25-ballot-calculation.patch
+
+ 3. Move the patch file to the production server, and place it in 
+    '/a/www/ietf-datatracker/patches/'
+
+ 4. Make a recursive copy of the production code to a new directory, named
+    with a patch number.  Assuming the production code is in 4.43/, and we
+    have web -> 4.43/::
+
+      /a/www/ietf-datatracker $ rsync -a web/ 4.43.p1/
+
+    (you could use 'cp -rp' instead, but rsync seems to do this faster).
+
+ 5. Apply the patch::
+
+      /a/www/ietf-datatracker $ cd 4.43.p1/      
+      /a/www/ietf-datatracker/4.43.p1 $ patch -p0 \
+			< ../patches/2013-03-25-ballot-calculation.patch
+
+    This should not produce any messages about failing to apply any chunks;
+    if it does, you probably should go back to 1. and figure
+    out why.
+
+ 6. Edit ``.../ietf/__init__.py`` in the new patched release to indicate the patch
+    version in the ``__version__`` string and current date and time in the ``__date__``
+    string.
+
+ 7. Change the 'web' symlink, reload etc. as described in
+    `General Instructions for Deployment of a New Release`_ .
+
+
+Installing a Secretariat Release
+================================
+
+Secretariat releases are based on a regular tagged release, but contain
+updated code in the ietf/secr/ tree.  They are named with a trailing
+'.secr<N>' version number.  If the tagged release is for instance 4.42, the
+first secretariat release based on that would have release number 4.42.secr1,
+and the second secretariat release based on it would be numbered 4.42.secr2.
+
+Secretariat releases are deployed in the same manner as regular releases, with
+the addition that patches made to the regular release needs to be picked up
+and applied.  Assuming we'releasing 4.42.p1.secr2, then after we've cd'd into
+the release diretory to run migrations, we'd also run a patch step to pick
+up the current patches to the current production release, which is pointed
+to by ../web::
+
+  /a/www/ietf-datatracker/4.42.p1.secr2 $ svn diff ../web/ | patch -p2
+
+Assuming that runs without error messages, the deployment is continued as
+usual, with re-pointing the symlink and restarting apache.
+
+Additional Version-Specific Instructions
+========================================
+
+Version 4.50
+------------
+
+Before you run step 3 (migration) of the general instructions, please run some specific
+initial migrations with the a --fake argument:
+
+      cd $releasenumber
+      PYTHONPATH=$PWD ietf/manage.py migrate group 0001 --fake
+      cd ..
+
+
+Version 4.42
+------------
+
+ - In order to serve the secretariat tools static pages (such as image, css and js
+   files) the exceptions to which urls need to be handled by the python-handler
+   must be updated.
+
+   In the datatracker test server, the following configuration has been used to
+   make apache handle the static files, and Django the dynamic pages.  The new
+   part is the <LocationMatch/> which mentions /secr/.  Adapt as needed for ietfa::
+
+       ----------------
+       # Logging and document root settings omitted
+
+	   <Location "/">
+	       PythonPath "['/srv/www/ietfdb'] + sys.path"
+	       SetHandler python-program
+	       PythonHandler django.core.handlers.modpython
+	       SetEnv DJANGO_SETTINGS_MODULE ietf.settings 
+	       PythonDebug On
+	   </Location>
+
+	   <LocationMatch "^/(robots.txt|favicon.ico|images|css|js|media|error)(/|$)"> 
+	       SetHandler None
+	   </LocationMatch> 
+
+	   # New for secretariat tools:
+	   <LocationMatch "^/secr/(img|css|js|error)"> 
+	       SetHandler None
+	   </LocationMatch> 
+
+	   <Location "/accounts/login/">
+	       AuthType Digest
+	       AuthName "IETF"
+	       AuthUserFile /var/local/loginmgr/digest
+	       AuthGroupFile /var/local/loginmgr/groups
+	       AuthDigestDomain http://tools.ietf.org/
+	       Require valid-user
+	   </Location>
+
+       # Caching and compression settings omitted
+       ----------------
+
+Version 4.40
+------------
+
+ - (DONE) Add ianasync user with an auth role in the "iana" group and an
+   rfceditorsync user with an auth role in the "rfceditor" group (don't
+   think Group(acronym="rfceditor") exists yet); IANA and RFC Editor need
+   to know the passwords for the poke mechanism
+
+
+ - (DONE) Make sure mailing list for iab-stream@iab.org is up (since we're now
+   emailing that)
+
+ - (DONE) Set rfc_must_published_later_than date in bin/iana-protocols-updates to today
+
+ - (DONE) Run the 3 new doc South migrations
+
+ - (DONE) New polling scripts, to be run every hour::
+
+     web/ietf/bin/iana-changes-updates
+     web/ietf/bin/iana-protocols-updates
+     web/ietf/bin/rfc-editor-index-updates (replaces mirror_rfc_index)
+     web/ietf/bin/rfc-editor-queue-updates (replaces mirror_rfc_queue)
+
+ - (DONE) Import old events from IANA::
+
+     bin/iana-changes-updates --from="2005-01-01 00:00:00" --to="2013-01-31 00:00:00" --no-email
+
+ - (DONE) Pipe IANA review emails to the datatracker.  There used to be an action to pipe
+   such mails to henrik@levkowetz.com, for testing this feature, but I haven't seen
+   any in a little while, so I don't know if this has broken.  Anyway, the iana review
+   emails should be piped into::
+
+     /www/ietf-datatracker/web/ietf/bin/iana-review-email
+
+
+ - (DONE) Tell IANA we're doing this for real now.
+
+
+Version 4.34
+------------
+
+The migration step you do as a part of the release sequence is going to take
+quite some time -- probably minutes.  It will generate some output while it's
+working, too.  As long as it doesn't halt and say that something failed or
+gave an error, this is as expected, and when it terminates, you should be OK
+to continue.
+
+Version 4.21
+------------
+
+This release will you to run migrations before moving the link to the new
+version and doing the apache reload.  I know you have a routine for the steps
+needed to deploy a new release by now, but thought I'd mention it, anyway.
+
+If there is any problem at all doing the migrations, then you'll need to
+do a fake initial migration, as follows::
+
+  web $ PYTHONPATH=PWD ietf/manage.py migrate --fake meeting 0001
+
+and then to the regular migration again.
+
+Version 4.20
+------------
+
+Some one-time actions that need to be taken are as follows::
+
+    Assuming that the release has been checked out in /a/www/ietf-datatracker/4.20:
+
+      cd /a/www/ietf-datatracker/4.20
+
+      PYTHONPATH=$PWD ietf/manage.py migrate --fake doc 0001
+      PYTHONPATH=$PWD ietf/manage.py migrate --fake name 0001
+
+      PYTHONPATH=$PWD ietf/manage.py dbshell <<< "delete from django_content_type where app_label='doc'
+		and model='groupballotpositiondocevent';"
+
+      PYTHONPATH=$PWD ietf/manage.py migrate doc || { \
+	PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'CREATE TABLE `doc_groupballotpositiondocevent`
+		(`block_comment` longtext NOT NULL, `comment` longtext NOT NULL,
+		`ad_id` integer NOT NULL, `comment_time` datetime NULL,
+		`block_comment_time` datetime NULL, `pos_id` varchar(8) NOT NULL DEFAULT "norecord",
+		`docevent_ptr_id` integer NOT NULL PRIMARY KEY);'
+	PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'DROP TABLE `doc_ballottype` CASCADE;'
+	PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'DROP TABLE `doc_ballottype_positions` CASCADE;'
+	PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'DROP TABLE `doc_ballotdocevent` CASCADE;'
+	PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'ALTER TABLE `doc_ballotpositiondocevent`
+		DROP COLUMN `ballot_id` CASCADE;'
+      }
+
+      PYTHONPATH=$PWD ietf/manage.py migrate doc
+      PYTHONPATH=$PWD ietf/manage.py migrate name
+      PYTHONPATH=$PWD python ietf/wgcharter/migrate.py | tee -a ~/charter-migration.log
+
+

r6328/LICENSE

@@ -0,0 +1,34 @@
+For code in the django/ directory, see LICENSE.django file).
+
+For files under static/js/yui/, static/css/yui/ and static/images/yui/,
+see http://developer.yahoo.net/yui/license.txt.
+
+---------------------------------------------------------------------------
+
+Copyright (c) 2008, The IETF Trust
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+    * Redistributions of source code must retain the above copyright notice,
+      this list of conditions and the following disclaimer.
+
+    * Redistributions in binary form must reproduce the above copyright
+      notice, this list of conditions and the following disclaimer in the
+      documentation and/or other materials provided with the distribution.
+
+    * Neither the name of the <ORGANIZATION> nor the names of its contributors
+      may be used to endorse or promote products derived from this software
+      without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

r6328/LICENSE.django

@@ -0,0 +1,30 @@
+(this applies to the django/ subdirectory)
+---------------------------------------------------------------------------
+
+Copyright (c) Django Software Foundation and individual contributors.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+    1. Redistributions of source code must retain the above copyright notice, 
+       this list of conditions and the following disclaimer.
+    
+    2. Redistributions in binary form must reproduce the above copyright 
+       notice, this list of conditions and the following disclaimer in the
+       documentation and/or other materials provided with the distribution.
+
+    3. Neither the name of Django nor the names of its contributors may be used
+       to endorse or promote products derived from this software without
+       specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

r6328/bin/commitlog

@@ -0,0 +1,3 @@
+#!/bin/bash
+svn log $PWD -r ${2:-HEAD}:${1:-BASE} | sed -n -e 's/^/  * /' -e '1,/^  \* Set version info and settings back to development mode/p' | egrep -v -- '^  \* (----------|r[0-9]+ |$)' | head -n -1 | tac | sed 's/$/\n/' | fold -s -w 76 | sed -r 's/^([^ ])/    \1/'
+date +" -- Henrik Levkowetz <henrik@netnod.se>  %d %b %Y %H:%M:%S %z"

r6328/bin/graph-models

@@ -0,0 +1,21 @@
+#!/bin/sh
+#
+# Copyright The IETF Trust 2007, All Rights Reserved
+#
+# Requires modelviz.py from
+#  http://code.djangoproject.com/wiki/DjangoGraphviz
+#
+PYTHONPATH=`dirname $PWD`
+export PYTHONPATH
+module=${PWD##*/}
+DJANGO_SETTINGS_MODULE=$module.settings
+export DJANGO_SETTINGS_MODULE
+for d in *
+do
+	if grep models.Model $d/models.py > /dev/null 2>&1
+	then
+		models="$models $d"
+	fi
+done
+modelviz.py $* $models > models.dot
+dot -Tpng models.dot