still incomplete, but much more verbose

Author: Alexander Smishlajev

I18N_PROGRESS.txt

@@ -2,36 +2,104 @@
 Roundup Internationalization Notes
 ==================================
 
-:Version: $Revision: 1.7 $
+:Version: $Revision: 1.8 $
 
-FIXME: add introduction - what's l10n, i18n, po, mo, pot, gettext...
+Overview
+--------
 
-Used Tools
-----------
+How stuff works:
 
-We use ``xpot`` utility from Francois Pinard free `PO utilities`_
-to build message template file.  ``pygettext`` utility included
-with Python distribution is not used because it has several limitations:
+1. Strings that may require translation (messages in human language)
+   are marked in the source code.  This step is discussed in
+   `Marking Strings for Translation`_ section.
 
- - no way to mark strings translated elsewhere
- - does not detect translation done by an object method
- - source references put in the comments are not recognized
-   by translation software.
+2. These strings are all extracted into Message Template File
+   ``locale/roundup.pot`` (POT file).  See `Extracting Translatable
+   Messages`_ below.
 
-Message translation can be done with `emacs`_ "po mode" provided
-by `PO utilities`_.  (als: i didn't try that personally.)
+3. Language teams use POT file to make Message Files for national
+   languages (PO files).  All PO files for Roundup are kept in
+   the ``locale`` directory.  Names of these files are target
+   locale names, usually just 2-letter language codes.  `Translating
+   Messages`_ section of this document gives useful hints for
+   message translators.
 
-Another tool for message translation (als: one that i use) is `poEdit`_
-by Vaclav Slavik.
+4. Translated Message Files are compiled into binary form (MO files)
+   and stored in ``locale`` directory (but not kept in the `Roundup
+   CVS`_ repository, as they may be easily made from PO files).
+   See `Compiling Message Catalogs`_ section.
 
-.. _PO utilities: http://po-utils.progiciels-bpi.ca/
-.. _emacs: http://www.gnu.org/software/emacs/
-.. _poEdit: http://poedit.sourceforge.net/
+5. Roundup installer creates runtime locale structure on the file
+   system, putting MO files in their appropriate places.
+
+6. Runtime internationalization (I18N) services use these MO files
+   to translate program messages into language selected by current
+   Roundup user.  Roundup command line interface uses locale name
+   set in OS environment variable ``LANGUAGE``, ``LC_ALL``,
+   ``LC_MESSAGES``, or ``LANG`` (in that order).  Roundup Web User
+   Interface uses language selected by currentluy authenticated user.
+
+Additional details may be found in `GNU gettext`_ and Python `gettext
+module`_ documentation.
+
+`Roundup source distribution`_ includes POT and PO files for message
+translators, and also pre-built MO files to facilitate installations
+from source.  `Roundup binary distribution`_ includes MO files only.
+
+.. _GNU gettext:
+
+GNU gettext package
+-------------------
+
+This document is full of references to `GNU`_ `gettext package`_.
+GNU gettext is a "must have" for nearly all steps of internationalizing
+any program, and it's manual is definetely a recommended reading
+for people involved in I18N.
+
+There are GNU gettext ports to all major OS platforms.
+Windows binaries are available from ``GNU mirror sites``.
+
+Roundup does not use GNU gettext at runtime, but it's tools
+are used for `extracting translatable messages`_, `compiling
+message catalogs`_ and, optionally, for `translating messages`_.
+
+Note that ``gettext`` package in some OS distributions (e.g.
+`FreeBSD`_) means just runtime libraries.  Development tools
+are usually distributed in separate package named ``gettext-devel``.
 
 Marking Strings for Translation
 -------------------------------
 
 Strings that need translation must be marked in the source code.
+Following sections of this chapter explain how this is done in
+different cases.
+
+Current status of the string marks in Roundup core files is shown
+in `I18 Status`_ section at the end of this document.
+
+If translatable string is used as a format string, it is recommended
+to always use named format specifiers::
+
+  _('Index of %(classname)s') % locals()
+
+This helps translators to better understand the context of the
+message and, in Python, remove format specifier altogether (which
+is sometimes useful, especially in singular cases of `Plural Forms`_).
+When there is more than one format specifier in the translatable
+format string, named format specifiers *must* be used almost always,
+because translation may require different order of items.
+
+It is better to *not* mark for translation strings that are not
+locale-dependent, as this makes it more difficult to keep track
+of translation completeness.  For example: following string in
+``index()`` method of the request handler in ``roundup_server``
+script::
+
+  </ol></body></html>
+
+has no human readable parts at all, and needs no translations.
+These strings are left untranslated in PO files, and are reported
+as such by PO status checkers (e.g. ``msgfmt --statistics``).
 
 Command Line Interfaces
 ~~~~~~~~~~~~~~~~~~~~~~~
@@ -47,20 +115,18 @@ For these interfaces, Python ``gettext`` engine must be initialized
 to use Roundup message catalogs.  This is normally done by including
 the following line in the module imports::
 
-  from i18n import _
+  from i18n import _, ngettext
 
 Simple translations are automatically marked by calls to builtin
 message translation function ``_()``::
 
   print _("This message is translated")
 
-*(not tested)* Translations for messages whose grammatical depends
-on a number must be done by ``ngettext()`` function::
+Translations for messages whose grammatical depends on a number
+must be done by ``ngettext()`` function::
 
   print ngettext("Nuked %i file", "Nuked %i files", number_of_files_nuked)
 
-*Discussion:* make ``i18n._()`` with the same interface as in ``config._()``?
-
 User Interfaces
 ~~~~~~~~~~~~~~~
 
@@ -69,17 +135,19 @@ User Interfaces
 This includes Mail Gateway and Web User Interfaces, where translation
 depends on the language of current Roundup User.  These translations
 will be done by the tracker configuration object.  Translatable strings
-will be automatically marked by calls to the ``_()`` method of that
-object::
+will be automatically marked by calls to the ``_()`` and ``ngettext()``
+methods of that object::
 
   self.config._("This message is translated")
-  self.config._("Nuked %i file", "Nuked %i files", number_of_files_nuked)
+  self.config.ngettext("Nuked %i file", "Nuked %i files",
+      number_of_files_nuked)
 
 Deferred Translations
 ~~~~~~~~~~~~~~~~~~~~~
 
 Sometimes translatable strings appear in the source code in untranslated
-form and must be translated elsewhere.  Example::
+form [#note_admin.py]_ and must be translated elsewhere.
+Example::
 
   for meal in ("spam", "egg", "beacon"):
       print _(meal)
@@ -97,23 +165,39 @@ types of string quotes::
       multiline string"""
   )
 
-Building Message Catalog Template
----------------------------------
+.. [#note_admin.py] In current Roundup sources, this feature is
+   extensively used in the ``admin`` module using method docstrings
+   as help messages.
+
+Extracting Translatable Messages
+--------------------------------
 
-Message catalog template ``roundup.pot`` is kept in `Roundup CVS`_
-and distributed with `Roundup Source`_.  If you wish to rebuild
-the template yourself, you will need ``xpot`` utility by Francois
-Pinard, included in `PO utilities`_ distribution.
+The most common tool for message extraction is ``xgettext`` utility
+from `GNU gettext package`_.  Unfortunately, this utility has no means
+of `Deferred Translations`_ in Python sources.
+There is ``xpot`` tool from Francois Pinard free `PO utilities`_
+that allows to mark strings for `deferred translations`_, but
+it does not handle `plural forms`_.
+Roundup overcomes these limitations by using both of these utilities.
+This means that you need both `GNU gettext`_ tools and `PO utilities`_
+to build the Message Template File yourself.
 
-To rebuild the template file, just run ``gmake`` (or ``make``, if you
-are on a `GNU`_ system like `linux`_ or `cygwin`_) in the 'locale'
-directory.
+Latest Message Template File is kept in `Roundup CVS`_ and distributed
+with `Roundup Source`_.
+If you wish to rebuild the template yourself, make sure that you have
+both ``xpot`` and ``xgettext`` installed and just run ``gmake`` (or
+``make``, if you are on a `GNU`_ system like `linux`_ or `cygwin`_)
+in the ``locale`` directory.
 
-.. Roundup CVS: http://sourceforge.net/cvs/?group_id=31577
-.. Roundup Source: http://sourceforge.net/project/showfiles.php?group_id=31577
-.. GNU: http://www.gnu.org/
-.. linux: http://www.linux.org/
-.. cygwin: http://www.cygwin.com/
+Translating Messages
+--------------------
+
+FIXME!
+
+Compiling Message Catalogs
+--------------------------
+
+FIXME!
 
 I18 Status
 ----------
@@ -200,3 +284,24 @@ test/test_schema.py
 test/test_templating.py
 test/unittest.py
 
+.. External hyperlink targets
+
+.. _cygwin: http://www.cygwin.com/
+.. _emacs: http://www.gnu.org/software/emacs/
+.. _gettext package: http://www.gnu.org/software/gettext/
+.. _gettext module: http://docs.python.org/lib/module-gettext.html
+.. _GNU: http://www.gnu.org/
+.. _GNU mirror sites: http://www.gnu.org/prep/ftp.html
+.. _FreeBSD: http://www.freebsd.org/
+.. _linux: http://www.linux.org/
+.. _Plural Forms:
+    http://www.gnu.org/software/gettext/manual/html_node/gettext_150.html
+.. _po filetype plugin:
+    http://vim.sourceforge.net/scripts/script.php?script_id=695
+.. _PO utilities: http://po-utils.progiciels-bpi.ca/
+.. _poEdit: http://poedit.sourceforge.net/
+.. _Roundup CVS: http://sourceforge.net/cvs/?group_id=31577
+.. _Roundup Source:
+.. _Roundup source distribution:
+.. _Roundup binary distribution:
+    http://sourceforge.net/project/showfiles.php?group_id=31577