doc: prep announcement.txt for 2.4.0beta1

Still have to add the classhelper web-component, but get the
rest of the beta announcement ready.

[skip travis]

Author: rouilj

doc/announcement.txt

@@ -1,4 +1,4 @@
-I'm proud to release version 2.3.0 of the Roundup issue
+I'm proud to release version 2.4.0 beta of the Roundup issue
 tracker.  This release is a bugfix and feature
 release, so make sure to read `docs/upgrading.txt
 <https://www.roundup-tracker.org/docs/upgrading.html>`_ to
@@ -12,131 +12,150 @@ update the database schema version. Do this before you use
 the web, command-line or mail interface and before any users
 access the tracker.
 
-You can download it with::
-
-   pip download roundup
-
-then unpack and test/install the tarball. Also::
+You can install it with::
 
    pip install roundup
 
-(preferably in a virtual environment) can be used.
+(preferably in a virtual environment). To download it, use::
 
-Among the notable improvements from the 2.2.0 release are:
+   pip download roundup
+
+then unpack and test/install from the tarball.
 
-* Dockerfile demo mode implemented. This allows quick evaluation as
-  well as the ability to spin up a configured tracker to customise.
+Among the notable improvements from the 2.3.0 release are:
 
-* SQLite backends can use WAL mode to reduce blocking between readers
-  and writers improving concurrent use.
+* new classhelper component thanks to a team of students
+  from CS682 at U-Mass Boston. This fixes many issues with
+  the old classhelper. It is implemented as a web-component
+  and needs REST interface access. It will fall back to the
+  classic classhelper if REST is not available or if the
+  browser does not support web-components.
 
-* Redis can be used for session database with SQLite and dbm
-  backends. Provides a major performance improvement.
+* fix Windows Python installation using pip. It used to go
+  into an infinite loop during install or download. Also fix
+  installation of shared files (templates) so roundup-admin
+  can find them.
 
-* roundup-mailgw can use OAUTH authentication to SMTP
-  server. (roundup-mailgw command line options changed as a result.)
+* using ``@current_user`` as a value in a search URL for a
+  user property will use the current logged in user. Now you
+  can share searches like: "My issues" as "my" will become
+  the current logged in user.
 
-* Postgres full text index can now be enabled.
+* login failures to the REST/Xmlrpc interfaces ae now rate
+  limited.
 
-* Modifications to in-reply-to threading when there are multiple
-  matches resulting in more predictable handling of messages.
+* utf8mb4 is the default charset for MySQL. This requires
+  migrating your database using the mysql client. You can
+  choose to keep the older character set in config.ini.
 
-* Many updates to documentation to make it scannable, useful and
-  work on mobile.
+* PostgreSQL services defined in pg_service.conf can be
+  used.  PostgreSQL schemas are supported to eliminate the
+  need for the roundup user to have database
+  creation/deletion privs.
 
-* Admin documentation includes a section on setting up Content
-  Security Policy (CSP) to better secure your Roundup trackers.
+* fix out of memory issue when importing larger trackers
+  into PostgreSQL.
 
-* REST now allows rate limiting headers to be accessed by client
-  JavaScript.
+* multiple roundup-admin improvements: display protected
+  properties (like creation date), better formatting of
+  output, command history. Also on windows, pyreadline3 is
+  supported to provide an editable interactive command line.
 
-* Default number of rounds for PBKDF2 updated to 2M to account for
-  improvements in password crackers and CPU power.
+* an experimental wsgi performance improvment in 2.3.0 is
+  now now the default and is opt-out.
 
-* Support PBKDF2 with SHA512 for password storage to improve
-  resistance to password crackers.
+* new template functions: utils.readfile and
+  utils.expandfile. Javascript that is included in the
+  Python core will be moved to external files and be able to
+  have values from Roundup substituted in the Javascript.
 
-* Deprecate SSHA password hash function.
+* allow content-type of a template to be set from inside the
+  template.  This allows returning json or xml from a
+  template without a .json or .xml extention.
 
-* roundup-admin reindex can be done in batches to manage load
-  incurred by reindexing.
+* fix import/export on windows to use Unix style line
+  endings fixing export/import on Windows and making exports
+  portable across platforms.
 
-* roundup-admin can list available templates and their installed
-  locations. This is useful when installing via pip or in a docker
-  container as supporting files are not stored in the usual locations
-  like /usr/share/roundup.
+* various other Windows platform fixes including test suite
+  fixes.
 
-* Crash fixes in detector handling
+* sqlite version 1 and StructuredText support removed.
 
-The file CHANGES.txt has a detailed list of feature additions and
-bug fixes (53) for each release. The most recent changes from
-there are at the end of this announcement. Also see the
-information in doc/upgrading.txt.
+The file CHANGES.txt has a detailed list of feature
+additions and bug fixes (58) for each release. The most
+recent changes from there are at the end of this
+announcement. Also see the information in doc/upgrading.txt.
 
-If you find bugs, please report them to issues AT roundup-tracker.org
-or create an account at https://issues.roundup-tracker.org and open a
-new ticket. If you have patches to fix the issues they can be attached
-to the email or uploaded to the tracker.
+If you find bugs, please report them to issues AT
+roundup-tracker.org or create an account at
+https://issues.roundup-tracker.org and open a new ticket. If
+you have patches to fix the issues they can be attached to
+the email or uploaded to the tracker.
 
 Upgrading
 =========
 
-If you're upgrading from an older version of Roundup you *must* follow
-all the "Software Upgrade" guidelines given in the doc/upgrading.txt
-documentation.
+If you're upgrading from an older version of Roundup you
+*must* follow all the "Software Upgrade" guidelines given in
+the doc/upgrading.txt documentation.
 
 Note that you should run ``roundup-admin ... migrate`` for
 all your trackers to update the database schema version. Do
 this before you use the web, command-line or mail interface
 and before any users access the tracker.
 
-Roundup requires Python 2 newer than version 2.7.12 or Python 3 newer
-than or equal to version 3.6 for correct operation. (Python
-3.4 or 3.5 may work, but are not tested.) Note that Python 2 support
-is being removed from the CI platforms, so you should deploy new
-trackers with Python 3 and plan on upgrading older trackers from Python
-2 to Python 3. See the upgrade guide.
+Roundup requires Python 2 newer than version 2.7.12 or
+Python 3 newer than or equal to version 3.6 for correct
+operation. (Python 3.4 or 3.5 may work, but are not tested.)
+Note that Python 2 support is being removed from the CI
+platforms, so you should deploy new trackers with Python 3
+and plan on upgrading older trackers from Python 2 to Python
+3. See the upgrade guide.
 
-To give Roundup a try, just download (directions above), unpack and run::
+To give Roundup a try, just download (directions above),
+unpack and run::
 
     python demo.py
 
 then open the url printed by the demo app.
 
 Release info and download page:
+
      https://pypi.org/project/roundup/
+
 Source and documentation is available at the website:
+
      https://www.roundup-tracker.org/
+
 Mailing lists - the place to ask questions:
+
      https://sourceforge.net/p/roundup/mailman/
 
 
 About Roundup
 =============
 
-Roundup is a simple-to-use and install issue-tracking system with
-command-line, web and e-mail interfaces. It is based on the winning design
-from Ka-Ping Yee in the Software Carpentry "Track" design competition.
-
-Note: Ping is not responsible for this project. The contact for this
-project is rouilj at users.sourceforge.net. Use this address for
-security or other sensitive issues. Development discussions occur on
-the roundup-devel at lists.sourceforge.net mailing list. Tickets can
-be opened at https://issues.roundup-tracker.org.
+Roundup is a simple-to-use and install issue-tracking system
+with command-line, web and e-mail interfaces. It is based on
+the winning design from Ka-Ping Yee in the Software
+Carpentry "Track" design competition.
 
-Roundup manages a number of issues (with flexible properties such as
-"description", "priority", and so on) and provides the ability to:
+Roundup manages a number of issues (with flexible properties
+such as "description", "priority", and so on) and provides
+the ability to:
 
 (a) submit new issues,
 (b) find and edit existing issues, and
 (c) discuss issues with other participants.
 
-The system facilitates communication among the participants by managing
-discussions and notifying interested parties when issues are edited. One of
-the major design goals for Roundup that it be simple to get going. Roundup
-is therefore usable "out of the box" with any Python 2.7.2+ (or 3.6+)
-installation. It doesn't even need to be "installed" to be operational,
-though an install script is provided.
+The system facilitates communication among the participants
+by managing discussions and notifying interested parties
+when issues are edited. One of the major design goals for
+Roundup that it be simple to get going. Roundup is therefore
+usable "out of the box" with any Python 3.6+
+installation. It doesn't even need to be "installed" to be
+operational, though an install script is provided.
 
 It comes with five basic issue tracker templates
 
@@ -146,161 +165,12 @@ It comes with five basic issue tracker templates
 * a jinja2 version of the devel template (work in progress)
 * a minimal skeleton
 
-and supports four database back-ends (anydbm, sqlite, mysql and postgresql).
+and supports four database back-ends (anydbm, sqlite, mysql
+and postgresql).
 
 Recent Changes
 ==============
 
-From 2.2.0 to 2.3.0
-
-Fixed:
-------
-
-- Updated directions for verifying Roundup distribution using pgp.
-- Dockerfile healthcheck fixed so it works when trackers are
-  specified on command line. Also cleanup of unneeded
-  packages. (John Rouillard)
-- issue2551224 - Replace dbm db for sessions and otks when using
-  sqlite. New databases are created for session data (db-session)
-  and one time key data (db-otk). The data is ephemeral so no
-  need to migrate. (John Rouillard)
-- issue2551223 - Timestamps are truncated in mysql and postgresql
-  for session and otk database tables. Modify db schema to use a
-  numeric type that preserves more significant figures. See
-  upgrading.txt for required steps. (John Rouillard)
-- added more testing of BasicDatabase to support use of SQLite
-  for that purpose. Had to fix memory, rdbms and dbm edge cases
-  due to new tests. (John Rouillard)
-- issue2551138 - roundup-server with ssl under python2 throws
-  traceback on socket close. Not sure how this got fixed,
-  but after fixing issue2551137 it was not an issue anymore.
-- issue2551137 - roundup-server won't run with ssl under python3
-  Fixed by using SocketIO and manually adding buffering io and
-  catching SSL.ZeroReturnError indicating SSL has been shut down.
-- add caching header for text/javascript in addition to depricated
-  application/javascript. (John Rouillard)
-- Enable postgres-fts: fix indexer-common::get_indexer so it returns a
-  postgresql-fts Test code paths in get_indexer. (John Rouillard)
-- Fix Postgres native-fts, implement a two phase initialization of the
-  indexer. The native-fts one gets assigned after the database
-  connection is open. (John Rouillard)
-- fix crash if postgresql native-fts backend is asked to index content
-  with null bytes. (John Rouillard)
-- issue2551232 - modify in-reply-to threading when multiple matches
-  Change how in-reply-to threading works in the mailgw. If there is
-  more than one issue with a matching parent message, fall back to
-  subject matching. See upgrading.txt for details. (John Rouillard)
-- issue2551195 - port scripts from optparse to argparse (Ralf Schlatterbeck)
-- issue2551246 - mitigation, document how -u doesn't work for
-  roundup-admin. (John Rouillard)
-- Document better that files in the template or static_files
-  directories accessed via @@file are available to any user with the
-  url. (John Rouillard)
-- Fix final exception handler in roundup-server to send proper
-  Content-Length header to the client. (John Rouillard)
-- Fix traceback if Origin header is missing. (John Rouillard)
-- issue2551250: Fix sorting of detectors even if there are two with the
-  same name and priority (can happen if they are created in two
-  different files).
-- Fix Traceback when a numeric order attribute is empty (Ralf
-  Schlatterbeck)
-- Update some template schema files to assign Register permissions for the
-  Anonymous user. Replaces the old Create permission. (John Rouillard)
-- Allow '*' and explicit origins in allowed_api_origins. Only return 
-  'Access-Control-Allow-Credentials' when not matching '*'. Fixes
-  security issue with rest when using '*'.  (John Rouillard)
-- issue2551263: In REST response expose rate limiting, sunset, allow
-  HTTP headers to calling JavaScript.  (John Rouillard)
-- issue2551257: When downloading an attached (user supplied file),
-  make sure that an 'X-Content-Type-Options: nosniff' header is sent.
-  (John Rouillard)
-- issue2551252 - default number of rounds for PKDF2 password increased
-  to 2,000,000.  (John Rouillard)
-- issue2551251 - migrate/re-encrypt PBKDF2 password if stored
-  password used a smaller number of rounds than set in
-  password_pbkdf2_default_rounds.  (John Rouillard)
-- upgrade from jquery-3.5.1 to jquery-3.6.3. Update user.help.html
-  to new version.  (John Rouillard)
-- Dockerfile scanned with hadolint. Fixed multiple issues. (John Rouillard)
-- fix crash due to invalid initialization/reset of configuration.py
-  option_validators. Crashed roundup-admin on second command if an
-  option_validator was added by a detector or extension. (John Rouillard)
-- Dockerfile uses dumb-init to properly wait for child/zombie
-  processes. Defense against child process starting from detector
-  and becoming a zombie when its roundup-server instance exits.
-  (John Rouillard)
-- Move installed frontend/Zope back to frontend/ZRoundup
-  directory. This better identifies the directory when copied into
-  the Zope framework. It also matches existing
-  documentation. (John Rouilard)
-- Multiple fixes/updates for installation documentation.
-  Including docker shell/admin/demo mdoes. (John Rouillard)
-- Invalid item identifiers passed to REST endpoint return a 404
-  rather than a 400 error. E.G. /rest/data/issue/issue4 (rather
-  than .../issue/4). (John Rouillard)
-- issue2551280 - sorted() method of MultilinkHTMLProperty is broken?
-  (Gabor Nagy report and fix; commit John Rouillard)
-
-Features:
----------
-
-- Add warning about limited Python 2 support lifetime to install and
-  upgrading docs. (John Rouillard)
-- Dockerfile supports demo mode for instant gratification
-  8-). Also supports shell and admin mode (John Rouillard)
-- Dockerfile build allows adding additional python packages via
-  pip, setting UID tracker is run under. (John Rouillard)
-- issue2551140 - Added redis as a session and otk database for use
-  with anydbm and sqlite primary databases. (John Rouillard)
-- issue2550559 - Pretty printing / formatting for Number types.
-  Added pretty(format='%0.3f') method to NumberHTMLProperty to
-  print numeric values. If value is None, return empty string
-  otherwise str() of value. (John Rouillard)
-- sqlite native-fts backend now uses the stopwords list in config.ini
-  to filter words from queries. (Stopwords are still indexed so that
-  phrase/proximity searches still work.) (John Rouillard)
-- sqlite databases use WAL mode when *created* to improve read
-  concurrency. Existing sqlite database still use rollback journal
-  mode. See upgrading.txt for details. (John Rouillard)
-- issue2551233 - create new roundup-admin command "templates" list all
-  template names, location and descriptions. Should help find where
-  /usr/share/roundup/templates is buried during some install
-  mechanisms. Does not need a tracker home to run. (John Rouillard)
-- Add OAuth authentication to the mailgw script. Now IMAPS can be used
-  with OAuth as required by several large cloud providers. Move command
-  line processing of the mailgw script to ``argparse``. Note that the
-  command line options of the mailgw have changed, see upgrading.txt for
-  details. (Ralf Schlatterbeck)
-- issue2551243: schema-dump.py enhanced with anti-CSRF headers. Flake8
-  cleanup and python2 support. (John Rouillard)
-- issue2551253 - new password hash PBDKF2-SHA512 added. Not
-  available by default. Follow directions in upgrading document
-  to use.  (John Rouillard)
-- roundup-admin migrate command reports the schema version.
-- issue2551262 - the mail gateway subject prefix now allows spaces
-  before/after prefix. Also allow spaces between classname and id
-  number in prefix designator. So "[ issue 23   ] subject" is parsed
-  like "[issue23] subject". (John Rouillard)
-- [doc]: add section on implementing CSP for Roundup to admin
-  doc. (John Rouillard)
-- issue2551265 - deprecate SSHA password hash method. Users using SSHA
-  passwords will have their passwords transprently upgraded to PBKDF2
-  derived hash on next login. (John Rouillard)
-- issue2551253 - Modify password PBKDF2 method to use SHA512. New
-  hash function using PBKDF2-SHA512 available. Will be default in
-  future. Directions for upgrading security by using it now is
-  included in upgrading.txt. (John Rouillard)
-- issue2551275 - Allow configuring max_children in roundup-server.
-  When using roundup-server in fork mode, allow raising number of
-  forked children above the default of 40. (Original patch by Joseph
-  Myers, config settings and docs by John Rouillard.)
-- roundup-admin genconfig does not need a tracker home to run. (John
-  Rouillard)
-- issue2551190 - Allow roundup-admin reindex to work in
-  batches. Running roundup-admin -i ... reindex issue:1-1000 will
-  reindex the first 1000 issues while reporting any missing issues
-  in the range. Also completion progress is reported when indexing a
-  specific class.
-- doc updates: add explanation for SQL code in 1.3.3->1.4.0 upgrade.
-  document schema table in rdbms backends and how to dump/extract
-  version from them. (John Rouillard)
+From 2.3.0 to 2.4.0
+
+TBA