Added support for accepting application/json payload in addition to

the existing application/x-www-form-urlencoded.

The key for this is that the third element of the FieldStorage is a
string as opposed to a list. So the code checks for the string and
that the Content-Type is exactly application/json. I do a string match
for the Content-Type.

This code also adds testing for the dispatch method of
RestfulInstance.  It tests dispatch using GET, PUT, POST, PATCH
methods with json and form data payloads. Existing tests bypass the
dispatch method.

It moves check for pretty printing till after the input payload is
checked to see if it's json. So you can set pretty in the json payload
if wanted.

Adds a new class: SimulateFieldStorageFromJson. This class
emulates the calling interface of FieldStorage. The json payload
is parsed into this class. Then the new object is passed off to the
code that expects a FieldStorage class. Note that this may or may not
work for file uploads, but for issue creation, setting properties,
patching objects, it seems to work.

Also refactored/replaced the etag header checks to use a more generic
method that will work for any header (e.g. Content-Type).

Future enhancements are to parse the full form of the Content-Type
mime type so something like: application/vnd.roundup.v1+json will also
work. Also the SimulateFieldStorageFromJson could be used to represent
XML format input, if so need to rename the class dropping
FromJson. But because of the issues with native xml parsers in python
parsing untrusted data, we may not want to go that route.

curl examples for my tracker is:

curl -s -u user:pass -X POST --header 'Content-Type: application/json' \
   --header 'Accept: application/json' \
   --data '{"title": "foo bar", "fyi": "text", "private": "true", "priority": "high" }' \
   -w "http status: %{http_code}\n" \
   "https://example.net/demo/rest/data/issue"
{
    "data": {
        "link":	"https://example.net/demo/rest/data/issue/2229",
        "id": "2229"
    }
}

http status: 201

Author: rouilj

CHANGES.txt

@@ -65,6 +65,8 @@ Features:
      collection property. Add @links property with self, next and prev
      links (where needed). Add @total_size with size of entire
      collection (unpaginated). Pagination index starts at 1 not 0.
+   - accept content-type application/json payload for PUT, PATCH, POST
+     requests in addition to application/x-www-form-urlencoded.
   (John Rouillard)
 - issue2550833: the export_csv web action now returns labels/names
   rather than id's. Replace calls to export_csv with the export_csv_id

roundup/rest.py

@@ -1331,12 +1331,6 @@ def dispatch(self, method, uri, input):
             # .../issue.json -> .../issue
             uri = uri[:-( len(ext_type) + 1 )]
 
-        # check for pretty print
-        try:
-            pretty_output = not input['pretty'].value.lower() == "false"
-        except KeyError:
-            pretty_output = True
-
         # add access-control-allow-* to support CORS
         self.client.setHeader("Access-Control-Allow-Origin", "*")
         self.client.setHeader(
@@ -1352,6 +1346,35 @@ def dispatch(self, method, uri, input):
             "HEAD, OPTIONS, GET, PUT, DELETE, PATCH"
         )
 
+        # Is there an input.value with format json data?
+        # If so turn it into an object that emulates enough
+        # of the FieldStorge methods/props to allow a response.
+        content_type_header = headers.getheader('Content-Type', None)
+        if type(input.value) == str and content_type_header:
+            parsed_content_type_header = content_type_header
+            # the structure of a content-type header
+            # is complex: mime-type; options(charset ...)
+            # for now we just accept application/json.
+            # FIXME there should be a function:
+            #   parse_content_type_header(content_type_header)
+            # that returns a tuple like the Accept header parser.
+            # Then the test below could use:
+            #   parsed_content_type_header[0].lower() == 'json'
+            # That way we could handle stuff like:
+            #  application/vnd.roundup-foo+json; charset=UTF8
+            # for example.
+            if content_type_header.lower() == "application/json":
+                try:
+                    input = SimulateFieldStorageFromJson(input.value)
+                except ValueError as msg:
+                    output = self.error_obj(400, msg)
+
+        # check for pretty print
+        try:
+            pretty_output = not input['pretty'].value.lower() == "false"
+        except KeyError:
+            pretty_output = True
+
         # Call the appropriate method
         try:
             # If output was defined by a prior error
@@ -1392,3 +1415,48 @@ def default(self, obj):
         except TypeError:
             result = str(obj)
         return result
+
+class SimulateFieldStorageFromJson():
+    '''
+    The internals of the rest interface assume the data was sent as 
+    application/x-www-form-urlencoded. So we should have a 
+    FieldStorage and MiniFieldStorage structure.
+
+    However if we want to handle json data, we need to:
+      1) create the Fieldstorage/MiniFieldStorage structure
+    or
+      2) simultate the interface parts of FieldStorage structure
+
+    To do 2, create a object that emulates the:
+
+          object['prop'].value
+
+    references used when accessing a FieldStorage structure.
+
+    That's what this class does.
+
+    '''
+    def __init__(self, json_string):
+        ''' Parse the json string into an internal dict. '''
+        def raise_error_on_constant(x):
+            raise ValueError, "Unacceptable number: %s"%x
+
+        self.json_dict = json.loads(json_string,
+                                    parse_constant = raise_error_on_constant)
+        self.value = [ self.FsValue(index, self.json_dict[index]) for index in self.json_dict.keys() ]
+
+    class FsValue:
+        '''Class that does nothing but response to a .value property '''
+        def __init__(self, name, val):
+            self.name=name
+            self.value=val
+
+    def __getitem__(self, index):
+        '''Return an FsValue created from the value of self.json_dict[index]
+        '''
+        return self.FsValue(index, self.json_dict[index])
+
+    def __contains__(self, index):
+        ''' implement: 'foo' in DICT '''
+        return index in self.json_dict
+