fix: Send Vary: Accept-Encoding on any data that could be compressed

This allows upstream caches to return the correct data even when
compression is not in use. It is not sent if the content would never
be compressed. I.E. size < 100 bytes, dynamic compression
disabled, file would not benefit from compression (img/jpeg, img/png).

Fix setVary to add header to vary list only if it's not already there.

Found by redbot.org testing.

References:

  https://www.stackpath.com/blog/accept-encoding-vary-important/

Author: rouilj

CHANGES.txt

@@ -161,6 +161,9 @@ Fixed:
   rather than reporting a TypeError. (John Rouillard)
 - Make Last-Modified header use GMT not -0000 timezone. Fix error
   reported by redbot testing. (John Rouillard)
+- Send Vary: Accept-Encoding on any file that could be compressed
+  even if the file is not encoded/compressed. Found by Redbot
+  testing. (John Rouillard)
 
 Features:
 

roundup/cgi/client.py

@@ -2355,7 +2355,8 @@ def setVary(self, header):
         '''Vary header will include the new header. This will append
            if Vary exists.'''
 
-        if ('Vary' in self.additional_headers):
+        if ('Vary' in self.additional_headers and 
+            header not in self.additional_headers['Vary']):
             self.additional_headers['Vary'] += ", %s" % header
         else:
             self.additional_headers['Vary'] = header
@@ -2373,6 +2374,7 @@ def compress_encode(self, byte_content, quality=4):
         # abort if already encoded (e.g. served from
         # precompressed file or cache on disk)
         if ('Content-Encoding' in self.additional_headers):
+            # Vary: 'Accept-Encoding' is set when Content-encoding set
             return byte_content
 
         # abort if file-type already compressed
@@ -2381,6 +2383,8 @@ def compress_encode(self, byte_content, quality=4):
                 self.precompressed_mime_types):
             return byte_content
 
+        self.setVary('Accept-Encoding')
+
         encoder = None
         # return same content if unable to compress
         new_content = byte_content
@@ -2419,7 +2423,6 @@ def compress_encode(self, byte_content, quality=4):
             # and add Content-Encoding and Vary header.
             self.additional_headers['Content-Length'] = str(len(new_content))
             self.additional_headers['Content-Encoding'] = encoder
-            self.setVary('Accept-Encoding')
             try:
                 current_etag = self.additional_headers['ETag']
             except KeyError: