1. 03 Sep, 2014 10 commits
    • Alex Vandiver's avatar
      Standardize on the stricter Encode::encode("UTF-8", ...) everywhere · 1d18663b
      Alex Vandiver authored
      This is not only for code consistency, but also for consistency of
      output.  Encode::encode_utf8(...) is equivalent to
      Encode::encode("utf8",...) which is the non-"strict" form of UTF-8.
      Strict UTF-8 encoding differs in that (from `perldoc Encode`):
      
          ...its range is much narrower (0 ..  0x10_FFFF to cover only 21 bits
          instead of 32 or 64 bits) and some sequences are not allowed, like
          those used in surrogate pairs, the 31 non-character code points
          0xFDD0 .. 0xFDEF, the last two code points in any plane (0xXX_FFFE
          and 0xXX_FFFF), all non-shortest encodings, etc.
      
      RT deals with interchange with databases, email, and other systems.  In
      dealing with encodings, it should ensure that it does not produce byte
      sequences that are invalid according to official Unicode standards.
      1d18663b
    • Alex Vandiver's avatar
      Verify that MIME::Entity headers are bytes, and remove _utf8_off call · ba110857
      Alex Vandiver authored
      See the prior commit for reasoning, which applies just as much to the
      header as the body.
      ba110857
    • Alex Vandiver's avatar
      Verify that MIME::Entity bodies are bytes, and remove _utf8_off call · 17702cde
      Alex Vandiver authored
      Use the newly-added RT::Util::assert_bytes function to verify that the
      body is indeed bytes, and not characters.
      
      We also remove the _utf8_off call -- because, contrary to what the
      comment implies, the presence or absence of the "UTF8" flag does _not_
      determine if a string is "encoded as octets and not as characters"; it
      merely states that the string is capable of holding codepoints > 255.
      If it happens to not contain any, the _utf8_off does nothing.  If it
      does, it effectively encodes all codepoints > 127 in UTF-8.
      
      Given the premise that the string contains bytes in some (probably
      non-UTF-8) encoding, re-encoding some bytes of it as UTF-8 cannot
      possibly produce valid output.  The flaw in this situation cannot be
      fixed by a simple _utf8_off, but instead must be fixed by ensuring that
      the body always contains bytes, not wide characters -- as it now does,
      thanks to the prior commits.  The call to RT::Util::assert_bytes serves
      as an additional safeguard against backsliding on that assumption.
      17702cde
    • Alex Vandiver's avatar
      Add a utility method to check that an input is bytes · a21eb81c
      Alex Vandiver authored
      Note that it is impossible to verify that an input is characters; here,
      we can only validate if it _could_ be bytes.
      
      First, any string with the "UTF8" flag off cannot contain codepoints
      above 255, and as such is safe.  Additionally, if the "UTF8" flag is on,
      having no codepoints above 127 means the bytes are unambigious.  Having
      codepoints above 255 is guaranteedly a sign that the input is not a byte
      string.
      
      This leaves only the case of a string with the "UTF8" flag on, and
      codepoints above 127 but below 255.  The "UTF8" flag is a sign that they
      were _likely_ touched by character data at some point.  In such cases we
      warn, suggesting that the bytes have the "UTF8" flag disabled by means
      of utf8::downgrade, if they are indeed bytes.
      a21eb81c
    • Alex Vandiver's avatar
      Make RT::Action::SendEmail->SetHeader take characters, not bytes · 12c2671c
      Alex Vandiver authored
      This helper method is used in a number of places in
      RT::Action::SendEmail, often without remembering that it should be
      passed bytes, not characters.  Change it to always take characters, and
      modify the two callsite which (correctly) passed it bytes to no longer
      do so.
      12c2671c
    • Alex Vandiver's avatar
      Ensure all MIME::Entity headers are UTF-8 encoded bytes · 41d084f1
      Alex Vandiver authored
      Placing wide characters into MIME::Entity objects can lead to
      double-encoding, as discovered most recently in d469cacc.  Explicitly
      decode all headers as UTF-8 when retrieving them with ->get(), and
      encode them as UTF-8 before updating them with ->set() or ->replace().
      This also applies to headers passed to ->build().  The only exceptions
      to this are fixed strings in the source (which, in the absence of "use
      utf8", are always bytes).
      
      While the majority of these headers will never have wide characters in
      them, always decoding and encoding ensures the proper disipline to
      guarantee that strings with the "UTF8" flag do not get placed in a
      header, which can cause double-encoding.
      41d084f1
    • Alex Vandiver's avatar
      Ensure all MIME::Entity bodies are UTF-8 encoded bytes · 6d9bd63c
      Alex Vandiver authored
      Placing wide characters into MIME::Entity objects can lead to
      double-encoding.  Always treat them as byte stores, encoding as UTF-8
      and noting their character set.
      
      In the case of Approvals/index.html, there was no need for an explicit
      MIME::Entity object; ->Correspond creates one as needed from a "Content"
      argument.
      6d9bd63c
    • Alex Vandiver's avatar
      The alluded-to deficiency is not a concern in perl ≥ 5.8.3 · 18ef9b24
      Alex Vandiver authored
      The comment and code were added in RT 2.1.38, which only required perl
      5.6.1; the perl version was increased to 5.8.3 to cover a large number
      of encoding bugs, such as the one this comment was likely alluding to.
      18ef9b24
    • Alex Vandiver's avatar
      Always log bytes, not characters · a275a7fa
      Alex Vandiver authored
      Ensure that we always send UTF-8 encoded bytes to loggers, and not wide
      characters.  This is correctly done via an explicit call to
      Encode::encode, and not via checks of utf8::is_utf8 (which may be false
      for character strings with codepoints > 127 but < 256), and not via
      _utf8_off (which would fail similarly for such characters).
      a275a7fa
    • Alex Vandiver's avatar
      Modernize and condense t/mail/sendmail.t and t/mail/sendmail-plaintext.t · 15dde68b
      Alex Vandiver authored
      t/data/emails/text-html-in-russian was removed because the original
      purpose of the test was removed in 46fd04d9, after 90f9c190 stopped
      attaching text/html incoming mail to autoreplies.
      15dde68b
  2. 02 Sep, 2014 4 commits
  3. 28 Aug, 2014 1 commit
    • Kevin Falcone's avatar
      Allow modification of which types of links are shown · 191f789c
      Kevin Falcone authored
      There's a callback to add data at the end of the Links portlet, but
      nothing to allow you to change what's display.  This provides a callback
      that would let you remove (for example) the Parents from the listing, so
      that you can list them in a separate portlet later, with a custom
      display.
      191f789c
  4. 27 Aug, 2014 4 commits
  5. 25 Aug, 2014 2 commits
  6. 19 Aug, 2014 2 commits
  7. 18 Aug, 2014 6 commits
  8. 15 Aug, 2014 4 commits
  9. 13 Aug, 2014 7 commits
    • Kevin Falcone's avatar
      Allow the SuccessfulLogin callback to change where a user Redirects to · 6a923e70
      Kevin Falcone authored
      This allows an extension to overwrite a user's intended destination, or
      more usefully, detect when someone is logging in at the top level and
      redirect instead to a Dashboard (or in RTIR's case, to /RTIR/).
      6a923e70
    • Kevin Falcone's avatar
      Allow extra arguments to be passed to ShowHistory · ec0bb0d9
      Kevin Falcone authored
      RTIR wants to customize a number of paths during history display (see
      RTIR/Display.html and RTIR/Incident/Display.html).  In order to use
      delayed history loading, RTIR either needs to pass those as GET
      arguments (allowing user injection of paths that we use to build links,
      which is dangerous without sanitation) or otherwise get them to
      ShowHistory.
      
      With this, RTIR can just callback in the custom arguments (you can also
      clobber any of the other arguments to ShowHistory with it).
      ec0bb0d9
    • Kevin Falcone's avatar
      Ignore vim swap files when testing · 1dd472e6
      Kevin Falcone authored
      mason-syntax already ignored swp files, but would try to compile swo
      files.  t/web/helper-http-cache-headers.t made no attempt to ignore
      anything, so if you're hacking on a Helper this test will fail
      spectacularly.
      1dd472e6
    • Kevin Falcone's avatar
      Still delay loading when showing headers · 1ff7f30c
      Kevin Falcone authored
      In 75a405b5 we added the ForceShowHistory flag to Click to load history
      so that even if your JS was off, clicking on the link would load the
      page.
      
      This flag is also found in the Show Headers links added in 1a3327ce
      and I think added so that when you clicked on ShowHeaders with Click to
      load enable, you didn't *then* have to click on "Load history" just to
      see the headers.
      
      However, with delay, we already passed ShowHeaders to the helper, but
      clicking on Show Headers forced us to do the page load inline instead.
      This negates all of the perceived speed boost from delayed loading.
      Removing it from the delay case hasn't immediately broken things, and
      the delayed loading really doesn't play well with having JS off anyway.
      1ff7f30c
    • sunnavy's avatar
      make attachment truncate/drop and error txns more noticeable · 4aaa1324
      sunnavy authored
      color scheme is from cloos++
      4aaa1324
    • sunnavy's avatar
      test attachment dropping/truncation · 15452374
      sunnavy authored
      15452374
    • sunnavy's avatar
      record attachments' dropping/truncation as txns · 15dddd2e
      sunnavy authored
      15dddd2e