GitLab

Oracle uses the MaxAttachmentSize to also control the maximum piece of
any BLOB column which is returned.  This hence also affects sessions,
and causes the attachment truncation tests to fail because login does
not succeed because the returned session data is truncated.

Set the test to use on-disk sessions to work around the spurious logouts
caused by a too-low MaxAttachmentSize on Oracle.

A queue name is easily changed within the WebUI and maybe the admin forget to
also update the OverrideOutgoingMailFrom setting. It more reliable to use the
queue id instead the queue name as the hash key.

This removes the preferences caching introduced in 4.2 in ddfeca8e,
replacing it with that of 4.0 from a273c51b and 5b707dfe.  It also
reolves a number of conflicts brought about by 4.0/utf8-reckoning and
its 4.2 counterpart.

Conflicts:
	lib/RT/Action/SendEmail.pm
	lib/RT/Attachment.pm
	lib/RT/Crypt/GnuPG.pm
	lib/RT/Dashboard/Mailer.pm
	lib/RT/I18N.pm
	lib/RT/I18N/fr.pm
	lib/RT/Interface/Email.pm
	lib/RT/Interface/Email/Auth/Crypt.pm
	lib/RT/Interface/Web.pm
	lib/RT/Record.pm
	lib/RT/Template.pm
	lib/RT/Ticket.pm
	lib/RT/Tickets.pm
	lib/RT/User.pm
	sbin/rt-test-dependencies.in
	share/html/Admin/Users/Modify.html
	share/html/Elements/EditCustomFieldDate
	share/html/Elements/EditCustomFieldDateTime
	share/html/NoAuth/css/aileron/ticket.css
	share/html/NoAuth/css/base/ticket.css
	share/html/Search/Results.tsv
	share/html/Ticket/Create.html
	share/html/Ticket/Elements/EditTransactionCustomFields
	share/html/Ticket/Elements/PreviewScrips
	share/html/Ticket/Update.html
	share/html/Tools/Offline.html
	t/api/i18n_guess.t
	t/mail/dashboard-chart-with-utf8.t
	t/mail/header-characters.t
	t/mail/sendmail.t
	t/web/offline_messages_utf8.t
	t/web/offline_utf8.t
	t/web/user_update.t

The link detection algorithm uses a depth-firsth traversal of links;
unfortunately, this can lead to connections being incorrectly omitted.
If a too-long pathway is encountered prior to a short one, the "seen"
check will omit it when it is encountered later.

While switching to breadth-first would resolve this more correctly,
simply tracking the depth that the link was found at is sufficient.

If we know we are not showing any requestor tickets there is no need for status
sorting and component lookups.

Previously the comp_exist check for "ShowRequestorTicketsNone" failed and set
$TicketTemplate to the default "ShowRequestorTicketsActive".
Fixing this by treating MoreAboutRequestorTicketList set to None the same way
as if $ShowTickets where set to a false value.

it's more right and secure.

so they don't get missed when user clicks "add more files" or something like that.

A reasonable use case for rt is to comment on a ticket (or
"correspond", which is just a comment that is also sent to the ticket
requestor).  In the web form for commenting, there is the ability to
update the status of the ticket at the same time.

This feature should be exposed in the command line client as well.

Bug-Debian: http://bugs.debian.org/760292

As input/output encoding has been cleaned up significantly, stop hiding
"Wide character in..." warnings, as they may conceal true problems.

Tests may produce characters (e.g. ok(1, "¡Oy!") ) which need to be
encoded before being output.  Use the suggested technique from the
Test::More POD to encode these strings.

Currently, any string which has the "UTF-8" flag is encoded as UTF-8
before being sent to the browser.  This requires that any output which
is binary, or has already been encoded to bytes, _not_ have the flag
accidentally set.

It also requires that all output character strings have the "UTF-8" flag
enabled; while necessary for codepoints > 255, it is not strictly
required for codepoints between 127 and 255.  As RT now consistently
uses Encode::decode() to produce character strings, which sets the
"UTF-8" flag even for characters in that range, this is likely safe.

The most correct fix would be to explicitly flag output that needs to be
encoded.  However, doing so in a backwards compatible manner is
extremely difficult; as is_utf8 is unlikely to be incorrect in this
context, the small potential additional correctness is deemed unworth
the cost of requiring all external modules to flag their binary (or
character) output as such.

This does require a version dump of the DBIx::SearchBuilder dependency
for PostgreSQL installs, as DBIx::SearchBuilder previous to that version
manually forced the "UTF8" off on all bound values before executing
statements.  When the character/byte distinction has been intentionally
made, manually (and unpredictably) applying an additional layer of UTF-8
encoding is incorrect.

RT::Interface::Web::Handler ensures that any code which contains
characters is encoded in UTF-8 before being sent to the browser.  As
such, the explicit encoding here is unnecessary.

929b4231 specifically documented _EncodeLOB and _DecodeLOB to take
bytes; enforce that by checking the arguments they are passed using
RT::Util::assert_bytes.

There is no need to check is_utf8 on the arguments passed to DecodeArgs,
as it is the first and only line of decoding of arguments over HTTP.

c95221e4 moved this comment from html/autohandler, but associated it
with the code it was above rather than the code above it.  Move it to
where it belongs, and update it slightly.

66930fd8 switched from an explicit _utf8_off to an explicit _utf8_on, in
an attempt to switch from splitting on bytes to splitting on characters.
However, the "UTF8" flag does not magically determine if a string is
bytes or characters.  Instead, only consistency in calling convention
can do so.  All callsites of RT::Interface::Email::EncodeToMIME and
RT::Action::SendEmail::MIMEEncodeString now pass character strings; all
that _utf8_on can do is incorrectly "decode" those strings as UTF-8 if
they happen to not have the "UTF8" flag set.

While there ay have been bugs surrounding encodings in LWP or
WWW::Mechanize previously, the ->content method correctly returns
characters and not bytes for all modern versions.  Remove the explicit
adjustment.

The dashboard mailer code replicates many of the original steps of Mason
parameter parsing -- but omitted the important step of decoding the
bytes once they had been un-URI-encoded.

All remaining cases of "use utf8" lie in the testsuite.  As "use utf8"
changes the semantics of dealing with Unicode strings, remove it to
allow programmers to always assume that literals are interpreted as
bytestrings, not characters.  To do otherwise means that one must always
ask if "use utf8" is in scope before performing operations on any
literals; instead, simply make the encodings and decodings explicit.

Note that wide characters may appear in editors, and that the encoding
of the characters on _disk_ will always be UTF-8.  The removal of "use
utf8" merely means that perl will generate a two-byte string from "é",
and not a one-character string.

"use utf8" causes the sourcecode (including all strings) to be
interpreted by perl as characters encoded in UTF-8, not bytes.  In
lib/RT/I18N/fr.pm, this was being used to substitude codepoint 160
(NO-BREAK SPACE, U+00A0) for commas.  The fact that the space character
was not 0x20, but rather 0xA0, was mostly hidden by use of "use utf8".

Remove the "use utf8" and make the replacement character clear.

This is not only for code consistency, but also for consistency of
output.  Encode::encode_utf8(...) is equivalent to
Encode::encode("utf8",...) which is the non-"strict" form of UTF-8.
Strict UTF-8 encoding differs in that (from `perldoc Encode`):

    ...its range is much narrower (0 ..  0x10_FFFF to cover only 21 bits
    instead of 32 or 64 bits) and some sequences are not allowed, like
    those used in surrogate pairs, the 31 non-character code points
    0xFDD0 .. 0xFDEF, the last two code points in any plane (0xXX_FFFE
    and 0xXX_FFFF), all non-shortest encodings, etc.

RT deals with interchange with databases, email, and other systems.  In
dealing with encodings, it should ensure that it does not produce byte
sequences that are invalid according to official Unicode standards.

See the prior commit for reasoning, which applies just as much to the
header as the body.