[jacorb-bugs] [Bug 969] New: Incorrect UTF-8 conversion for non-BMP characters

[bugzilla-daemon at jacorb.org]

http://www.jacorb.org/bugzilla/show_bug.cgi?id=969

            Bug ID: 969
           Summary: Incorrect UTF-8 conversion for non-BMP characters
           Product: JacORB
           Version: 3.3
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: ORB
          Assignee: jacorb-bugs at lists.spline.inf.fu-berlin.de
          Reporter: peter.klotz at ith-icoserve.com

Methods read_char() and write_char() in class Utf8CodeSet convert data
characterwise from/to UTF-8. This presents a problem when dealing with UTF-16
characters outside the Basic Multilingual Plane (all code points beyond
U+FFFF). Here UTF-16 requires the use of surrogate pairs which means that two
Java chars form a single character.

The following currently happens in JacORB 3.3 when sending Unicode Character
U+1044F (DESERET SMALL LETTER EW, see
http://www.fileformat.info/info/unicode/char/1044f/index.htm):

Java UTF-16 string: "\uD801\uDC4F"
Converted into UTF-8 and received by omniORB: "\xed\xa0\x81" "\xed\xb1\x8f"

The correct UTF-8 encoding would be: "\xf0\x90\x91\x8f"

So JacORB simply sees each surrogate as a character of its own and encodes it
into UTF-8. This leads to 6 bytes whereas the correct encoding would be 4 byte
in length.

To fix this, it would be necessary that JacORB no longer performs its
conversion solely on Java char basis. The conversion classes should be able to
handle Java strings. This would allow the conversion class to detect parts of
surrogate pairs and convert them in a single step into the correct destination
encoding.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spline.inf.fu-berlin.de/pipermail/jacorb-bugs/attachments/20131113/c433ecc4/attachment.html>