Ways to detect that XXXX in JSON \uXXXX does not correspond to a Unicode character?
Costello, Roger L.
costello at mitre.org
Fri May 8 04:27:03 CDT 2015
Philippe Verdy wrote:
Ø implementations just support JSON as plain 16-bit streams
Ø Try by yourself, you can perfectly send JSON text containing
Ø '\uFFFF' (non-character) or '\uD800' (unpaired surrogate) and
Ø I've not seen any JSON implementation complaining about one
Ø or the other
Okay, I gave it a try. I created this string which contains binary data (sequence of arbitrary unsigned integers):
When I validated that string against this JSON Schema:
"type" : "string"
using this online validator: https://json-schema-validator.herokuapp.com/
I got an error: Invalid JSON: parse error, line 1
I am pretty sure that Daniel is correct, JSON cannot contain arbitrary bit streams.
Ø The RFC is just informative not normative
Interesting! What does that mean? JSON vendors are free to ignore the JSON RFC and do as they see fit?
From: verdyp at gmail.com [mailto:verdyp at gmail.com] On Behalf Of Philippe Verdy
Sent: Thursday, May 07, 2015 11:08 PM
To: Daniel Bünzli
Cc: Unicode at unicode.org; Costello, Roger L.; Markus Scherer
Subject: Re: Ways to detect that XXXX in JSON \uXXXX does not correspond to a Unicode character?
The RFC is jsut informative not normative, and thez effective usage and implementations just support JSON as plain 16-bit streams, even if the transport syntax requires encoding it in plain-text (using some UTF, not necessarily UTF-8 even if this is the default).
2015-05-08 3:22 GMT+02:00 Daniel Bünzli <daniel.buenzli at erratique.ch<mailto:daniel.buenzli at erratique.ch>>:
Le vendredi, 8 mai 2015 à 02:16, Philippe Verdy a écrit :
I suggest you have a careful read at RFC 7159 as it specifically implies that this is not the model it supports (albeit using broken or let's say ambiguous/imprecise Unicode terminology).
> Then the JSON processor will decode this text and will remap it to an internal UTF-16 encoding (for characters that are not escaped) and the "\uXXXX" will be decoded as plain 16-bit code units. The result will be a stream of 16-bit code units, which can then externally be outpout and encoded or stored in any convenient encoding that preserves this stream, EVEN if this is not valid UTF-16.
I don't know where you get this from but you won't find any mention of this in the standard. We are dealing with text, Unicode scalar values, not encodings. At the risk of repeating myself, read section 8.2 of RFC 7159.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Unicode