Unicode character property

From Wikipedia, the free encyclopedia

The Unicode Standard assigngs various properties to each Unicode character and code point.[1][2]:Name

The properties can be used to handle characters (code points) in processes, like in line-breaking, script direction right-to-left or applying controls. Some "character properties" are also defined for code points that have no character assigned, and code points that are labeled like "<not a character>". The character properties are described in Standard Annex #44.[2]

Properties have levels of forcefulness: normative, informative, contributory, or provisional. For simplicity of specification, a character property can be assigned by specifying a continuous range of code points that have the same property.[citation needed]

Semantic elements[]

Properties are displayed[where?] in the following order:

[code];[name];[gc];[cc];[bc];[decomposition];;;[nv];[bm];[alias];;;;
  • 'alias' = corrected name
  • 'bc' = bidi (bidirectional) category [L, R etc]
  • 'bm' = bidi mirrored [N or Y]
  • 'cc' = combining class [position of diacritic]
  • decomposition = letter + diacritic, ligature X Y, superscript X, font X, initial X, medial X, final X, isolated X, vertical X, etc.
  • 'gc' = general category [letter, symbol, digit, punctuation, case behavior, etc.]
  • 'nv' = numeric value [of a digit]

Name[]

A Unicode character is assigned a unique Name (na).[1] The name is composed of uppercase letters A–Z, digits 0–9, - (hyphen-minus) and <space>. Some sequences are excluded: names beginning with a space or hyphen, names ending with a space or hyphen, repeated spaces or hyphens, and space after hyphen are not allowed. The name is guaranteed to be unique within Unicode, and can be used to identify a code point and its character. Ideographic characters, of which there are tens of thousands, are named in the pattern "cjk unified ideograph-hhhh". For example, U+4E00 CJK UNIFIED IDEOGRAPH-4E00. Formatting characters are named too: U+00A0   NO-BREAK SPACE.

The following classes of code point do not have a Name (na=""): Controls (General Category: Cc), Private use (Co), Surrogate (Cs), Non-characters (Cn) and Reserved (Cn). They may be referenced, informally, by a generic or specific meta-name, called "Code Point Labels": <control>, <control-0088>, <reserved>, <noncharacter-hhhh>, <private-use-hhhh>, <surrogate>. Since these labels contain <>-brackets, they can never appear as a Name, which prevents confusion.

Version 1.0 names[]

In version 2.0 of Unicode, many names were changed. From then on the rule "a name will never change" came into effect, including the strict (normative) use of alias names. Disused version 1.0-names were moved to the property Alias, to provide some backward compatibility.

Character name alias[]

Starting from Unicode version 2.0, the published name for a code point will never change. Therefore, in the event of a character name being misspelled or if the character name is completely wrong or seriously misleading, a formal Character Name Alias may be assigned to the character, and this alias may be used by applications instead of the actual defective character name.[1] For example, U+FE18 PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRAKCET has the character name alias "PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRACKET" in order to mitigate the misspelling of "bracket" as "brakcet" in the actual character name; U+A015 YI SYLLABLE WU has the character name alias "YI SYLLABLE ITERATION MARK" because contrary to the character name it does not have a fixed syllabic value.

In addition to character name aliases which are corrections to defective character names, some characters are assigned aliases which are alternative names or abbreviations. Five types of character name aliases are defined in the Unicode Standard:

  • Correction: corrections for misspelled or seriously incorrect character names;
  • Control: ISO 6429 names for C0 and C1 control functions (which are not assigned character names in the Unicode Standard);
  • Alternate: alternative names for some format characters (only U+FEFF "ZERO WIDTH NO-BREAK SPACE" which has the alias "BYTE ORDER MARK");
  • Figment: Documented labels for some C1 control code functions which are not actual names in any standard;
  • Abbreviation: Abbreviations or acronyms for control codes, format characters, spaces, and variation selectors.

All formal character name aliases follow the rules for permissible character names, and are guaranteed to be unique within both the character name alias and the character name namespaces (for this reason, the ISO 6429 name "BELL" is not defined as an alias for U+0007 because U+1F514 is named "BELL").[1]

As of Unicode version 12.1, twenty-eight formal character name aliases are defined as corrections for defective character names.[3] These are listed below.

Apart from these normative names, informal names may be shown in the Unicode code charts. These are other commonly used names for a character, and need not be restricted to letters A–Z, digits 0–9, - (hyphen-minus) and <space>. These informal names are not guaranteed to be unique, and may be changed or removed in later versions of the standard.

General Category[]

Each code point is assigned a value for General Category. This is one of the character properties that are also defined for unassigned code points, and code points that are defined "not a character".

hideGeneral Category (Unicode Character Property)[a]
Value Category Major, minor Basic type[b] Character assigned[b] Count
(as of 13.0)
Remarks
 
L, Letter; LC, Cased Letter (Lu, Ll, and Lt only)[c]
Lu Letter, uppercase Graphic Character 1,791
Ll Letter, lowercase Graphic Character 2,155
Lt Letter, titlecase Graphic Character 31 Ligatures containing uppercase followed by lowercase letters (e.g., Dž, Lj, Nj, and Dz)
Lm Letter, modifier Graphic Character 260 A modifier letter
Lo Letter, other Graphic Character 127,004 An ideograph or a letter in a unicase alphabet
M, Mark
Mn Mark, nonspacing Graphic Character 1,839
Mc Mark, spacing combining Graphic Character 443
Me Mark, enclosing Graphic Character 13
N, Number
Nd Number, decimal digit Graphic Character 650 All these, and only these, have Numeric Type = De[d]
Nl Number, letter Graphic Character 236 Numerals composed of letters or letterlike symbols (e.g., Roman numerals)
No Number, other Graphic Character 895 E.g., vulgar fractions, superscript and subscript digits
P, Punctuation
Pc Punctuation, connector Graphic Character 10 Includes "_" underscore
Pd Punctuation, dash Graphic Character 25 Includes several hyphen characters
Ps Punctuation, open Graphic Character 75 Opening bracket characters
Pe Punctuation, close Graphic Character 73 Closing bracket characters
Pi Punctuation, initial quote Graphic Character 12 Opening quotation mark. Does not include the ASCII "neutral" quotation mark. May behave like Ps or Pe depending on usage
Pf Punctuation, final quote Graphic Character 10 Closing quotation mark. May behave like Ps or Pe depending on usage
Po Punctuation, other Graphic Character 593
S, Symbol
Sm Symbol, math Graphic Character 948 Mathematical symbols (e.g., +, , =, ×, ÷, , , ). Does not include parentheses and brackets, which are in categories Ps and Pe. Also does not include !, *, -, or /, which despite frequent use as mathematical operators, are primarily considered to be "punctuation".
Sc Symbol, currency Graphic Character 62 Currency symbols
Sk Symbol, modifier Graphic Character 123
So Symbol, other Graphic Character 6,431
Z, Separator
Zs Separator, space Graphic Character 17 Includes the space, but not TAB, CR, or LF, which are Cc
Zl Separator, line Format Character 1 Only U+2028 LINE SEPARATOR (LSEP)
Zp Separator, paragraph Format Character 1 Only U+2029 PARAGRAPH SEPARATOR (PSEP)
C, Other
Cc Other, control Control Character 65 (will never change)[d] No name,[e] <control>
Cf Other, format Format Character 161 Includes the soft hyphen, joining control characters (zwnj and zwj), control characters to support bi-directional text, and language tag characters
Cs Other, surrogate Surrogate Not (only used in UTF-16) 2,048 (will never change)[d] No name,[e] <surrogate>
Co Other, private use Private-use Character (but no interpretation specified) 137,468 total (will never change)[d] (6,400 in BMP, 131,068 in Planes 15–16) No name,[e] <private-use>
Cn Other, not assigned Noncharacter Not 66 (will never change)[d] No name,[e] <noncharacter>
Reserved Not 830,606 No name,[e] <reserved>
  1. ^ "Table 4-4: General Category" (PDF). The Unicode Standard. Unicode Consortium. March 2020.
  2. ^ Jump up to: a b "Table 2-3: Types of code points" (PDF). The Unicode Standard. Unicode Consortium. March 2020.
  3. ^ "5.7.1 General Category Values". UTR #44: Unicode Character Database. Unicode Consortium. 2020-03-04.
  4. ^ Jump up to: a b c d e Unicode Character Encoding Stability Policies: Property Value Stability Stability policy: Some gc groups will never change. gc=Nd corresponds with Numeric Type=De (decimal).
  5. ^ Jump up to: a b c d e "Table 4-9: Construction of Code Point Labels" (PDF). The Unicode Standard. Unicode Consortium. March 2020. A Code Point Label may be used to identify a nameless code point. E.g. <control-hhhh>, <control-0088>. The Name remains blank, which can prevent inadvertently replacing, in documentation, a Control Name with a true Control code. Unicode also uses <not a character> for <noncharacter>.

Punctuation[]

Characters have separate properties to denote they are a punctuation character. The properties all have a Yes/No values: Dash, Quotation_Mark, Sentence_Terminal, Terminal_Punctuation.

Whitespace[]

Whitespace is a commonly used concept for a typographic effect. Basically it covers invisible characters that have a spacing effect in rendered text. It includes spaces, tabs, and new line formatting controls. In Unicode, such a character has the property set "WSpace=yes". In version 13.0, there are 25 whitespace characters.

Name Code point Width box May break? In
IDN?
Script Block General
category
Notes
 Name  Code point Width box May break? In
IDN?
Script Block General
category
Notes
  1. ^ White_Space is a binary Unicode property.[16]
  2. ^ "Unicode 13.0 UCD: PropList.txt". 2019-11-27. Retrieved 2020-03-12.
  3. ^ Although &ZeroWidthSpace; is one HTML5 named entity for U+200B, the additional names NegativeMediumSpace, NegativeThickSpace, NegativeThinSpace and NegativeVeryThinSpace (which are names used in the Wolfram Language for negative-advance spaces, which it maps to the Private Use Area)[10][11][12][13] are also defined by HTML5 as aliases for U+200B (e.g. &NegativeMediumSpace;).[9]


Other general characteristics[]

Ideographic, alphabetic, noncharacter.

Combining class[]

Some common codes:

0 = spacing letter, symbol or modifier (e.g. a, (, ʰ)
1 = overlay
6 = Han reading (CJK diacritic reading marks)
7 = nukta (diacritic nukta in Brahmic scripts)
8 = kana voicing marks
9 = virama

10–199 = various fixed-position classes

Marks which attach to the base letter:

200 = attached at bottom left
202 = attached directly below (e.g. cedilla on ç)
204 = attached at bottom right
208 = attached to left
210 = attached to right
212 = attached to top left
214 = attached directly above
216 = attached at top right

Marks which do not attach to the base letter:

218 = bottom left
220 = directly below (e.g. ring on n̥)
222 = below right
224 = left
226 = right
228 = above left
230 = above (e.g. acute accent on á)
232 = above right
233 = double below (subtends two bases)
234 = double above (extends two bases)
240 = iota subscript (only that Greek diacritic)

Display-related properties[]

Shaping, width.

Bidirectional writing[]

Six character properties pertain to bi-directional writing: Bidi_Class, Bidi_Control, Bidi_Mirrored, Bidi_Mirroring_Glyph, Bidi_Paired_Bracket and Bidi_Paired_Bracket_Type.

One of Unicode's major features is support of bi-directional (Bidi) text display right-to-left (R-to-L) and left-to-right (L-to-R). The Unicode Bidirectional Algorithm UAX9[17] describes the process of presenting text with altering script directions. For example, it enables a Hebrew quote in an English text. The Bidi_Character_Type marks a character's behaviour in directional writing. To override a direction, Unicode has defined special formatting control characters (Bidi-Controls). These characters can enforce a direction, and by definition only affect bi-directional writing.

Each code point has a property called Bidi_Class. It defines its behaviour in a bidirectional text as interpreted by the algorithm:

Bidirectional character type (Unicode character property Bidi_Class)[1]

In normal situations, the algorithm can determine the direction of a text by this character property. To control more complex Bidi situations, e.g. when an English text has a Hebrew quote, extra options are added to Unicode. Twelve characters have the property Bidi_Control=Yes: ALM, FSI, LRE, LRI, LRM, LRO, PDF, PDI, RLE, RLI, RLM and RLO as named in the table. These are invisible formatting control characters, only used by the algorithm and with no effect outside of bidirectional formatting.[17] Despite the name, they are formatting characters, not control characters, and have General category "Other, format (Cf)" in the Unicode definition.

Basically, the algorithm determines a sequence of characters with the same strong direction type (R-to-L or L-to-R), taking in account an overruling by the special Bidi-controls. Number strings (Weak types) are assigned a direction according to their strong environment, as are Neutral characters. Finally, the characters are displayed per a string's direction.

Two character properties are relevant to determining a mirror image of a glyph in bidirectional text: Bidi_Mirrored=Yes indicates that the glyph should be mirrored when written R-to-L. The property Bidi_Mirroring_Glyph=U+hhhh can then point to the mirrored character. For example, brackets "()" are mirrored this way. Shaping cursive scripts such as Arabic, and mirroring glyphs that have a direction, is not part of the algorithm.

Casing[]

The Case value is Normative in Unicode. It pertains to those scripts with uppercase (aka capital, majuscule) and the lowercase (aka small, minuscule) letters. Case-difference occurs in Adlam, Armenian, Cherokee, Coptic, Cyrillic, Deseret, Glagolitic, Greek, Khutsuri and Mkhedruli Georgian, Latin, Medefaidrin, Old Hungarian, Osage and Warang Citi scripts.

(upper, lower, title, folding—both simple and full)

Numeric values and types[]

Decimal[]

Characters are classified with a Numeric type.[1] Characters such as fractions, subscripts, superscripts, Roman numerals, currency numerators, encircled numbers, and script-specific digits are type Numeric. They have a numeric value that can be decimal, including zero and negatives, or a vulgar fraction. If there is not such a value, as with most of the characters, the numeric type is "None".

The characters that do have a numeric value are separated in three groups: Decimal (De), Digit (Di) and Numeric (Nu, i.e. all other). "Decimal" means the character is a straight decimal digit. Only characters that are part of a contiguous encoded range 0..9 have numeric type Decimal. Other digits, like superscripts, have numeric type Digit. All numeric characters like fractions and Roman numerals end up with the type "Numeric". The intended effect is that a simple parser can use these decimal numeric values, without being distracted by say a numeric superscript or a fraction. Seventy-three CJK Ideographs that represent a number, including those used for accounting, are typed Numeric.

On the other hand, characters that could have a numeric value as a second meaning are still marked Numeric type "None", and have no numeric value (""). E.g. Latin letters can be used in paragraph numbering like "II.A.1.b", but the letters "I", "A" and "b" are not numeric (type "None") and have no numeric value.

Numeric Type[a][b] (Unicode character property)
Numeric type Code Has numeric value Example Remarks
Not numeric None No
  • A
  • X (Latin)
  • !
  • Д
  • μ
Numeric Value="NaN"
Decimal De Yes
  • 0
  • 1
  • 9
  •  (Devanagari 6)
  •  (Kannada 6)
WIKI