Class HexCodePoints

java.lang.Object
opennlp.tools.util.normalizer.HexCodePoints

@Internal public final class HexCodePoints extends Object
Reads the hex code point notation of Unicode data files and similar code point tables: one code point as hex digits (1F600), a sequence as hex digits separated by whitespace (1F468 200D 1F469), and an inclusive range as two code points joined by two dots (1F3FB..1F3FF). Also removes the # comments these files carry.
  • Method Details

    • stripComment

      public static String stripComment(String line)
      Removes a trailing comment from a data file line.
      Parameters:
      line - The raw line. Must not be null.
      Returns:
      The line up to but excluding the first #, or line itself when it has none. Surrounding whitespace is kept.
      Throws:
      IllegalArgumentException - Thrown if line is null.
    • parseCodePoint

      public static int parseCodePoint(CharSequence hex, int start, int end)
      Parses the hex digits between two indexes as one code point.
      Parameters:
      hex - The text holding the digits. Must not be null.
      start - The index of the first digit.
      end - The index after the last digit.
      Returns:
      The code point.
      Throws:
      IllegalArgumentException - Thrown if hex is null, the region lies outside hex, is empty, holds a character that is not a hex digit, or names a value outside [0, U+10FFFF].
    • parseCodePoint

      public static int parseCodePoint(CharSequence hex)
      Parses hex digits as one code point.
      Parameters:
      hex - The digits. Must not be null.
      Returns:
      The code point.
      Throws:
      IllegalArgumentException - Thrown if hex is null or empty, holds a character that is not a hex digit, or names a value outside [0, U+10FFFF].
    • decodeSequence

      public static String decodeSequence(CharSequence hex)
      Decodes a sequence of hex code points separated by runs of Unicode whitespace into the characters they name. Leading and trailing whitespace is ignored.
      Parameters:
      hex - The sequence. Must not be null.
      Returns:
      The decoded characters, in order.
      Throws:
      IllegalArgumentException - Thrown if hex is null, holds no code point, or one of its code points is malformed.
    • decodeSequence

      public static String decodeSequence(CharSequence hex, char separator)
      Decodes a sequence of hex code points separated by exactly one separator each into the characters they name. Two adjacent separators or a leading separator leave an empty code point, which is rejected; trailing separators are ignored.
      Parameters:
      hex - The sequence. Must not be null.
      separator - The character between two code points. Must not be a surrogate.
      Returns:
      The decoded characters, in order.
      Throws:
      IllegalArgumentException - Thrown if hex is null or empty, separator is a surrogate, or one of the code points is malformed.
    • codePointTokens

      public static String[] codePointTokens(CharSequence hex)
      Splits a sequence into its hex code point tokens.
      Parameters:
      hex - The sequence. Must not be null.
      Returns:
      The tokens in order, at least one.
      Throws:
      IllegalArgumentException - Thrown if hex is null or holds no token.
    • parseRange

      public static int[] parseRange(CharSequence hex)
      Parses one code point or an inclusive XXXX..YYYY range.
      Parameters:
      hex - The code point or range. Must not be null.
      Returns:
      The first and the last code point of the range; both the same for one code point.
      Throws:
      IllegalArgumentException - Thrown if hex is null, a code point is malformed, or the range ends before it starts.