Class BundledUnicodeData

java.lang.Object
opennlp.tools.util.normalizer.BundledUnicodeData

@Internal public final class BundledUnicodeData extends Object
Loads the line-based data files bundled with OpenNLP, such as the Unicode Character Database extracts and the project-authored emoji tables.

A load failure surfaces from every accessor call that triggers a load: an IllegalStateException for a missing resource, an UncheckedIOException for a resource that cannot be read, and the parser's IllegalArgumentException for malformed data. A BundledUnicodeData.Lazy holder does not cache a failed load, so a later call tries again.

  • Method Details

    • load

      public static <T> T load(Class<?> owner, String resource, String description, BundledUnicodeData.Parser<T> parser)
      Opens a resource next to owner and parses it.
      Type Parameters:
      T - The type of the parsed data.
      Parameters:
      owner - The class the resource name is resolved against. Must not be null.
      resource - The resource name. Must not be null.
      description - The kind of data, used in error messages, for example "Word_Break". Must not be null.
      parser - Parses the opened stream. Must not be null.
      Returns:
      The parsed data.
      Throws:
      IllegalArgumentException - Thrown if an argument is null, or rethrown from parser for malformed data.
      IllegalStateException - Thrown if the resource is missing.
      UncheckedIOException - Thrown if the resource cannot be read.
    • forEachLine

      public static void forEachLine(InputStream in, Charset charset, BundledUnicodeData.LineConsumer consumer) throws IOException
      Reads every line of in and closes it.
      Parameters:
      in - The stream to read. Must not be null.
      charset - The encoding of in. Must not be null.
      consumer - Receives each line with its number. Must not be null.
      Throws:
      IOException - Thrown if reading in fails.
      IllegalArgumentException - Thrown if an argument is null.
    • forEachContentLine

      public static void forEachContentLine(InputStream in, BundledUnicodeData.LineConsumer consumer) throws IOException
      Reads the content lines of a UTF-8 data file in the Unicode Character Database layout and closes the stream. Each line is passed without its # comment and surrounding whitespace; lines that are empty after that are skipped but still counted.
      Parameters:
      in - The stream to read. Must not be null.
      consumer - Receives each non-empty content line with its number. Must not be null.
      Throws:
      IOException - Thrown if reading in fails.
      IllegalArgumentException - Thrown if an argument is null.