Class FullCaseFoldCharSequenceNormalizer

java.lang.Object
opennlp.tools.util.normalizer.FullCaseFoldCharSequenceNormalizer
All Implemented Interfaces:
Serializable, opennlp.tools.util.normalizer.CharSequenceNormalizer, opennlp.tools.util.normalizer.OffsetAwareNormalizer

public final class FullCaseFoldCharSequenceNormalizer extends Object implements opennlp.tools.util.normalizer.OffsetAwareNormalizer
A CharSequenceNormalizer that applies Unicode full case folding for case-insensitive matching, as defined by the Default Case Algorithms in Section 3.13 of the Unicode Standard, using the bundled CaseFolding.txt data of the Unicode Character Database.

Unlike CaseFoldCharSequenceNormalizer, which lower cases with Locale.ROOT, this applies the full case foldings (the C common and F full status mappings), including the expanding folds that plain lower casing does not perform: the sharp s (U+00DF) to ss, the Latin ligatures (for example U+FB00 to ff), and the Greek and Armenian multi-character folds. It is therefore an expanding, offset-changing transform, so it is offset-aware: normalizeAligned(CharSequence) reports the Alignment from the folded text back to the input.

The S simple and T Turkic status mappings are excluded, so the Turkish and Azerbaijani dotless-i rule is not applied here. Input is expected in NFC: the fold matches precomposed code points, so decomposed sequences pass through unchanged.

The folding table is loaded on first use. If the bundled CaseFolding.txt is missing or cannot be read, every normalize call throws IllegalStateException or UncheckedIOException.

See Also: