Class SentenceVectorsDL
- All Implemented Interfaces:
Closeable, AutoCloseable, opennlp.tools.embeddings.TextEmbedder
The model inputs follow the standard single-segment BERT
encoding: attention_mask is 1 for every real
token and token_type_ids is 0 throughout.
The sentence vector is read from a sentence_embedding output of shape
[batch, hidden] if the model has one. Otherwise the token vectors of the first
[batch, tokens, hidden] output are pooled with the configured Pooling. By
default the token vectors are averaged and the result is scaled to unit length, as
sentence-transformers does for the MiniLM family. Text longer than the maximum length is
truncated, keeping the final [SEP] token.
Release note (OpenNLP 3.0.0): prior releases sent an
all-zero attention_mask and all-one token_type_ids,
so the encoder attended to nothing and the output vectors were
incorrect, and they returned the raw [CLS] vector instead of the pooled
sentence vector. Additionally, tokenization now performs BERT basic
tokenization (lower casing and accent stripping by default, see
WordpieceEncoder) before wordpiece.
Output vectors change with the corrected encoding, tokenization and pooling;
any embeddings persisted from the previous behavior are not
comparable with the corrected output and must be re-embedded.
This class is thread-safe and may be shared across threads: the inference methods hold no
per-call instance state and the underlying OrtSession supports concurrent execution.
This thread-safety guarantee applies until close() is called; callers must not race
close() with inference methods.
embedAll(List) batches its texts according to the configured
PaddingStrategy. By default it does not pad on the CPU and runs one inference per
distinct tokenized length. On a GPU it pads to the longest text of a batch, which runs texts of
mixed lengths together, except for a model with a sentence_embedding output, which is
not padded by default because such a model may ignore the attention mask. No inference exceeds
16384 token positions, except that a single text whose encoding is wider runs as an inference
of its own.
-
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final PaddingStrategyThePaddingStrategyon the CPU, or on a GPU for a pooled output, if none is given.static final PaddingStrategyThePaddingStrategyon a GPU if none is given and the model has no pooled output, so padding is masked out of the pooling, and the vocabulary has a padding token.static final intThe maximum number of tokens per input if none is given, the BERT position limit.Fields inherited from class AbstractDL
ATTENTION_MASK, INPUT_IDS, TOKEN_TYPE_IDS -
Constructor Summary
ConstructorsConstructorDescriptionSentenceVectorsDL(File model, File vocabulary) Instantiates asentence vector generatorfor an uncased model.SentenceVectorsDL(File model, File vocabulary, boolean lowerCase) Instantiates asentence vector generatorthat averages the token vectors, scales the result to unit length and truncates input to 512 tokens.SentenceVectorsDL(File model, File vocabulary, boolean lowerCase, Pooling pooling, boolean normalize, int maxLength) Instantiates asentence vector generatorusing ONNX models.SentenceVectorsDL(File model, File vocabulary, boolean lowerCase, SentenceVectorsOptions vectorOptions, InferenceOptions inferenceOptions) Instantiates a sentence vector generator with vector settings and ONNX session settings. -
Method Summary
Modifier and TypeMethodDescriptionvoidclose()Closes the ONNX session.intfloat[]embed(CharSequence text) float[][]embedAll(List<? extends CharSequence> texts) float[]getVectors(String sentence) Generates vectors given a sentence.Methods inherited from class AbstractDL
loadVocab
-
Field Details
-
DEFAULT_MAX_LENGTH
public static final int DEFAULT_MAX_LENGTHThe maximum number of tokens per input if none is given, the BERT position limit.- See Also:
-
DEFAULT_CPU_PADDING
ThePaddingStrategyon the CPU, or on a GPU for a pooled output, if none is given. -
DEFAULT_GPU_PADDING
ThePaddingStrategyon a GPU if none is given and the model has no pooled output, so padding is masked out of the pooling, and the vocabulary has a padding token. A GPU runs one wide inference faster than many narrow ones.
-
-
Constructor Details
-
SentenceVectorsDL
public SentenceVectorsDL(File model, File vocabulary) throws ai.onnxruntime.OrtException, IOException Instantiates asentence vector generatorfor an uncased model. Input text is lower cased and accent stripped during tokenization, as required by uncased models such as the sentence-transformers MiniLM family.- Parameters:
model- The file name of a sentence vectors ONNX model.vocabulary- The file name of the vocabulary file for the model.- Throws:
ai.onnxruntime.OrtException- Thrown if themodelcannot be loaded.IOException- Thrown if errors occurred loading themodelorvocabulary.opennlp.tools.util.InvalidFormatException- Thrown if a JSONvocabularyis malformed, has an unsupported layout, or sets a lower casing that disagrees with the lower casing the component is configured with.
-
SentenceVectorsDL
public SentenceVectorsDL(File model, File vocabulary, boolean lowerCase) throws ai.onnxruntime.OrtException, IOException Instantiates asentence vector generatorthat averages the token vectors, scales the result to unit length and truncates input to 512 tokens.- Parameters:
model- The file name of a sentence vectors ONNX model.vocabulary- The file name of the vocabulary file for the model.lowerCase-truefor uncased models (lower casing and accent stripping during tokenization),falsefor cased models.- Throws:
ai.onnxruntime.OrtException- Thrown if themodelcannot be loaded.IOException- Thrown if errors occurred loading themodelorvocabulary.opennlp.tools.util.InvalidFormatException- Thrown if a JSONvocabularyis malformed, has an unsupported layout, or sets a lower casing that disagrees with the lower casing the component is configured with.
-
SentenceVectorsDL
public SentenceVectorsDL(File model, File vocabulary, boolean lowerCase, Pooling pooling, boolean normalize, int maxLength) throws ai.onnxruntime.OrtException, IOException Instantiates asentence vector generatorusing ONNX models.- Parameters:
model- The file name of a sentence vectors ONNX model.vocabulary- The file name of the vocabulary file for the model.lowerCase-truefor uncased models (lower casing and accent stripping during tokenization),falsefor cased models.pooling- How token vectors are pooled. Not used for a model with asentence_embeddingoutput. Must not benull.normalize-trueto scale every vector to unit length.maxLength- The maximum number of tokens per input,[CLS]and[SEP]included; longer input is truncated. Must be at least2.- Throws:
IllegalArgumentException- Thrown ifpoolingisnull, ifmaxLengthis less than2, or if the model has no output of shape[batch, hidden]or[batch, tokens, hidden].ai.onnxruntime.OrtException- Thrown if themodelcannot be loaded.IOException- Thrown if errors occurred loading themodelorvocabulary.opennlp.tools.util.InvalidFormatException- Thrown if a JSONvocabularyis malformed, has an unsupported layout, or sets a lower casing that disagrees with the lower casing the component is configured with.
-
SentenceVectorsDL
public SentenceVectorsDL(File model, File vocabulary, boolean lowerCase, SentenceVectorsOptions vectorOptions, InferenceOptions inferenceOptions) throws ai.onnxruntime.OrtException, IOException Instantiates a sentence vector generator with vector settings and ONNX session settings. WithPaddingStrategy.MAX_LENGTH, a probe for an unknown output dimension also runs at the configured maximum length and can fail if it exceeds the model's position limit.- Parameters:
model- The file name of a sentence vectors ONNX model.vocabulary- The file name of the vocabulary file for the model.lowerCase-truefor uncased models,falsefor cased models. A lower case setting ininferenceOptionstakes precedence.vectorOptions- The pooling, normalization, truncation and padding settings. Must not benull.PaddingStrategy.AUTOselects the device and model default.inferenceOptions- The ONNX session settings. Must not benull.- Throws:
IllegalArgumentException- Thrown ifvectorOptionsorinferenceOptionsisnull, if the inference options contain invalid split settings or a negative GPU device id, if the model has no output of shape[batch, hidden]or[batch, tokens, hidden], or if padding is requested without a vocabulary padding token.ai.onnxruntime.OrtException- Thrown if the model cannot be loaded, the output dimension probe fails, or the requested GPU is unavailable. ONNX Runtime does not fall back to the CPU.IOException- Thrown if the model or vocabulary cannot be read.opennlp.tools.util.InvalidFormatException- Thrown if a JSON vocabulary is malformed, has an unsupported layout, or disagrees with the configured lower casing.- Since:
- 3.0.0
-
-
Method Details
-
getVectors
Generates vectors given a sentence.- Parameters:
sentence- The input sentence.- Returns:
- The sentence vector.
- Throws:
IllegalArgumentException- Thrown ifsentenceisnull.ai.onnxruntime.OrtException- Thrown if an error occurs during inference.
-
embed
Empty or unrecognized input is still run through the model as the wrapped
[CLS] [SEP]sequence rather than returning a zero vector.- Specified by:
embedin interfaceopennlp.tools.embeddings.TextEmbedder
-
embedAll
The inputs are tokenized up front and batched according to the configured
PaddingStrategy, so that no inference exceeds 16384 token positions; a single text whose encoding is wider runs alone. Without padding, every row is computed from the tensors its single-input call would have used. A padded position is0in the attention mask, so vectorimatches whatembed(CharSequence)returns for inputiup to floating-point rounding, provided the model honors the attention mask. Asentence_embeddingoutput that ignores the mask changes with padding.- Specified by:
embedAllin interfaceopennlp.tools.embeddings.TextEmbedder
-
dimension
public int dimension()Read from the model's declared output shape, or from one inference at construction if the model declares the hidden dimension dynamically.
- Specified by:
dimensionin interfaceopennlp.tools.embeddings.TextEmbedder
-
close
public void close()Closes the ONNX session. AnOrtExceptionfrom the session is rethrown as the cause of anUncheckedIOException.- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceCloseable- Specified by:
closein interfaceopennlp.tools.embeddings.TextEmbedder- Overrides:
closein classAbstractDL
-