Class DependencyCrossValidator

java.lang.Object
opennlp.tools.depparse.DependencyCrossValidator

public class DependencyCrossValidator extends Object
Cross validator for DependencyParserME.

The samples are split into k folds. For each fold a parser is trained with DependencyParserME.train(String, ObjectStream, TrainingParameters) on the other k - 1 folds and scored on the held-out fold with a DependencyEvaluator. The unlabeled and labeled attachment scores are accumulated over all evaluated tokens, so every token of the input counts once, in the fold it was held out from. The listeners given at construction are told about each held-out sample.

Since:
3.0.0
See Also:
  • Constructor Details

    • DependencyCrossValidator

      public DependencyCrossValidator(String languageCode, opennlp.tools.util.TrainingParameters params, opennlp.tools.depparse.DependencyEvaluationMonitor... listeners)
      Initializes a DependencyCrossValidator that treats tokens tagged DependencyEvaluator.UNIVERSAL_PUNCTUATION_TAG as punctuation.
      Parameters:
      languageCode - The ISO language code of the samples. Must not be null.
      params - The TrainingParameters for the parser of each fold. Must not be null.
      listeners - The listeners told about each held-out sample; null entries are ignored.
      Throws:
      IllegalArgumentException - Thrown if languageCode or params is null.
    • DependencyCrossValidator

      public DependencyCrossValidator(String languageCode, opennlp.tools.util.TrainingParameters params, Predicate<String> punctuationTag, opennlp.tools.depparse.DependencyEvaluationMonitor... listeners)
      Initializes a DependencyCrossValidator with a custom notion of punctuation.
      Parameters:
      languageCode - The ISO language code of the samples. Must not be null.
      params - The TrainingParameters for the parser of each fold. Must not be null.
      punctuationTag - Decides from a gold part-of-speech tag whether the token is punctuation and therefore left out of the punctuation-free scores. Must not be null.
      listeners - The listeners told about each held-out sample; null entries are ignored.
      Throws:
      IllegalArgumentException - Thrown if languageCode, params or punctuationTag is null.
  • Method Details

    • evaluate

      public void evaluate(opennlp.tools.util.ObjectStream<opennlp.tools.depparse.DependencySample> samples, int folds) throws IOException
      Runs the cross validation and adds the scores of every fold to the totals.
      Parameters:
      samples - The samples to train and test with. Must not be null and must support ObjectStream.reset(), because every fold reads the stream from the start. The stream is not closed.
      folds - The number of folds. Must be at least 2.
      Throws:
      IOException - Thrown if reading the samples fails.
      IllegalArgumentException - Thrown if samples is null, folds is below 2, or the training parameters do not select an event model trainer.
    • getUas

      public double getUas()
      Returns:
      The unlabeled attachment score over all tokens evaluated so far.
    • getLas

      public double getLas()
      Returns:
      The labeled attachment score over all tokens evaluated so far.
    • getWordCount

      public long getWordCount()
      Returns:
      The number of tokens evaluated so far; over one complete run this is the number of tokens in the samples, because every token is held out once.
    • getUasExcludingPunctuation

      public double getUasExcludingPunctuation()
      Returns:
      The unlabeled attachment score over the evaluated tokens that are not punctuation.
    • getLasExcludingPunctuation

      public double getLasExcludingPunctuation()
      Returns:
      The labeled attachment score over the evaluated tokens that are not punctuation.
    • getWordCountExcludingPunctuation

      public long getWordCountExcludingPunctuation()
      Returns:
      The number of evaluated tokens that are not punctuation.