Class FeedforwardDependencyTrainer
FeedforwardDependencyModel: oracle-derived transition examples,
minibatch AdaGrad over a softmax cross-entropy loss, cube activation, and inverted
dropout on the hidden layer, the training recipe of
Chen and Manning (2014).
The hyperparameters are read from TrainingParameters: the keys of this
class plus Parameters.ITERATIONS_PARAM for the number of epochs,
Parameters.CUTOFF_PARAM for the minimum word frequency,
QNTrainer.L2COST_PARAM for the L2 penalty on the dense weights (default
1.0E-8), and BeamSearch.BEAM_SIZE_PARAMETER for the refinement
beam. Other keys are ignored.
Words below the frequency cutoff share the reserved unknown row, which is
therefore trained only when such words exist; the unknown rows of the tag and label
vocabularies keep their random initialization. Absent template positions share a
trained padding row. Non-projective samples have no arc-standard derivation and are
skipped. Training is deterministic for a fixed SEED_PARAM.
- Since:
- 3.0.0
-
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final StringThe training parameter key of the minibatch size; default 256.static final intThe default minibatch size.static final intThe default refinement beam size.static final doubleThe default hidden dropout probability.static final intThe default embedding dimensionality.static final intThe default number of epochs.static final intThe default hidden layer width.static final doubleThe default L2 penalty.static final doubleThe default AdaGrad step size.static final intThe default random seed.static final intThe default minimum word frequency for an embedding row.static final StringThe training parameter key of the hidden dropout probability; default 0.5.static final StringThe training parameter key of the embedding dimensionality; default 50.static final StringThe training parameter key of the hidden layer width; default 200.static final StringThe training parameter key of the AdaGrad step size; default 0.02.static final StringThe training parameter key of the random seed, read as anint; default 17. -
Method Summary
Modifier and TypeMethodDescriptionstatic FeedforwardDependencyModelrefine(FeedforwardDependencyModel model, opennlp.tools.util.ObjectStream<opennlp.tools.depparse.DependencySample> samples, opennlp.tools.util.TrainingParameters parameters) Refines a locally trained model with beam search and the early-update method of Collins and Roark (2004).static FeedforwardDependencyModeltrain(String languageCode, opennlp.tools.util.ObjectStream<opennlp.tools.depparse.DependencySample> samples, opennlp.tools.util.TrainingParameters parameters) Trains a model from dependency samples.
-
Field Details
-
EMBEDDING_SIZE_PARAM
-
HIDDEN_SIZE_PARAM
-
BATCH_SIZE_PARAM
-
LEARNING_RATE_PARAM
-
DROPOUT_PARAM
-
SEED_PARAM
-
DEFAULT_EMBEDDING_SIZE
public static final int DEFAULT_EMBEDDING_SIZEThe default embedding dimensionality.- See Also:
-
DEFAULT_HIDDEN_SIZE
public static final int DEFAULT_HIDDEN_SIZEThe default hidden layer width.- See Also:
-
DEFAULT_EPOCHS
public static final int DEFAULT_EPOCHSThe default number of epochs.- See Also:
-
DEFAULT_BATCH_SIZE
public static final int DEFAULT_BATCH_SIZEThe default minibatch size.- See Also:
-
DEFAULT_LEARNING_RATE
public static final double DEFAULT_LEARNING_RATEThe default AdaGrad step size.- See Also:
-
DEFAULT_L2
public static final double DEFAULT_L2The default L2 penalty.- See Also:
-
DEFAULT_DROPOUT
public static final double DEFAULT_DROPOUTThe default hidden dropout probability.- See Also:
-
DEFAULT_WORD_CUTOFF
public static final int DEFAULT_WORD_CUTOFFThe default minimum word frequency for an embedding row.- See Also:
-
DEFAULT_SEED
public static final int DEFAULT_SEEDThe default random seed.- See Also:
-
DEFAULT_BEAM_SIZE
public static final int DEFAULT_BEAM_SIZEThe default refinement beam size.- See Also:
-
-
Method Details
-
train
public static FeedforwardDependencyModel train(String languageCode, opennlp.tools.util.ObjectStream<opennlp.tools.depparse.DependencySample> samples, opennlp.tools.util.TrainingParameters parameters) throws IOException Trains a model from dependency samples.- Parameters:
languageCode- The ISO language code of the training data. Must not benull.samples- The training samples. Must not benull.parameters- TheTrainingParameters. Must not benull.- Returns:
- A trained
FeedforwardDependencyModel. Nevernull. - Throws:
IOException- Thrown if reading the samples fails.IllegalArgumentException- Thrown if a parameter isnull, a hyperparameter is out of range, no trainable example can be derived from the samples, a vocabulary or the three vocabularies together exceedResourceLimits.MAX_ENTRIESentries or a symbol as many UTF-8 bytes, or the network would store more thanResourceLimits.MAX_MATRIX_CELLSvalues.IllegalStateException- Thrown if an epoch produces a non-finite parameter.
-
refine
public static FeedforwardDependencyModel refine(FeedforwardDependencyModel model, opennlp.tools.util.ObjectStream<opennlp.tools.depparse.DependencySample> samples, opennlp.tools.util.TrainingParameters parameters) throws IOException Refines a locally trained model with beam search and the early-update method of Collins and Roark (2004). The loss is conditional likelihood over candidate paths scored by summed log-probabilities.Refinement trains a copy with per-sentence AdaGrad steps and no dropout.
Parameters.ITERATIONS_PARAMsets the number of passes andBeamSearch.BEAM_SIZE_PARAMETERthe beam width, which the manifest of the result names as the beam size to parse it with. The vocabularies and the transition inventory ofmodelare not extended, so the refined model carries the word cutoff ofmodel.- Parameters:
model- The locally trained model. Not modified. Must not benull.samples- The training samples. Must not benull.parameters- TheTrainingParameters; the epochs, learning rate, L2 penalty, seed, and beam size apply. Must not benull.- Returns:
- A new refined model, distinct from
model. Nevernull. - Throws:
IOException- Thrown if reading the samples fails.IllegalArgumentException- Thrown if a parameter isnull, a hyperparameter is out of range, the beam size is below 2, no trainable sample can be derived, or a sample requires a transitionmodeldoes not know.IllegalStateException- Thrown if an epoch produces a non-finite parameter, or the network produces a non-finite transition score while a sentence is decoded.
-