Examples of TwoPassDataIndexer

opennlp.maxent.TwoPassDataIndexer
opennlp.model.TwoPassDataIndexer
Collecting event and context counts by making two passes over the events. The first pass determines which contexts will be used by the model, and the second pass creates the events in memory containing only the contexts which will be used. This greatly reduces the amount of memory required for storing the events. During the first pass a temporary event file is created which is read during the second pass.
opennlp.tools.ml.model.TwoPassDataIndexer
Collecting event and context counts by making two passes over the events. The first pass determines which contexts will be used by the model, and the second pass creates the events in memory containing only the contexts which will be used. This greatly reduces the amount of memory required for storing the events. During the first pass a temporary event file is created which is read during the second pass.

Examples of opennlp.model.TwoPassDataIndexer

  
  @Test
  public void testMaxentOnPrepAttachData2Threads() throws IOException {
    AbstractModel model = 
        new GISTrainer(true).trainModel(100,
            new TwoPassDataIndexer(createTrainingStream(), 1),
            new UniformPrior(), 1, 2);
    
    testModel(model, 0.7997028967566229);
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

   *
   * @return the new model
   */
   @Deprecated
  public static AbstractModel train(DocumentCategorizerEventStream eventStream) throws IOException {
    return GIS.trainModel(100, new TwoPassDataIndexer(eventStream, 5));
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

     return model;
   }
   
  @Deprecated
  public static GISModel train(EventStream es, int iterations, int cut) throws IOException {
    return GIS.trainModel(iterations, new TwoPassDataIndexer(es, cut));
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

   */
  public static POSModel train(String languageCode, ObjectStream<POSSample> samples, POSDictionary tagDictionary,
      Dictionary ngramDictionary, int cutoff, int iterations) throws IOException {


    GISModel posModel = opennlp.maxent.GIS.trainModel(iterations,
        new TwoPassDataIndexer(new POSSampleEventStream(samples,
        new DefaultPOSContextGenerator(ngramDictionary)), cutoff));


    return new POSModel(languageCode, posModel, tagDictionary, ngramDictionary);
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

   * @return the new model
   * @throws IOException
   */
  @Deprecated
  public static AbstractModel trainMaxentModel(EventStream es, int iterations, int cut) throws IOException {
    return opennlp.maxent.GIS.trainModel(iterations, new TwoPassDataIndexer(es, cut));
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

  public static AbstractModel trainMaxentModel(EventStream es, int iterations, int cut) throws IOException {
    return opennlp.maxent.GIS.trainModel(iterations, new TwoPassDataIndexer(es, cut));
  }


  public static AbstractModel trainPerceptronModel(EventStream es, int iterations, int cut, boolean useAverage) throws IOException {
    return new opennlp.perceptron.PerceptronTrainer().trainModel(iterations, new TwoPassDataIndexer(es, cut, false), cut, useAverage);
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

   * @deprecated Please do not use anymore, use the ObjectStream train methods instead! This method
   * will be removed soon.
   */
  @Deprecated
  public static AbstractModel train(opennlp.model.EventStream es, int iterations, int cut) throws java.io.IOException {
    return opennlp.maxent.GIS.trainModel(iterations, new TwoPassDataIndexer(es, cut));
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

    return train(languageCode, parseSamples, rules, params);
  }
  
  @Deprecated
  public static AbstractModel train(opennlp.model.EventStream es, int iterations, int cut) throws java.io.IOException {
    return opennlp.maxent.GIS.trainModel(iterations, new TwoPassDataIndexer(es, cut));
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

   *
   * @return the new model
   */
   @Deprecated
  public static AbstractModel train(DocumentCategorizerEventStream eventStream) throws IOException {
    return GIS.trainModel(100, new TwoPassDataIndexer(eventStream, 5));
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

   */
  public static POSModel train(String languageCode, ObjectStream<POSSample> samples, POSDictionary tagDictionary,
      Dictionary ngramDictionary, int cutoff, int iterations) throws IOException {


    GISModel posModel = opennlp.maxent.GIS.trainModel(iterations,
        new TwoPassDataIndexer(new POSSampleEventStream(samples,
        new DefaultPOSContextGenerator(ngramDictionary)), cutoff));


    return new POSModel(languageCode, posModel, tagDictionary, ngramDictionary);
  }

View Full Code Here

0 1 2 3 4

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.