Examples of TwoPassDataIndexer

opennlp.maxent.TwoPassDataIndexer
opennlp.model.TwoPassDataIndexer
Collecting event and context counts by making two passes over the events. The first pass determines which contexts will be used by the model, and the second pass creates the events in memory containing only the contexts which will be used. This greatly reduces the amount of memory required for storing the events. During the first pass a temporary event file is created which is read during the second pass.
opennlp.tools.ml.model.TwoPassDataIndexer
Collecting event and context counts by making two passes over the events. The first pass determines which contexts will be used by the model, and the second pass creates the events in memory containing only the contexts which will be used. This greatly reduces the amount of memory required for storing the events. During the first pass a temporary event file is created which is read during the second pass.

Examples of opennlp.model.TwoPassDataIndexer

   * @return the new model
   * @throws IOException
   */
  @Deprecated
  public static AbstractModel trainMaxentModel(EventStream es, int iterations, int cut) throws IOException {
    return opennlp.maxent.GIS.trainModel(iterations, new TwoPassDataIndexer(es, cut));
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

  public static AbstractModel trainMaxentModel(EventStream es, int iterations, int cut) throws IOException {
    return opennlp.maxent.GIS.trainModel(iterations, new TwoPassDataIndexer(es, cut));
  }


  public static AbstractModel trainPerceptronModel(EventStream es, int iterations, int cut, boolean useAverage) throws IOException {
    return new opennlp.perceptron.PerceptronTrainer().trainModel(iterations, new TwoPassDataIndexer(es, cut, false), cut, useAverage);
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

     return model;
   }


  @Deprecated
  public static GISModel train(EventStream es, int iterations, int cut) throws IOException {
    return GIS.trainModel(iterations, new TwoPassDataIndexer(es, cut));
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

    assertNotNull(eval);
  }


  @Test
  public void testInBigDevSet() throws IOException {
    QNModel trainedModel = new QNTrainer(10, 1000, true).trainModel(new TwoPassDataIndexer(createTrainingStream()));
    // then
    testModel(trainedModel);
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

    return newParses;
  }


  @Deprecated
  public static AbstractModel train(opennlp.model.EventStream es, int iterations, int cut) throws java.io.IOException {
    return opennlp.maxent.GIS.trainModel(iterations, new TwoPassDataIndexer(es, cut));
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

        (opennlp.tools.parser.lang.en.HeadRules) rules, ParserType.TREEINSERT);
  }
  
  @Deprecated
  public static AbstractModel train(opennlp.model.EventStream es, int iterations, int cut) throws java.io.IOException {
    return opennlp.maxent.GIS.trainModel(iterations, new TwoPassDataIndexer(es, cut));
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

    
    EventStream es = new ChunkerEventStream(in, contextGenerator);
    HashSumEventStream hses = new HashSumEventStream(es);
    
    AbstractModel maxentModel = opennlp.maxent.GIS.trainModel(iterations, 
        new TwoPassDataIndexer(hses, cutoff));
    
    manifestInfoEntries.put(BaseModel.TRAINING_EVENTHASH_PROPERTY, 
        hses.calculateHashSum().toString(16));
    
    return new ChunkerModel(lang, maxentModel, manifestInfoEntries);

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

       featureGenerator = createFeatureGenerator();
     
     EventStream eventStream = new NameFinderEventStream(samples, type,
         new DefaultNameContextGenerator(featureGenerator));
     HashSumEventStream hses = new HashSumEventStream(eventStream);
     AbstractModel nameFinderModel = GIS.trainModel(iterations, new TwoPassDataIndexer(hses, cutoff));
     
     manifestInfoEntries.put(BaseModel.TRAINING_EVENTHASH_PROPERTY, 
         hses.calculateHashSum().toString(16));
     
     return new TokenNameFinderModel(languageCode, nameFinderModel,

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

     return NameFinderME.train(languageCode, type, samples, resources, 100, 5);
   }
   
  @Deprecated
  public static GISModel train(EventStream es, int iterations, int cut) throws IOException {
    return GIS.trainModel(iterations, new TwoPassDataIndexer(es, cut));
  }

View Full Code Here

Examples of opennlp.model.TwoPassDataIndexer

   *
   * @return the new model
   */
   @Deprecated
  public static AbstractModel train(DocumentCategorizerEventStream eventStream) throws IOException {
    return GIS.trainModel(100, new TwoPassDataIndexer(eventStream, 5));
  }

View Full Code Here

0 1 2 3 4

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.