Examples of opennlp.ccg.parse.tagger.io.PipeDelimitedFactoredBundleCorpusIterator

opennlp.ccg.parse.tagger.io.PipeDelimitedFactoredBundleCorpusIterator
Assuming an input file of n different sentences of the form: ~~wordbundle1 wordbundle2 ... wordbundleM~~ ... [n-2 lines] ~~wordbundle1 ... wordbundleQ~~ where the 'wordbundle's are SRILM factored LM-compliant bundles of factors (wordform, lemma, POS, supertag, semantic class, etc.). We assume one sentence per line, so the ~~...~~ bracketing is just a formality (what SRILM expects). @author Dennis N. Mehay @version $Revision: 1.2 $, $Date: 2010/09/26 05:50:15 $

            pbr = new BufferedWriter(new FileWriter(posd));
            minCatFreq = catFreq;
            if (tokenisationType.equalsIgnoreCase("srilm")) {
                incorp = new SRILMFactoredBundleCorpusIterator(new BufferedReader(new FileReader(corpus)));
            } else {
                incorp = new PipeDelimitedFactoredBundleCorpusIterator(new BufferedReader(new FileReader(corpus)));
            }
        } catch (IOException ex) {
            Logger.getLogger(TaggingDictionaryExtractor.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

View Full Code Here

              Iterator<List<Word>> goldCorpus = null;
              
              if(options.valueOf(tokenisation).equalsIgnoreCase("srilm")) {
                  corpus = new SRILMFactoredBundleCorpusIterator(new BufferedReader(new FileReader(options.valueOf(inputspec))));
              } else if(options.valueOf(tokenisation).equalsIgnoreCase("candc")) {
                  corpus = new PipeDelimitedFactoredBundleCorpusIterator(new BufferedReader(new FileReader(options.valueOf(inputspec))));
              }
              if(options.has("test") && options.valueOf(tokenisation).equalsIgnoreCase("srilm")) {
                  goldCorpus = new SRILMFactoredBundleCorpusIterator(new BufferedReader(new FileReader(options.valueOf(goldstandspec))));
              } else if(options.has("test") && options.valueOf(tokenisation).equalsIgnoreCase("candc")) {
                  goldCorpus = new PipeDelimitedFactoredBundleCorpusIterator(new BufferedReader(new FileReader(options.valueOf(goldstandspec))));
              }
              
              BufferedWriter outf = new BufferedWriter(new FileWriter(options.valueOf(outputspec)));
              
              boolean test = options.has("test");

View Full Code Here

        this.outputF = outputF;
        try {
            if (tokenisation.equalsIgnoreCase("srilm")) {
                incorp = new SRILMFactoredBundleCorpusIterator(new BufferedReader(new FileReader(corpus)));
            } else {
                incorp = new PipeDelimitedFactoredBundleCorpusIterator(new BufferedReader(new FileReader(corpus)));
            }
        } catch (FileNotFoundException ex) {
            Logger.getLogger(ZhangLeTrainingExtractor.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

View Full Code Here

TOP

Related Classes of opennlp.ccg.parse.tagger.io.PipeDelimitedFactoredBundleCorpusIterator

opennlp.ccg.parse.supertagger.JavaSupertaggingApp

opennlp.ccg.parse.supertagger.ml.ZhangLeTrainingExtractor

opennlp.ccg.parse.supertagger.util.TaggingDictionaryExtractor

java.io.IOException

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.