Examples of org.apache.lucene.analysis.en.PorterStemFilter

org.apache.lucene.analysis.en.PorterStemFilter
Transforms the token stream as per the Porter stemming algorithm. Note: the input to the stemming filter must already be in lower case, so you will need to use LowerCaseFilter or LowerCaseTokenizer farther down the Tokenizer chain in order for this to work properly!
To use this filter with other analyzers, you'll want to write an Analyzer class that sets up the TokenStream chain as you want it. To use this with LowerCaseTokenizer, for example, you'd write an analyzer like this:
```
 class MyAnalyzer extends Analyzer { {@literal @Override}protected TokenStreamComponents createComponents(String fieldName, Reader reader) { Tokenizer source = new LowerCaseTokenizer(version, reader); return new TokenStreamComponents(source, new PorterStemFilter(source)); } } 
```
Note: This filter is aware of the {@link KeywordAttribute}. To prevent certain terms from being passed to the stemmer {@link KeywordAttribute#isKeyword()} should be set to truein a previous {@link TokenStream}.

    Tokenizer source = new WhitespaceTokenizer(matchVersion, reader);
    TokenStream filter = new LowerCaseEntityPreservingFilter(source);


    if (stemming) {
      // Porter stemmer ignores words which are marked as keywords
      filter = new PorterStemFilter(filter);
    }
    return new TokenStreamComponents(source, filter);
  }

View Full Code Here

    Tokenizer source = new WhitespaceTokenizer(matchVersion, reader);
    TokenStream filter = new LowerCaseEntityPreservingFilter(source);


    if (stemming) {
      // Porter stemmer ignores words which are marked as keywords
      filter = new PorterStemFilter(filter);
    }
    return new TokenStreamComponents(source, filter);
  }

View Full Code Here

    Tokenizer source = new WhitespaceTokenizer(matchVersion, reader);
    TokenStream filter = new LowerCaseEntityPreservingFilter(source);


    if (stemming) {
      // Porter stemmer ignores words which are marked as keywords
      filter = new PorterStemFilter(filter);
    }
    return new TokenStreamComponents(source, filter);
  }

View Full Code Here

            return new SnowballFilter(tokenStream, new KpStemmer());


            // English stemmers
        } else if ("english".equalsIgnoreCase(language)) {
            if (indexVersion.onOrAfter(Version.V_1_3_0)) {
                return new PorterStemFilter(tokenStream);
            } else {
                return new SnowballFilter(tokenStream, new EnglishStemmer());
            }
        } else if ("light_english".equalsIgnoreCase(language) || "lightEnglish".equalsIgnoreCase(language)
                || "kstem".equalsIgnoreCase(language)) {
            return new KStemFilter(tokenStream);
        } else if ("lovins".equalsIgnoreCase(language)) {
            return new SnowballFilter(tokenStream, new LovinsStemmer());
        } else if ("porter".equalsIgnoreCase(language)) {
            return new PorterStemFilter(tokenStream);
        } else if ("porter2".equalsIgnoreCase(language)) {
            if (indexVersion.onOrAfter(Version.V_1_3_0)) {
                return new SnowballFilter(tokenStream, new EnglishStemmer());
            } else {
                return new SnowballFilter(tokenStream, new PorterStemmer());

View Full Code Here

        super(index, indexSettings, name, settings);
    }


    @Override
    public TokenStream create(TokenStream tokenStream) {
        return new PorterStemFilter(tokenStream);
    }

View Full Code Here

0 1 2 3

TOP

Related Classes of org.apache.lucene.analysis.en.PorterStemFilter

cc.twittertools.index.TweetAnalyzer

cc.twittertools.search.indexing.TweetAnalyzer

com.github.rnewson.couchdb.lucene.util.PorterStemAnalyzer

com.googlecode.lucene.PorterAnalyzer

org.apache.lucene.analysis.cn.smart.SmartChineseAnalyzer

org.apache.lucene.analysis.en.PorterStemFilterFactory

org.apache.lucene.analysis.miscellaneous.TestKeywordMarkerFilterFactory

org.apache.lucene.analysis.miscellaneous.TestStemmerOverrideFilter

org.apache.lucene.analysis.miscellaneous.TestStemmerOverrideFilterFactory

org.apache.mahout.text.MailArchivesClusteringAnalyzer

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.