Examples of org.sindice.siren.analysis.filter.URIDecodingFilter

Package org.sindice.siren.analysis.filter

Examples of org.sindice.siren.analysis.filter.URIDecodingFilter

org.sindice.siren.analysis.filter.URIDecodingFilter
Decode the URI encoding format of special characters such as '?' or '<'.
Special characters (except of the SPACE that can be encoded with '+' and '%20') begins with a '%' and are followed by two characters in hexadecimal format. If a special character cannot be decoded, it is just skipped and the decoding process just continue.
When a URI has special characters, two versions of the URI are produced (both tokens have the same position):
- the original URI
- the decoded URI

  }


  @Override
  protected TokenStreamComponents createComponents(final String fieldName, final Reader reader) {
    final WhitespaceTokenizer source = new WhitespaceTokenizer(matchVersion, reader);
    TokenStream sink = new URIDecodingFilter(source, "UTF-8");
    sink = this.applyURINormalisation(sink);
    sink = new MailtoFilter(sink);
    sink = new LowerCaseFilter(matchVersion, sink );
    sink = new StopFilter(matchVersion, sink, stopSet);
    sink = new LengthFilter(true, sink, 2, 256);

View Full Code Here

   super.init(args);
  }


  @Override
  public TokenStream create(final TokenStream input) {
    return new URIDecodingFilter(input, DEFAULT_ENCODING);
  }

View Full Code Here

TOP

Related Classes of org.sindice.siren.analysis.filter.URIDecodingFilter

org.sindice.siren.analysis.AnyURIAnalyzer

org.sindice.siren.solr.analysis.URIEncodingFilterFactory

java.nio.charset.UnsupportedCharsetException

java.nio.charset.Charset

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.