Examples of StrTokenizer

org.apache.commons.lang.text.StrTokenizer
Tokenizes a string based based on delimiters (separators) and supporting quoting and ignored character concepts.
This class can split a String into many smaller strings. It aims to do a similar job to {@link java.util.StringTokenizer StringTokenizer}, however it offers much more control and flexibility including implementing the ListIterator interface. By default, it is set up like StringTokenizer.
The input String is split into a number of tokens. Each token is separated from the next String by a delimiter. One or more delimiter characters must be specified.
Each token may be surrounded by quotes. The quote matcher specifies the quote character(s). A quote may be escaped within a quoted section by duplicating itself.
Between each token and the delimiter are potentially characters that need trimming. The trimmer matcher specifies these characters. One usage might be to trim whitespace characters.
At any point outside the quotes there might potentially be invalid characters. The ignored matcher specifies these characters to be removed. One usage might be to remove new line characters.
Empty tokens may be removed or returned as null.
```
 "a,b,c"         - Three tokens "a","b","c"   (comma delimiter) " a, b , c "    - Three tokens "a","b","c"   (default CSV processing trims whitespace) "a, ", b ,", c" - Three tokens "a, " , " b ", ", c" (quoted text untouched) 
```
This tokenizer has the following properties and options:

Property Type Default

delim CharSetMatcher { \t\n\r\f}

quote NoneMatcher {}

ignore NoneMatcher {}

emptyTokenAsNull boolean false

ignoreEmptyTokens boolean true

@author Apache Software Foundation @author Matthew Inger @author Gary D. Gregory @since 2.2 @version $Id: StrTokenizer.java 907631 2010-02-08 12:22:48Z sebb $
org.apache.commons.lang3.text.StrTokenizer
Tokenizes a string based based on delimiters (separators) and supporting quoting and ignored character concepts.
This class can split a String into many smaller strings. It aims to do a similar job to {@link java.util.StringTokenizer StringTokenizer}, however it offers much more control and flexibility including implementing the ListIterator interface. By default, it is set up like StringTokenizer.
The input String is split into a number of tokens. Each token is separated from the next String by a delimiter. One or more delimiter characters must be specified.
Each token may be surrounded by quotes. The quote matcher specifies the quote character(s). A quote may be escaped within a quoted section by duplicating itself.
Between each token and the delimiter are potentially characters that need trimming. The trimmer matcher specifies these characters. One usage might be to trim whitespace characters.
At any point outside the quotes there might potentially be invalid characters. The ignored matcher specifies these characters to be removed. One usage might be to remove new line characters.
Empty tokens may be removed or returned as null.
```
 "a,b,c"         - Three tokens "a","b","c"   (comma delimiter) " a, b , c "    - Three tokens "a","b","c"   (default CSV processing trims whitespace) "a, ", b ,", c" - Three tokens "a, " , " b ", ", c" (quoted text untouched) 
```
This tokenizer has the following properties and options:

Property Type Default

delim CharSetMatcher { \t\n\r\f}

quote NoneMatcher {}

ignore NoneMatcher {}

emptyTokenAsNull boolean false

ignoreEmptyTokens boolean true

@since 2.2 @version $Id: StrTokenizer.java 1153241 2011-08-02 18:49:52Z ggregory $

Property	Type	Default
delim	CharSetMatcher	{ \t\n\r\f}
quote	NoneMatcher	{}
ignore	NoneMatcher	{}
emptyTokenAsNull	boolean	false
ignoreEmptyTokens	boolean	true

Property	Type	Default
delim	CharSetMatcher	{ \t\n\r\f}
quote	NoneMatcher	{}
ignore	NoneMatcher	{}
emptyTokenAsNull	boolean	false
ignoreEmptyTokens	boolean	true

Examples of org.apache.commons.lang3.text.StrTokenizer

    return success;
  }


  @Override
  protected int executeCommand(String line) {
    String[] tokens = new StrTokenizer(line).getTokenArray();
    String action = tokens[0];
    String[] actionArgs = Arrays.copyOfRange(tokens, 1, tokens.length);
    if (logger.isDebugEnabled()) {
      logger.debug("Executing command action: {}, Tokens: {}", action, tokens.length);
    }

View Full Code Here

Examples of org.apache.commons.lang3.text.StrTokenizer

                fieldResult.add(makeAction((String) arrActionResult.get(x), prefixCount + splitCampiVal[x].replaceAll("/\\*", ""), builder));
                // appo=prefix+prefix_count+element.getText().replaceAll("/\\*","");
              } else if (StringUtils.defaultString(splitCampiVal[x]).contains("/*/")) {
                String appoSt = splitCampiVal[x];
                int count = 0;
                StrTokenizer strTokenizerGenericField = new StrTokenizer(genericField, StrMatcher.charSetMatcher("[]"));
                while (strTokenizerGenericField.hasNext()) {
                  String tokenStr2 = strTokenizerGenericField.nextToken();
                  try {
                    Integer.parseInt(tokenStr2);
                    appoSt = appoSt.replaceFirst("/\\*", "[" + tokenStr2 + "]");
                  } catch (Exception e) {
                    // TODO: handle exception

View Full Code Here

Examples of org.apache.commons.lang3.text.StrTokenizer


    return returnValue;
  }


  private static void analizeAction(String fieldName, StrBuilder bufFieldName, ArrayList arrActionResult) {
    StrTokenizer strTokenizer = new StrTokenizer(fieldName, ",");
    while (strTokenizer.hasNext()) {
      String tokenStr = strTokenizer.nextToken();
      try {
        String parseAct = tokenStr.substring(0, tokenStr.indexOf(":"));
        if (parseAct.equals("name")) {
          bufFieldName.replaceAll("name:", "");// (0,fieldName.substring(fieldName.indexOf(":")+1));
          arrActionResult.add("name");

View Full Code Here

Examples of org.apache.commons.lang3.text.StrTokenizer

    }
  }


  public static String scriptingResolver(String genericField, String fieldValue, XMLBuilder builder) {
    StringBuffer stringBuffer = new StringBuffer();
    StrTokenizer strTokenizer = new StrTokenizer(fieldValue, StrMatcher.charSetMatcher("{}"));
    while (strTokenizer.hasNext()) {
      String tokenStr = strTokenizer.nextToken();
      ArrayList arrayList = getFieldName(genericField, tokenStr, builder, "");
      if (arrayList.size() > 0) {
        for (int x = 0; x < arrayList.size(); x++) {
          stringBuffer.append(StringEscapeUtils.escapeEcmaScript((String) arrayList.get(x)));
          if (arrayList.size() - 1 != x) {

View Full Code Here

Examples of org.apache.commons.lang3.text.StrTokenizer

    return stringBuffer.toString();
  }


  public static String scriptingResolver(String genericField, String fieldValue, XMLBuilder builder, String prefixCount) {
    StringBuffer stringBuffer = new StringBuffer();
    StrTokenizer strTokenizer = new StrTokenizer(fieldValue, StrMatcher.charSetMatcher("{}"));
    while (strTokenizer.hasNext()) {
      String tokenStr = strTokenizer.nextToken();
      ArrayList arrayList = getFieldName(genericField, tokenStr, builder, prefixCount);
      if (arrayList.size() > 0) {
        for (int x = 0; x < arrayList.size(); x++) {
          stringBuffer.append(StringEscapeUtils.escapeEcmaScript((String) arrayList.get(x)));
          if (arrayList.size() - 1 != x) {

View Full Code Here

Examples of org.apache.commons.lang3.text.StrTokenizer

    decomponeQuery();
  }


  // PRIVATE METHODS
  private void decomponeQuery() {
    StrTokenizer tokenizer = new StrTokenizer(originalQuery);
    Pattern pattern = Pattern.compile("\\$\\{.*?\\}");


    String partialStr = "";
    String logicOperator = "";
    String prevOperator = "";
    while (tokenizer.hasNext()) {
      String elem = (String) tokenizer.next();


      if (logicOperators.containsKey(elem)) {
        logicOperator += elem + " ";
      } else {
        int iterations = 1;
        if (!tokenizer.hasNext())
          iterations = 2;


        for (int i = 0; i < iterations; i++) {
          if (!logicOperator.equals("") || !tokenizer.hasNext()) {
            StringBuilder partStr = new StringBuilder(partialStr);


            QueryPart queryPart = new QueryPart();


            Matcher matcher = pattern.matcher(partStr.toString());

View Full Code Here

Examples of org.apache.commons.lang3.text.StrTokenizer

            final Map<String, String> uriTemplateVariables) {
        if (path.startsWith(pathSeparator) != pattern.startsWith(pathSeparator)) {
            return false;
        }


        final String[] patternDirs = new StrTokenizer(pattern, pathSeparator)
                .setIgnoreEmptyTokens(true).getTokenArray();
        final String[] pathDirs = new StrTokenizer(path, pathSeparator)
                .setIgnoreEmptyTokens(true).getTokenArray();


        int pattIdxStart = 0;
        int pattIdxEnd = patternDirs.length - 1;
        int pathIdxStart = 0;

View Full Code Here

Examples of org.apache.commons.lang3.text.StrTokenizer

     * <code>pattern</code>' and '<code>path</code>', but does
     * <strong>not</strong> enforce this.
     */
    public String extractPathWithinPattern(final String pattern,
            final String path) {
        final String[] patternParts = new StrTokenizer(pattern, pathSeparator)
                .setIgnoreEmptyTokens(true).getTokenArray();
        final String[] pathParts = new StrTokenizer(path, pathSeparator)
                .setIgnoreEmptyTokens(true).getTokenArray();


        final StringBuilder builder = new StringBuilder();


        // Add any path parts that have a wildcarded pattern part.

View Full Code Here

Examples of org.apache.commons.lang3.text.StrTokenizer

            return;
        }


        final Set<String> disallowedOperationSet = new HashSet<String>();
        if (!"".equals(disallowedOperations)) {
            final String[] disallowedOperationsTokens = new StrTokenizer(
                    disallowedOperations, ",").getTokenArray();
            for (final String operation : disallowedOperationsTokens) {
                if (!("create".equals(operation) || "update".equals(operation) || "delete"
                        .equals(operation))) {
                    LOGGER.warning("-disallowedOperations options can only contain 'create', 'update', 'delete': -disallowedOperations update,delete");

View Full Code Here

Examples of org.apache.commons.lang3.text.StrTokenizer

        if (StringUtils.isBlank(filter)) {
            return finders;
        }


        final Set<String> requiredEntries = new HashSet<String>();
        final String[] filterTokens = new StrTokenizer(filter, ",")
                .getTokenArray();
        for (final String requiredString : filterTokens) {
            requiredEntries.add(requiredString.toLowerCase());
        }
        if (requiredEntries.isEmpty()) {

View Full Code Here

0 1 2 3

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.