Examples of edu.stanford.nlp.trees.TreeNormalizer

edu.stanford.nlp.trees.TreeNormalizer

A class for tree normalization. The default one does no normalization. Other tree normalizers will change various node labels, or perhaps the whole tree geometry (by doing such things as deleting functional tags or empty elements). Another operation that a TreeNormalizer may wish to perform is interning the Strings passed to it. Can be reused as a Singleton. Designed to be extended.

The TreeNormalizer methods are in two groups. The contract for this class is that first normalizeTerminal or normalizeNonterminal will be called on each String that will be put into a Tree, when they are read from files or otherwise created. Then normalizeWholeTree will be called on the Tree. It normally walks the Tree making whatever modifications it wishes to. A TreeNormalizer need not make a deep copy of a Tree. It is assumed to be able to work destructively, because afterwards we will only use the normalized Tree.

Implementation note: This is a very old legacy class used in conjunction with PennTreeReader. It seems now that it would be better to move the String normalization into the tokenizer, and then we are just left with a (possibly destructive) TreeTransformer. @author Christopher Manning

    String file = pieces[pieces.length - 1];
    Format format = Format.TEXT;
    String encoding = getEncoding(config);
    String tagSeparator = getTagSeparator(config);
    TreeTransformer treeTransformer = null;
    TreeNormalizer treeNormalizer = null;
    TreeReaderFactory trf = null;
    NumberRangesFileFilter treeRange = null;
    Predicate<Tree> treeFilter = null;
    Integer wordColumn = null, tagColumn = null;

  }


  public TreeReader newTreeReader(Reader in) {
    TreeReader tr = null;
    if(noNormalization) {
      tr = new PennTreeReader(in, new LabeledScoredTreeFactory(), new TreeNormalizer(), new ArabicTreebankTokenizer(in));
    } else
      tr = new PennTreeReader(in, new LabeledScoredTreeFactory(), new ArabicTreeNormalizer(retainNPTmp,retainPRD,changeNoLabels, retainNPSbj, retainPPClr), new ArabicTreebankTokenizer(in));


    if (filterX)
      tr = new FilteringTreeReader(tr, new XFilter());

    List<SemanticGraph> graphs = Generics.newArrayList();
    // TODO: allow other sources of graphs, such as dependency files
    if (argsMap.containsKey(TREE_FILE) && argsMap.get(TREE_FILE).length > 0) {
      for (String treeFile : argsMap.get(TREE_FILE)) {
        System.err.println("Loading file " + treeFile);
        MemoryTreebank treebank = new MemoryTreebank(new TreeNormalizer());
        treebank.loadPath(treeFile);
        for (Tree tree : treebank) {
          // TODO: allow other languages... this defaults to English
          SemanticGraph graph = SemanticGraphFactory.makeFromTree(tree, mode, useExtras, true, null);
          graphs.add(graph);

Examples of edu.stanford.nlp.trees.TreeNormalizer

Related Classes of edu.stanford.nlp.trees.TreeNormalizer