Examples of fri.patterns.interpreter.parsergenerator.builder.SerializedLexer

Package fri.patterns.interpreter.parsergenerator.builder

Examples of fri.patterns.interpreter.parsergenerator.builder.SerializedLexer

fri.patterns.interpreter.parsergenerator.builder.SerializedLexer
Buffering Lexers. SerializedLexer will build the Lexer from scratch only the first time. Every following call will load the serialized Lexer from filesystem.
The time to build a lexer from scratch is equal to deserializing it in most cases. So a standalone lexer can be built without this class. When needed for a Parser, use the SerializedParser factory!
This factory will separate the passed syntax into parser and lexer syntax if token and ignored symbol Lists are null. So take care to use "token" and "ignored" rules within syntax to achieve the desired result!
Example (syntax input from a file):
```
 File ebnfFile = ...; Lexer lexer = new SerializedLexer().get(ebnfFile); 
```
or (syntax input from a Reader, must pass a filename):
```
 Reader ebnfReader = ...; Lexer lexer = new SerializedLexer().get(ebnfReader, "MyLexer.ser"); 
```
@author (c) 2002, Fritz Ritzberger

    // override the Parser builder to install the new Lexer implementation
    SerializedParser builder = new SerializedParser()  {
      protected SerializedLexer newSerializedLexer()  // override SerializedLexer factory method
        throws Exception
      {
        return new SerializedLexer()  {
          protected LexerBuilder newLexerBuilder(Syntax syntax, List ignoredSymbols)  // override LexerBuilder factory method
            throws LexerException, SyntaxException
          {
            return new LexerBuilder(syntax, ignoredSymbols)  {
              public Lexer getLexer()  {  // override Lexer factory method

View Full Code Here

      System.exit(1);
    }


    TimeStopper timer = new TimeStopper();


    SerializedLexer builder = new SerializedLexer();
    LexerImpl lexer;
    
    if ((lexer = (LexerImpl) builder.readLexer(null, "Dtd")) == null)  {  // try to get serialized lexer
      // read the DTD syntax (fragment) from EBNF file
      Syntax dtdSyntax = new SyntaxBuilder(new InputStreamReader(DtdLexer.class.getResourceAsStream("Dtd.syntax"))).getSyntax();
      // read the XML syntax from EBNF file
      Syntax xmlSyntax = new SyntaxBuilder(new InputStreamReader(DtdLexer.class.getResourceAsStream("Xml.syntax"))).getSyntax();
      
      // resolve DTD syntax from XML syntax (which contains most of DTD rules for processing embedded DTDs)
      dtdSyntax.resolveFrom(xmlSyntax);
      //System.err.println("DTD Syntax is:\n"+dtdSyntax);
      
      // put the lexer to file cache for next call
      SyntaxSeparation separation = new SyntaxSeparation(dtdSyntax);
      lexer = (LexerImpl) builder.buildAndStoreLexer(  // store the lexer for next call
          separation.getLexerSyntax(),
          "Dtd",
          separation.getTokenSymbols(),
          separation.getIgnoredSymbols());
    }

View Full Code Here

    // Building lexer from scratch takes 840 millis. Parsing takes 60 millis for a 70 line XML file.


    // read the syntax from EBNF file
    Reader syntaxInput = new InputStreamReader(XmlLexer.class.getResourceAsStream("Xml.syntax"));
    boolean PRODUCTION = false;  // always build from scratch at development time
    LexerImpl lexer = (LexerImpl) new SerializedLexer(PRODUCTION).get(syntaxInput, "Xml");


    System.err.println("time to build XML file parser was "+timer.getInterval());


    for (int i = 0; i < args.length; i++)  {
      String parseFile = args[i];

View Full Code Here

TOP

Related Classes of fri.patterns.interpreter.parsergenerator.builder.SerializedLexer

fri.patterns.interpreter.parsergenerator.examples.DtdLexer

fri.patterns.interpreter.parsergenerator.examples.HowToOverrideLexerImpl

fri.patterns.interpreter.parsergenerator.examples.XmlLexer

fri.patterns.interpreter.parsergenerator.Lexer

fri.patterns.interpreter.parsergenerator.syntax.builder.SyntaxSeparation

fri.util.TimeStopper

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.