Package com.tangosol.coherence.dsltools.precedence

Class OPToken

  • Direct Known Subclasses:
    BetweenOPToken, ColonToken, ContainsOPToken, IdentifierOPToken, InfixOPToken, InfixRightOPToken, KeywordOPToken, LiteralOPToken, NestingOPToken, NotOPToken, PathOPToken, PeekOPToken, PrefixOPToken, PunctuationOPToken
    public class OPToken
extends Object
    OPToken is the root class for the Top Down Operator Precedence Parsing framework's tokens. This framework was first done by Vaughan Pratt in 1973 an is undergoing a bit of a renaissance with people that find the typical formal grammer tool a bit to heavyweight. The fundamental idea behind "Pratt Parsers" is that Tokens are objects that posses methods that allow them to make precedence decisions, match other tokens, and build abstract syntax trees (AST). The central issue of the precedence problem is that given an operand object between two operators, should the operand be bound to the left operator or the right operator? obj1 OP1 obj2 OP2 obj3 like: (1 + 2 * 3) Does obj2 bind to OP1 or to OP2? The technique we will use has Token objects "know" their precedence levels, and implement methods called "nud" (the null denotation in Pratt speak) and "led" (the left denotation). A nud method "should" have no interest in the tokens to the left while a "led" method does. A nud method is typically used on values such as variables and literals and by prefix operators like '-', or 'not'. A led method is used by infix operators and suffix operators. A token will often have both a nud method and a led method with the canonical example being '-' which is both a prefix operator and an infix operator. The heart of Pratt's technique is the "expression" function. It takes a right binding power that controls how aggressively that it should bind to tokens on its right. We also pass the parser to the token objects so that their functions may look at context and have access to the tokenizer.
    Author:
    djl 2009.03.14
    See Also:
    OPParser

    • Field Detail

      • BINARY_OPERATOR_NODE

        public static String BINARY_OPERATOR_NODE
        The AST node name for a Binary Operator node.

      • BINDING_NODE

        public static String BINDING_NODE
        The AST node name for a Binding Node.

      • CALL_NODE

        public static String CALL_NODE
        The AST node name for a Method Call Node.

      • DEREF_NODE

        public static String DEREF_NODE
        The AST node name for a De-referencing Node.

      • FIELD_LIST

        public static String FIELD_LIST
        The AST node name for a Field List Node.

      • IDENTIFIER_NODE

        public static String IDENTIFIER_NODE
        The AST node name for an Identifier Node.

      • LIST_NODE

        public static String LIST_NODE
        The AST node name for a List of Values Node.

      • LITERAL_NODE

        public static String LITERAL_NODE
        The AST node name for a Literal Node.

      • UNARY_OPERATOR_NODE

        public static String UNARY_OPERATOR_NODE
        The AST node name for a Unary Operator Node.

      • PRECEDENCE_KEYWORD

        public static final int PRECEDENCE_KEYWORD
        The binding precedence for keyword tokens
        See Also:
        Constant Field Values

      • PRECEDENCE_IDENTIFIER

        public static final int PRECEDENCE_IDENTIFIER
        The binding precedence for identifier tokens
        See Also:
        Constant Field Values

      • PRECEDENCE_ASSIGNMENT

        public static final int PRECEDENCE_ASSIGNMENT
        The binding precedence for assignment operators assignment such as = += -= *= /= %= &= ^= |= <<= >>= >>>=
        See Also:
        Constant Field Values

      • PRECEDENCE_LOGICAL

        public static final int PRECEDENCE_LOGICAL
        The binding precedence for logical tokens such as &&, ||, etc
        See Also:
        Constant Field Values

      • PRECEDENCE_LOGICAL_BITWISE

        public static final int PRECEDENCE_LOGICAL_BITWISE
        The binding precedence for bitwise logical tokens such as &, |, ^ etc
        See Also:
        Constant Field Values

      • PRECEDENCE_RELATIONAL

        public static final int PRECEDENCE_RELATIONAL
        The binding precedence for relational operators such as ==, <=, like, contains etc
        See Also:
        Constant Field Values

      • PRECEDENCE_BITWISE

        public static final int PRECEDENCE_BITWISE
        The binding precedence for bitwise operators such as >>> >> and <<
        See Also:
        Constant Field Values

      • PRECEDENCE_SUM

        public static final int PRECEDENCE_SUM
        The binding precedence for sum arithmetic, i.e. + and -
        See Also:
        Constant Field Values

      • PRECEDENCE_PRODUCT

        public static final int PRECEDENCE_PRODUCT
        The binding precedence for product arithmetic, multiplication, division, mod * / %
        See Also:
        Constant Field Values

      • PRECEDENCE_EXPONENT

        public static final int PRECEDENCE_EXPONENT
        The binding precedence for exponent arithmetic, i.e. raising by a power
        See Also:
        Constant Field Values

      • PRECEDENCE_UNARY

        public static final int PRECEDENCE_UNARY
        The binding precedence for other unary operators: pre-increment, pre-decrement, plus, minus, logical negation, bitwise complement, type cast ++expr --expr +expr -expr ! ~ (type)
        See Also:
        Constant Field Values

      • PRECEDENCE_UNARY_POST

        public static final int PRECEDENCE_UNARY_POST
        The binding precedence for unary post operators such as post-increment and post-decrement of the form expr++ or expr--
        See Also:
        Constant Field Values

      • PRECEDENCE_PARENTHESES

        public static final int PRECEDENCE_PARENTHESES
        The binding precedence for parentheses ( ) [ ]
        See Also:
        Constant Field Values

      • m_sValue

        protected String m_sValue
        The string value of the literal.

      • m_sLedASTName

        protected String m_sLedASTName
        A functor for building ast for led method.

      • m_sNudASTName

        protected String m_sNudASTName
        A functor for building ast for nud method.

      • m_nBindingPower

        protected int m_nBindingPower
        The power that this token binds, typically to the left.

    • Constructor Detail

      • OPToken

        public OPToken()
        Construct a new OPToken.

      • OPToken

        public OPToken​(String sId)
        Construct a new OPToken with the given parameters.
        Parameters:
        sId - string representation of the literal

      • OPToken

        public OPToken​(String sId,
               int nBp)
        Construct a new OPToken with the given parameters.
        Parameters:
        sId - string representation of the token
        nBp - The binding precedence for this token

      • OPToken

        public OPToken​(String sId,
               String sAstName)
        Construct a new OPToken with the given parameters.
        Parameters:
        sId - string representation of the token
        sAstName - the type code for this literal token

      • OPToken

        public OPToken​(String sId,
               int nBp,
               String sAstName)
        Construct a new OPToken with the given parameters.
        Parameters:
        sId - string representation of the token
        nBp - The binding precedence for this token
        sAstName - the name for this tokens AST

      • OPToken

        public OPToken​(String sId,
               int nBp,
               String sLedASTName,
               String sNudASTName)
        Construct a new OPToken with the given parameters.
        Parameters:
        sId - string representation of the token
        nBp - The binding precedence for this token
        sLedASTName - the name for this tokens AST
        sNudASTName - the name for this tokens AST

    • Method Detail

      • leftBindingPower

        public int leftBindingPower()
        Obtain the power that this token will bind to the left.
        Returns:
        the left binding power

      • nud

        public Term nud​(OPParser parser)
        Process this token in the context of parser p with the null denotation. A nud method typically will have no interest in the token to the left. The processing results in an Abstract Syntax Tree Node that captures the meaning
        Parameters:
        parser - the parser that is the context for parsing
        Returns:
        an AstNode

      • led

        public Term led​(OPParser parser,
                Term leftNode)
        Process this token and possibly the given leftNodein the context of a parser with the left denotation. A led method typically will be interested t in the token to the left. The processing results in an Abstract Syntax Tree Node that captures the meaning
        Parameters:
        parser - the parser that is the context for parsing
        leftNode - an ast Term that the token is possibly interested in
        Returns:
        an AstNode

      • newAST

        protected Term newAST​(String sAstName,
                      String sFunctor)
        Create an Abstract Syntax Node for the given arguments. If the astName argument is not null then use it for the functor and the given functor argument become the first child Term.
        Parameters:
        sAstName - classification functor or null
        sFunctor - functor for ast node to be constructed
        Returns:
        a Term representing the AST

      • newAST

        protected Term newAST​(String sAstName,
                      Term term)
        Create an Abstract Syntax Node for the given arguments. If the astName argument is not null then use it for the functor otherwise just assume the Term t is good.
        Parameters:
        sAstName - classification functor or null
        term - an Term that is part of the ast
        Returns:
        a Term representing the AST

      • newAST

        protected Term newAST​(String sAstName,
                      String sFunctor,
                      Term term)
        Create an Abstract Syntax Node for the given arguments. If the astName argument is not null then use it for the functor and the given functor argument become the first child Term.
        Parameters:
        sAstName - classification functor or null
        sFunctor - functor for ast node to be constructed
        term - an Term that is part of the ast
        Returns:
        a Term representing the AST

      • newAST

        protected Term newAST​(String sAstName,
                      String sFunctor,
                      Term t1,
                      Term t2)
        Create an Abstract Syntax Node for the given arguments. If the astName argument is not null then use it for the functor and the given functor argument become the first child Term.
        Parameters:
        sAstName - classification functor or null
        sFunctor - functor for ast node to be constructed
        t1 - an Term that is part of the ast
        t2 - an Term that is part of the ast
        Returns:
        a Term representing the AST

      • newAST

        protected Term newAST​(String sAstName,
                      String sFunctor,
                      Term t1,
                      Term t2,
                      Term t3)
        Create an Abstract Syntax Node for the given arguments. If the astName argument is not null then use it for the functor and the given functor argument become the first child Term.
        Parameters:
        sAstName - classification functor or null
        sFunctor - functor for ast node to be constructed
        t1 - an Term that is part of the ast
        t2 - an Term that is part of the ast
        t3 - an Term that is part of the ast
        Returns:
        a Term representing the AST

      • getId

        public String getId()
        Obtain the string representation of this token.
        Returns:
        the string representation

      • setId

        public void setId​(String sId)
        Set the string representation of this token to the given id.
        Parameters:
        sId - the string representation for the token

      • getBindingPower

        public int getBindingPower()
        Get The binding precedence of this token.
        Returns:
        The binding precedence

      • setBindingPower

        public void setBindingPower​(int nBp)
        Set The binding precedence that this token will use for binding to the right.
        Parameters:
        nBp - the power power for this token

      • getNudASTName

        public String getNudASTName()
        Get nud AST Name for this token
        Returns:
        the nud ast name for this token

      • setNudASTName

        public void setNudASTName​(String sAstName)
        Set the nud AST Name for this token to be the given string
        Parameters:
        sAstName - the nud ast name for this token

      • getLedASTName

        public String getLedASTName()
        Get led AST Name for this token
        Returns:
        the led ast name for this token

      • setLedASTName

        public void setLedASTName​(String sAstName)
        Set the led AST Name for this token to be the given string
        Parameters:
        sAstName - the led ast name for this token

      • getValue

        public String getValue()
        Get a string value that identifies this token
        Returns:
        the a string that identifies this token

      • setValue

        public void setValue​(String s)
        Set the AST Name for this token to be the given string
        Parameters:
        s - the ast name for this token

      • toString

        public String toString()
        Return a human-readable description for this token.
        Overrides:
        toString in class Object
        Returns:
        a String description of the token

      • hashCode

        public int hashCode()
        Overrides:
        hashCode in class Object