Features In Flex Start Conditions Start Condition Example Start ...
cs 445 - System Program Design
03/26/2002
Features in Flex
Start conditions
• Start conditions
•
flex provides a mechanism for
• Values available to the user
conditionally activating rules
• C++ lexical analyzer
•
Any rule whose pattern is prefixed with
• Interfacing with yacc
"<sc>" will only be active when the
scanner is in the start condition named
"sc“
Start condition example
Start condition declaration
<STRING>[^"]* { /* eat up the string body */
•
Start conditions are declared in the
... }
definitions (first) section of the input
will be active only when the scanner is in the
using unindented lines beginning with
"STRING" start condition
either `%s' or `%x' followed by a list of
names
<INITIAL,STRING,QUOTE>\. { /* handle an escape */
–
The former (%s) declares inclusive start
... }
conditions
will be active only when the current start condition is
–
the latter (%x) declares exclusive start
either "INITIAL", "STRING", or "QUOTE".
conditions
Activation
Exclusive
• A start condition is activated using the BEGIN
• If it is exclusive, then only rules qualified with
action
the start condition will be active
• Until the next BEGIN action is executed, rules
• A set of rules contingent on the same
exclusive start condition describe a scanner
with the given start condition will be active
which is independent of any of the other rules
and rules with other start conditions will be
in the flex input
inactive
– Because of this, exclusive start conditions make it
• If the start condition is inclusive, then rules
easy to specify "mini-scanners" which scan
with no start conditions at all will also be
portions of the input that are syntactically different
from the rest (e.g., comments).
active
John Dickinson
1
cs 445 - System Program Design
03/26/2002
Little Example
Controlling Start Conditions
• Part of a flex file with start conditions:
• Once activated, a start condition is in
effect until another start condition is
activated
%x example
declares example
– So how do we get back to our initial state?
%%
<example>foo
do_something();
only rule used
• BEGIN(INITIAL) or BEGIN(0) reset the
bar
something_else();
start condition back to the original state
“==“
BEGIN(example);
activates example
• Also <*> can be used as a start
%%
condition that will match any start
condition
C comment example
Values available to user
•
Here is a scanner which recognizes (and discards) C comments whi le
• ‘char *yytext’ holds the text of the current token.
maintaining a count of the current input line
• ‘int yyleng’ holds the length of the current token.
• ‘FILE *yyin’ is the file which by default flex reads from.
%x comment
%%
• ‘void yyrestart( FILE *new_file )’ may be called to point
int line_num = 1;
yyin at the new input file.
"/*"
BEGIN(comment);
• ‘FILE *yyout’ is the file to which `ECHO' actions are
<comment>[^*\n]*
/* eat anything that's not a '*' */
done.
<comment>"*"+[^*/\n]* /* eat up '*'s not followed by '/'s */
• YY_START returns an integer value corresponding to
<comment>\n
++line_num;
the current start condition.
<comment>"*"+"/"
BEGIN(INITIAL);
C++ lexical analyzer
C++ lexical anlyzer
• flex provides two different ways to generate
• You can also use flex to generate a C++
scanners for use with C++
scanner class, using the `-+' option
• The first way is to simply compile a scanner
– This option is automatically specified if the name of
generated by flex using a C++ compiler instead of a
the flex executable ends in a `+', such as flex++
C compiler
• When using this option, flex defaults to
• You can then use C++ code in your rule actions
instead of C code
generating the scanner to the file `lex.yy.cc'
– Note that the default input source for your scanner remains
instead of `lex.yy.c‘
yyin, and default echoing is still done to yyout . Both of these
• The generated scanner includes the header file
remain `FILE *' variables and not C++ streams .
`FlexLexer.h', which defines the interface to two
C++ classes.
John Dickinson
2
cs 445 - System Program Design
03/26/2002
C++ classes
C++ classes
• The first class, FlexLexer, provides an abstract
• The first class, FlexLexer, provides an abstract
base class defining the general scanner class
base class defining the general scanner class
interface. It provides the following member
interface.
functions:
•
The second class defined in `FlexLexer.h' is
– ‘const char* YYText()’ – returns the text of the most recently
matched token, the equivalent of yytext.
yyFlexLexer, which is derived from FlexLexer.
– ‘int YYLeng()’ – returns the length of the most recently matched
token, the equivalent of yyleng.
• For details on these classes you should visit
– ‘int lineno() const’ – returns the current input line number
http://dinosaur.compilertools.net/flex/
– ‘void set_debug( int flag )’ – sets the debugging flag for the
scanner, equivalent to assigning to yy_flex_debug
– ‘int debug() const’ – returns the current setting of the debugging
flag.
Interfacing with yacc/bison
Introducing Bison
• One of the main uses of flex is as a
• Bison is like yacc (I’d suggest using
companion to the yacc parser-generator
bison, but if you only have access to
• yacc parsers expect to call a routine
yacc, then use yacc)
named `yylex()' to find the next input
• Input to Bison specifies the grammar
token
rules and the semantic actions
• The simplest way is to use an include
statement in the yacc/bison file (I’ll show
• Often flex is used for the lexical analysis
an example)
General Bison file format
Example
• The general form of a Bison grammar file is as
• Consider the grammar
follows:
S ? LET OP A
%{
C declarations
A ? A DIG | DIG
%}
Bison declarations
%%
Grammar rules
%%
Additional C code
– The `%%', `%{' and `%}' are punctuation that appear in every Bison
grammar file to separate the sections.
John Dickinson
3
cs 445 - System Program Design
03/26/2002
Bison file
• In bison.y there would be:
%{
%}
%token NUM, LET, OP
%%
S:
LET OP A
{printf(“Found S\n”);}
;
A:
A NUM
{printf(“Found A NUM\n”);}
| NUM
{printf(“Found NUM\n”);}
;
%%
#include “lex.yy.c”
John Dickinson
4
