lex Command lex
Lexical analyzer generator
lex [-t][-v][file]
cc lex.yy.c -ll
Many programs, e.g., compilers, process highly structured input
according to rules. Two of the most complicated parts of such
programs are lexical analysis and parsing (also called syntax
analysis). The COHERENT system includes two powerful tools
called lex and yacc to help you construct these parts of a
program. lex converts a set of lexical rules into a lexical
analyzer, and yacc converts a set of parsing rules into a parser.
The output of lex may be used directly, or may be used by a par-
ser generated by yacc.
lex reads a specification from the given file (or from the stan-
dard input if none), and generates a C function called yylex().
lex writes the generated function in the file lex.yy.c, or on
standard output if you use the -t option. The -v option prints
some statistics about the generated tables.
The tutorial on lex that appear in this manual describes lex in
detail. In brief, the generated function yylex() matches por-
tions of its input to one pattern (sometimes called a regular
expression) from a set of rules, or context, and executes as-
sociated C commands. Unmatched portions of the input are copied
to the output stream. yylex() returns EOF when input has been
exhausted.
lex uses the following macros that you may replace with the
preprocessor directive #undef if you wish: input() (read the
standard input stream), and output(c) (write the character c to
the standard output stream). You may also replace the following
functions if you wish: main() (main function), error(...) (print
error messages; takes same arguments as printf), and yywrap()
(handle events at the end of a file). If an action is desired on
end of file, such as arranging for more input, yywrap() should
perform it, returning zero to keep going.
A full lex specification has the following format:
* Macro definitions, of the form: name pattern
* Start condition declarations: %S NAME ...
* Context declarations: %C NAME ...
* Code to be included in the header section: %{
anything
%}
<tab or space> anything
COHERENT Lexicon Page 1
lex Command lex
* Rules section delimiter (must always be present): %%
* Code to appear at the start of yylex(): <tab or space> anything
* Rulesfor initialcontext, inanyof theforms: rule
action;
rule | (means use next action)
rule {
<tab or space> action;
<tab or space> }
* For each additional context: %C NAME
...rules for this context...
* End of rules section delimiter: %%
* Code to be copied verbatim, such as user provided input(),
output(), yywrap(), or other.
lex matches the longest string possible; if two rules match the
same length string, the rule specified first takes precedence.
lex puts the matched string, or token, in the char array
yytext[], and sets the variable yyleng to its length.
Actions may use the following:
ECHO Output the token
REJECT Perform action for lower precedence match
BEGIN NAME Set start condition to NAME
BEGIN 0 Clear start condition
yyswitch(NAME) Switch to context NAME, return current
yyswitch(0) Switch to initial context
yynext() Steal next character from input
yyback(c) Put character c back into input
yyless(n) Reduce token length to n, put rest back
yymore() Append next token to this one
yylook() Returns number of chars in input buffer
lex rules are contiguous strings of the form
[ <NAME,...> ][ ^ ] token [ /lookahead ][ $ ]
where brackets `[]' indicate optional items.
<NAME,...> Match only under given start conditions
^ Match the beginning of a line
$ Match the end of a line
token Pattern that a given token is to match
/lookahead Pattern that given trailing text is to match
COHERENT Lexicon Page 2
lex Command lex
Pattern elements:
a The character a
\a The character a, even if special
. Any character except newline
[abx-z] Any of a, b, or x through z
[^abx-z]Any except a, b, or x through z
abc The string abc, even if any are special
{name} The macro definition name
(exp) The pattern exp (grouping operator)
Optional operators on elements:
e? Zero or one occurrence of e
e* Zero or more consecutive es
e+ One or more consecutive es
e{n} n (a decimal number) consecutive es
e{m,n} m through n consecutive es
Patterns may be of the form:
e1e2 Matches the sequence e1 e2
e1|e2 Matches either e1 or e2
lex recognizes the standard C escapes: \n, \t, \r, \b, \f, and
\ooo (octal representation). The special characters
\ ( ) < > { } % * + ? [ - ] ^ / $ . |
must be prefixed with \ or enclosed within quotation marks (ex-
cepting " and \) to be normal. Within classes, only the charac-
ters . ^ - \ and ] are special.
***** Files *****
/usr/lib/libl.a
***** See Also *****
commands, yacc
Introduction to lex, the Lexical Analyzer
COHERENT Lexicon Page 3