Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(217)

Side by Side Diff: bison/src/bison/2.4.1/bison-2.4.1-src/doc/bison.texinfo

Issue 10807020: Add native Windows binary for bison. (Closed) Base URL: svn://chrome-svn/chrome/trunk/deps/third_party/
Patch Set: Created 8 years, 5 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
OLDNEW
(Empty)
1 \input texinfo @c -*-texinfo-*-
2 @comment %**start of header
3 @setfilename bison.info
4 @include version.texi
5 @settitle Bison @value{VERSION}
6 @setchapternewpage odd
7
8 @finalout
9
10 @c SMALL BOOK version
11 @c This edition has been formatted so that you can format and print it in
12 @c the smallbook format.
13 @c @smallbook
14
15 @c Set following if you want to document %default-prec and %no-default-prec.
16 @c This feature is experimental and may change in future Bison versions.
17 @c @set defaultprec
18
19 @ifnotinfo
20 @syncodeindex fn cp
21 @syncodeindex vr cp
22 @syncodeindex tp cp
23 @end ifnotinfo
24 @ifinfo
25 @synindex fn cp
26 @synindex vr cp
27 @synindex tp cp
28 @end ifinfo
29 @comment %**end of header
30
31 @copying
32
33 This manual (@value{UPDATED}) is for @acronym{GNU} Bison (version
34 @value{VERSION}), the @acronym{GNU} parser generator.
35
36 Copyright @copyright{} 1988, 1989, 1990, 1991, 1992, 1993, 1995, 1998,
37 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008 Free Software
38 Foundation, Inc.
39
40 @quotation
41 Permission is granted to copy, distribute and/or modify this document
42 under the terms of the @acronym{GNU} Free Documentation License,
43 Version 1.2 or any later version published by the Free Software
44 Foundation; with no Invariant Sections, with the Front-Cover texts
45 being ``A @acronym{GNU} Manual,'' and with the Back-Cover Texts as in
46 (a) below. A copy of the license is included in the section entitled
47 ``@acronym{GNU} Free Documentation License.''
48
49 (a) The FSF's Back-Cover Text is: ``You have the freedom to copy and
50 modify this @acronym{GNU} manual. Buying copies from the @acronym{FSF}
51 supports it in developing @acronym{GNU} and promoting software
52 freedom.''
53 @end quotation
54 @end copying
55
56 @dircategory Software development
57 @direntry
58 * bison: (bison). @acronym{GNU} parser generator (Yacc replacement).
59 @end direntry
60
61 @titlepage
62 @title Bison
63 @subtitle The Yacc-compatible Parser Generator
64 @subtitle @value{UPDATED}, Bison Version @value{VERSION}
65
66 @author by Charles Donnelly and Richard Stallman
67
68 @page
69 @vskip 0pt plus 1filll
70 @insertcopying
71 @sp 2
72 Published by the Free Software Foundation @*
73 51 Franklin Street, Fifth Floor @*
74 Boston, MA 02110-1301 USA @*
75 Printed copies are available from the Free Software Foundation.@*
76 @acronym{ISBN} 1-882114-44-2
77 @sp 2
78 Cover art by Etienne Suvasa.
79 @end titlepage
80
81 @contents
82
83 @ifnottex
84 @node Top
85 @top Bison
86 @insertcopying
87 @end ifnottex
88
89 @menu
90 * Introduction::
91 * Conditions::
92 * Copying:: The @acronym{GNU} General Public License says
93 how you can copy and share Bison.
94
95 Tutorial sections:
96 * Concepts:: Basic concepts for understanding Bison.
97 * Examples:: Three simple explained examples of using Bison.
98
99 Reference sections:
100 * Grammar File:: Writing Bison declarations and rules.
101 * Interface:: C-language interface to the parser function @code{yypars e}.
102 * Algorithm:: How the Bison parser works at run-time.
103 * Error Recovery:: Writing rules for error recovery.
104 * Context Dependency:: What to do if your language syntax is too
105 messy for Bison to handle straightforwardly.
106 * Debugging:: Understanding or debugging Bison parsers.
107 * Invocation:: How to run Bison (to produce the parser source file).
108 * Other Languages:: Creating C++ and Java parsers.
109 * FAQ:: Frequently Asked Questions
110 * Table of Symbols:: All the keywords of the Bison language are explained.
111 * Glossary:: Basic concepts are explained.
112 * Copying This Manual:: License for copying this manual.
113 * Index:: Cross-references to the text.
114
115 @detailmenu
116 --- The Detailed Node Listing ---
117
118 The Concepts of Bison
119
120 * Language and Grammar:: Languages and context-free grammars,
121 as mathematical ideas.
122 * Grammar in Bison:: How we represent grammars for Bison's sake.
123 * Semantic Values:: Each token or syntactic grouping can have
124 a semantic value (the value of an integer,
125 the name of an identifier, etc.).
126 * Semantic Actions:: Each rule can have an action containing C code.
127 * GLR Parsers:: Writing parsers for general context-free languages.
128 * Locations Overview:: Tracking Locations.
129 * Bison Parser:: What are Bison's input and output,
130 how is the output used?
131 * Stages:: Stages in writing and running Bison grammars.
132 * Grammar Layout:: Overall structure of a Bison grammar file.
133
134 Writing @acronym{GLR} Parsers
135
136 * Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
137 * Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
138 * GLR Semantic Actions:: Deferred semantic actions have special concerns.
139 * Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
140
141 Examples
142
143 * RPN Calc:: Reverse polish notation calculator;
144 a first example with no operator precedence.
145 * Infix Calc:: Infix (algebraic) notation calculator.
146 Operator precedence is introduced.
147 * Simple Error Recovery:: Continuing after syntax errors.
148 * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
149 * Multi-function Calc:: Calculator with memory and trig functions.
150 It uses multiple data-types for semantic values.
151 * Exercises:: Ideas for improving the multi-function calculator.
152
153 Reverse Polish Notation Calculator
154
155 * Rpcalc Declarations:: Prologue (declarations) for rpcalc.
156 * Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
157 * Rpcalc Lexer:: The lexical analyzer.
158 * Rpcalc Main:: The controlling function.
159 * Rpcalc Error:: The error reporting function.
160 * Rpcalc Generate:: Running Bison on the grammar file.
161 * Rpcalc Compile:: Run the C compiler on the output code.
162
163 Grammar Rules for @code{rpcalc}
164
165 * Rpcalc Input::
166 * Rpcalc Line::
167 * Rpcalc Expr::
168
169 Location Tracking Calculator: @code{ltcalc}
170
171 * Ltcalc Declarations:: Bison and C declarations for ltcalc.
172 * Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
173 * Ltcalc Lexer:: The lexical analyzer.
174
175 Multi-Function Calculator: @code{mfcalc}
176
177 * Mfcalc Declarations:: Bison declarations for multi-function calculator.
178 * Mfcalc Rules:: Grammar rules for the calculator.
179 * Mfcalc Symbol Table:: Symbol table management subroutines.
180
181 Bison Grammar Files
182
183 * Grammar Outline:: Overall layout of the grammar file.
184 * Symbols:: Terminal and nonterminal symbols.
185 * Rules:: How to write grammar rules.
186 * Recursion:: Writing recursive rules.
187 * Semantics:: Semantic values and actions.
188 * Locations:: Locations and actions.
189 * Declarations:: All kinds of Bison declarations are described here.
190 * Multiple Parsers:: Putting more than one Bison parser in one program.
191
192 Outline of a Bison Grammar
193
194 * Prologue:: Syntax and usage of the prologue.
195 * Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
196 * Bison Declarations:: Syntax and usage of the Bison declarations section.
197 * Grammar Rules:: Syntax and usage of the grammar rules section.
198 * Epilogue:: Syntax and usage of the epilogue.
199
200 Defining Language Semantics
201
202 * Value Type:: Specifying one data type for all semantic values.
203 * Multiple Types:: Specifying several alternative data types.
204 * Actions:: An action is the semantic definition of a grammar rule.
205 * Action Types:: Specifying data types for actions to operate on.
206 * Mid-Rule Actions:: Most actions go at the end of a rule.
207 This says when, why and how to use the exceptional
208 action in the middle of a rule.
209
210 Tracking Locations
211
212 * Location Type:: Specifying a data type for locations.
213 * Actions and Locations:: Using locations in actions.
214 * Location Default Action:: Defining a general way to compute locations.
215
216 Bison Declarations
217
218 * Require Decl:: Requiring a Bison version.
219 * Token Decl:: Declaring terminal symbols.
220 * Precedence Decl:: Declaring terminals with precedence and associativity.
221 * Union Decl:: Declaring the set of all semantic value types.
222 * Type Decl:: Declaring the choice of type for a nonterminal symbol.
223 * Initial Action Decl:: Code run before parsing starts.
224 * Destructor Decl:: Declaring how symbols are freed.
225 * Expect Decl:: Suppressing warnings about parsing conflicts.
226 * Start Decl:: Specifying the start symbol.
227 * Pure Decl:: Requesting a reentrant parser.
228 * Push Decl:: Requesting a push parser.
229 * Decl Summary:: Table of all Bison declarations.
230
231 Parser C-Language Interface
232
233 * Parser Function:: How to call @code{yyparse} and what it returns.
234 * Push Parser Function:: How to call @code{yypush_parse} and what it returns.
235 * Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
236 * Parser Create Function:: How to call @code{yypstate_new} and what it returns.
237 * Parser Delete Function:: How to call @code{yypstate_delete} and what it retur ns.
238 * Lexical:: You must supply a function @code{yylex}
239 which reads tokens.
240 * Error Reporting:: You must supply a function @code{yyerror}.
241 * Action Features:: Special features for use in actions.
242 * Internationalization:: How to let the parser speak in the user's
243 native language.
244
245 The Lexical Analyzer Function @code{yylex}
246
247 * Calling Convention:: How @code{yyparse} calls @code{yylex}.
248 * Token Values:: How @code{yylex} must return the semantic value
249 of the token it has read.
250 * Token Locations:: How @code{yylex} must return the text location
251 (line number, etc.) of the token, if the
252 actions want that.
253 * Pure Calling:: How the calling convention differs in a pure parser
254 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
255
256 The Bison Parser Algorithm
257
258 * Lookahead:: Parser looks one token ahead when deciding what to do.
259 * Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
260 * Precedence:: Operator precedence works by resolving conflicts.
261 * Contextual Precedence:: When an operator's precedence depends on context.
262 * Parser States:: The parser is a finite-state-machine with stack.
263 * Reduce/Reduce:: When two rules are applicable in the same situation.
264 * Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
265 * Generalized LR Parsing:: Parsing arbitrary context-free grammars.
266 * Memory Management:: What happens when memory is exhausted. How to avoid it.
267
268 Operator Precedence
269
270 * Why Precedence:: An example showing why precedence is needed.
271 * Using Precedence:: How to specify precedence in Bison grammars.
272 * Precedence Examples:: How these features are used in the previous example.
273 * How Precedence:: How they work.
274
275 Handling Context Dependencies
276
277 * Semantic Tokens:: Token parsing can depend on the semantic context.
278 * Lexical Tie-ins:: Token parsing can depend on the syntactic context.
279 * Tie-in Recovery:: Lexical tie-ins have implications for how
280 error recovery rules must be written.
281
282 Debugging Your Parser
283
284 * Understanding:: Understanding the structure of your parser.
285 * Tracing:: Tracing the execution of your parser.
286
287 Invoking Bison
288
289 * Bison Options:: All the options described in detail,
290 in alphabetical order by short options.
291 * Option Cross Key:: Alphabetical list of long options.
292 * Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
293
294 Parsers Written In Other Languages
295
296 * C++ Parsers:: The interface to generate C++ parser classes
297 * Java Parsers:: The interface to generate Java parser classes
298
299 C++ Parsers
300
301 * C++ Bison Interface:: Asking for C++ parser generation
302 * C++ Semantic Values:: %union vs. C++
303 * C++ Location Values:: The position and location classes
304 * C++ Parser Interface:: Instantiating and running the parser
305 * C++ Scanner Interface:: Exchanges between yylex and parse
306 * A Complete C++ Example:: Demonstrating their use
307
308 A Complete C++ Example
309
310 * Calc++ --- C++ Calculator:: The specifications
311 * Calc++ Parsing Driver:: An active parsing context
312 * Calc++ Parser:: A parser class
313 * Calc++ Scanner:: A pure C++ Flex scanner
314 * Calc++ Top Level:: Conducting the band
315
316 Java Parsers
317
318 * Java Bison Interface:: Asking for Java parser generation
319 * Java Semantic Values:: %type and %token vs. Java
320 * Java Location Values:: The position and location classes
321 * Java Parser Interface:: Instantiating and running the parser
322 * Java Scanner Interface:: Specifying the scanner for the parser
323 * Java Action Features:: Special features for use in actions
324 * Java Differences:: Differences between C/C++ and Java Grammars
325 * Java Declarations Summary:: List of Bison declarations used with Java
326
327 Frequently Asked Questions
328
329 * Memory Exhausted:: Breaking the Stack Limits
330 * How Can I Reset the Parser:: @code{yyparse} Keeps some State
331 * Strings are Destroyed:: @code{yylval} Loses Track of Strings
332 * Implementing Gotos/Loops:: Control Flow in the Calculator
333 * Multiple start-symbols:: Factoring closely related grammars
334 * Secure? Conform?:: Is Bison @acronym{POSIX} safe?
335 * I can't build Bison:: Troubleshooting
336 * Where can I find help?:: Troubleshouting
337 * Bug Reports:: Troublereporting
338 * More Languages:: Parsers in C++, Java, and so on
339 * Beta Testing:: Experimenting development versions
340 * Mailing Lists:: Meeting other Bison users
341
342 Copying This Manual
343
344 * Copying This Manual:: License for copying this manual.
345
346 @end detailmenu
347 @end menu
348
349 @node Introduction
350 @unnumbered Introduction
351 @cindex introduction
352
353 @dfn{Bison} is a general-purpose parser generator that converts an
354 annotated context-free grammar into an @acronym{LALR}(1) or
355 @acronym{GLR} parser for that grammar. Once you are proficient with
356 Bison, you can use it to develop a wide range of language parsers, from those
357 used in simple desk calculators to complex programming languages.
358
359 Bison is upward compatible with Yacc: all properly-written Yacc grammars
360 ought to work with Bison with no change. Anyone familiar with Yacc
361 should be able to use Bison with little trouble. You need to be fluent in
362 C or C++ programming in order to use Bison or to understand this manual.
363
364 We begin with tutorial chapters that explain the basic concepts of using
365 Bison and show three explained examples, each building on the last. If you
366 don't know Bison or Yacc, start by reading these chapters. Reference
367 chapters follow which describe specific aspects of Bison in detail.
368
369 Bison was written primarily by Robert Corbett; Richard Stallman made it
370 Yacc-compatible. Wilfred Hansen of Carnegie Mellon University added
371 multi-character string literals and other features.
372
373 This edition corresponds to version @value{VERSION} of Bison.
374
375 @node Conditions
376 @unnumbered Conditions for Using Bison
377
378 The distribution terms for Bison-generated parsers permit using the
379 parsers in nonfree programs. Before Bison version 2.2, these extra
380 permissions applied only when Bison was generating @acronym{LALR}(1)
381 parsers in C@. And before Bison version 1.24, Bison-generated
382 parsers could be used only in programs that were free software.
383
384 The other @acronym{GNU} programming tools, such as the @acronym{GNU} C
385 compiler, have never
386 had such a requirement. They could always be used for nonfree
387 software. The reason Bison was different was not due to a special
388 policy decision; it resulted from applying the usual General Public
389 License to all of the Bison source code.
390
391 The output of the Bison utility---the Bison parser file---contains a
392 verbatim copy of a sizable piece of Bison, which is the code for the
393 parser's implementation. (The actions from your grammar are inserted
394 into this implementation at one point, but most of the rest of the
395 implementation is not changed.) When we applied the @acronym{GPL}
396 terms to the skeleton code for the parser's implementation,
397 the effect was to restrict the use of Bison output to free software.
398
399 We didn't change the terms because of sympathy for people who want to
400 make software proprietary. @strong{Software should be free.} But we
401 concluded that limiting Bison's use to free software was doing little to
402 encourage people to make other software free. So we decided to make the
403 practical conditions for using Bison match the practical conditions for
404 using the other @acronym{GNU} tools.
405
406 This exception applies when Bison is generating code for a parser.
407 You can tell whether the exception applies to a Bison output file by
408 inspecting the file for text beginning with ``As a special
409 exception@dots{}''. The text spells out the exact terms of the
410 exception.
411
412 @node Copying
413 @unnumbered GNU GENERAL PUBLIC LICENSE
414 @include gpl-3.0.texi
415
416 @node Concepts
417 @chapter The Concepts of Bison
418
419 This chapter introduces many of the basic concepts without which the
420 details of Bison will not make sense. If you do not already know how to
421 use Bison or Yacc, we suggest you start by reading this chapter carefully.
422
423 @menu
424 * Language and Grammar:: Languages and context-free grammars,
425 as mathematical ideas.
426 * Grammar in Bison:: How we represent grammars for Bison's sake.
427 * Semantic Values:: Each token or syntactic grouping can have
428 a semantic value (the value of an integer,
429 the name of an identifier, etc.).
430 * Semantic Actions:: Each rule can have an action containing C code.
431 * GLR Parsers:: Writing parsers for general context-free languages.
432 * Locations Overview:: Tracking Locations.
433 * Bison Parser:: What are Bison's input and output,
434 how is the output used?
435 * Stages:: Stages in writing and running Bison grammars.
436 * Grammar Layout:: Overall structure of a Bison grammar file.
437 @end menu
438
439 @node Language and Grammar
440 @section Languages and Context-Free Grammars
441
442 @cindex context-free grammar
443 @cindex grammar, context-free
444 In order for Bison to parse a language, it must be described by a
445 @dfn{context-free grammar}. This means that you specify one or more
446 @dfn{syntactic groupings} and give rules for constructing them from their
447 parts. For example, in the C language, one kind of grouping is called an
448 `expression'. One rule for making an expression might be, ``An expression
449 can be made of a minus sign and another expression''. Another would be,
450 ``An expression can be an integer''. As you can see, rules are often
451 recursive, but there must be at least one rule which leads out of the
452 recursion.
453
454 @cindex @acronym{BNF}
455 @cindex Backus-Naur form
456 The most common formal system for presenting such rules for humans to read
457 is @dfn{Backus-Naur Form} or ``@acronym{BNF}'', which was developed in
458 order to specify the language Algol 60. Any grammar expressed in
459 @acronym{BNF} is a context-free grammar. The input to Bison is
460 essentially machine-readable @acronym{BNF}.
461
462 @cindex @acronym{LALR}(1) grammars
463 @cindex @acronym{LR}(1) grammars
464 There are various important subclasses of context-free grammar. Although it
465 can handle almost all context-free grammars, Bison is optimized for what
466 are called @acronym{LALR}(1) grammars.
467 In brief, in these grammars, it must be possible to
468 tell how to parse any portion of an input string with just a single
469 token of lookahead. Strictly speaking, that is a description of an
470 @acronym{LR}(1) grammar, and @acronym{LALR}(1) involves additional
471 restrictions that are
472 hard to explain simply; but it is rare in actual practice to find an
473 @acronym{LR}(1) grammar that fails to be @acronym{LALR}(1).
474 @xref{Mystery Conflicts, ,Mysterious Reduce/Reduce Conflicts}, for
475 more information on this.
476
477 @cindex @acronym{GLR} parsing
478 @cindex generalized @acronym{LR} (@acronym{GLR}) parsing
479 @cindex ambiguous grammars
480 @cindex nondeterministic parsing
481
482 Parsers for @acronym{LALR}(1) grammars are @dfn{deterministic}, meaning
483 roughly that the next grammar rule to apply at any point in the input is
484 uniquely determined by the preceding input and a fixed, finite portion
485 (called a @dfn{lookahead}) of the remaining input. A context-free
486 grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
487 apply the grammar rules to get the same inputs. Even unambiguous
488 grammars can be @dfn{nondeterministic}, meaning that no fixed
489 lookahead always suffices to determine the next grammar rule to apply.
490 With the proper declarations, Bison is also able to parse these more
491 general context-free grammars, using a technique known as @acronym{GLR}
492 parsing (for Generalized @acronym{LR}). Bison's @acronym{GLR} parsers
493 are able to handle any context-free grammar for which the number of
494 possible parses of any given string is finite.
495
496 @cindex symbols (abstract)
497 @cindex token
498 @cindex syntactic grouping
499 @cindex grouping, syntactic
500 In the formal grammatical rules for a language, each kind of syntactic
501 unit or grouping is named by a @dfn{symbol}. Those which are built by
502 grouping smaller constructs according to grammatical rules are called
503 @dfn{nonterminal symbols}; those which can't be subdivided are called
504 @dfn{terminal symbols} or @dfn{token types}. We call a piece of input
505 corresponding to a single terminal symbol a @dfn{token}, and a piece
506 corresponding to a single nonterminal symbol a @dfn{grouping}.
507
508 We can use the C language as an example of what symbols, terminal and
509 nonterminal, mean. The tokens of C are identifiers, constants (numeric
510 and string), and the various keywords, arithmetic operators and
511 punctuation marks. So the terminal symbols of a grammar for C include
512 `identifier', `number', `string', plus one symbol for each keyword,
513 operator or punctuation mark: `if', `return', `const', `static', `int',
514 `char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
515 (These tokens can be subdivided into characters, but that is a matter of
516 lexicography, not grammar.)
517
518 Here is a simple C function subdivided into tokens:
519
520 @ifinfo
521 @example
522 int /* @r{keyword `int'} */
523 square (int x) /* @r{identifier, open-paren, keyword `int',}
524 @r{identifier, close-paren} */
525 @{ /* @r{open-brace} */
526 return x * x; /* @r{keyword `return', identifier, asterisk,}
527 @r{identifier, semicolon} */
528 @} /* @r{close-brace} */
529 @end example
530 @end ifinfo
531 @ifnotinfo
532 @example
533 int /* @r{keyword `int'} */
534 square (int x) /* @r{identifier, open-paren, keyword `int', identifier, close-p aren} */
535 @{ /* @r{open-brace} */
536 return x * x; /* @r{keyword `return', identifier, asterisk, identifier, semico lon} */
537 @} /* @r{close-brace} */
538 @end example
539 @end ifnotinfo
540
541 The syntactic groupings of C include the expression, the statement, the
542 declaration, and the function definition. These are represented in the
543 grammar of C by nonterminal symbols `expression', `statement',
544 `declaration' and `function definition'. The full grammar uses dozens of
545 additional language constructs, each with its own nonterminal symbol, in
546 order to express the meanings of these four. The example above is a
547 function definition; it contains one declaration, and one statement. In
548 the statement, each @samp{x} is an expression and so is @samp{x * x}.
549
550 Each nonterminal symbol must have grammatical rules showing how it is made
551 out of simpler constructs. For example, one kind of C statement is the
552 @code{return} statement; this would be described with a grammar rule which
553 reads informally as follows:
554
555 @quotation
556 A `statement' can be made of a `return' keyword, an `expression' and a
557 `semicolon'.
558 @end quotation
559
560 @noindent
561 There would be many other rules for `statement', one for each kind of
562 statement in C.
563
564 @cindex start symbol
565 One nonterminal symbol must be distinguished as the special one which
566 defines a complete utterance in the language. It is called the @dfn{start
567 symbol}. In a compiler, this means a complete input program. In the C
568 language, the nonterminal symbol `sequence of definitions and declarations'
569 plays this role.
570
571 For example, @samp{1 + 2} is a valid C expression---a valid part of a C
572 program---but it is not valid as an @emph{entire} C program. In the
573 context-free grammar of C, this follows from the fact that `expression' is
574 not the start symbol.
575
576 The Bison parser reads a sequence of tokens as its input, and groups the
577 tokens using the grammar rules. If the input is valid, the end result is
578 that the entire token sequence reduces to a single grouping whose symbol is
579 the grammar's start symbol. If we use a grammar for C, the entire input
580 must be a `sequence of definitions and declarations'. If not, the parser
581 reports a syntax error.
582
583 @node Grammar in Bison
584 @section From Formal Rules to Bison Input
585 @cindex Bison grammar
586 @cindex grammar, Bison
587 @cindex formal grammar
588
589 A formal grammar is a mathematical construct. To define the language
590 for Bison, you must write a file expressing the grammar in Bison syntax:
591 a @dfn{Bison grammar} file. @xref{Grammar File, ,Bison Grammar Files}.
592
593 A nonterminal symbol in the formal grammar is represented in Bison input
594 as an identifier, like an identifier in C@. By convention, it should be
595 in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
596
597 The Bison representation for a terminal symbol is also called a @dfn{token
598 type}. Token types as well can be represented as C-like identifiers. By
599 convention, these identifiers should be upper case to distinguish them from
600 nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
601 @code{RETURN}. A terminal symbol that stands for a particular keyword in
602 the language should be named after that keyword converted to upper case.
603 The terminal symbol @code{error} is reserved for error recovery.
604 @xref{Symbols}.
605
606 A terminal symbol can also be represented as a character literal, just like
607 a C character constant. You should do this whenever a token is just a
608 single character (parenthesis, plus-sign, etc.): use that same character in
609 a literal as the terminal symbol for that token.
610
611 A third way to represent a terminal symbol is with a C string constant
612 containing several characters. @xref{Symbols}, for more information.
613
614 The grammar rules also have an expression in Bison syntax. For example,
615 here is the Bison rule for a C @code{return} statement. The semicolon in
616 quotes is a literal character token, representing part of the C syntax for
617 the statement; the naked semicolon, and the colon, are Bison punctuation
618 used in every rule.
619
620 @example
621 stmt: RETURN expr ';'
622 ;
623 @end example
624
625 @noindent
626 @xref{Rules, ,Syntax of Grammar Rules}.
627
628 @node Semantic Values
629 @section Semantic Values
630 @cindex semantic value
631 @cindex value, semantic
632
633 A formal grammar selects tokens only by their classifications: for example,
634 if a rule mentions the terminal symbol `integer constant', it means that
635 @emph{any} integer constant is grammatically valid in that position. The
636 precise value of the constant is irrelevant to how to parse the input: if
637 @samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
638 grammatical.
639
640 But the precise value is very important for what the input means once it is
641 parsed. A compiler is useless if it fails to distinguish between 4, 1 and
642 3989 as constants in the program! Therefore, each token in a Bison grammar
643 has both a token type and a @dfn{semantic value}. @xref{Semantics,
644 ,Defining Language Semantics},
645 for details.
646
647 The token type is a terminal symbol defined in the grammar, such as
648 @code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything
649 you need to know to decide where the token may validly appear and how to
650 group it with other tokens. The grammar rules know nothing about tokens
651 except their types.
652
653 The semantic value has all the rest of the information about the
654 meaning of the token, such as the value of an integer, or the name of an
655 identifier. (A token such as @code{','} which is just punctuation doesn't
656 need to have any semantic value.)
657
658 For example, an input token might be classified as token type
659 @code{INTEGER} and have the semantic value 4. Another input token might
660 have the same token type @code{INTEGER} but value 3989. When a grammar
661 rule says that @code{INTEGER} is allowed, either of these tokens is
662 acceptable because each is an @code{INTEGER}. When the parser accepts the
663 token, it keeps track of the token's semantic value.
664
665 Each grouping can also have a semantic value as well as its nonterminal
666 symbol. For example, in a calculator, an expression typically has a
667 semantic value that is a number. In a compiler for a programming
668 language, an expression typically has a semantic value that is a tree
669 structure describing the meaning of the expression.
670
671 @node Semantic Actions
672 @section Semantic Actions
673 @cindex semantic actions
674 @cindex actions, semantic
675
676 In order to be useful, a program must do more than parse input; it must
677 also produce some output based on the input. In a Bison grammar, a grammar
678 rule can have an @dfn{action} made up of C statements. Each time the
679 parser recognizes a match for that rule, the action is executed.
680 @xref{Actions}.
681
682 Most of the time, the purpose of an action is to compute the semantic value
683 of the whole construct from the semantic values of its parts. For example,
684 suppose we have a rule which says an expression can be the sum of two
685 expressions. When the parser recognizes such a sum, each of the
686 subexpressions has a semantic value which describes how it was built up.
687 The action for this rule should create a similar sort of value for the
688 newly recognized larger expression.
689
690 For example, here is a rule that says an expression can be the sum of
691 two subexpressions:
692
693 @example
694 expr: expr '+' expr @{ $$ = $1 + $3; @}
695 ;
696 @end example
697
698 @noindent
699 The action says how to produce the semantic value of the sum expression
700 from the values of the two subexpressions.
701
702 @node GLR Parsers
703 @section Writing @acronym{GLR} Parsers
704 @cindex @acronym{GLR} parsing
705 @cindex generalized @acronym{LR} (@acronym{GLR}) parsing
706 @findex %glr-parser
707 @cindex conflicts
708 @cindex shift/reduce conflicts
709 @cindex reduce/reduce conflicts
710
711 In some grammars, Bison's standard
712 @acronym{LALR}(1) parsing algorithm cannot decide whether to apply a
713 certain grammar rule at a given point. That is, it may not be able to
714 decide (on the basis of the input read so far) which of two possible
715 reductions (applications of a grammar rule) applies, or whether to apply
716 a reduction or read more of the input and apply a reduction later in the
717 input. These are known respectively as @dfn{reduce/reduce} conflicts
718 (@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
719 (@pxref{Shift/Reduce}).
720
721 To use a grammar that is not easily modified to be @acronym{LALR}(1), a
722 more general parsing algorithm is sometimes necessary. If you include
723 @code{%glr-parser} among the Bison declarations in your file
724 (@pxref{Grammar Outline}), the result is a Generalized @acronym{LR}
725 (@acronym{GLR}) parser. These parsers handle Bison grammars that
726 contain no unresolved conflicts (i.e., after applying precedence
727 declarations) identically to @acronym{LALR}(1) parsers. However, when
728 faced with unresolved shift/reduce and reduce/reduce conflicts,
729 @acronym{GLR} parsers use the simple expedient of doing both,
730 effectively cloning the parser to follow both possibilities. Each of
731 the resulting parsers can again split, so that at any given time, there
732 can be any number of possible parses being explored. The parsers
733 proceed in lockstep; that is, all of them consume (shift) a given input
734 symbol before any of them proceed to the next. Each of the cloned
735 parsers eventually meets one of two possible fates: either it runs into
736 a parsing error, in which case it simply vanishes, or it merges with
737 another parser, because the two of them have reduced the input to an
738 identical set of symbols.
739
740 During the time that there are multiple parsers, semantic actions are
741 recorded, but not performed. When a parser disappears, its recorded
742 semantic actions disappear as well, and are never performed. When a
743 reduction makes two parsers identical, causing them to merge, Bison
744 records both sets of semantic actions. Whenever the last two parsers
745 merge, reverting to the single-parser case, Bison resolves all the
746 outstanding actions either by precedences given to the grammar rules
747 involved, or by performing both actions, and then calling a designated
748 user-defined function on the resulting values to produce an arbitrary
749 merged result.
750
751 @menu
752 * Simple GLR Parsers:: Using @acronym{GLR} parsers on unambiguous grammars.
753 * Merging GLR Parses:: Using @acronym{GLR} parsers to resolve ambiguities.
754 * GLR Semantic Actions:: Deferred semantic actions have special concerns.
755 * Compiler Requirements:: @acronym{GLR} parsers require a modern C compiler.
756 @end menu
757
758 @node Simple GLR Parsers
759 @subsection Using @acronym{GLR} on Unambiguous Grammars
760 @cindex @acronym{GLR} parsing, unambiguous grammars
761 @cindex generalized @acronym{LR} (@acronym{GLR}) parsing, unambiguous grammars
762 @findex %glr-parser
763 @findex %expect-rr
764 @cindex conflicts
765 @cindex reduce/reduce conflicts
766 @cindex shift/reduce conflicts
767
768 In the simplest cases, you can use the @acronym{GLR} algorithm
769 to parse grammars that are unambiguous, but fail to be @acronym{LALR}(1).
770 Such grammars typically require more than one symbol of lookahead,
771 or (in rare cases) fall into the category of grammars in which the
772 @acronym{LALR}(1) algorithm throws away too much information (they are in
773 @acronym{LR}(1), but not @acronym{LALR}(1), @ref{Mystery Conflicts}).
774
775 Consider a problem that
776 arises in the declaration of enumerated and subrange types in the
777 programming language Pascal. Here are some examples:
778
779 @example
780 type subrange = lo .. hi;
781 type enum = (a, b, c);
782 @end example
783
784 @noindent
785 The original language standard allows only numeric
786 literals and constant identifiers for the subrange bounds (@samp{lo}
787 and @samp{hi}), but Extended Pascal (@acronym{ISO}/@acronym{IEC}
788 10206) and many other
789 Pascal implementations allow arbitrary expressions there. This gives
790 rise to the following situation, containing a superfluous pair of
791 parentheses:
792
793 @example
794 type subrange = (a) .. b;
795 @end example
796
797 @noindent
798 Compare this to the following declaration of an enumerated
799 type with only one value:
800
801 @example
802 type enum = (a);
803 @end example
804
805 @noindent
806 (These declarations are contrived, but they are syntactically
807 valid, and more-complicated cases can come up in practical programs.)
808
809 These two declarations look identical until the @samp{..} token.
810 With normal @acronym{LALR}(1) one-token lookahead it is not
811 possible to decide between the two forms when the identifier
812 @samp{a} is parsed. It is, however, desirable
813 for a parser to decide this, since in the latter case
814 @samp{a} must become a new identifier to represent the enumeration
815 value, while in the former case @samp{a} must be evaluated with its
816 current meaning, which may be a constant or even a function call.
817
818 You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
819 to be resolved later, but this typically requires substantial
820 contortions in both semantic actions and large parts of the
821 grammar, where the parentheses are nested in the recursive rules for
822 expressions.
823
824 You might think of using the lexer to distinguish between the two
825 forms by returning different tokens for currently defined and
826 undefined identifiers. But if these declarations occur in a local
827 scope, and @samp{a} is defined in an outer scope, then both forms
828 are possible---either locally redefining @samp{a}, or using the
829 value of @samp{a} from the outer scope. So this approach cannot
830 work.
831
832 A simple solution to this problem is to declare the parser to
833 use the @acronym{GLR} algorithm.
834 When the @acronym{GLR} parser reaches the critical state, it
835 merely splits into two branches and pursues both syntax rules
836 simultaneously. Sooner or later, one of them runs into a parsing
837 error. If there is a @samp{..} token before the next
838 @samp{;}, the rule for enumerated types fails since it cannot
839 accept @samp{..} anywhere; otherwise, the subrange type rule
840 fails since it requires a @samp{..} token. So one of the branches
841 fails silently, and the other one continues normally, performing
842 all the intermediate actions that were postponed during the split.
843
844 If the input is syntactically incorrect, both branches fail and the parser
845 reports a syntax error as usual.
846
847 The effect of all this is that the parser seems to ``guess'' the
848 correct branch to take, or in other words, it seems to use more
849 lookahead than the underlying @acronym{LALR}(1) algorithm actually allows
850 for. In this example, @acronym{LALR}(2) would suffice, but also some cases
851 that are not @acronym{LALR}(@math{k}) for any @math{k} can be handled this way.
852
853 In general, a @acronym{GLR} parser can take quadratic or cubic worst-case time,
854 and the current Bison parser even takes exponential time and space
855 for some grammars. In practice, this rarely happens, and for many
856 grammars it is possible to prove that it cannot happen.
857 The present example contains only one conflict between two
858 rules, and the type-declaration context containing the conflict
859 cannot be nested. So the number of
860 branches that can exist at any time is limited by the constant 2,
861 and the parsing time is still linear.
862
863 Here is a Bison grammar corresponding to the example above. It
864 parses a vastly simplified form of Pascal type declarations.
865
866 @example
867 %token TYPE DOTDOT ID
868
869 @group
870 %left '+' '-'
871 %left '*' '/'
872 @end group
873
874 %%
875
876 @group
877 type_decl : TYPE ID '=' type ';'
878 ;
879 @end group
880
881 @group
882 type : '(' id_list ')'
883 | expr DOTDOT expr
884 ;
885 @end group
886
887 @group
888 id_list : ID
889 | id_list ',' ID
890 ;
891 @end group
892
893 @group
894 expr : '(' expr ')'
895 | expr '+' expr
896 | expr '-' expr
897 | expr '*' expr
898 | expr '/' expr
899 | ID
900 ;
901 @end group
902 @end example
903
904 When used as a normal @acronym{LALR}(1) grammar, Bison correctly complains
905 about one reduce/reduce conflict. In the conflicting situation the
906 parser chooses one of the alternatives, arbitrarily the one
907 declared first. Therefore the following correct input is not
908 recognized:
909
910 @example
911 type t = (a) .. b;
912 @end example
913
914 The parser can be turned into a @acronym{GLR} parser, while also telling Bison
915 to be silent about the one known reduce/reduce conflict, by
916 adding these two declarations to the Bison input file (before the first
917 @samp{%%}):
918
919 @example
920 %glr-parser
921 %expect-rr 1
922 @end example
923
924 @noindent
925 No change in the grammar itself is required. Now the
926 parser recognizes all valid declarations, according to the
927 limited syntax above, transparently. In fact, the user does not even
928 notice when the parser splits.
929
930 So here we have a case where we can use the benefits of @acronym{GLR},
931 almost without disadvantages. Even in simple cases like this, however,
932 there are at least two potential problems to beware. First, always
933 analyze the conflicts reported by Bison to make sure that @acronym{GLR}
934 splitting is only done where it is intended. A @acronym{GLR} parser
935 splitting inadvertently may cause problems less obvious than an
936 @acronym{LALR} parser statically choosing the wrong alternative in a
937 conflict. Second, consider interactions with the lexer (@pxref{Semantic
938 Tokens}) with great care. Since a split parser consumes tokens without
939 performing any actions during the split, the lexer cannot obtain
940 information via parser actions. Some cases of lexer interactions can be
941 eliminated by using @acronym{GLR} to shift the complications from the
942 lexer to the parser. You must check the remaining cases for
943 correctness.
944
945 In our example, it would be safe for the lexer to return tokens based on
946 their current meanings in some symbol table, because no new symbols are
947 defined in the middle of a type declaration. Though it is possible for
948 a parser to define the enumeration constants as they are parsed, before
949 the type declaration is completed, it actually makes no difference since
950 they cannot be used within the same enumerated type declaration.
951
952 @node Merging GLR Parses
953 @subsection Using @acronym{GLR} to Resolve Ambiguities
954 @cindex @acronym{GLR} parsing, ambiguous grammars
955 @cindex generalized @acronym{LR} (@acronym{GLR}) parsing, ambiguous grammars
956 @findex %dprec
957 @findex %merge
958 @cindex conflicts
959 @cindex reduce/reduce conflicts
960
961 Let's consider an example, vastly simplified from a C++ grammar.
962
963 @example
964 %@{
965 #include <stdio.h>
966 #define YYSTYPE char const *
967 int yylex (void);
968 void yyerror (char const *);
969 %@}
970
971 %token TYPENAME ID
972
973 %right '='
974 %left '+'
975
976 %glr-parser
977
978 %%
979
980 prog :
981 | prog stmt @{ printf ("\n"); @}
982 ;
983
984 stmt : expr ';' %dprec 1
985 | decl %dprec 2
986 ;
987
988 expr : ID @{ printf ("%s ", $$); @}
989 | TYPENAME '(' expr ')'
990 @{ printf ("%s <cast> ", $1); @}
991 | expr '+' expr @{ printf ("+ "); @}
992 | expr '=' expr @{ printf ("= "); @}
993 ;
994
995 decl : TYPENAME declarator ';'
996 @{ printf ("%s <declare> ", $1); @}
997 | TYPENAME declarator '=' expr ';'
998 @{ printf ("%s <init-declare> ", $1); @}
999 ;
1000
1001 declarator : ID @{ printf ("\"%s\" ", $1); @}
1002 | '(' declarator ')'
1003 ;
1004 @end example
1005
1006 @noindent
1007 This models a problematic part of the C++ grammar---the ambiguity between
1008 certain declarations and statements. For example,
1009
1010 @example
1011 T (x) = y+z;
1012 @end example
1013
1014 @noindent
1015 parses as either an @code{expr} or a @code{stmt}
1016 (assuming that @samp{T} is recognized as a @code{TYPENAME} and
1017 @samp{x} as an @code{ID}).
1018 Bison detects this as a reduce/reduce conflict between the rules
1019 @code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
1020 time it encounters @code{x} in the example above. Since this is a
1021 @acronym{GLR} parser, it therefore splits the problem into two parses, one for
1022 each choice of resolving the reduce/reduce conflict.
1023 Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1024 however, neither of these parses ``dies,'' because the grammar as it stands is
1025 ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1026 the other reduces @code{stmt : decl}, after which both parsers are in an
1027 identical state: they've seen @samp{prog stmt} and have the same unprocessed
1028 input remaining. We say that these parses have @dfn{merged.}
1029
1030 At this point, the @acronym{GLR} parser requires a specification in the
1031 grammar of how to choose between the competing parses.
1032 In the example above, the two @code{%dprec}
1033 declarations specify that Bison is to give precedence
1034 to the parse that interprets the example as a
1035 @code{decl}, which implies that @code{x} is a declarator.
1036 The parser therefore prints
1037
1038 @example
1039 "x" y z + T <init-declare>
1040 @end example
1041
1042 The @code{%dprec} declarations only come into play when more than one
1043 parse survives. Consider a different input string for this parser:
1044
1045 @example
1046 T (x) + y;
1047 @end example
1048
1049 @noindent
1050 This is another example of using @acronym{GLR} to parse an unambiguous
1051 construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
1052 Here, there is no ambiguity (this cannot be parsed as a declaration).
1053 However, at the time the Bison parser encounters @code{x}, it does not
1054 have enough information to resolve the reduce/reduce conflict (again,
1055 between @code{x} as an @code{expr} or a @code{declarator}). In this
1056 case, no precedence declaration is used. Again, the parser splits
1057 into two, one assuming that @code{x} is an @code{expr}, and the other
1058 assuming @code{x} is a @code{declarator}. The second of these parsers
1059 then vanishes when it sees @code{+}, and the parser prints
1060
1061 @example
1062 x T <cast> y +
1063 @end example
1064
1065 Suppose that instead of resolving the ambiguity, you wanted to see all
1066 the possibilities. For this purpose, you must merge the semantic
1067 actions of the two possible parsers, rather than choosing one over the
1068 other. To do so, you could change the declaration of @code{stmt} as
1069 follows:
1070
1071 @example
1072 stmt : expr ';' %merge <stmtMerge>
1073 | decl %merge <stmtMerge>
1074 ;
1075 @end example
1076
1077 @noindent
1078 and define the @code{stmtMerge} function as:
1079
1080 @example
1081 static YYSTYPE
1082 stmtMerge (YYSTYPE x0, YYSTYPE x1)
1083 @{
1084 printf ("<OR> ");
1085 return "";
1086 @}
1087 @end example
1088
1089 @noindent
1090 with an accompanying forward declaration
1091 in the C declarations at the beginning of the file:
1092
1093 @example
1094 %@{
1095 #define YYSTYPE char const *
1096 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1097 %@}
1098 @end example
1099
1100 @noindent
1101 With these declarations, the resulting parser parses the first example
1102 as both an @code{expr} and a @code{decl}, and prints
1103
1104 @example
1105 "x" y z + T <init-declare> x T <cast> y z + = <OR>
1106 @end example
1107
1108 Bison requires that all of the
1109 productions that participate in any particular merge have identical
1110 @samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1111 and the parser will report an error during any parse that results in
1112 the offending merge.
1113
1114 @node GLR Semantic Actions
1115 @subsection GLR Semantic Actions
1116
1117 @cindex deferred semantic actions
1118 By definition, a deferred semantic action is not performed at the same time as
1119 the associated reduction.
1120 This raises caveats for several Bison features you might use in a semantic
1121 action in a @acronym{GLR} parser.
1122
1123 @vindex yychar
1124 @cindex @acronym{GLR} parsers and @code{yychar}
1125 @vindex yylval
1126 @cindex @acronym{GLR} parsers and @code{yylval}
1127 @vindex yylloc
1128 @cindex @acronym{GLR} parsers and @code{yylloc}
1129 In any semantic action, you can examine @code{yychar} to determine the type of
1130 the lookahead token present at the time of the associated reduction.
1131 After checking that @code{yychar} is not set to @code{YYEMPTY} or @code{YYEOF},
1132 you can then examine @code{yylval} and @code{yylloc} to determine the
1133 lookahead token's semantic value and location, if any.
1134 In a nondeferred semantic action, you can also modify any of these variables to
1135 influence syntax analysis.
1136 @xref{Lookahead, ,Lookahead Tokens}.
1137
1138 @findex yyclearin
1139 @cindex @acronym{GLR} parsers and @code{yyclearin}
1140 In a deferred semantic action, it's too late to influence syntax analysis.
1141 In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1142 shallow copies of the values they had at the time of the associated reduction.
1143 For this reason alone, modifying them is dangerous.
1144 Moreover, the result of modifying them is undefined and subject to change with
1145 future versions of Bison.
1146 For example, if a semantic action might be deferred, you should never write it
1147 to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1148 memory referenced by @code{yylval}.
1149
1150 @findex YYERROR
1151 @cindex @acronym{GLR} parsers and @code{YYERROR}
1152 Another Bison feature requiring special consideration is @code{YYERROR}
1153 (@pxref{Action Features}), which you can invoke in a semantic action to
1154 initiate error recovery.
1155 During deterministic @acronym{GLR} operation, the effect of @code{YYERROR} is
1156 the same as its effect in an @acronym{LALR}(1) parser.
1157 In a deferred semantic action, its effect is undefined.
1158 @c The effect is probably a syntax error at the split point.
1159
1160 Also, see @ref{Location Default Action, ,Default Action for Locations}, which
1161 describes a special usage of @code{YYLLOC_DEFAULT} in @acronym{GLR} parsers.
1162
1163 @node Compiler Requirements
1164 @subsection Considerations when Compiling @acronym{GLR} Parsers
1165 @cindex @code{inline}
1166 @cindex @acronym{GLR} parsers and @code{inline}
1167
1168 The @acronym{GLR} parsers require a compiler for @acronym{ISO} C89 or
1169 later. In addition, they use the @code{inline} keyword, which is not
1170 C89, but is C99 and is a common extension in pre-C99 compilers. It is
1171 up to the user of these parsers to handle
1172 portability issues. For instance, if using Autoconf and the Autoconf
1173 macro @code{AC_C_INLINE}, a mere
1174
1175 @example
1176 %@{
1177 #include <config.h>
1178 %@}
1179 @end example
1180
1181 @noindent
1182 will suffice. Otherwise, we suggest
1183
1184 @example
1185 %@{
1186 #if __STDC_VERSION__ < 199901 && ! defined __GNUC__ && ! defined inline
1187 #define inline
1188 #endif
1189 %@}
1190 @end example
1191
1192 @node Locations Overview
1193 @section Locations
1194 @cindex location
1195 @cindex textual location
1196 @cindex location, textual
1197
1198 Many applications, like interpreters or compilers, have to produce verbose
1199 and useful error messages. To achieve this, one must be able to keep track of
1200 the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
1201 Bison provides a mechanism for handling these locations.
1202
1203 Each token has a semantic value. In a similar fashion, each token has an
1204 associated location, but the type of locations is the same for all tokens and
1205 groupings. Moreover, the output parser is equipped with a default data
1206 structure for storing locations (@pxref{Locations}, for more details).
1207
1208 Like semantic values, locations can be reached in actions using a dedicated
1209 set of constructs. In the example above, the location of the whole grouping
1210 is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1211 @code{@@3}.
1212
1213 When a rule is matched, a default action is used to compute the semantic value
1214 of its left hand side (@pxref{Actions}). In the same way, another default
1215 action is used for locations. However, the action for locations is general
1216 enough for most cases, meaning there is usually no need to describe for each
1217 rule how @code{@@$} should be formed. When building a new location for a given
1218 grouping, the default behavior of the output parser is to take the beginning
1219 of the first symbol, and the end of the last symbol.
1220
1221 @node Bison Parser
1222 @section Bison Output: the Parser File
1223 @cindex Bison parser
1224 @cindex Bison utility
1225 @cindex lexical analyzer, purpose
1226 @cindex parser
1227
1228 When you run Bison, you give it a Bison grammar file as input. The output
1229 is a C source file that parses the language described by the grammar.
1230 This file is called a @dfn{Bison parser}. Keep in mind that the Bison
1231 utility and the Bison parser are two distinct programs: the Bison utility
1232 is a program whose output is the Bison parser that becomes part of your
1233 program.
1234
1235 The job of the Bison parser is to group tokens into groupings according to
1236 the grammar rules---for example, to build identifiers and operators into
1237 expressions. As it does this, it runs the actions for the grammar rules it
1238 uses.
1239
1240 The tokens come from a function called the @dfn{lexical analyzer} that
1241 you must supply in some fashion (such as by writing it in C). The Bison
1242 parser calls the lexical analyzer each time it wants a new token. It
1243 doesn't know what is ``inside'' the tokens (though their semantic values
1244 may reflect this). Typically the lexical analyzer makes the tokens by
1245 parsing characters of text, but Bison does not depend on this.
1246 @xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
1247
1248 The Bison parser file is C code which defines a function named
1249 @code{yyparse} which implements that grammar. This function does not make
1250 a complete C program: you must supply some additional functions. One is
1251 the lexical analyzer. Another is an error-reporting function which the
1252 parser calls to report an error. In addition, a complete C program must
1253 start with a function called @code{main}; you have to provide this, and
1254 arrange for it to call @code{yyparse} or the parser will never run.
1255 @xref{Interface, ,Parser C-Language Interface}.
1256
1257 Aside from the token type names and the symbols in the actions you
1258 write, all symbols defined in the Bison parser file itself
1259 begin with @samp{yy} or @samp{YY}. This includes interface functions
1260 such as the lexical analyzer function @code{yylex}, the error reporting
1261 function @code{yyerror} and the parser function @code{yyparse} itself.
1262 This also includes numerous identifiers used for internal purposes.
1263 Therefore, you should avoid using C identifiers starting with @samp{yy}
1264 or @samp{YY} in the Bison grammar file except for the ones defined in
1265 this manual. Also, you should avoid using the C identifiers
1266 @samp{malloc} and @samp{free} for anything other than their usual
1267 meanings.
1268
1269 In some cases the Bison parser file includes system headers, and in
1270 those cases your code should respect the identifiers reserved by those
1271 headers. On some non-@acronym{GNU} hosts, @code{<alloca.h>}, @code{<malloc.h>},
1272 @code{<stddef.h>}, and @code{<stdlib.h>} are included as needed to
1273 declare memory allocators and related types. @code{<libintl.h>} is
1274 included if message translation is in use
1275 (@pxref{Internationalization}). Other system headers may
1276 be included if you define @code{YYDEBUG} to a nonzero value
1277 (@pxref{Tracing, ,Tracing Your Parser}).
1278
1279 @node Stages
1280 @section Stages in Using Bison
1281 @cindex stages in using Bison
1282 @cindex using Bison
1283
1284 The actual language-design process using Bison, from grammar specification
1285 to a working compiler or interpreter, has these parts:
1286
1287 @enumerate
1288 @item
1289 Formally specify the grammar in a form recognized by Bison
1290 (@pxref{Grammar File, ,Bison Grammar Files}). For each grammatical rule
1291 in the language, describe the action that is to be taken when an
1292 instance of that rule is recognized. The action is described by a
1293 sequence of C statements.
1294
1295 @item
1296 Write a lexical analyzer to process input and pass tokens to the parser.
1297 The lexical analyzer may be written by hand in C (@pxref{Lexical, ,The
1298 Lexical Analyzer Function @code{yylex}}). It could also be produced
1299 using Lex, but the use of Lex is not discussed in this manual.
1300
1301 @item
1302 Write a controlling function that calls the Bison-produced parser.
1303
1304 @item
1305 Write error-reporting routines.
1306 @end enumerate
1307
1308 To turn this source code as written into a runnable program, you
1309 must follow these steps:
1310
1311 @enumerate
1312 @item
1313 Run Bison on the grammar to produce the parser.
1314
1315 @item
1316 Compile the code output by Bison, as well as any other source files.
1317
1318 @item
1319 Link the object files to produce the finished product.
1320 @end enumerate
1321
1322 @node Grammar Layout
1323 @section The Overall Layout of a Bison Grammar
1324 @cindex grammar file
1325 @cindex file format
1326 @cindex format of grammar file
1327 @cindex layout of Bison grammar
1328
1329 The input file for the Bison utility is a @dfn{Bison grammar file}. The
1330 general form of a Bison grammar file is as follows:
1331
1332 @example
1333 %@{
1334 @var{Prologue}
1335 %@}
1336
1337 @var{Bison declarations}
1338
1339 %%
1340 @var{Grammar rules}
1341 %%
1342 @var{Epilogue}
1343 @end example
1344
1345 @noindent
1346 The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1347 in every Bison grammar file to separate the sections.
1348
1349 The prologue may define types and variables used in the actions. You can
1350 also use preprocessor commands to define macros used there, and use
1351 @code{#include} to include header files that do any of these things.
1352 You need to declare the lexical analyzer @code{yylex} and the error
1353 printer @code{yyerror} here, along with any other global identifiers
1354 used by the actions in the grammar rules.
1355
1356 The Bison declarations declare the names of the terminal and nonterminal
1357 symbols, and may also describe operator precedence and the data types of
1358 semantic values of various symbols.
1359
1360 The grammar rules define how to construct each nonterminal symbol from its
1361 parts.
1362
1363 The epilogue can contain any code you want to use. Often the
1364 definitions of functions declared in the prologue go here. In a
1365 simple program, all the rest of the program can go here.
1366
1367 @node Examples
1368 @chapter Examples
1369 @cindex simple examples
1370 @cindex examples, simple
1371
1372 Now we show and explain three sample programs written using Bison: a
1373 reverse polish notation calculator, an algebraic (infix) notation
1374 calculator, and a multi-function calculator. All three have been tested
1375 under BSD Unix 4.3; each produces a usable, though limited, interactive
1376 desk-top calculator.
1377
1378 These examples are simple, but Bison grammars for real programming
1379 languages are written the same way. You can copy these examples into a
1380 source file to try them.
1381
1382 @menu
1383 * RPN Calc:: Reverse polish notation calculator;
1384 a first example with no operator precedence.
1385 * Infix Calc:: Infix (algebraic) notation calculator.
1386 Operator precedence is introduced.
1387 * Simple Error Recovery:: Continuing after syntax errors.
1388 * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
1389 * Multi-function Calc:: Calculator with memory and trig functions.
1390 It uses multiple data-types for semantic values.
1391 * Exercises:: Ideas for improving the multi-function calculator.
1392 @end menu
1393
1394 @node RPN Calc
1395 @section Reverse Polish Notation Calculator
1396 @cindex reverse polish notation
1397 @cindex polish notation calculator
1398 @cindex @code{rpcalc}
1399 @cindex calculator, simple
1400
1401 The first example is that of a simple double-precision @dfn{reverse polish
1402 notation} calculator (a calculator using postfix operators). This example
1403 provides a good starting point, since operator precedence is not an issue.
1404 The second example will illustrate how operator precedence is handled.
1405
1406 The source code for this calculator is named @file{rpcalc.y}. The
1407 @samp{.y} extension is a convention used for Bison input files.
1408
1409 @menu
1410 * Rpcalc Declarations:: Prologue (declarations) for rpcalc.
1411 * Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
1412 * Rpcalc Lexer:: The lexical analyzer.
1413 * Rpcalc Main:: The controlling function.
1414 * Rpcalc Error:: The error reporting function.
1415 * Rpcalc Generate:: Running Bison on the grammar file.
1416 * Rpcalc Compile:: Run the C compiler on the output code.
1417 @end menu
1418
1419 @node Rpcalc Declarations
1420 @subsection Declarations for @code{rpcalc}
1421
1422 Here are the C and Bison declarations for the reverse polish notation
1423 calculator. As in C, comments are placed between @samp{/*@dots{}*/}.
1424
1425 @example
1426 /* Reverse polish notation calculator. */
1427
1428 %@{
1429 #define YYSTYPE double
1430 #include <math.h>
1431 int yylex (void);
1432 void yyerror (char const *);
1433 %@}
1434
1435 %token NUM
1436
1437 %% /* Grammar rules and actions follow. */
1438 @end example
1439
1440 The declarations section (@pxref{Prologue, , The prologue}) contains two
1441 preprocessor directives and two forward declarations.
1442
1443 The @code{#define} directive defines the macro @code{YYSTYPE}, thus
1444 specifying the C data type for semantic values of both tokens and
1445 groupings (@pxref{Value Type, ,Data Types of Semantic Values}). The
1446 Bison parser will use whatever type @code{YYSTYPE} is defined as; if you
1447 don't define it, @code{int} is the default. Because we specify
1448 @code{double}, each token and each expression has an associated value,
1449 which is a floating point number.
1450
1451 The @code{#include} directive is used to declare the exponentiation
1452 function @code{pow}.
1453
1454 The forward declarations for @code{yylex} and @code{yyerror} are
1455 needed because the C language requires that functions be declared
1456 before they are used. These functions will be defined in the
1457 epilogue, but the parser calls them so they must be declared in the
1458 prologue.
1459
1460 The second section, Bison declarations, provides information to Bison
1461 about the token types (@pxref{Bison Declarations, ,The Bison
1462 Declarations Section}). Each terminal symbol that is not a
1463 single-character literal must be declared here. (Single-character
1464 literals normally don't need to be declared.) In this example, all the
1465 arithmetic operators are designated by single-character literals, so the
1466 only terminal symbol that needs to be declared is @code{NUM}, the token
1467 type for numeric constants.
1468
1469 @node Rpcalc Rules
1470 @subsection Grammar Rules for @code{rpcalc}
1471
1472 Here are the grammar rules for the reverse polish notation calculator.
1473
1474 @example
1475 input: /* empty */
1476 | input line
1477 ;
1478
1479 line: '\n'
1480 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1481 ;
1482
1483 exp: NUM @{ $$ = $1; @}
1484 | exp exp '+' @{ $$ = $1 + $2; @}
1485 | exp exp '-' @{ $$ = $1 - $2; @}
1486 | exp exp '*' @{ $$ = $1 * $2; @}
1487 | exp exp '/' @{ $$ = $1 / $2; @}
1488 /* Exponentiation */
1489 | exp exp '^' @{ $$ = pow ($1, $2); @}
1490 /* Unary minus */
1491 | exp 'n' @{ $$ = -$1; @}
1492 ;
1493 %%
1494 @end example
1495
1496 The groupings of the rpcalc ``language'' defined here are the expression
1497 (given the name @code{exp}), the line of input (@code{line}), and the
1498 complete input transcript (@code{input}). Each of these nonterminal
1499 symbols has several alternate rules, joined by the vertical bar @samp{|}
1500 which is read as ``or''. The following sections explain what these rules
1501 mean.
1502
1503 The semantics of the language is determined by the actions taken when a
1504 grouping is recognized. The actions are the C code that appears inside
1505 braces. @xref{Actions}.
1506
1507 You must specify these actions in C, but Bison provides the means for
1508 passing semantic values between the rules. In each action, the
1509 pseudo-variable @code{$$} stands for the semantic value for the grouping
1510 that the rule is going to construct. Assigning a value to @code{$$} is the
1511 main job of most actions. The semantic values of the components of the
1512 rule are referred to as @code{$1}, @code{$2}, and so on.
1513
1514 @menu
1515 * Rpcalc Input::
1516 * Rpcalc Line::
1517 * Rpcalc Expr::
1518 @end menu
1519
1520 @node Rpcalc Input
1521 @subsubsection Explanation of @code{input}
1522
1523 Consider the definition of @code{input}:
1524
1525 @example
1526 input: /* empty */
1527 | input line
1528 ;
1529 @end example
1530
1531 This definition reads as follows: ``A complete input is either an empty
1532 string, or a complete input followed by an input line''. Notice that
1533 ``complete input'' is defined in terms of itself. This definition is said
1534 to be @dfn{left recursive} since @code{input} appears always as the
1535 leftmost symbol in the sequence. @xref{Recursion, ,Recursive Rules}.
1536
1537 The first alternative is empty because there are no symbols between the
1538 colon and the first @samp{|}; this means that @code{input} can match an
1539 empty string of input (no tokens). We write the rules this way because it
1540 is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1541 It's conventional to put an empty alternative first and write the comment
1542 @samp{/* empty */} in it.
1543
1544 The second alternate rule (@code{input line}) handles all nontrivial input.
1545 It means, ``After reading any number of lines, read one more line if
1546 possible.'' The left recursion makes this rule into a loop. Since the
1547 first alternative matches empty input, the loop can be executed zero or
1548 more times.
1549
1550 The parser function @code{yyparse} continues to process input until a
1551 grammatical error is seen or the lexical analyzer says there are no more
1552 input tokens; we will arrange for the latter to happen at end-of-input.
1553
1554 @node Rpcalc Line
1555 @subsubsection Explanation of @code{line}
1556
1557 Now consider the definition of @code{line}:
1558
1559 @example
1560 line: '\n'
1561 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1562 ;
1563 @end example
1564
1565 The first alternative is a token which is a newline character; this means
1566 that rpcalc accepts a blank line (and ignores it, since there is no
1567 action). The second alternative is an expression followed by a newline.
1568 This is the alternative that makes rpcalc useful. The semantic value of
1569 the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1570 question is the first symbol in the alternative. The action prints this
1571 value, which is the result of the computation the user asked for.
1572
1573 This action is unusual because it does not assign a value to @code{$$}. As
1574 a consequence, the semantic value associated with the @code{line} is
1575 uninitialized (its value will be unpredictable). This would be a bug if
1576 that value were ever used, but we don't use it: once rpcalc has printed the
1577 value of the user's input line, that value is no longer needed.
1578
1579 @node Rpcalc Expr
1580 @subsubsection Explanation of @code{expr}
1581
1582 The @code{exp} grouping has several rules, one for each kind of expression.
1583 The first rule handles the simplest expressions: those that are just numbers.
1584 The second handles an addition-expression, which looks like two expressions
1585 followed by a plus-sign. The third handles subtraction, and so on.
1586
1587 @example
1588 exp: NUM
1589 | exp exp '+' @{ $$ = $1 + $2; @}
1590 | exp exp '-' @{ $$ = $1 - $2; @}
1591 @dots{}
1592 ;
1593 @end example
1594
1595 We have used @samp{|} to join all the rules for @code{exp}, but we could
1596 equally well have written them separately:
1597
1598 @example
1599 exp: NUM ;
1600 exp: exp exp '+' @{ $$ = $1 + $2; @} ;
1601 exp: exp exp '-' @{ $$ = $1 - $2; @} ;
1602 @dots{}
1603 @end example
1604
1605 Most of the rules have actions that compute the value of the expression in
1606 terms of the value of its parts. For example, in the rule for addition,
1607 @code{$1} refers to the first component @code{exp} and @code{$2} refers to
1608 the second one. The third component, @code{'+'}, has no meaningful
1609 associated semantic value, but if it had one you could refer to it as
1610 @code{$3}. When @code{yyparse} recognizes a sum expression using this
1611 rule, the sum of the two subexpressions' values is produced as the value of
1612 the entire expression. @xref{Actions}.
1613
1614 You don't have to give an action for every rule. When a rule has no
1615 action, Bison by default copies the value of @code{$1} into @code{$$}.
1616 This is what happens in the first rule (the one that uses @code{NUM}).
1617
1618 The formatting shown here is the recommended convention, but Bison does
1619 not require it. You can add or change white space as much as you wish.
1620 For example, this:
1621
1622 @example
1623 exp : NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
1624 @end example
1625
1626 @noindent
1627 means the same thing as this:
1628
1629 @example
1630 exp: NUM
1631 | exp exp '+' @{ $$ = $1 + $2; @}
1632 | @dots{}
1633 ;
1634 @end example
1635
1636 @noindent
1637 The latter, however, is much more readable.
1638
1639 @node Rpcalc Lexer
1640 @subsection The @code{rpcalc} Lexical Analyzer
1641 @cindex writing a lexical analyzer
1642 @cindex lexical analyzer, writing
1643
1644 The lexical analyzer's job is low-level parsing: converting characters
1645 or sequences of characters into tokens. The Bison parser gets its
1646 tokens by calling the lexical analyzer. @xref{Lexical, ,The Lexical
1647 Analyzer Function @code{yylex}}.
1648
1649 Only a simple lexical analyzer is needed for the @acronym{RPN}
1650 calculator. This
1651 lexical analyzer skips blanks and tabs, then reads in numbers as
1652 @code{double} and returns them as @code{NUM} tokens. Any other character
1653 that isn't part of a number is a separate token. Note that the token-code
1654 for such a single-character token is the character itself.
1655
1656 The return value of the lexical analyzer function is a numeric code which
1657 represents a token type. The same text used in Bison rules to stand for
1658 this token type is also a C expression for the numeric code for the type.
1659 This works in two ways. If the token type is a character literal, then its
1660 numeric code is that of the character; you can use the same
1661 character literal in the lexical analyzer to express the number. If the
1662 token type is an identifier, that identifier is defined by Bison as a C
1663 macro whose definition is the appropriate number. In this example,
1664 therefore, @code{NUM} becomes a macro for @code{yylex} to use.
1665
1666 The semantic value of the token (if it has one) is stored into the
1667 global variable @code{yylval}, which is where the Bison parser will look
1668 for it. (The C data type of @code{yylval} is @code{YYSTYPE}, which was
1669 defined at the beginning of the grammar; @pxref{Rpcalc Declarations,
1670 ,Declarations for @code{rpcalc}}.)
1671
1672 A token type code of zero is returned if the end-of-input is encountered.
1673 (Bison recognizes any nonpositive value as indicating end-of-input.)
1674
1675 Here is the code for the lexical analyzer:
1676
1677 @example
1678 @group
1679 /* The lexical analyzer returns a double floating point
1680 number on the stack and the token NUM, or the numeric code
1681 of the character read if not a number. It skips all blanks
1682 and tabs, and returns 0 for end-of-input. */
1683
1684 #include <ctype.h>
1685 @end group
1686
1687 @group
1688 int
1689 yylex (void)
1690 @{
1691 int c;
1692
1693 /* Skip white space. */
1694 while ((c = getchar ()) == ' ' || c == '\t')
1695 ;
1696 @end group
1697 @group
1698 /* Process numbers. */
1699 if (c == '.' || isdigit (c))
1700 @{
1701 ungetc (c, stdin);
1702 scanf ("%lf", &yylval);
1703 return NUM;
1704 @}
1705 @end group
1706 @group
1707 /* Return end-of-input. */
1708 if (c == EOF)
1709 return 0;
1710 /* Return a single char. */
1711 return c;
1712 @}
1713 @end group
1714 @end example
1715
1716 @node Rpcalc Main
1717 @subsection The Controlling Function
1718 @cindex controlling function
1719 @cindex main function in simple example
1720
1721 In keeping with the spirit of this example, the controlling function is
1722 kept to the bare minimum. The only requirement is that it call
1723 @code{yyparse} to start the process of parsing.
1724
1725 @example
1726 @group
1727 int
1728 main (void)
1729 @{
1730 return yyparse ();
1731 @}
1732 @end group
1733 @end example
1734
1735 @node Rpcalc Error
1736 @subsection The Error Reporting Routine
1737 @cindex error reporting routine
1738
1739 When @code{yyparse} detects a syntax error, it calls the error reporting
1740 function @code{yyerror} to print an error message (usually but not
1741 always @code{"syntax error"}). It is up to the programmer to supply
1742 @code{yyerror} (@pxref{Interface, ,Parser C-Language Interface}), so
1743 here is the definition we will use:
1744
1745 @example
1746 @group
1747 #include <stdio.h>
1748
1749 /* Called by yyparse on error. */
1750 void
1751 yyerror (char const *s)
1752 @{
1753 fprintf (stderr, "%s\n", s);
1754 @}
1755 @end group
1756 @end example
1757
1758 After @code{yyerror} returns, the Bison parser may recover from the error
1759 and continue parsing if the grammar contains a suitable error rule
1760 (@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
1761 have not written any error rules in this example, so any invalid input will
1762 cause the calculator program to exit. This is not clean behavior for a
1763 real calculator, but it is adequate for the first example.
1764
1765 @node Rpcalc Generate
1766 @subsection Running Bison to Make the Parser
1767 @cindex running Bison (introduction)
1768
1769 Before running Bison to produce a parser, we need to decide how to
1770 arrange all the source code in one or more source files. For such a
1771 simple example, the easiest thing is to put everything in one file. The
1772 definitions of @code{yylex}, @code{yyerror} and @code{main} go at the
1773 end, in the epilogue of the file
1774 (@pxref{Grammar Layout, ,The Overall Layout of a Bison Grammar}).
1775
1776 For a large project, you would probably have several source files, and use
1777 @code{make} to arrange to recompile them.
1778
1779 With all the source in a single file, you use the following command to
1780 convert it into a parser file:
1781
1782 @example
1783 bison @var{file}.y
1784 @end example
1785
1786 @noindent
1787 In this example the file was called @file{rpcalc.y} (for ``Reverse Polish
1788 @sc{calc}ulator''). Bison produces a file named @file{@var{file}.tab.c},
1789 removing the @samp{.y} from the original file name. The file output by
1790 Bison contains the source code for @code{yyparse}. The additional
1791 functions in the input file (@code{yylex}, @code{yyerror} and @code{main})
1792 are copied verbatim to the output.
1793
1794 @node Rpcalc Compile
1795 @subsection Compiling the Parser File
1796 @cindex compiling the parser
1797
1798 Here is how to compile and run the parser file:
1799
1800 @example
1801 @group
1802 # @r{List files in current directory.}
1803 $ @kbd{ls}
1804 rpcalc.tab.c rpcalc.y
1805 @end group
1806
1807 @group
1808 # @r{Compile the Bison parser.}
1809 # @r{@samp{-lm} tells compiler to search math library for @code{pow}.}
1810 $ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
1811 @end group
1812
1813 @group
1814 # @r{List files again.}
1815 $ @kbd{ls}
1816 rpcalc rpcalc.tab.c rpcalc.y
1817 @end group
1818 @end example
1819
1820 The file @file{rpcalc} now contains the executable code. Here is an
1821 example session using @code{rpcalc}.
1822
1823 @example
1824 $ @kbd{rpcalc}
1825 @kbd{4 9 +}
1826 13
1827 @kbd{3 7 + 3 4 5 *+-}
1828 -13
1829 @kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
1830 13
1831 @kbd{5 6 / 4 n +}
1832 -3.166666667
1833 @kbd{3 4 ^} @r{Exponentiation}
1834 81
1835 @kbd{^D} @r{End-of-file indicator}
1836 $
1837 @end example
1838
1839 @node Infix Calc
1840 @section Infix Notation Calculator: @code{calc}
1841 @cindex infix notation calculator
1842 @cindex @code{calc}
1843 @cindex calculator, infix notation
1844
1845 We now modify rpcalc to handle infix operators instead of postfix. Infix
1846 notation involves the concept of operator precedence and the need for
1847 parentheses nested to arbitrary depth. Here is the Bison code for
1848 @file{calc.y}, an infix desk-top calculator.
1849
1850 @example
1851 /* Infix notation calculator. */
1852
1853 %@{
1854 #define YYSTYPE double
1855 #include <math.h>
1856 #include <stdio.h>
1857 int yylex (void);
1858 void yyerror (char const *);
1859 %@}
1860
1861 /* Bison declarations. */
1862 %token NUM
1863 %left '-' '+'
1864 %left '*' '/'
1865 %left NEG /* negation--unary minus */
1866 %right '^' /* exponentiation */
1867
1868 %% /* The grammar follows. */
1869 input: /* empty */
1870 | input line
1871 ;
1872
1873 line: '\n'
1874 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1875 ;
1876
1877 exp: NUM @{ $$ = $1; @}
1878 | exp '+' exp @{ $$ = $1 + $3; @}
1879 | exp '-' exp @{ $$ = $1 - $3; @}
1880 | exp '*' exp @{ $$ = $1 * $3; @}
1881 | exp '/' exp @{ $$ = $1 / $3; @}
1882 | '-' exp %prec NEG @{ $$ = -$2; @}
1883 | exp '^' exp @{ $$ = pow ($1, $3); @}
1884 | '(' exp ')' @{ $$ = $2; @}
1885 ;
1886 %%
1887 @end example
1888
1889 @noindent
1890 The functions @code{yylex}, @code{yyerror} and @code{main} can be the
1891 same as before.
1892
1893 There are two important new features shown in this code.
1894
1895 In the second section (Bison declarations), @code{%left} declares token
1896 types and says they are left-associative operators. The declarations
1897 @code{%left} and @code{%right} (right associativity) take the place of
1898 @code{%token} which is used to declare a token type name without
1899 associativity. (These tokens are single-character literals, which
1900 ordinarily don't need to be declared. We declare them here to specify
1901 the associativity.)
1902
1903 Operator precedence is determined by the line ordering of the
1904 declarations; the higher the line number of the declaration (lower on
1905 the page or screen), the higher the precedence. Hence, exponentiation
1906 has the highest precedence, unary minus (@code{NEG}) is next, followed
1907 by @samp{*} and @samp{/}, and so on. @xref{Precedence, ,Operator
1908 Precedence}.
1909
1910 The other important new feature is the @code{%prec} in the grammar
1911 section for the unary minus operator. The @code{%prec} simply instructs
1912 Bison that the rule @samp{| '-' exp} has the same precedence as
1913 @code{NEG}---in this case the next-to-highest. @xref{Contextual
1914 Precedence, ,Context-Dependent Precedence}.
1915
1916 Here is a sample run of @file{calc.y}:
1917
1918 @need 500
1919 @example
1920 $ @kbd{calc}
1921 @kbd{4 + 4.5 - (34/(8*3+-3))}
1922 6.880952381
1923 @kbd{-56 + 2}
1924 -54
1925 @kbd{3 ^ 2}
1926 9
1927 @end example
1928
1929 @node Simple Error Recovery
1930 @section Simple Error Recovery
1931 @cindex error recovery, simple
1932
1933 Up to this point, this manual has not addressed the issue of @dfn{error
1934 recovery}---how to continue parsing after the parser detects a syntax
1935 error. All we have handled is error reporting with @code{yyerror}.
1936 Recall that by default @code{yyparse} returns after calling
1937 @code{yyerror}. This means that an erroneous input line causes the
1938 calculator program to exit. Now we show how to rectify this deficiency.
1939
1940 The Bison language itself includes the reserved word @code{error}, which
1941 may be included in the grammar rules. In the example below it has
1942 been added to one of the alternatives for @code{line}:
1943
1944 @example
1945 @group
1946 line: '\n'
1947 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
1948 | error '\n' @{ yyerrok; @}
1949 ;
1950 @end group
1951 @end example
1952
1953 This addition to the grammar allows for simple error recovery in the
1954 event of a syntax error. If an expression that cannot be evaluated is
1955 read, the error will be recognized by the third rule for @code{line},
1956 and parsing will continue. (The @code{yyerror} function is still called
1957 upon to print its message as well.) The action executes the statement
1958 @code{yyerrok}, a macro defined automatically by Bison; its meaning is
1959 that error recovery is complete (@pxref{Error Recovery}). Note the
1960 difference between @code{yyerrok} and @code{yyerror}; neither one is a
1961 misprint.
1962
1963 This form of error recovery deals with syntax errors. There are other
1964 kinds of errors; for example, division by zero, which raises an exception
1965 signal that is normally fatal. A real calculator program must handle this
1966 signal and use @code{longjmp} to return to @code{main} and resume parsing
1967 input lines; it would also have to discard the rest of the current line of
1968 input. We won't discuss this issue further because it is not specific to
1969 Bison programs.
1970
1971 @node Location Tracking Calc
1972 @section Location Tracking Calculator: @code{ltcalc}
1973 @cindex location tracking calculator
1974 @cindex @code{ltcalc}
1975 @cindex calculator, location tracking
1976
1977 This example extends the infix notation calculator with location
1978 tracking. This feature will be used to improve the error messages. For
1979 the sake of clarity, this example is a simple integer calculator, since
1980 most of the work needed to use locations will be done in the lexical
1981 analyzer.
1982
1983 @menu
1984 * Ltcalc Declarations:: Bison and C declarations for ltcalc.
1985 * Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
1986 * Ltcalc Lexer:: The lexical analyzer.
1987 @end menu
1988
1989 @node Ltcalc Declarations
1990 @subsection Declarations for @code{ltcalc}
1991
1992 The C and Bison declarations for the location tracking calculator are
1993 the same as the declarations for the infix notation calculator.
1994
1995 @example
1996 /* Location tracking calculator. */
1997
1998 %@{
1999 #define YYSTYPE int
2000 #include <math.h>
2001 int yylex (void);
2002 void yyerror (char const *);
2003 %@}
2004
2005 /* Bison declarations. */
2006 %token NUM
2007
2008 %left '-' '+'
2009 %left '*' '/'
2010 %left NEG
2011 %right '^'
2012
2013 %% /* The grammar follows. */
2014 @end example
2015
2016 @noindent
2017 Note there are no declarations specific to locations. Defining a data
2018 type for storing locations is not needed: we will use the type provided
2019 by default (@pxref{Location Type, ,Data Types of Locations}), which is a
2020 four member structure with the following integer fields:
2021 @code{first_line}, @code{first_column}, @code{last_line} and
2022 @code{last_column}. By conventions, and in accordance with the GNU
2023 Coding Standards and common practice, the line and column count both
2024 start at 1.
2025
2026 @node Ltcalc Rules
2027 @subsection Grammar Rules for @code{ltcalc}
2028
2029 Whether handling locations or not has no effect on the syntax of your
2030 language. Therefore, grammar rules for this example will be very close
2031 to those of the previous example: we will only modify them to benefit
2032 from the new information.
2033
2034 Here, we will use locations to report divisions by zero, and locate the
2035 wrong expressions or subexpressions.
2036
2037 @example
2038 @group
2039 input : /* empty */
2040 | input line
2041 ;
2042 @end group
2043
2044 @group
2045 line : '\n'
2046 | exp '\n' @{ printf ("%d\n", $1); @}
2047 ;
2048 @end group
2049
2050 @group
2051 exp : NUM @{ $$ = $1; @}
2052 | exp '+' exp @{ $$ = $1 + $3; @}
2053 | exp '-' exp @{ $$ = $1 - $3; @}
2054 | exp '*' exp @{ $$ = $1 * $3; @}
2055 @end group
2056 @group
2057 | exp '/' exp
2058 @{
2059 if ($3)
2060 $$ = $1 / $3;
2061 else
2062 @{
2063 $$ = 1;
2064 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2065 @@3.first_line, @@3.first_column,
2066 @@3.last_line, @@3.last_column);
2067 @}
2068 @}
2069 @end group
2070 @group
2071 | '-' exp %prec NEG @{ $$ = -$2; @}
2072 | exp '^' exp @{ $$ = pow ($1, $3); @}
2073 | '(' exp ')' @{ $$ = $2; @}
2074 @end group
2075 @end example
2076
2077 This code shows how to reach locations inside of semantic actions, by
2078 using the pseudo-variables @code{@@@var{n}} for rule components, and the
2079 pseudo-variable @code{@@$} for groupings.
2080
2081 We don't need to assign a value to @code{@@$}: the output parser does it
2082 automatically. By default, before executing the C code of each action,
2083 @code{@@$} is set to range from the beginning of @code{@@1} to the end
2084 of @code{@@@var{n}}, for a rule with @var{n} components. This behavior
2085 can be redefined (@pxref{Location Default Action, , Default Action for
2086 Locations}), and for very specific rules, @code{@@$} can be computed by
2087 hand.
2088
2089 @node Ltcalc Lexer
2090 @subsection The @code{ltcalc} Lexical Analyzer.
2091
2092 Until now, we relied on Bison's defaults to enable location
2093 tracking. The next step is to rewrite the lexical analyzer, and make it
2094 able to feed the parser with the token locations, as it already does for
2095 semantic values.
2096
2097 To this end, we must take into account every single character of the
2098 input text, to avoid the computed locations of being fuzzy or wrong:
2099
2100 @example
2101 @group
2102 int
2103 yylex (void)
2104 @{
2105 int c;
2106 @end group
2107
2108 @group
2109 /* Skip white space. */
2110 while ((c = getchar ()) == ' ' || c == '\t')
2111 ++yylloc.last_column;
2112 @end group
2113
2114 @group
2115 /* Step. */
2116 yylloc.first_line = yylloc.last_line;
2117 yylloc.first_column = yylloc.last_column;
2118 @end group
2119
2120 @group
2121 /* Process numbers. */
2122 if (isdigit (c))
2123 @{
2124 yylval = c - '0';
2125 ++yylloc.last_column;
2126 while (isdigit (c = getchar ()))
2127 @{
2128 ++yylloc.last_column;
2129 yylval = yylval * 10 + c - '0';
2130 @}
2131 ungetc (c, stdin);
2132 return NUM;
2133 @}
2134 @end group
2135
2136 /* Return end-of-input. */
2137 if (c == EOF)
2138 return 0;
2139
2140 /* Return a single char, and update location. */
2141 if (c == '\n')
2142 @{
2143 ++yylloc.last_line;
2144 yylloc.last_column = 0;
2145 @}
2146 else
2147 ++yylloc.last_column;
2148 return c;
2149 @}
2150 @end example
2151
2152 Basically, the lexical analyzer performs the same processing as before:
2153 it skips blanks and tabs, and reads numbers or single-character tokens.
2154 In addition, it updates @code{yylloc}, the global variable (of type
2155 @code{YYLTYPE}) containing the token's location.
2156
2157 Now, each time this function returns a token, the parser has its number
2158 as well as its semantic value, and its location in the text. The last
2159 needed change is to initialize @code{yylloc}, for example in the
2160 controlling function:
2161
2162 @example
2163 @group
2164 int
2165 main (void)
2166 @{
2167 yylloc.first_line = yylloc.last_line = 1;
2168 yylloc.first_column = yylloc.last_column = 0;
2169 return yyparse ();
2170 @}
2171 @end group
2172 @end example
2173
2174 Remember that computing locations is not a matter of syntax. Every
2175 character must be associated to a location update, whether it is in
2176 valid input, in comments, in literal strings, and so on.
2177
2178 @node Multi-function Calc
2179 @section Multi-Function Calculator: @code{mfcalc}
2180 @cindex multi-function calculator
2181 @cindex @code{mfcalc}
2182 @cindex calculator, multi-function
2183
2184 Now that the basics of Bison have been discussed, it is time to move on to
2185 a more advanced problem. The above calculators provided only five
2186 functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2187 be nice to have a calculator that provides other mathematical functions such
2188 as @code{sin}, @code{cos}, etc.
2189
2190 It is easy to add new operators to the infix calculator as long as they are
2191 only single-character literals. The lexical analyzer @code{yylex} passes
2192 back all nonnumeric characters as tokens, so new grammar rules suffice for
2193 adding a new operator. But we want something more flexible: built-in
2194 functions whose syntax has this form:
2195
2196 @example
2197 @var{function_name} (@var{argument})
2198 @end example
2199
2200 @noindent
2201 At the same time, we will add memory to the calculator, by allowing you
2202 to create named variables, store values in them, and use them later.
2203 Here is a sample session with the multi-function calculator:
2204
2205 @example
2206 $ @kbd{mfcalc}
2207 @kbd{pi = 3.141592653589}
2208 3.1415926536
2209 @kbd{sin(pi)}
2210 0.0000000000
2211 @kbd{alpha = beta1 = 2.3}
2212 2.3000000000
2213 @kbd{alpha}
2214 2.3000000000
2215 @kbd{ln(alpha)}
2216 0.8329091229
2217 @kbd{exp(ln(beta1))}
2218 2.3000000000
2219 $
2220 @end example
2221
2222 Note that multiple assignment and nested function calls are permitted.
2223
2224 @menu
2225 * Mfcalc Declarations:: Bison declarations for multi-function calculator.
2226 * Mfcalc Rules:: Grammar rules for the calculator.
2227 * Mfcalc Symbol Table:: Symbol table management subroutines.
2228 @end menu
2229
2230 @node Mfcalc Declarations
2231 @subsection Declarations for @code{mfcalc}
2232
2233 Here are the C and Bison declarations for the multi-function calculator.
2234
2235 @smallexample
2236 @group
2237 %@{
2238 #include <math.h> /* For math functions, cos(), sin(), etc. */
2239 #include "calc.h" /* Contains definition of `symrec'. */
2240 int yylex (void);
2241 void yyerror (char const *);
2242 %@}
2243 @end group
2244 @group
2245 %union @{
2246 double val; /* For returning numbers. */
2247 symrec *tptr; /* For returning symbol-table pointers. */
2248 @}
2249 @end group
2250 %token <val> NUM /* Simple double precision number. */
2251 %token <tptr> VAR FNCT /* Variable and Function. */
2252 %type <val> exp
2253
2254 @group
2255 %right '='
2256 %left '-' '+'
2257 %left '*' '/'
2258 %left NEG /* negation--unary minus */
2259 %right '^' /* exponentiation */
2260 @end group
2261 %% /* The grammar follows. */
2262 @end smallexample
2263
2264 The above grammar introduces only two new features of the Bison language.
2265 These features allow semantic values to have various data types
2266 (@pxref{Multiple Types, ,More Than One Value Type}).
2267
2268 The @code{%union} declaration specifies the entire list of possible types;
2269 this is instead of defining @code{YYSTYPE}. The allowable types are now
2270 double-floats (for @code{exp} and @code{NUM}) and pointers to entries in
2271 the symbol table. @xref{Union Decl, ,The Collection of Value Types}.
2272
2273 Since values can now have various types, it is necessary to associate a
2274 type with each grammar symbol whose semantic value is used. These symbols
2275 are @code{NUM}, @code{VAR}, @code{FNCT}, and @code{exp}. Their
2276 declarations are augmented with information about their data type (placed
2277 between angle brackets).
2278
2279 The Bison construct @code{%type} is used for declaring nonterminal
2280 symbols, just as @code{%token} is used for declaring token types. We
2281 have not used @code{%type} before because nonterminal symbols are
2282 normally declared implicitly by the rules that define them. But
2283 @code{exp} must be declared explicitly so we can specify its value type.
2284 @xref{Type Decl, ,Nonterminal Symbols}.
2285
2286 @node Mfcalc Rules
2287 @subsection Grammar Rules for @code{mfcalc}
2288
2289 Here are the grammar rules for the multi-function calculator.
2290 Most of them are copied directly from @code{calc}; three rules,
2291 those which mention @code{VAR} or @code{FNCT}, are new.
2292
2293 @smallexample
2294 @group
2295 input: /* empty */
2296 | input line
2297 ;
2298 @end group
2299
2300 @group
2301 line:
2302 '\n'
2303 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2304 | error '\n' @{ yyerrok; @}
2305 ;
2306 @end group
2307
2308 @group
2309 exp: NUM @{ $$ = $1; @}
2310 | VAR @{ $$ = $1->value.var; @}
2311 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2312 | FNCT '(' exp ')' @{ $$ = (*($1->value.fnctptr))($3); @}
2313 | exp '+' exp @{ $$ = $1 + $3; @}
2314 | exp '-' exp @{ $$ = $1 - $3; @}
2315 | exp '*' exp @{ $$ = $1 * $3; @}
2316 | exp '/' exp @{ $$ = $1 / $3; @}
2317 | '-' exp %prec NEG @{ $$ = -$2; @}
2318 | exp '^' exp @{ $$ = pow ($1, $3); @}
2319 | '(' exp ')' @{ $$ = $2; @}
2320 ;
2321 @end group
2322 /* End of grammar. */
2323 %%
2324 @end smallexample
2325
2326 @node Mfcalc Symbol Table
2327 @subsection The @code{mfcalc} Symbol Table
2328 @cindex symbol table example
2329
2330 The multi-function calculator requires a symbol table to keep track of the
2331 names and meanings of variables and functions. This doesn't affect the
2332 grammar rules (except for the actions) or the Bison declarations, but it
2333 requires some additional C functions for support.
2334
2335 The symbol table itself consists of a linked list of records. Its
2336 definition, which is kept in the header @file{calc.h}, is as follows. It
2337 provides for either functions or variables to be placed in the table.
2338
2339 @smallexample
2340 @group
2341 /* Function type. */
2342 typedef double (*func_t) (double);
2343 @end group
2344
2345 @group
2346 /* Data type for links in the chain of symbols. */
2347 struct symrec
2348 @{
2349 char *name; /* name of symbol */
2350 int type; /* type of symbol: either VAR or FNCT */
2351 union
2352 @{
2353 double var; /* value of a VAR */
2354 func_t fnctptr; /* value of a FNCT */
2355 @} value;
2356 struct symrec *next; /* link field */
2357 @};
2358 @end group
2359
2360 @group
2361 typedef struct symrec symrec;
2362
2363 /* The symbol table: a chain of `struct symrec'. */
2364 extern symrec *sym_table;
2365
2366 symrec *putsym (char const *, int);
2367 symrec *getsym (char const *);
2368 @end group
2369 @end smallexample
2370
2371 The new version of @code{main} includes a call to @code{init_table}, a
2372 function that initializes the symbol table. Here it is, and
2373 @code{init_table} as well:
2374
2375 @smallexample
2376 #include <stdio.h>
2377
2378 @group
2379 /* Called by yyparse on error. */
2380 void
2381 yyerror (char const *s)
2382 @{
2383 printf ("%s\n", s);
2384 @}
2385 @end group
2386
2387 @group
2388 struct init
2389 @{
2390 char const *fname;
2391 double (*fnct) (double);
2392 @};
2393 @end group
2394
2395 @group
2396 struct init const arith_fncts[] =
2397 @{
2398 "sin", sin,
2399 "cos", cos,
2400 "atan", atan,
2401 "ln", log,
2402 "exp", exp,
2403 "sqrt", sqrt,
2404 0, 0
2405 @};
2406 @end group
2407
2408 @group
2409 /* The symbol table: a chain of `struct symrec'. */
2410 symrec *sym_table;
2411 @end group
2412
2413 @group
2414 /* Put arithmetic functions in table. */
2415 void
2416 init_table (void)
2417 @{
2418 int i;
2419 symrec *ptr;
2420 for (i = 0; arith_fncts[i].fname != 0; i++)
2421 @{
2422 ptr = putsym (arith_fncts[i].fname, FNCT);
2423 ptr->value.fnctptr = arith_fncts[i].fnct;
2424 @}
2425 @}
2426 @end group
2427
2428 @group
2429 int
2430 main (void)
2431 @{
2432 init_table ();
2433 return yyparse ();
2434 @}
2435 @end group
2436 @end smallexample
2437
2438 By simply editing the initialization list and adding the necessary include
2439 files, you can add additional functions to the calculator.
2440
2441 Two important functions allow look-up and installation of symbols in the
2442 symbol table. The function @code{putsym} is passed a name and the type
2443 (@code{VAR} or @code{FNCT}) of the object to be installed. The object is
2444 linked to the front of the list, and a pointer to the object is returned.
2445 The function @code{getsym} is passed the name of the symbol to look up. If
2446 found, a pointer to that symbol is returned; otherwise zero is returned.
2447
2448 @smallexample
2449 symrec *
2450 putsym (char const *sym_name, int sym_type)
2451 @{
2452 symrec *ptr;
2453 ptr = (symrec *) malloc (sizeof (symrec));
2454 ptr->name = (char *) malloc (strlen (sym_name) + 1);
2455 strcpy (ptr->name,sym_name);
2456 ptr->type = sym_type;
2457 ptr->value.var = 0; /* Set value to 0 even if fctn. */
2458 ptr->next = (struct symrec *)sym_table;
2459 sym_table = ptr;
2460 return ptr;
2461 @}
2462
2463 symrec *
2464 getsym (char const *sym_name)
2465 @{
2466 symrec *ptr;
2467 for (ptr = sym_table; ptr != (symrec *) 0;
2468 ptr = (symrec *)ptr->next)
2469 if (strcmp (ptr->name,sym_name) == 0)
2470 return ptr;
2471 return 0;
2472 @}
2473 @end smallexample
2474
2475 The function @code{yylex} must now recognize variables, numeric values, and
2476 the single-character arithmetic operators. Strings of alphanumeric
2477 characters with a leading letter are recognized as either variables or
2478 functions depending on what the symbol table says about them.
2479
2480 The string is passed to @code{getsym} for look up in the symbol table. If
2481 the name appears in the table, a pointer to its location and its type
2482 (@code{VAR} or @code{FNCT}) is returned to @code{yyparse}. If it is not
2483 already in the table, then it is installed as a @code{VAR} using
2484 @code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
2485 returned to @code{yyparse}.
2486
2487 No change is needed in the handling of numeric values and arithmetic
2488 operators in @code{yylex}.
2489
2490 @smallexample
2491 @group
2492 #include <ctype.h>
2493 @end group
2494
2495 @group
2496 int
2497 yylex (void)
2498 @{
2499 int c;
2500
2501 /* Ignore white space, get first nonwhite character. */
2502 while ((c = getchar ()) == ' ' || c == '\t');
2503
2504 if (c == EOF)
2505 return 0;
2506 @end group
2507
2508 @group
2509 /* Char starts a number => parse the number. */
2510 if (c == '.' || isdigit (c))
2511 @{
2512 ungetc (c, stdin);
2513 scanf ("%lf", &yylval.val);
2514 return NUM;
2515 @}
2516 @end group
2517
2518 @group
2519 /* Char starts an identifier => read the name. */
2520 if (isalpha (c))
2521 @{
2522 symrec *s;
2523 static char *symbuf = 0;
2524 static int length = 0;
2525 int i;
2526 @end group
2527
2528 @group
2529 /* Initially make the buffer long enough
2530 for a 40-character symbol name. */
2531 if (length == 0)
2532 length = 40, symbuf = (char *)malloc (length + 1);
2533
2534 i = 0;
2535 do
2536 @end group
2537 @group
2538 @{
2539 /* If buffer is full, make it bigger. */
2540 if (i == length)
2541 @{
2542 length *= 2;
2543 symbuf = (char *) realloc (symbuf, length + 1);
2544 @}
2545 /* Add this character to the buffer. */
2546 symbuf[i++] = c;
2547 /* Get another character. */
2548 c = getchar ();
2549 @}
2550 @end group
2551 @group
2552 while (isalnum (c));
2553
2554 ungetc (c, stdin);
2555 symbuf[i] = '\0';
2556 @end group
2557
2558 @group
2559 s = getsym (symbuf);
2560 if (s == 0)
2561 s = putsym (symbuf, VAR);
2562 yylval.tptr = s;
2563 return s->type;
2564 @}
2565
2566 /* Any other character is a token by itself. */
2567 return c;
2568 @}
2569 @end group
2570 @end smallexample
2571
2572 This program is both powerful and flexible. You may easily add new
2573 functions, and it is a simple job to modify this code to install
2574 predefined variables such as @code{pi} or @code{e} as well.
2575
2576 @node Exercises
2577 @section Exercises
2578 @cindex exercises
2579
2580 @enumerate
2581 @item
2582 Add some new functions from @file{math.h} to the initialization list.
2583
2584 @item
2585 Add another array that contains constants and their values. Then
2586 modify @code{init_table} to add these constants to the symbol table.
2587 It will be easiest to give the constants type @code{VAR}.
2588
2589 @item
2590 Make the program report an error if the user refers to an
2591 uninitialized variable in any way except to store a value in it.
2592 @end enumerate
2593
2594 @node Grammar File
2595 @chapter Bison Grammar Files
2596
2597 Bison takes as input a context-free grammar specification and produces a
2598 C-language function that recognizes correct instances of the grammar.
2599
2600 The Bison grammar input file conventionally has a name ending in @samp{.y}.
2601 @xref{Invocation, ,Invoking Bison}.
2602
2603 @menu
2604 * Grammar Outline:: Overall layout of the grammar file.
2605 * Symbols:: Terminal and nonterminal symbols.
2606 * Rules:: How to write grammar rules.
2607 * Recursion:: Writing recursive rules.
2608 * Semantics:: Semantic values and actions.
2609 * Locations:: Locations and actions.
2610 * Declarations:: All kinds of Bison declarations are described here.
2611 * Multiple Parsers:: Putting more than one Bison parser in one program.
2612 @end menu
2613
2614 @node Grammar Outline
2615 @section Outline of a Bison Grammar
2616
2617 A Bison grammar file has four main sections, shown here with the
2618 appropriate delimiters:
2619
2620 @example
2621 %@{
2622 @var{Prologue}
2623 %@}
2624
2625 @var{Bison declarations}
2626
2627 %%
2628 @var{Grammar rules}
2629 %%
2630
2631 @var{Epilogue}
2632 @end example
2633
2634 Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
2635 As a @acronym{GNU} extension, @samp{//} introduces a comment that
2636 continues until end of line.
2637
2638 @menu
2639 * Prologue:: Syntax and usage of the prologue.
2640 * Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
2641 * Bison Declarations:: Syntax and usage of the Bison declarations section.
2642 * Grammar Rules:: Syntax and usage of the grammar rules section.
2643 * Epilogue:: Syntax and usage of the epilogue.
2644 @end menu
2645
2646 @node Prologue
2647 @subsection The prologue
2648 @cindex declarations section
2649 @cindex Prologue
2650 @cindex declarations
2651
2652 The @var{Prologue} section contains macro definitions and declarations
2653 of functions and variables that are used in the actions in the grammar
2654 rules. These are copied to the beginning of the parser file so that
2655 they precede the definition of @code{yyparse}. You can use
2656 @samp{#include} to get the declarations from a header file. If you
2657 don't need any C declarations, you may omit the @samp{%@{} and
2658 @samp{%@}} delimiters that bracket this section.
2659
2660 The @var{Prologue} section is terminated by the first occurrence
2661 of @samp{%@}} that is outside a comment, a string literal, or a
2662 character constant.
2663
2664 You may have more than one @var{Prologue} section, intermixed with the
2665 @var{Bison declarations}. This allows you to have C and Bison
2666 declarations that refer to each other. For example, the @code{%union}
2667 declaration may use types defined in a header file, and you may wish to
2668 prototype functions that take arguments of type @code{YYSTYPE}. This
2669 can be done with two @var{Prologue} blocks, one before and one after the
2670 @code{%union} declaration.
2671
2672 @smallexample
2673 %@{
2674 #define _GNU_SOURCE
2675 #include <stdio.h>
2676 #include "ptypes.h"
2677 %@}
2678
2679 %union @{
2680 long int n;
2681 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2682 @}
2683
2684 %@{
2685 static void print_token_value (FILE *, int, YYSTYPE);
2686 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2687 %@}
2688
2689 @dots{}
2690 @end smallexample
2691
2692 When in doubt, it is usually safer to put prologue code before all
2693 Bison declarations, rather than after. For example, any definitions
2694 of feature test macros like @code{_GNU_SOURCE} or
2695 @code{_POSIX_C_SOURCE} should appear before all Bison declarations, as
2696 feature test macros can affect the behavior of Bison-generated
2697 @code{#include} directives.
2698
2699 @node Prologue Alternatives
2700 @subsection Prologue Alternatives
2701 @cindex Prologue Alternatives
2702
2703 @findex %code
2704 @findex %code requires
2705 @findex %code provides
2706 @findex %code top
2707 (The prologue alternatives described here are experimental.
2708 More user feedback will help to determine whether they should become permanent
2709 features.)
2710
2711 The functionality of @var{Prologue} sections can often be subtle and
2712 inflexible.
2713 As an alternative, Bison provides a %code directive with an explicit qualifier
2714 field, which identifies the purpose of the code and thus the location(s) where
2715 Bison should generate it.
2716 For C/C++, the qualifier can be omitted for the default location, or it can be
2717 one of @code{requires}, @code{provides}, @code{top}.
2718 @xref{Decl Summary,,%code}.
2719
2720 Look again at the example of the previous section:
2721
2722 @smallexample
2723 %@{
2724 #define _GNU_SOURCE
2725 #include <stdio.h>
2726 #include "ptypes.h"
2727 %@}
2728
2729 %union @{
2730 long int n;
2731 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2732 @}
2733
2734 %@{
2735 static void print_token_value (FILE *, int, YYSTYPE);
2736 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2737 %@}
2738
2739 @dots{}
2740 @end smallexample
2741
2742 @noindent
2743 Notice that there are two @var{Prologue} sections here, but there's a subtle
2744 distinction between their functionality.
2745 For example, if you decide to override Bison's default definition for
2746 @code{YYLTYPE}, in which @var{Prologue} section should you write your new
2747 definition?
2748 You should write it in the first since Bison will insert that code into the
2749 parser source code file @emph{before} the default @code{YYLTYPE} definition.
2750 In which @var{Prologue} section should you prototype an internal function,
2751 @code{trace_token}, that accepts @code{YYLTYPE} and @code{yytokentype} as
2752 arguments?
2753 You should prototype it in the second since Bison will insert that code
2754 @emph{after} the @code{YYLTYPE} and @code{yytokentype} definitions.
2755
2756 This distinction in functionality between the two @var{Prologue} sections is
2757 established by the appearance of the @code{%union} between them.
2758 This behavior raises a few questions.
2759 First, why should the position of a @code{%union} affect definitions related to
2760 @code{YYLTYPE} and @code{yytokentype}?
2761 Second, what if there is no @code{%union}?
2762 In that case, the second kind of @var{Prologue} section is not available.
2763 This behavior is not intuitive.
2764
2765 To avoid this subtle @code{%union} dependency, rewrite the example using a
2766 @code{%code top} and an unqualified @code{%code}.
2767 Let's go ahead and add the new @code{YYLTYPE} definition and the
2768 @code{trace_token} prototype at the same time:
2769
2770 @smallexample
2771 %code top @{
2772 #define _GNU_SOURCE
2773 #include <stdio.h>
2774
2775 /* WARNING: The following code really belongs
2776 * in a `%code requires'; see below. */
2777
2778 #include "ptypes.h"
2779 #define YYLTYPE YYLTYPE
2780 typedef struct YYLTYPE
2781 @{
2782 int first_line;
2783 int first_column;
2784 int last_line;
2785 int last_column;
2786 char *filename;
2787 @} YYLTYPE;
2788 @}
2789
2790 %union @{
2791 long int n;
2792 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2793 @}
2794
2795 %code @{
2796 static void print_token_value (FILE *, int, YYSTYPE);
2797 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2798 static void trace_token (enum yytokentype token, YYLTYPE loc);
2799 @}
2800
2801 @dots{}
2802 @end smallexample
2803
2804 @noindent
2805 In this way, @code{%code top} and the unqualified @code{%code} achieve the same
2806 functionality as the two kinds of @var{Prologue} sections, but it's always
2807 explicit which kind you intend.
2808 Moreover, both kinds are always available even in the absence of @code{%union}.
2809
2810 The @code{%code top} block above logically contains two parts.
2811 The first two lines before the warning need to appear near the top of the
2812 parser source code file.
2813 The first line after the warning is required by @code{YYSTYPE} and thus also
2814 needs to appear in the parser source code file.
2815 However, if you've instructed Bison to generate a parser header file
2816 (@pxref{Decl Summary, ,%defines}), you probably want that line to appear before
2817 the @code{YYSTYPE} definition in that header file as well.
2818 The @code{YYLTYPE} definition should also appear in the parser header file to
2819 override the default @code{YYLTYPE} definition there.
2820
2821 In other words, in the @code{%code top} block above, all but the first two
2822 lines are dependency code required by the @code{YYSTYPE} and @code{YYLTYPE}
2823 definitions.
2824 Thus, they belong in one or more @code{%code requires}:
2825
2826 @smallexample
2827 %code top @{
2828 #define _GNU_SOURCE
2829 #include <stdio.h>
2830 @}
2831
2832 %code requires @{
2833 #include "ptypes.h"
2834 @}
2835 %union @{
2836 long int n;
2837 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2838 @}
2839
2840 %code requires @{
2841 #define YYLTYPE YYLTYPE
2842 typedef struct YYLTYPE
2843 @{
2844 int first_line;
2845 int first_column;
2846 int last_line;
2847 int last_column;
2848 char *filename;
2849 @} YYLTYPE;
2850 @}
2851
2852 %code @{
2853 static void print_token_value (FILE *, int, YYSTYPE);
2854 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2855 static void trace_token (enum yytokentype token, YYLTYPE loc);
2856 @}
2857
2858 @dots{}
2859 @end smallexample
2860
2861 @noindent
2862 Now Bison will insert @code{#include "ptypes.h"} and the new @code{YYLTYPE}
2863 definition before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
2864 definitions in both the parser source code file and the parser header file.
2865 (By the same reasoning, @code{%code requires} would also be the appropriate
2866 place to write your own definition for @code{YYSTYPE}.)
2867
2868 When you are writing dependency code for @code{YYSTYPE} and @code{YYLTYPE}, you
2869 should prefer @code{%code requires} over @code{%code top} regardless of whether
2870 you instruct Bison to generate a parser header file.
2871 When you are writing code that you need Bison to insert only into the parser
2872 source code file and that has no special need to appear at the top of that
2873 file, you should prefer the unqualified @code{%code} over @code{%code top}.
2874 These practices will make the purpose of each block of your code explicit to
2875 Bison and to other developers reading your grammar file.
2876 Following these practices, we expect the unqualified @code{%code} and
2877 @code{%code requires} to be the most important of the four @var{Prologue}
2878 alternatives.
2879
2880 At some point while developing your parser, you might decide to provide
2881 @code{trace_token} to modules that are external to your parser.
2882 Thus, you might wish for Bison to insert the prototype into both the parser
2883 header file and the parser source code file.
2884 Since this function is not a dependency required by @code{YYSTYPE} or
2885 @code{YYLTYPE}, it doesn't make sense to move its prototype to a
2886 @code{%code requires}.
2887 More importantly, since it depends upon @code{YYLTYPE} and @code{yytokentype},
2888 @code{%code requires} is not sufficient.
2889 Instead, move its prototype from the unqualified @code{%code} to a
2890 @code{%code provides}:
2891
2892 @smallexample
2893 %code top @{
2894 #define _GNU_SOURCE
2895 #include <stdio.h>
2896 @}
2897
2898 %code requires @{
2899 #include "ptypes.h"
2900 @}
2901 %union @{
2902 long int n;
2903 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
2904 @}
2905
2906 %code requires @{
2907 #define YYLTYPE YYLTYPE
2908 typedef struct YYLTYPE
2909 @{
2910 int first_line;
2911 int first_column;
2912 int last_line;
2913 int last_column;
2914 char *filename;
2915 @} YYLTYPE;
2916 @}
2917
2918 %code provides @{
2919 void trace_token (enum yytokentype token, YYLTYPE loc);
2920 @}
2921
2922 %code @{
2923 static void print_token_value (FILE *, int, YYSTYPE);
2924 #define YYPRINT(F, N, L) print_token_value (F, N, L)
2925 @}
2926
2927 @dots{}
2928 @end smallexample
2929
2930 @noindent
2931 Bison will insert the @code{trace_token} prototype into both the parser header
2932 file and the parser source code file after the definitions for
2933 @code{yytokentype}, @code{YYLTYPE}, and @code{YYSTYPE}.
2934
2935 The above examples are careful to write directives in an order that reflects
2936 the layout of the generated parser source code and header files:
2937 @code{%code top}, @code{%code requires}, @code{%code provides}, and then
2938 @code{%code}.
2939 While your grammar files may generally be easier to read if you also follow
2940 this order, Bison does not require it.
2941 Instead, Bison lets you choose an organization that makes sense to you.
2942
2943 You may declare any of these directives multiple times in the grammar file.
2944 In that case, Bison concatenates the contained code in declaration order.
2945 This is the only way in which the position of one of these directives within
2946 the grammar file affects its functionality.
2947
2948 The result of the previous two properties is greater flexibility in how you may
2949 organize your grammar file.
2950 For example, you may organize semantic-type-related directives by semantic
2951 type:
2952
2953 @smallexample
2954 %code requires @{ #include "type1.h" @}
2955 %union @{ type1 field1; @}
2956 %destructor @{ type1_free ($$); @} <field1>
2957 %printer @{ type1_print ($$); @} <field1>
2958
2959 %code requires @{ #include "type2.h" @}
2960 %union @{ type2 field2; @}
2961 %destructor @{ type2_free ($$); @} <field2>
2962 %printer @{ type2_print ($$); @} <field2>
2963 @end smallexample
2964
2965 @noindent
2966 You could even place each of the above directive groups in the rules section of
2967 the grammar file next to the set of rules that uses the associated semantic
2968 type.
2969 (In the rules section, you must terminate each of those directives with a
2970 semicolon.)
2971 And you don't have to worry that some directive (like a @code{%union}) in the
2972 definitions section is going to adversely affect their functionality in some
2973 counter-intuitive manner just because it comes first.
2974 Such an organization is not possible using @var{Prologue} sections.
2975
2976 This section has been concerned with explaining the advantages of the four
2977 @var{Prologue} alternatives over the original Yacc @var{Prologue}.
2978 However, in most cases when using these directives, you shouldn't need to
2979 think about all the low-level ordering issues discussed here.
2980 Instead, you should simply use these directives to label each block of your
2981 code according to its purpose and let Bison handle the ordering.
2982 @code{%code} is the most generic label.
2983 Move code to @code{%code requires}, @code{%code provides}, or @code{%code top}
2984 as needed.
2985
2986 @node Bison Declarations
2987 @subsection The Bison Declarations Section
2988 @cindex Bison declarations (introduction)
2989 @cindex declarations, Bison (introduction)
2990
2991 The @var{Bison declarations} section contains declarations that define
2992 terminal and nonterminal symbols, specify precedence, and so on.
2993 In some simple grammars you may not need any declarations.
2994 @xref{Declarations, ,Bison Declarations}.
2995
2996 @node Grammar Rules
2997 @subsection The Grammar Rules Section
2998 @cindex grammar rules section
2999 @cindex rules section for grammar
3000
3001 The @dfn{grammar rules} section contains one or more Bison grammar
3002 rules, and nothing else. @xref{Rules, ,Syntax of Grammar Rules}.
3003
3004 There must always be at least one grammar rule, and the first
3005 @samp{%%} (which precedes the grammar rules) may never be omitted even
3006 if it is the first thing in the file.
3007
3008 @node Epilogue
3009 @subsection The epilogue
3010 @cindex additional C code section
3011 @cindex epilogue
3012 @cindex C code, section for additional
3013
3014 The @var{Epilogue} is copied verbatim to the end of the parser file, just as
3015 the @var{Prologue} is copied to the beginning. This is the most convenient
3016 place to put anything that you want to have in the parser file but which need
3017 not come before the definition of @code{yyparse}. For example, the
3018 definitions of @code{yylex} and @code{yyerror} often go here. Because
3019 C requires functions to be declared before being used, you often need
3020 to declare functions like @code{yylex} and @code{yyerror} in the Prologue,
3021 even if you define them in the Epilogue.
3022 @xref{Interface, ,Parser C-Language Interface}.
3023
3024 If the last section is empty, you may omit the @samp{%%} that separates it
3025 from the grammar rules.
3026
3027 The Bison parser itself contains many macros and identifiers whose names
3028 start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
3029 any such names (except those documented in this manual) in the epilogue
3030 of the grammar file.
3031
3032 @node Symbols
3033 @section Symbols, Terminal and Nonterminal
3034 @cindex nonterminal symbol
3035 @cindex terminal symbol
3036 @cindex token type
3037 @cindex symbol
3038
3039 @dfn{Symbols} in Bison grammars represent the grammatical classifications
3040 of the language.
3041
3042 A @dfn{terminal symbol} (also known as a @dfn{token type}) represents a
3043 class of syntactically equivalent tokens. You use the symbol in grammar
3044 rules to mean that a token in that class is allowed. The symbol is
3045 represented in the Bison parser by a numeric code, and the @code{yylex}
3046 function returns a token type code to indicate what kind of token has
3047 been read. You don't need to know what the code value is; you can use
3048 the symbol to stand for it.
3049
3050 A @dfn{nonterminal symbol} stands for a class of syntactically
3051 equivalent groupings. The symbol name is used in writing grammar rules.
3052 By convention, it should be all lower case.
3053
3054 Symbol names can contain letters, digits (not at the beginning),
3055 underscores and periods. Periods make sense only in nonterminals.
3056
3057 There are three ways of writing terminal symbols in the grammar:
3058
3059 @itemize @bullet
3060 @item
3061 A @dfn{named token type} is written with an identifier, like an
3062 identifier in C@. By convention, it should be all upper case. Each
3063 such name must be defined with a Bison declaration such as
3064 @code{%token}. @xref{Token Decl, ,Token Type Names}.
3065
3066 @item
3067 @cindex character token
3068 @cindex literal token
3069 @cindex single-character literal
3070 A @dfn{character token type} (or @dfn{literal character token}) is
3071 written in the grammar using the same syntax used in C for character
3072 constants; for example, @code{'+'} is a character token type. A
3073 character token type doesn't need to be declared unless you need to
3074 specify its semantic value data type (@pxref{Value Type, ,Data Types of
3075 Semantic Values}), associativity, or precedence (@pxref{Precedence,
3076 ,Operator Precedence}).
3077
3078 By convention, a character token type is used only to represent a
3079 token that consists of that particular character. Thus, the token
3080 type @code{'+'} is used to represent the character @samp{+} as a
3081 token. Nothing enforces this convention, but if you depart from it,
3082 your program will confuse other readers.
3083
3084 All the usual escape sequences used in character literals in C can be
3085 used in Bison as well, but you must not use the null character as a
3086 character literal because its numeric code, zero, signifies
3087 end-of-input (@pxref{Calling Convention, ,Calling Convention
3088 for @code{yylex}}). Also, unlike standard C, trigraphs have no
3089 special meaning in Bison character literals, nor is backslash-newline
3090 allowed.
3091
3092 @item
3093 @cindex string token
3094 @cindex literal string token
3095 @cindex multicharacter literal
3096 A @dfn{literal string token} is written like a C string constant; for
3097 example, @code{"<="} is a literal string token. A literal string token
3098 doesn't need to be declared unless you need to specify its semantic
3099 value data type (@pxref{Value Type}), associativity, or precedence
3100 (@pxref{Precedence}).
3101
3102 You can associate the literal string token with a symbolic name as an
3103 alias, using the @code{%token} declaration (@pxref{Token Decl, ,Token
3104 Declarations}). If you don't do that, the lexical analyzer has to
3105 retrieve the token number for the literal string token from the
3106 @code{yytname} table (@pxref{Calling Convention}).
3107
3108 @strong{Warning}: literal string tokens do not work in Yacc.
3109
3110 By convention, a literal string token is used only to represent a token
3111 that consists of that particular string. Thus, you should use the token
3112 type @code{"<="} to represent the string @samp{<=} as a token. Bison
3113 does not enforce this convention, but if you depart from it, people who
3114 read your program will be confused.
3115
3116 All the escape sequences used in string literals in C can be used in
3117 Bison as well, except that you must not use a null character within a
3118 string literal. Also, unlike Standard C, trigraphs have no special
3119 meaning in Bison string literals, nor is backslash-newline allowed. A
3120 literal string token must contain two or more characters; for a token
3121 containing just one character, use a character token (see above).
3122 @end itemize
3123
3124 How you choose to write a terminal symbol has no effect on its
3125 grammatical meaning. That depends only on where it appears in rules and
3126 on when the parser function returns that symbol.
3127
3128 The value returned by @code{yylex} is always one of the terminal
3129 symbols, except that a zero or negative value signifies end-of-input.
3130 Whichever way you write the token type in the grammar rules, you write
3131 it the same way in the definition of @code{yylex}. The numeric code
3132 for a character token type is simply the positive numeric code of the
3133 character, so @code{yylex} can use the identical value to generate the
3134 requisite code, though you may need to convert it to @code{unsigned
3135 char} to avoid sign-extension on hosts where @code{char} is signed.
3136 Each named token type becomes a C macro in
3137 the parser file, so @code{yylex} can use the name to stand for the code.
3138 (This is why periods don't make sense in terminal symbols.)
3139 @xref{Calling Convention, ,Calling Convention for @code{yylex}}.
3140
3141 If @code{yylex} is defined in a separate file, you need to arrange for the
3142 token-type macro definitions to be available there. Use the @samp{-d}
3143 option when you run Bison, so that it will write these macro definitions
3144 into a separate header file @file{@var{name}.tab.h} which you can include
3145 in the other source files that need it. @xref{Invocation, ,Invoking Bison}.
3146
3147 If you want to write a grammar that is portable to any Standard C
3148 host, you must use only nonnull character tokens taken from the basic
3149 execution character set of Standard C@. This set consists of the ten
3150 digits, the 52 lower- and upper-case English letters, and the
3151 characters in the following C-language string:
3152
3153 @example
3154 "\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
3155 @end example
3156
3157 The @code{yylex} function and Bison must use a consistent character set
3158 and encoding for character tokens. For example, if you run Bison in an
3159 @acronym{ASCII} environment, but then compile and run the resulting
3160 program in an environment that uses an incompatible character set like
3161 @acronym{EBCDIC}, the resulting program may not work because the tables
3162 generated by Bison will assume @acronym{ASCII} numeric values for
3163 character tokens. It is standard practice for software distributions to
3164 contain C source files that were generated by Bison in an
3165 @acronym{ASCII} environment, so installers on platforms that are
3166 incompatible with @acronym{ASCII} must rebuild those files before
3167 compiling them.
3168
3169 The symbol @code{error} is a terminal symbol reserved for error recovery
3170 (@pxref{Error Recovery}); you shouldn't use it for any other purpose.
3171 In particular, @code{yylex} should never return this value. The default
3172 value of the error token is 256, unless you explicitly assigned 256 to
3173 one of your tokens with a @code{%token} declaration.
3174
3175 @node Rules
3176 @section Syntax of Grammar Rules
3177 @cindex rule syntax
3178 @cindex grammar rule syntax
3179 @cindex syntax of grammar rules
3180
3181 A Bison grammar rule has the following general form:
3182
3183 @example
3184 @group
3185 @var{result}: @var{components}@dots{}
3186 ;
3187 @end group
3188 @end example
3189
3190 @noindent
3191 where @var{result} is the nonterminal symbol that this rule describes,
3192 and @var{components} are various terminal and nonterminal symbols that
3193 are put together by this rule (@pxref{Symbols}).
3194
3195 For example,
3196
3197 @example
3198 @group
3199 exp: exp '+' exp
3200 ;
3201 @end group
3202 @end example
3203
3204 @noindent
3205 says that two groupings of type @code{exp}, with a @samp{+} token in between,
3206 can be combined into a larger grouping of type @code{exp}.
3207
3208 White space in rules is significant only to separate symbols. You can add
3209 extra white space as you wish.
3210
3211 Scattered among the components can be @var{actions} that determine
3212 the semantics of the rule. An action looks like this:
3213
3214 @example
3215 @{@var{C statements}@}
3216 @end example
3217
3218 @noindent
3219 @cindex braced code
3220 This is an example of @dfn{braced code}, that is, C code surrounded by
3221 braces, much like a compound statement in C@. Braced code can contain
3222 any sequence of C tokens, so long as its braces are balanced. Bison
3223 does not check the braced code for correctness directly; it merely
3224 copies the code to the output file, where the C compiler can check it.
3225
3226 Within braced code, the balanced-brace count is not affected by braces
3227 within comments, string literals, or character constants, but it is
3228 affected by the C digraphs @samp{<%} and @samp{%>} that represent
3229 braces. At the top level braced code must be terminated by @samp{@}}
3230 and not by a digraph. Bison does not look for trigraphs, so if braced
3231 code uses trigraphs you should ensure that they do not affect the
3232 nesting of braces or the boundaries of comments, string literals, or
3233 character constants.
3234
3235 Usually there is only one action and it follows the components.
3236 @xref{Actions}.
3237
3238 @findex |
3239 Multiple rules for the same @var{result} can be written separately or can
3240 be joined with the vertical-bar character @samp{|} as follows:
3241
3242 @example
3243 @group
3244 @var{result}: @var{rule1-components}@dots{}
3245 | @var{rule2-components}@dots{}
3246 @dots{}
3247 ;
3248 @end group
3249 @end example
3250
3251 @noindent
3252 They are still considered distinct rules even when joined in this way.
3253
3254 If @var{components} in a rule is empty, it means that @var{result} can
3255 match the empty string. For example, here is how to define a
3256 comma-separated sequence of zero or more @code{exp} groupings:
3257
3258 @example
3259 @group
3260 expseq: /* empty */
3261 | expseq1
3262 ;
3263 @end group
3264
3265 @group
3266 expseq1: exp
3267 | expseq1 ',' exp
3268 ;
3269 @end group
3270 @end example
3271
3272 @noindent
3273 It is customary to write a comment @samp{/* empty */} in each rule
3274 with no components.
3275
3276 @node Recursion
3277 @section Recursive Rules
3278 @cindex recursive rule
3279
3280 A rule is called @dfn{recursive} when its @var{result} nonterminal
3281 appears also on its right hand side. Nearly all Bison grammars need to
3282 use recursion, because that is the only way to define a sequence of any
3283 number of a particular thing. Consider this recursive definition of a
3284 comma-separated sequence of one or more expressions:
3285
3286 @example
3287 @group
3288 expseq1: exp
3289 | expseq1 ',' exp
3290 ;
3291 @end group
3292 @end example
3293
3294 @cindex left recursion
3295 @cindex right recursion
3296 @noindent
3297 Since the recursive use of @code{expseq1} is the leftmost symbol in the
3298 right hand side, we call this @dfn{left recursion}. By contrast, here
3299 the same construct is defined using @dfn{right recursion}:
3300
3301 @example
3302 @group
3303 expseq1: exp
3304 | exp ',' expseq1
3305 ;
3306 @end group
3307 @end example
3308
3309 @noindent
3310 Any kind of sequence can be defined using either left recursion or right
3311 recursion, but you should always use left recursion, because it can
3312 parse a sequence of any number of elements with bounded stack space.
3313 Right recursion uses up space on the Bison stack in proportion to the
3314 number of elements in the sequence, because all the elements must be
3315 shifted onto the stack before the rule can be applied even once.
3316 @xref{Algorithm, ,The Bison Parser Algorithm}, for further explanation
3317 of this.
3318
3319 @cindex mutual recursion
3320 @dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3321 rule does not appear directly on its right hand side, but does appear
3322 in rules for other nonterminals which do appear on its right hand
3323 side.
3324
3325 For example:
3326
3327 @example
3328 @group
3329 expr: primary
3330 | primary '+' primary
3331 ;
3332 @end group
3333
3334 @group
3335 primary: constant
3336 | '(' expr ')'
3337 ;
3338 @end group
3339 @end example
3340
3341 @noindent
3342 defines two mutually-recursive nonterminals, since each refers to the
3343 other.
3344
3345 @node Semantics
3346 @section Defining Language Semantics
3347 @cindex defining language semantics
3348 @cindex language semantics, defining
3349
3350 The grammar rules for a language determine only the syntax. The semantics
3351 are determined by the semantic values associated with various tokens and
3352 groupings, and by the actions taken when various groupings are recognized.
3353
3354 For example, the calculator calculates properly because the value
3355 associated with each expression is the proper number; it adds properly
3356 because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3357 the numbers associated with @var{x} and @var{y}.
3358
3359 @menu
3360 * Value Type:: Specifying one data type for all semantic values.
3361 * Multiple Types:: Specifying several alternative data types.
3362 * Actions:: An action is the semantic definition of a grammar rule.
3363 * Action Types:: Specifying data types for actions to operate on.
3364 * Mid-Rule Actions:: Most actions go at the end of a rule.
3365 This says when, why and how to use the exceptional
3366 action in the middle of a rule.
3367 @end menu
3368
3369 @node Value Type
3370 @subsection Data Types of Semantic Values
3371 @cindex semantic value type
3372 @cindex value type, semantic
3373 @cindex data types of semantic values
3374 @cindex default data type
3375
3376 In a simple program it may be sufficient to use the same data type for
3377 the semantic values of all language constructs. This was true in the
3378 @acronym{RPN} and infix calculator examples (@pxref{RPN Calc, ,Reverse Polish
3379 Notation Calculator}).
3380
3381 Bison normally uses the type @code{int} for semantic values if your
3382 program uses the same data type for all language constructs. To
3383 specify some other type, define @code{YYSTYPE} as a macro, like this:
3384
3385 @example
3386 #define YYSTYPE double
3387 @end example
3388
3389 @noindent
3390 @code{YYSTYPE}'s replacement list should be a type name
3391 that does not contain parentheses or square brackets.
3392 This macro definition must go in the prologue of the grammar file
3393 (@pxref{Grammar Outline, ,Outline of a Bison Grammar}).
3394
3395 @node Multiple Types
3396 @subsection More Than One Value Type
3397
3398 In most programs, you will need different data types for different kinds
3399 of tokens and groupings. For example, a numeric constant may need type
3400 @code{int} or @code{long int}, while a string constant needs type
3401 @code{char *}, and an identifier might need a pointer to an entry in the
3402 symbol table.
3403
3404 To use more than one data type for semantic values in one parser, Bison
3405 requires you to do two things:
3406
3407 @itemize @bullet
3408 @item
3409 Specify the entire collection of possible data types, either by using the
3410 @code{%union} Bison declaration (@pxref{Union Decl, ,The Collection of
3411 Value Types}), or by using a @code{typedef} or a @code{#define} to
3412 define @code{YYSTYPE} to be a union type whose member names are
3413 the type tags.
3414
3415 @item
3416 Choose one of those types for each symbol (terminal or nonterminal) for
3417 which semantic values are used. This is done for tokens with the
3418 @code{%token} Bison declaration (@pxref{Token Decl, ,Token Type Names})
3419 and for groupings with the @code{%type} Bison declaration (@pxref{Type
3420 Decl, ,Nonterminal Symbols}).
3421 @end itemize
3422
3423 @node Actions
3424 @subsection Actions
3425 @cindex action
3426 @vindex $$
3427 @vindex $@var{n}
3428
3429 An action accompanies a syntactic rule and contains C code to be executed
3430 each time an instance of that rule is recognized. The task of most actions
3431 is to compute a semantic value for the grouping built by the rule from the
3432 semantic values associated with tokens or smaller groupings.
3433
3434 An action consists of braced code containing C statements, and can be
3435 placed at any position in the rule;
3436 it is executed at that position. Most rules have just one action at the
3437 end of the rule, following all the components. Actions in the middle of
3438 a rule are tricky and used only for special purposes (@pxref{Mid-Rule
3439 Actions, ,Actions in Mid-Rule}).
3440
3441 The C code in an action can refer to the semantic values of the components
3442 matched by the rule with the construct @code{$@var{n}}, which stands for
3443 the value of the @var{n}th component. The semantic value for the grouping
3444 being constructed is @code{$$}. Bison translates both of these
3445 constructs into expressions of the appropriate type when it copies the
3446 actions into the parser file. @code{$$} is translated to a modifiable
3447 lvalue, so it can be assigned to.
3448
3449 Here is a typical example:
3450
3451 @example
3452 @group
3453 exp: @dots{}
3454 | exp '+' exp
3455 @{ $$ = $1 + $3; @}
3456 @end group
3457 @end example
3458
3459 @noindent
3460 This rule constructs an @code{exp} from two smaller @code{exp} groupings
3461 connected by a plus-sign token. In the action, @code{$1} and @code{$3}
3462 refer to the semantic values of the two component @code{exp} groupings,
3463 which are the first and third symbols on the right hand side of the rule.
3464 The sum is stored into @code{$$} so that it becomes the semantic value of
3465 the addition-expression just recognized by the rule. If there were a
3466 useful semantic value associated with the @samp{+} token, it could be
3467 referred to as @code{$2}.
3468
3469 Note that the vertical-bar character @samp{|} is really a rule
3470 separator, and actions are attached to a single rule. This is a
3471 difference with tools like Flex, for which @samp{|} stands for either
3472 ``or'', or ``the same action as that of the next rule''. In the
3473 following example, the action is triggered only when @samp{b} is found:
3474
3475 @example
3476 @group
3477 a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
3478 @end group
3479 @end example
3480
3481 @cindex default action
3482 If you don't specify an action for a rule, Bison supplies a default:
3483 @w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
3484 becomes the value of the whole rule. Of course, the default action is
3485 valid only if the two data types match. There is no meaningful default
3486 action for an empty rule; every empty rule must have an explicit action
3487 unless the rule's value does not matter.
3488
3489 @code{$@var{n}} with @var{n} zero or negative is allowed for reference
3490 to tokens and groupings on the stack @emph{before} those that match the
3491 current rule. This is a very risky practice, and to use it reliably
3492 you must be certain of the context in which the rule is applied. Here
3493 is a case in which you can use this reliably:
3494
3495 @example
3496 @group
3497 foo: expr bar '+' expr @{ @dots{} @}
3498 | expr bar '-' expr @{ @dots{} @}
3499 ;
3500 @end group
3501
3502 @group
3503 bar: /* empty */
3504 @{ previous_expr = $0; @}
3505 ;
3506 @end group
3507 @end example
3508
3509 As long as @code{bar} is used only in the fashion shown here, @code{$0}
3510 always refers to the @code{expr} which precedes @code{bar} in the
3511 definition of @code{foo}.
3512
3513 @vindex yylval
3514 It is also possible to access the semantic value of the lookahead token, if
3515 any, from a semantic action.
3516 This semantic value is stored in @code{yylval}.
3517 @xref{Action Features, ,Special Features for Use in Actions}.
3518
3519 @node Action Types
3520 @subsection Data Types of Values in Actions
3521 @cindex action data types
3522 @cindex data types in actions
3523
3524 If you have chosen a single data type for semantic values, the @code{$$}
3525 and @code{$@var{n}} constructs always have that data type.
3526
3527 If you have used @code{%union} to specify a variety of data types, then you
3528 must declare a choice among these types for each terminal or nonterminal
3529 symbol that can have a semantic value. Then each time you use @code{$$} or
3530 @code{$@var{n}}, its data type is determined by which symbol it refers to
3531 in the rule. In this example,
3532
3533 @example
3534 @group
3535 exp: @dots{}
3536 | exp '+' exp
3537 @{ $$ = $1 + $3; @}
3538 @end group
3539 @end example
3540
3541 @noindent
3542 @code{$1} and @code{$3} refer to instances of @code{exp}, so they all
3543 have the data type declared for the nonterminal symbol @code{exp}. If
3544 @code{$2} were used, it would have the data type declared for the
3545 terminal symbol @code{'+'}, whatever that might be.
3546
3547 Alternatively, you can specify the data type when you refer to the value,
3548 by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
3549 reference. For example, if you have defined types as shown here:
3550
3551 @example
3552 @group
3553 %union @{
3554 int itype;
3555 double dtype;
3556 @}
3557 @end group
3558 @end example
3559
3560 @noindent
3561 then you can write @code{$<itype>1} to refer to the first subunit of the
3562 rule as an integer, or @code{$<dtype>1} to refer to it as a double.
3563
3564 @node Mid-Rule Actions
3565 @subsection Actions in Mid-Rule
3566 @cindex actions in mid-rule
3567 @cindex mid-rule actions
3568
3569 Occasionally it is useful to put an action in the middle of a rule.
3570 These actions are written just like usual end-of-rule actions, but they
3571 are executed before the parser even recognizes the following components.
3572
3573 A mid-rule action may refer to the components preceding it using
3574 @code{$@var{n}}, but it may not refer to subsequent components because
3575 it is run before they are parsed.
3576
3577 The mid-rule action itself counts as one of the components of the rule.
3578 This makes a difference when there is another action later in the same rule
3579 (and usually there is another at the end): you have to count the actions
3580 along with the symbols when working out which number @var{n} to use in
3581 @code{$@var{n}}.
3582
3583 The mid-rule action can also have a semantic value. The action can set
3584 its value with an assignment to @code{$$}, and actions later in the rule
3585 can refer to the value using @code{$@var{n}}. Since there is no symbol
3586 to name the action, there is no way to declare a data type for the value
3587 in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
3588 specify a data type each time you refer to this value.
3589
3590 There is no way to set the value of the entire rule with a mid-rule
3591 action, because assignments to @code{$$} do not have that effect. The
3592 only way to set the value for the entire rule is with an ordinary action
3593 at the end of the rule.
3594
3595 Here is an example from a hypothetical compiler, handling a @code{let}
3596 statement that looks like @samp{let (@var{variable}) @var{statement}} and
3597 serves to create a variable named @var{variable} temporarily for the
3598 duration of @var{statement}. To parse this construct, we must put
3599 @var{variable} into the symbol table while @var{statement} is parsed, then
3600 remove it afterward. Here is how it is done:
3601
3602 @example
3603 @group
3604 stmt: LET '(' var ')'
3605 @{ $<context>$ = push_context ();
3606 declare_variable ($3); @}
3607 stmt @{ $$ = $6;
3608 pop_context ($<context>5); @}
3609 @end group
3610 @end example
3611
3612 @noindent
3613 As soon as @samp{let (@var{variable})} has been recognized, the first
3614 action is run. It saves a copy of the current semantic context (the
3615 list of accessible variables) as its semantic value, using alternative
3616 @code{context} in the data-type union. Then it calls
3617 @code{declare_variable} to add the new variable to that list. Once the
3618 first action is finished, the embedded statement @code{stmt} can be
3619 parsed. Note that the mid-rule action is component number 5, so the
3620 @samp{stmt} is component number 6.
3621
3622 After the embedded statement is parsed, its semantic value becomes the
3623 value of the entire @code{let}-statement. Then the semantic value from the
3624 earlier action is used to restore the prior list of variables. This
3625 removes the temporary @code{let}-variable from the list so that it won't
3626 appear to exist while the rest of the program is parsed.
3627
3628 @findex %destructor
3629 @cindex discarded symbols, mid-rule actions
3630 @cindex error recovery, mid-rule actions
3631 In the above example, if the parser initiates error recovery (@pxref{Error
3632 Recovery}) while parsing the tokens in the embedded statement @code{stmt},
3633 it might discard the previous semantic context @code{$<context>5} without
3634 restoring it.
3635 Thus, @code{$<context>5} needs a destructor (@pxref{Destructor Decl, , Freeing
3636 Discarded Symbols}).
3637 However, Bison currently provides no means to declare a destructor specific to
3638 a particular mid-rule action's semantic value.
3639
3640 One solution is to bury the mid-rule action inside a nonterminal symbol and to
3641 declare a destructor for that symbol:
3642
3643 @example
3644 @group
3645 %type <context> let
3646 %destructor @{ pop_context ($$); @} let
3647
3648 %%
3649
3650 stmt: let stmt
3651 @{ $$ = $2;
3652 pop_context ($1); @}
3653 ;
3654
3655 let: LET '(' var ')'
3656 @{ $$ = push_context ();
3657 declare_variable ($3); @}
3658 ;
3659
3660 @end group
3661 @end example
3662
3663 @noindent
3664 Note that the action is now at the end of its rule.
3665 Any mid-rule action can be converted to an end-of-rule action in this way, and
3666 this is what Bison actually does to implement mid-rule actions.
3667
3668 Taking action before a rule is completely recognized often leads to
3669 conflicts since the parser must commit to a parse in order to execute the
3670 action. For example, the following two rules, without mid-rule actions,
3671 can coexist in a working parser because the parser can shift the open-brace
3672 token and look at what follows before deciding whether there is a
3673 declaration or not:
3674
3675 @example
3676 @group
3677 compound: '@{' declarations statements '@}'
3678 | '@{' statements '@}'
3679 ;
3680 @end group
3681 @end example
3682
3683 @noindent
3684 But when we add a mid-rule action as follows, the rules become nonfunctional:
3685
3686 @example
3687 @group
3688 compound: @{ prepare_for_local_variables (); @}
3689 '@{' declarations statements '@}'
3690 @end group
3691 @group
3692 | '@{' statements '@}'
3693 ;
3694 @end group
3695 @end example
3696
3697 @noindent
3698 Now the parser is forced to decide whether to run the mid-rule action
3699 when it has read no farther than the open-brace. In other words, it
3700 must commit to using one rule or the other, without sufficient
3701 information to do it correctly. (The open-brace token is what is called
3702 the @dfn{lookahead} token at this time, since the parser is still
3703 deciding what to do about it. @xref{Lookahead, ,Lookahead Tokens}.)
3704
3705 You might think that you could correct the problem by putting identical
3706 actions into the two rules, like this:
3707
3708 @example
3709 @group
3710 compound: @{ prepare_for_local_variables (); @}
3711 '@{' declarations statements '@}'
3712 | @{ prepare_for_local_variables (); @}
3713 '@{' statements '@}'
3714 ;
3715 @end group
3716 @end example
3717
3718 @noindent
3719 But this does not help, because Bison does not realize that the two actions
3720 are identical. (Bison never tries to understand the C code in an action.)
3721
3722 If the grammar is such that a declaration can be distinguished from a
3723 statement by the first token (which is true in C), then one solution which
3724 does work is to put the action after the open-brace, like this:
3725
3726 @example
3727 @group
3728 compound: '@{' @{ prepare_for_local_variables (); @}
3729 declarations statements '@}'
3730 | '@{' statements '@}'
3731 ;
3732 @end group
3733 @end example
3734
3735 @noindent
3736 Now the first token of the following declaration or statement,
3737 which would in any case tell Bison which rule to use, can still do so.
3738
3739 Another solution is to bury the action inside a nonterminal symbol which
3740 serves as a subroutine:
3741
3742 @example
3743 @group
3744 subroutine: /* empty */
3745 @{ prepare_for_local_variables (); @}
3746 ;
3747
3748 @end group
3749
3750 @group
3751 compound: subroutine
3752 '@{' declarations statements '@}'
3753 | subroutine
3754 '@{' statements '@}'
3755 ;
3756 @end group
3757 @end example
3758
3759 @noindent
3760 Now Bison can execute the action in the rule for @code{subroutine} without
3761 deciding which rule for @code{compound} it will eventually use.
3762
3763 @node Locations
3764 @section Tracking Locations
3765 @cindex location
3766 @cindex textual location
3767 @cindex location, textual
3768
3769 Though grammar rules and semantic actions are enough to write a fully
3770 functional parser, it can be useful to process some additional information,
3771 especially symbol locations.
3772
3773 The way locations are handled is defined by providing a data type, and
3774 actions to take when rules are matched.
3775
3776 @menu
3777 * Location Type:: Specifying a data type for locations.
3778 * Actions and Locations:: Using locations in actions.
3779 * Location Default Action:: Defining a general way to compute locations.
3780 @end menu
3781
3782 @node Location Type
3783 @subsection Data Type of Locations
3784 @cindex data type of locations
3785 @cindex default location type
3786
3787 Defining a data type for locations is much simpler than for semantic values,
3788 since all tokens and groupings always use the same type.
3789
3790 You can specify the type of locations by defining a macro called
3791 @code{YYLTYPE}, just as you can specify the semantic value type by
3792 defining a @code{YYSTYPE} macro (@pxref{Value Type}).
3793 When @code{YYLTYPE} is not defined, Bison uses a default structure type with
3794 four members:
3795
3796 @example
3797 typedef struct YYLTYPE
3798 @{
3799 int first_line;
3800 int first_column;
3801 int last_line;
3802 int last_column;
3803 @} YYLTYPE;
3804 @end example
3805
3806 At the beginning of the parsing, Bison initializes all these fields to 1
3807 for @code{yylloc}.
3808
3809 @node Actions and Locations
3810 @subsection Actions and Locations
3811 @cindex location actions
3812 @cindex actions, location
3813 @vindex @@$
3814 @vindex @@@var{n}
3815
3816 Actions are not only useful for defining language semantics, but also for
3817 describing the behavior of the output parser with locations.
3818
3819 The most obvious way for building locations of syntactic groupings is very
3820 similar to the way semantic values are computed. In a given rule, several
3821 constructs can be used to access the locations of the elements being matched.
3822 The location of the @var{n}th component of the right hand side is
3823 @code{@@@var{n}}, while the location of the left hand side grouping is
3824 @code{@@$}.
3825
3826 Here is a basic example using the default data type for locations:
3827
3828 @example
3829 @group
3830 exp: @dots{}
3831 | exp '/' exp
3832 @{
3833 @@$.first_column = @@1.first_column;
3834 @@$.first_line = @@1.first_line;
3835 @@$.last_column = @@3.last_column;
3836 @@$.last_line = @@3.last_line;
3837 if ($3)
3838 $$ = $1 / $3;
3839 else
3840 @{
3841 $$ = 1;
3842 fprintf (stderr,
3843 "Division by zero, l%d,c%d-l%d,c%d",
3844 @@3.first_line, @@3.first_column,
3845 @@3.last_line, @@3.last_column);
3846 @}
3847 @}
3848 @end group
3849 @end example
3850
3851 As for semantic values, there is a default action for locations that is
3852 run each time a rule is matched. It sets the beginning of @code{@@$} to the
3853 beginning of the first symbol, and the end of @code{@@$} to the end of the
3854 last symbol.
3855
3856 With this default action, the location tracking can be fully automatic. The
3857 example above simply rewrites this way:
3858
3859 @example
3860 @group
3861 exp: @dots{}
3862 | exp '/' exp
3863 @{
3864 if ($3)
3865 $$ = $1 / $3;
3866 else
3867 @{
3868 $$ = 1;
3869 fprintf (stderr,
3870 "Division by zero, l%d,c%d-l%d,c%d",
3871 @@3.first_line, @@3.first_column,
3872 @@3.last_line, @@3.last_column);
3873 @}
3874 @}
3875 @end group
3876 @end example
3877
3878 @vindex yylloc
3879 It is also possible to access the location of the lookahead token, if any,
3880 from a semantic action.
3881 This location is stored in @code{yylloc}.
3882 @xref{Action Features, ,Special Features for Use in Actions}.
3883
3884 @node Location Default Action
3885 @subsection Default Action for Locations
3886 @vindex YYLLOC_DEFAULT
3887 @cindex @acronym{GLR} parsers and @code{YYLLOC_DEFAULT}
3888
3889 Actually, actions are not the best place to compute locations. Since
3890 locations are much more general than semantic values, there is room in
3891 the output parser to redefine the default action to take for each
3892 rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
3893 matched, before the associated action is run. It is also invoked
3894 while processing a syntax error, to compute the error's location.
3895 Before reporting an unresolvable syntactic ambiguity, a @acronym{GLR}
3896 parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
3897 of that ambiguity.
3898
3899 Most of the time, this macro is general enough to suppress location
3900 dedicated code from semantic actions.
3901
3902 The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
3903 the location of the grouping (the result of the computation). When a
3904 rule is matched, the second parameter identifies locations of
3905 all right hand side elements of the rule being matched, and the third
3906 parameter is the size of the rule's right hand side.
3907 When a @acronym{GLR} parser reports an ambiguity, which of multiple candidate
3908 right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
3909 When processing a syntax error, the second parameter identifies locations
3910 of the symbols that were discarded during error processing, and the third
3911 parameter is the number of discarded symbols.
3912
3913 By default, @code{YYLLOC_DEFAULT} is defined this way:
3914
3915 @smallexample
3916 @group
3917 # define YYLLOC_DEFAULT(Current, Rhs, N) \
3918 do \
3919 if (N) \
3920 @{ \
3921 (Current).first_line = YYRHSLOC(Rhs, 1).first_line; \
3922 (Current).first_column = YYRHSLOC(Rhs, 1).first_column; \
3923 (Current).last_line = YYRHSLOC(Rhs, N).last_line; \
3924 (Current).last_column = YYRHSLOC(Rhs, N).last_column; \
3925 @} \
3926 else \
3927 @{ \
3928 (Current).first_line = (Current).last_line = \
3929 YYRHSLOC(Rhs, 0).last_line; \
3930 (Current).first_column = (Current).last_column = \
3931 YYRHSLOC(Rhs, 0).last_column; \
3932 @} \
3933 while (0)
3934 @end group
3935 @end smallexample
3936
3937 where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
3938 in @var{rhs} when @var{k} is positive, and the location of the symbol
3939 just before the reduction when @var{k} and @var{n} are both zero.
3940
3941 When defining @code{YYLLOC_DEFAULT}, you should consider that:
3942
3943 @itemize @bullet
3944 @item
3945 All arguments are free of side-effects. However, only the first one (the
3946 result) should be modified by @code{YYLLOC_DEFAULT}.
3947
3948 @item
3949 For consistency with semantic actions, valid indexes within the
3950 right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
3951 valid index, and it refers to the symbol just before the reduction.
3952 During error processing @var{n} is always positive.
3953
3954 @item
3955 Your macro should parenthesize its arguments, if need be, since the
3956 actual arguments may not be surrounded by parentheses. Also, your
3957 macro should expand to something that can be used as a single
3958 statement when it is followed by a semicolon.
3959 @end itemize
3960
3961 @node Declarations
3962 @section Bison Declarations
3963 @cindex declarations, Bison
3964 @cindex Bison declarations
3965
3966 The @dfn{Bison declarations} section of a Bison grammar defines the symbols
3967 used in formulating the grammar and the data types of semantic values.
3968 @xref{Symbols}.
3969
3970 All token type names (but not single-character literal tokens such as
3971 @code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
3972 declared if you need to specify which data type to use for the semantic
3973 value (@pxref{Multiple Types, ,More Than One Value Type}).
3974
3975 The first rule in the file also specifies the start symbol, by default.
3976 If you want some other symbol to be the start symbol, you must declare
3977 it explicitly (@pxref{Language and Grammar, ,Languages and Context-Free
3978 Grammars}).
3979
3980 @menu
3981 * Require Decl:: Requiring a Bison version.
3982 * Token Decl:: Declaring terminal symbols.
3983 * Precedence Decl:: Declaring terminals with precedence and associativity.
3984 * Union Decl:: Declaring the set of all semantic value types.
3985 * Type Decl:: Declaring the choice of type for a nonterminal symbol.
3986 * Initial Action Decl:: Code run before parsing starts.
3987 * Destructor Decl:: Declaring how symbols are freed.
3988 * Expect Decl:: Suppressing warnings about parsing conflicts.
3989 * Start Decl:: Specifying the start symbol.
3990 * Pure Decl:: Requesting a reentrant parser.
3991 * Push Decl:: Requesting a push parser.
3992 * Decl Summary:: Table of all Bison declarations.
3993 @end menu
3994
3995 @node Require Decl
3996 @subsection Require a Version of Bison
3997 @cindex version requirement
3998 @cindex requiring a version of Bison
3999 @findex %require
4000
4001 You may require the minimum version of Bison to process the grammar. If
4002 the requirement is not met, @command{bison} exits with an error (exit
4003 status 63).
4004
4005 @example
4006 %require "@var{version}"
4007 @end example
4008
4009 @node Token Decl
4010 @subsection Token Type Names
4011 @cindex declaring token type names
4012 @cindex token type names, declaring
4013 @cindex declaring literal string tokens
4014 @findex %token
4015
4016 The basic way to declare a token type name (terminal symbol) is as follows:
4017
4018 @example
4019 %token @var{name}
4020 @end example
4021
4022 Bison will convert this into a @code{#define} directive in
4023 the parser, so that the function @code{yylex} (if it is in this file)
4024 can use the name @var{name} to stand for this token type's code.
4025
4026 Alternatively, you can use @code{%left}, @code{%right}, or
4027 @code{%nonassoc} instead of @code{%token}, if you wish to specify
4028 associativity and precedence. @xref{Precedence Decl, ,Operator
4029 Precedence}.
4030
4031 You can explicitly specify the numeric code for a token type by appending
4032 a nonnegative decimal or hexadecimal integer value in the field immediately
4033 following the token name:
4034
4035 @example
4036 %token NUM 300
4037 %token XNUM 0x12d // a GNU extension
4038 @end example
4039
4040 @noindent
4041 It is generally best, however, to let Bison choose the numeric codes for
4042 all token types. Bison will automatically select codes that don't conflict
4043 with each other or with normal characters.
4044
4045 In the event that the stack type is a union, you must augment the
4046 @code{%token} or other token declaration to include the data type
4047 alternative delimited by angle-brackets (@pxref{Multiple Types, ,More
4048 Than One Value Type}).
4049
4050 For example:
4051
4052 @example
4053 @group
4054 %union @{ /* define stack type */
4055 double val;
4056 symrec *tptr;
4057 @}
4058 %token <val> NUM /* define token NUM and its type */
4059 @end group
4060 @end example
4061
4062 You can associate a literal string token with a token type name by
4063 writing the literal string at the end of a @code{%token}
4064 declaration which declares the name. For example:
4065
4066 @example
4067 %token arrow "=>"
4068 @end example
4069
4070 @noindent
4071 For example, a grammar for the C language might specify these names with
4072 equivalent literal string tokens:
4073
4074 @example
4075 %token <operator> OR "||"
4076 %token <operator> LE 134 "<="
4077 %left OR "<="
4078 @end example
4079
4080 @noindent
4081 Once you equate the literal string and the token name, you can use them
4082 interchangeably in further declarations or the grammar rules. The
4083 @code{yylex} function can use the token name or the literal string to
4084 obtain the token type code number (@pxref{Calling Convention}).
4085 Syntax error messages passed to @code{yyerror} from the parser will reference
4086 the literal string instead of the token name.
4087
4088 The token numbered as 0 corresponds to end of file; the following line
4089 allows for nicer error messages referring to ``end of file'' instead
4090 of ``$end'':
4091
4092 @example
4093 %token END 0 "end of file"
4094 @end example
4095
4096 @node Precedence Decl
4097 @subsection Operator Precedence
4098 @cindex precedence declarations
4099 @cindex declaring operator precedence
4100 @cindex operator precedence, declaring
4101
4102 Use the @code{%left}, @code{%right} or @code{%nonassoc} declaration to
4103 declare a token and specify its precedence and associativity, all at
4104 once. These are called @dfn{precedence declarations}.
4105 @xref{Precedence, ,Operator Precedence}, for general information on
4106 operator precedence.
4107
4108 The syntax of a precedence declaration is nearly the same as that of
4109 @code{%token}: either
4110
4111 @example
4112 %left @var{symbols}@dots{}
4113 @end example
4114
4115 @noindent
4116 or
4117
4118 @example
4119 %left <@var{type}> @var{symbols}@dots{}
4120 @end example
4121
4122 And indeed any of these declarations serves the purposes of @code{%token}.
4123 But in addition, they specify the associativity and relative precedence for
4124 all the @var{symbols}:
4125
4126 @itemize @bullet
4127 @item
4128 The associativity of an operator @var{op} determines how repeated uses
4129 of the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op}
4130 @var{z}} is parsed by grouping @var{x} with @var{y} first or by
4131 grouping @var{y} with @var{z} first. @code{%left} specifies
4132 left-associativity (grouping @var{x} with @var{y} first) and
4133 @code{%right} specifies right-associativity (grouping @var{y} with
4134 @var{z} first). @code{%nonassoc} specifies no associativity, which
4135 means that @samp{@var{x} @var{op} @var{y} @var{op} @var{z}} is
4136 considered a syntax error.
4137
4138 @item
4139 The precedence of an operator determines how it nests with other operators.
4140 All the tokens declared in a single precedence declaration have equal
4141 precedence and nest together according to their associativity.
4142 When two tokens declared in different precedence declarations associate,
4143 the one declared later has the higher precedence and is grouped first.
4144 @end itemize
4145
4146 For backward compatibility, there is a confusing difference between the
4147 argument lists of @code{%token} and precedence declarations.
4148 Only a @code{%token} can associate a literal string with a token type name.
4149 A precedence declaration always interprets a literal string as a reference to a
4150 separate token.
4151 For example:
4152
4153 @example
4154 %left OR "<=" // Does not declare an alias.
4155 %left OR 134 "<=" 135 // Declares 134 for OR and 135 for "<=".
4156 @end example
4157
4158 @node Union Decl
4159 @subsection The Collection of Value Types
4160 @cindex declaring value types
4161 @cindex value types, declaring
4162 @findex %union
4163
4164 The @code{%union} declaration specifies the entire collection of
4165 possible data types for semantic values. The keyword @code{%union} is
4166 followed by braced code containing the same thing that goes inside a
4167 @code{union} in C@.
4168
4169 For example:
4170
4171 @example
4172 @group
4173 %union @{
4174 double val;
4175 symrec *tptr;
4176 @}
4177 @end group
4178 @end example
4179
4180 @noindent
4181 This says that the two alternative types are @code{double} and @code{symrec
4182 *}. They are given names @code{val} and @code{tptr}; these names are used
4183 in the @code{%token} and @code{%type} declarations to pick one of the types
4184 for a terminal or nonterminal symbol (@pxref{Type Decl, ,Nonterminal Symbols}).
4185
4186 As an extension to @acronym{POSIX}, a tag is allowed after the
4187 @code{union}. For example:
4188
4189 @example
4190 @group
4191 %union value @{
4192 double val;
4193 symrec *tptr;
4194 @}
4195 @end group
4196 @end example
4197
4198 @noindent
4199 specifies the union tag @code{value}, so the corresponding C type is
4200 @code{union value}. If you do not specify a tag, it defaults to
4201 @code{YYSTYPE}.
4202
4203 As another extension to @acronym{POSIX}, you may specify multiple
4204 @code{%union} declarations; their contents are concatenated. However,
4205 only the first @code{%union} declaration can specify a tag.
4206
4207 Note that, unlike making a @code{union} declaration in C, you need not write
4208 a semicolon after the closing brace.
4209
4210 Instead of @code{%union}, you can define and use your own union type
4211 @code{YYSTYPE} if your grammar contains at least one
4212 @samp{<@var{type}>} tag. For example, you can put the following into
4213 a header file @file{parser.h}:
4214
4215 @example
4216 @group
4217 union YYSTYPE @{
4218 double val;
4219 symrec *tptr;
4220 @};
4221 typedef union YYSTYPE YYSTYPE;
4222 @end group
4223 @end example
4224
4225 @noindent
4226 and then your grammar can use the following
4227 instead of @code{%union}:
4228
4229 @example
4230 @group
4231 %@{
4232 #include "parser.h"
4233 %@}
4234 %type <val> expr
4235 %token <tptr> ID
4236 @end group
4237 @end example
4238
4239 @node Type Decl
4240 @subsection Nonterminal Symbols
4241 @cindex declaring value types, nonterminals
4242 @cindex value types, nonterminals, declaring
4243 @findex %type
4244
4245 @noindent
4246 When you use @code{%union} to specify multiple value types, you must
4247 declare the value type of each nonterminal symbol for which values are
4248 used. This is done with a @code{%type} declaration, like this:
4249
4250 @example
4251 %type <@var{type}> @var{nonterminal}@dots{}
4252 @end example
4253
4254 @noindent
4255 Here @var{nonterminal} is the name of a nonterminal symbol, and
4256 @var{type} is the name given in the @code{%union} to the alternative
4257 that you want (@pxref{Union Decl, ,The Collection of Value Types}). You
4258 can give any number of nonterminal symbols in the same @code{%type}
4259 declaration, if they have the same value type. Use spaces to separate
4260 the symbol names.
4261
4262 You can also declare the value type of a terminal symbol. To do this,
4263 use the same @code{<@var{type}>} construction in a declaration for the
4264 terminal symbol. All kinds of token declarations allow
4265 @code{<@var{type}>}.
4266
4267 @node Initial Action Decl
4268 @subsection Performing Actions before Parsing
4269 @findex %initial-action
4270
4271 Sometimes your parser needs to perform some initializations before
4272 parsing. The @code{%initial-action} directive allows for such arbitrary
4273 code.
4274
4275 @deffn {Directive} %initial-action @{ @var{code} @}
4276 @findex %initial-action
4277 Declare that the braced @var{code} must be invoked before parsing each time
4278 @code{yyparse} is called. The @var{code} may use @code{$$} and
4279 @code{@@$} --- initial value and location of the lookahead --- and the
4280 @code{%parse-param}.
4281 @end deffn
4282
4283 For instance, if your locations use a file name, you may use
4284
4285 @example
4286 %parse-param @{ char const *file_name @};
4287 %initial-action
4288 @{
4289 @@$.initialize (file_name);
4290 @};
4291 @end example
4292
4293
4294 @node Destructor Decl
4295 @subsection Freeing Discarded Symbols
4296 @cindex freeing discarded symbols
4297 @findex %destructor
4298 @findex <*>
4299 @findex <>
4300 During error recovery (@pxref{Error Recovery}), symbols already pushed
4301 on the stack and tokens coming from the rest of the file are discarded
4302 until the parser falls on its feet. If the parser runs out of memory,
4303 or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
4304 symbols on the stack must be discarded. Even if the parser succeeds, it
4305 must discard the start symbol.
4306
4307 When discarded symbols convey heap based information, this memory is
4308 lost. While this behavior can be tolerable for batch parsers, such as
4309 in traditional compilers, it is unacceptable for programs like shells or
4310 protocol implementations that may parse and execute indefinitely.
4311
4312 The @code{%destructor} directive defines code that is called when a
4313 symbol is automatically discarded.
4314
4315 @deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
4316 @findex %destructor
4317 Invoke the braced @var{code} whenever the parser discards one of the
4318 @var{symbols}.
4319 Within @var{code}, @code{$$} designates the semantic value associated
4320 with the discarded symbol, and @code{@@$} designates its location.
4321 The additional parser parameters are also available (@pxref{Parser Function, ,
4322 The Parser Function @code{yyparse}}).
4323
4324 When a symbol is listed among @var{symbols}, its @code{%destructor} is called a
4325 per-symbol @code{%destructor}.
4326 You may also define a per-type @code{%destructor} by listing a semantic type
4327 tag among @var{symbols}.
4328 In that case, the parser will invoke this @var{code} whenever it discards any
4329 grammar symbol that has that semantic type tag unless that symbol has its own
4330 per-symbol @code{%destructor}.
4331
4332 Finally, you can define two different kinds of default @code{%destructor}s.
4333 (These default forms are experimental.
4334 More user feedback will help to determine whether they should become permanent
4335 features.)
4336 You can place each of @code{<*>} and @code{<>} in the @var{symbols} list of
4337 exactly one @code{%destructor} declaration in your grammar file.
4338 The parser will invoke the @var{code} associated with one of these whenever it
4339 discards any user-defined grammar symbol that has no per-symbol and no per-type
4340 @code{%destructor}.
4341 The parser uses the @var{code} for @code{<*>} in the case of such a grammar
4342 symbol for which you have formally declared a semantic type tag (@code{%type}
4343 counts as such a declaration, but @code{$<tag>$} does not).
4344 The parser uses the @var{code} for @code{<>} in the case of such a grammar
4345 symbol that has no declared semantic type tag.
4346 @end deffn
4347
4348 @noindent
4349 For example:
4350
4351 @smallexample
4352 %union @{ char *string; @}
4353 %token <string> STRING1
4354 %token <string> STRING2
4355 %type <string> string1
4356 %type <string> string2
4357 %union @{ char character; @}
4358 %token <character> CHR
4359 %type <character> chr
4360 %token TAGLESS
4361
4362 %destructor @{ @} <character>
4363 %destructor @{ free ($$); @} <*>
4364 %destructor @{ free ($$); printf ("%d", @@$.first_line); @} STRING1 string1
4365 %destructor @{ printf ("Discarding tagless symbol.\n"); @} <>
4366 @end smallexample
4367
4368 @noindent
4369 guarantees that, when the parser discards any user-defined symbol that has a
4370 semantic type tag other than @code{<character>}, it passes its semantic value
4371 to @code{free} by default.
4372 However, when the parser discards a @code{STRING1} or a @code{string1}, it also
4373 prints its line number to @code{stdout}.
4374 It performs only the second @code{%destructor} in this case, so it invokes
4375 @code{free} only once.
4376 Finally, the parser merely prints a message whenever it discards any symbol,
4377 such as @code{TAGLESS}, that has no semantic type tag.
4378
4379 A Bison-generated parser invokes the default @code{%destructor}s only for
4380 user-defined as opposed to Bison-defined symbols.
4381 For example, the parser will not invoke either kind of default
4382 @code{%destructor} for the special Bison-defined symbols @code{$accept},
4383 @code{$undefined}, or @code{$end} (@pxref{Table of Symbols, ,Bison Symbols}),
4384 none of which you can reference in your grammar.
4385 It also will not invoke either for the @code{error} token (@pxref{Table of
4386 Symbols, ,error}), which is always defined by Bison regardless of whether you
4387 reference it in your grammar.
4388 However, it may invoke one of them for the end token (token 0) if you
4389 redefine it from @code{$end} to, for example, @code{END}:
4390
4391 @smallexample
4392 %token END 0
4393 @end smallexample
4394
4395 @cindex actions in mid-rule
4396 @cindex mid-rule actions
4397 Finally, Bison will never invoke a @code{%destructor} for an unreferenced
4398 mid-rule semantic value (@pxref{Mid-Rule Actions,,Actions in Mid-Rule}).
4399 That is, Bison does not consider a mid-rule to have a semantic value if you do
4400 not reference @code{$$} in the mid-rule's action or @code{$@var{n}} (where
4401 @var{n} is the RHS symbol position of the mid-rule) in any later action in that
4402 rule.
4403 However, if you do reference either, the Bison-generated parser will invoke the
4404 @code{<>} @code{%destructor} whenever it discards the mid-rule symbol.
4405
4406 @ignore
4407 @noindent
4408 In the future, it may be possible to redefine the @code{error} token as a
4409 nonterminal that captures the discarded symbols.
4410 In that case, the parser will invoke the default destructor for it as well.
4411 @end ignore
4412
4413 @sp 1
4414
4415 @cindex discarded symbols
4416 @dfn{Discarded symbols} are the following:
4417
4418 @itemize
4419 @item
4420 stacked symbols popped during the first phase of error recovery,
4421 @item
4422 incoming terminals during the second phase of error recovery,
4423 @item
4424 the current lookahead and the entire stack (except the current
4425 right-hand side symbols) when the parser returns immediately, and
4426 @item
4427 the start symbol, when the parser succeeds.
4428 @end itemize
4429
4430 The parser can @dfn{return immediately} because of an explicit call to
4431 @code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
4432 exhaustion.
4433
4434 Right-hand side symbols of a rule that explicitly triggers a syntax
4435 error via @code{YYERROR} are not discarded automatically. As a rule
4436 of thumb, destructors are invoked only when user actions cannot manage
4437 the memory.
4438
4439 @node Expect Decl
4440 @subsection Suppressing Conflict Warnings
4441 @cindex suppressing conflict warnings
4442 @cindex preventing warnings about conflicts
4443 @cindex warnings, preventing
4444 @cindex conflicts, suppressing warnings of
4445 @findex %expect
4446 @findex %expect-rr
4447
4448 Bison normally warns if there are any conflicts in the grammar
4449 (@pxref{Shift/Reduce, ,Shift/Reduce Conflicts}), but most real grammars
4450 have harmless shift/reduce conflicts which are resolved in a predictable
4451 way and would be difficult to eliminate. It is desirable to suppress
4452 the warning about these conflicts unless the number of conflicts
4453 changes. You can do this with the @code{%expect} declaration.
4454
4455 The declaration looks like this:
4456
4457 @example
4458 %expect @var{n}
4459 @end example
4460
4461 Here @var{n} is a decimal integer. The declaration says there should
4462 be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
4463 Bison reports an error if the number of shift/reduce conflicts differs
4464 from @var{n}, or if there are any reduce/reduce conflicts.
4465
4466 For normal @acronym{LALR}(1) parsers, reduce/reduce conflicts are more
4467 serious, and should be eliminated entirely. Bison will always report
4468 reduce/reduce conflicts for these parsers. With @acronym{GLR}
4469 parsers, however, both kinds of conflicts are routine; otherwise,
4470 there would be no need to use @acronym{GLR} parsing. Therefore, it is
4471 also possible to specify an expected number of reduce/reduce conflicts
4472 in @acronym{GLR} parsers, using the declaration:
4473
4474 @example
4475 %expect-rr @var{n}
4476 @end example
4477
4478 In general, using @code{%expect} involves these steps:
4479
4480 @itemize @bullet
4481 @item
4482 Compile your grammar without @code{%expect}. Use the @samp{-v} option
4483 to get a verbose list of where the conflicts occur. Bison will also
4484 print the number of conflicts.
4485
4486 @item
4487 Check each of the conflicts to make sure that Bison's default
4488 resolution is what you really want. If not, rewrite the grammar and
4489 go back to the beginning.
4490
4491 @item
4492 Add an @code{%expect} declaration, copying the number @var{n} from the
4493 number which Bison printed. With @acronym{GLR} parsers, add an
4494 @code{%expect-rr} declaration as well.
4495 @end itemize
4496
4497 Now Bison will warn you if you introduce an unexpected conflict, but
4498 will keep silent otherwise.
4499
4500 @node Start Decl
4501 @subsection The Start-Symbol
4502 @cindex declaring the start symbol
4503 @cindex start symbol, declaring
4504 @cindex default start symbol
4505 @findex %start
4506
4507 Bison assumes by default that the start symbol for the grammar is the first
4508 nonterminal specified in the grammar specification section. The programmer
4509 may override this restriction with the @code{%start} declaration as follows:
4510
4511 @example
4512 %start @var{symbol}
4513 @end example
4514
4515 @node Pure Decl
4516 @subsection A Pure (Reentrant) Parser
4517 @cindex reentrant parser
4518 @cindex pure parser
4519 @findex %define api.pure
4520
4521 A @dfn{reentrant} program is one which does not alter in the course of
4522 execution; in other words, it consists entirely of @dfn{pure} (read-only)
4523 code. Reentrancy is important whenever asynchronous execution is possible;
4524 for example, a nonreentrant program may not be safe to call from a signal
4525 handler. In systems with multiple threads of control, a nonreentrant
4526 program must be called only within interlocks.
4527
4528 Normally, Bison generates a parser which is not reentrant. This is
4529 suitable for most uses, and it permits compatibility with Yacc. (The
4530 standard Yacc interfaces are inherently nonreentrant, because they use
4531 statically allocated variables for communication with @code{yylex},
4532 including @code{yylval} and @code{yylloc}.)
4533
4534 Alternatively, you can generate a pure, reentrant parser. The Bison
4535 declaration @code{%define api.pure} says that you want the parser to be
4536 reentrant. It looks like this:
4537
4538 @example
4539 %define api.pure
4540 @end example
4541
4542 The result is that the communication variables @code{yylval} and
4543 @code{yylloc} become local variables in @code{yyparse}, and a different
4544 calling convention is used for the lexical analyzer function
4545 @code{yylex}. @xref{Pure Calling, ,Calling Conventions for Pure
4546 Parsers}, for the details of this. The variable @code{yynerrs}
4547 becomes local in @code{yyparse} in pull mode but it becomes a member
4548 of yypstate in push mode. (@pxref{Error Reporting, ,The Error
4549 Reporting Function @code{yyerror}}). The convention for calling
4550 @code{yyparse} itself is unchanged.
4551
4552 Whether the parser is pure has nothing to do with the grammar rules.
4553 You can generate either a pure parser or a nonreentrant parser from any
4554 valid grammar.
4555
4556 @node Push Decl
4557 @subsection A Push Parser
4558 @cindex push parser
4559 @cindex push parser
4560 @findex %define api.push_pull
4561
4562 (The current push parsing interface is experimental and may evolve.
4563 More user feedback will help to stabilize it.)
4564
4565 A pull parser is called once and it takes control until all its input
4566 is completely parsed. A push parser, on the other hand, is called
4567 each time a new token is made available.
4568
4569 A push parser is typically useful when the parser is part of a
4570 main event loop in the client's application. This is typically
4571 a requirement of a GUI, when the main event loop needs to be triggered
4572 within a certain time period.
4573
4574 Normally, Bison generates a pull parser.
4575 The following Bison declaration says that you want the parser to be a push
4576 parser (@pxref{Decl Summary,,%define api.push_pull}):
4577
4578 @example
4579 %define api.push_pull "push"
4580 @end example
4581
4582 In almost all cases, you want to ensure that your push parser is also
4583 a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}). The only
4584 time you should create an impure push parser is to have backwards
4585 compatibility with the impure Yacc pull mode interface. Unless you know
4586 what you are doing, your declarations should look like this:
4587
4588 @example
4589 %define api.pure
4590 %define api.push_pull "push"
4591 @end example
4592
4593 There is a major notable functional difference between the pure push parser
4594 and the impure push parser. It is acceptable for a pure push parser to have
4595 many parser instances, of the same type of parser, in memory at the same time.
4596 An impure push parser should only use one parser at a time.
4597
4598 When a push parser is selected, Bison will generate some new symbols in
4599 the generated parser. @code{yypstate} is a structure that the generated
4600 parser uses to store the parser's state. @code{yypstate_new} is the
4601 function that will create a new parser instance. @code{yypstate_delete}
4602 will free the resources associated with the corresponding parser instance.
4603 Finally, @code{yypush_parse} is the function that should be called whenever a
4604 token is available to provide the parser. A trivial example
4605 of using a pure push parser would look like this:
4606
4607 @example
4608 int status;
4609 yypstate *ps = yypstate_new ();
4610 do @{
4611 status = yypush_parse (ps, yylex (), NULL);
4612 @} while (status == YYPUSH_MORE);
4613 yypstate_delete (ps);
4614 @end example
4615
4616 If the user decided to use an impure push parser, a few things about
4617 the generated parser will change. The @code{yychar} variable becomes
4618 a global variable instead of a variable in the @code{yypush_parse} function.
4619 For this reason, the signature of the @code{yypush_parse} function is
4620 changed to remove the token as a parameter. A nonreentrant push parser
4621 example would thus look like this:
4622
4623 @example
4624 extern int yychar;
4625 int status;
4626 yypstate *ps = yypstate_new ();
4627 do @{
4628 yychar = yylex ();
4629 status = yypush_parse (ps);
4630 @} while (status == YYPUSH_MORE);
4631 yypstate_delete (ps);
4632 @end example
4633
4634 That's it. Notice the next token is put into the global variable @code{yychar}
4635 for use by the next invocation of the @code{yypush_parse} function.
4636
4637 Bison also supports both the push parser interface along with the pull parser
4638 interface in the same generated parser. In order to get this functionality,
4639 you should replace the @code{%define api.push_pull "push"} declaration with the
4640 @code{%define api.push_pull "both"} declaration. Doing this will create all of
4641 the symbols mentioned earlier along with the two extra symbols, @code{yyparse}
4642 and @code{yypull_parse}. @code{yyparse} can be used exactly as it normally
4643 would be used. However, the user should note that it is implemented in the
4644 generated parser by calling @code{yypull_parse}.
4645 This makes the @code{yyparse} function that is generated with the
4646 @code{%define api.push_pull "both"} declaration slower than the normal
4647 @code{yyparse} function. If the user
4648 calls the @code{yypull_parse} function it will parse the rest of the input
4649 stream. It is possible to @code{yypush_parse} tokens to select a subgrammar
4650 and then @code{yypull_parse} the rest of the input stream. If you would like
4651 to switch back and forth between between parsing styles, you would have to
4652 write your own @code{yypull_parse} function that knows when to quit looking
4653 for input. An example of using the @code{yypull_parse} function would look
4654 like this:
4655
4656 @example
4657 yypstate *ps = yypstate_new ();
4658 yypull_parse (ps); /* Will call the lexer */
4659 yypstate_delete (ps);
4660 @end example
4661
4662 Adding the @code{%define api.pure} declaration does exactly the same thing to
4663 the generated parser with @code{%define api.push_pull "both"} as it did for
4664 @code{%define api.push_pull "push"}.
4665
4666 @node Decl Summary
4667 @subsection Bison Declaration Summary
4668 @cindex Bison declaration summary
4669 @cindex declaration summary
4670 @cindex summary, Bison declaration
4671
4672 Here is a summary of the declarations used to define a grammar:
4673
4674 @deffn {Directive} %union
4675 Declare the collection of data types that semantic values may have
4676 (@pxref{Union Decl, ,The Collection of Value Types}).
4677 @end deffn
4678
4679 @deffn {Directive} %token
4680 Declare a terminal symbol (token type name) with no precedence
4681 or associativity specified (@pxref{Token Decl, ,Token Type Names}).
4682 @end deffn
4683
4684 @deffn {Directive} %right
4685 Declare a terminal symbol (token type name) that is right-associative
4686 (@pxref{Precedence Decl, ,Operator Precedence}).
4687 @end deffn
4688
4689 @deffn {Directive} %left
4690 Declare a terminal symbol (token type name) that is left-associative
4691 (@pxref{Precedence Decl, ,Operator Precedence}).
4692 @end deffn
4693
4694 @deffn {Directive} %nonassoc
4695 Declare a terminal symbol (token type name) that is nonassociative
4696 (@pxref{Precedence Decl, ,Operator Precedence}).
4697 Using it in a way that would be associative is a syntax error.
4698 @end deffn
4699
4700 @ifset defaultprec
4701 @deffn {Directive} %default-prec
4702 Assign a precedence to rules lacking an explicit @code{%prec} modifier
4703 (@pxref{Contextual Precedence, ,Context-Dependent Precedence}).
4704 @end deffn
4705 @end ifset
4706
4707 @deffn {Directive} %type
4708 Declare the type of semantic values for a nonterminal symbol
4709 (@pxref{Type Decl, ,Nonterminal Symbols}).
4710 @end deffn
4711
4712 @deffn {Directive} %start
4713 Specify the grammar's start symbol (@pxref{Start Decl, ,The
4714 Start-Symbol}).
4715 @end deffn
4716
4717 @deffn {Directive} %expect
4718 Declare the expected number of shift-reduce conflicts
4719 (@pxref{Expect Decl, ,Suppressing Conflict Warnings}).
4720 @end deffn
4721
4722
4723 @sp 1
4724 @noindent
4725 In order to change the behavior of @command{bison}, use the following
4726 directives:
4727
4728 @deffn {Directive} %code @{@var{code}@}
4729 @findex %code
4730 This is the unqualified form of the @code{%code} directive.
4731 It inserts @var{code} verbatim at a language-dependent default location in the
4732 output@footnote{The default location is actually skeleton-dependent;
4733 writers of non-standard skeletons however should choose the default location
4734 consistently with the behavior of the standard Bison skeletons.}.
4735
4736 @cindex Prologue
4737 For C/C++, the default location is the parser source code
4738 file after the usual contents of the parser header file.
4739 Thus, @code{%code} replaces the traditional Yacc prologue,
4740 @code{%@{@var{code}%@}}, for most purposes.
4741 For a detailed discussion, see @ref{Prologue Alternatives}.
4742
4743 For Java, the default location is inside the parser class.
4744
4745 (Like all the Yacc prologue alternatives, this directive is experimental.
4746 More user feedback will help to determine whether it should become a permanent
4747 feature.)
4748 @end deffn
4749
4750 @deffn {Directive} %code @var{qualifier} @{@var{code}@}
4751 This is the qualified form of the @code{%code} directive.
4752 If you need to specify location-sensitive verbatim @var{code} that does not
4753 belong at the default location selected by the unqualified @code{%code} form,
4754 use this form instead.
4755
4756 @var{qualifier} identifies the purpose of @var{code} and thus the location(s)
4757 where Bison should generate it.
4758 Not all values of @var{qualifier} are available for all target languages:
4759
4760 @itemize @bullet
4761 @item requires
4762 @findex %code requires
4763
4764 @itemize @bullet
4765 @item Language(s): C, C++
4766
4767 @item Purpose: This is the best place to write dependency code required for
4768 @code{YYSTYPE} and @code{YYLTYPE}.
4769 In other words, it's the best place to define types referenced in @code{%union}
4770 directives, and it's the best place to override Bison's default @code{YYSTYPE}
4771 and @code{YYLTYPE} definitions.
4772
4773 @item Location(s): The parser header file and the parser source code file
4774 before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE} definitions.
4775 @end itemize
4776
4777 @item provides
4778 @findex %code provides
4779
4780 @itemize @bullet
4781 @item Language(s): C, C++
4782
4783 @item Purpose: This is the best place to write additional definitions and
4784 declarations that should be provided to other modules.
4785
4786 @item Location(s): The parser header file and the parser source code file after
4787 the Bison-generated @code{YYSTYPE}, @code{YYLTYPE}, and token definitions.
4788 @end itemize
4789
4790 @item top
4791 @findex %code top
4792
4793 @itemize @bullet
4794 @item Language(s): C, C++
4795
4796 @item Purpose: The unqualified @code{%code} or @code{%code requires} should
4797 usually be more appropriate than @code{%code top}.
4798 However, occasionally it is necessary to insert code much nearer the top of the
4799 parser source code file.
4800 For example:
4801
4802 @smallexample
4803 %code top @{
4804 #define _GNU_SOURCE
4805 #include <stdio.h>
4806 @}
4807 @end smallexample
4808
4809 @item Location(s): Near the top of the parser source code file.
4810 @end itemize
4811
4812 @item imports
4813 @findex %code imports
4814
4815 @itemize @bullet
4816 @item Language(s): Java
4817
4818 @item Purpose: This is the best place to write Java import directives.
4819
4820 @item Location(s): The parser Java file after any Java package directive and
4821 before any class definitions.
4822 @end itemize
4823 @end itemize
4824
4825 (Like all the Yacc prologue alternatives, this directive is experimental.
4826 More user feedback will help to determine whether it should become a permanent
4827 feature.)
4828
4829 @cindex Prologue
4830 For a detailed discussion of how to use @code{%code} in place of the
4831 traditional Yacc prologue for C/C++, see @ref{Prologue Alternatives}.
4832 @end deffn
4833
4834 @deffn {Directive} %debug
4835 In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
4836 already defined, so that the debugging facilities are compiled.
4837 @end deffn
4838 @xref{Tracing, ,Tracing Your Parser}.
4839
4840 @deffn {Directive} %define @var{variable}
4841 @deffnx {Directive} %define @var{variable} "@var{value}"
4842 Define a variable to adjust Bison's behavior.
4843 The possible choices for @var{variable}, as well as their meanings, depend on
4844 the selected target language and/or the parser skeleton (@pxref{Decl
4845 Summary,,%language}, @pxref{Decl Summary,,%skeleton}).
4846
4847 Bison will warn if a @var{variable} is defined multiple times.
4848
4849 Omitting @code{"@var{value}"} is always equivalent to specifying it as
4850 @code{""}.
4851
4852 Some @var{variable}s may be used as Booleans.
4853 In this case, Bison will complain if the variable definition does not meet one
4854 of the following four conditions:
4855
4856 @enumerate
4857 @item @code{"@var{value}"} is @code{"true"}
4858
4859 @item @code{"@var{value}"} is omitted (or is @code{""}).
4860 This is equivalent to @code{"true"}.
4861
4862 @item @code{"@var{value}"} is @code{"false"}.
4863
4864 @item @var{variable} is never defined.
4865 In this case, Bison selects a default value, which may depend on the selected
4866 target language and/or parser skeleton.
4867 @end enumerate
4868
4869 Some of the accepted @var{variable}s are:
4870
4871 @itemize @bullet
4872 @item api.pure
4873 @findex %define api.pure
4874
4875 @itemize @bullet
4876 @item Language(s): C
4877
4878 @item Purpose: Request a pure (reentrant) parser program.
4879 @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
4880
4881 @item Accepted Values: Boolean
4882
4883 @item Default Value: @code{"false"}
4884 @end itemize
4885
4886 @item api.push_pull
4887 @findex %define api.push_pull
4888
4889 @itemize @bullet
4890 @item Language(s): C (LALR(1) only)
4891
4892 @item Purpose: Requests a pull parser, a push parser, or both.
4893 @xref{Push Decl, ,A Push Parser}.
4894 (The current push parsing interface is experimental and may evolve.
4895 More user feedback will help to stabilize it.)
4896
4897 @item Accepted Values: @code{"pull"}, @code{"push"}, @code{"both"}
4898
4899 @item Default Value: @code{"pull"}
4900 @end itemize
4901
4902 @item lr.keep_unreachable_states
4903 @findex %define lr.keep_unreachable_states
4904
4905 @itemize @bullet
4906 @item Language(s): all
4907
4908 @item Purpose: Requests that Bison allow unreachable parser states to remain in
4909 the parser tables.
4910 Bison considers a state to be unreachable if there exists no sequence of
4911 transitions from the start state to that state.
4912 A state can become unreachable during conflict resolution if Bison disables a
4913 shift action leading to it from a predecessor state.
4914 Keeping unreachable states is sometimes useful for analysis purposes, but they
4915 are useless in the generated parser.
4916
4917 @item Accepted Values: Boolean
4918
4919 @item Default Value: @code{"false"}
4920
4921 @item Caveats:
4922
4923 @itemize @bullet
4924
4925 @item Unreachable states may contain conflicts and may use rules not used in
4926 any other state.
4927 Thus, keeping unreachable states may induce warnings that are irrelevant to
4928 your parser's behavior, and it may eliminate warnings that are relevant.
4929 Of course, the change in warnings may actually be relevant to a parser table
4930 analysis that wants to keep unreachable states, so this behavior will likely
4931 remain in future Bison releases.
4932
4933 @item While Bison is able to remove unreachable states, it is not guaranteed to
4934 remove other kinds of useless states.
4935 Specifically, when Bison disables reduce actions during conflict resolution,
4936 some goto actions may become useless, and thus some additional states may
4937 become useless.
4938 If Bison were to compute which goto actions were useless and then disable those
4939 actions, it could identify such states as unreachable and then remove those
4940 states.
4941 However, Bison does not compute which goto actions are useless.
4942 @end itemize
4943 @end itemize
4944
4945 @item namespace
4946 @findex %define namespace
4947
4948 @itemize
4949 @item Languages(s): C++
4950
4951 @item Purpose: Specifies the namespace for the parser class.
4952 For example, if you specify:
4953
4954 @smallexample
4955 %define namespace "foo::bar"
4956 @end smallexample
4957
4958 Bison uses @code{foo::bar} verbatim in references such as:
4959
4960 @smallexample
4961 foo::bar::parser::semantic_type
4962 @end smallexample
4963
4964 However, to open a namespace, Bison removes any leading @code{::} and then
4965 splits on any remaining occurrences:
4966
4967 @smallexample
4968 namespace foo @{ namespace bar @{
4969 class position;
4970 class location;
4971 @} @}
4972 @end smallexample
4973
4974 @item Accepted Values: Any absolute or relative C++ namespace reference without
4975 a trailing @code{"::"}.
4976 For example, @code{"foo"} or @code{"::foo::bar"}.
4977
4978 @item Default Value: The value specified by @code{%name-prefix}, which defaults
4979 to @code{yy}.
4980 This usage of @code{%name-prefix} is for backward compatibility and can be
4981 confusing since @code{%name-prefix} also specifies the textual prefix for the
4982 lexical analyzer function.
4983 Thus, if you specify @code{%name-prefix}, it is best to also specify
4984 @code{%define namespace} so that @code{%name-prefix} @emph{only} affects the
4985 lexical analyzer function.
4986 For example, if you specify:
4987
4988 @smallexample
4989 %define namespace "foo"
4990 %name-prefix "bar::"
4991 @end smallexample
4992
4993 The parser namespace is @code{foo} and @code{yylex} is referenced as
4994 @code{bar::lex}.
4995 @end itemize
4996 @end itemize
4997
4998 @end deffn
4999
5000 @deffn {Directive} %defines
5001 Write a header file containing macro definitions for the token type
5002 names defined in the grammar as well as a few other declarations.
5003 If the parser output file is named @file{@var{name}.c} then this file
5004 is named @file{@var{name}.h}.
5005
5006 For C parsers, the output header declares @code{YYSTYPE} unless
5007 @code{YYSTYPE} is already defined as a macro or you have used a
5008 @code{<@var{type}>} tag without using @code{%union}.
5009 Therefore, if you are using a @code{%union}
5010 (@pxref{Multiple Types, ,More Than One Value Type}) with components that
5011 require other definitions, or if you have defined a @code{YYSTYPE} macro
5012 or type definition
5013 (@pxref{Value Type, ,Data Types of Semantic Values}), you need to
5014 arrange for these definitions to be propagated to all modules, e.g., by
5015 putting them in a prerequisite header that is included both by your
5016 parser and by any other module that needs @code{YYSTYPE}.
5017
5018 Unless your parser is pure, the output header declares @code{yylval}
5019 as an external variable. @xref{Pure Decl, ,A Pure (Reentrant)
5020 Parser}.
5021
5022 If you have also used locations, the output header declares
5023 @code{YYLTYPE} and @code{yylloc} using a protocol similar to that of
5024 the @code{YYSTYPE} macro and @code{yylval}. @xref{Locations, ,Tracking
5025 Locations}.
5026
5027 This output file is normally essential if you wish to put the definition
5028 of @code{yylex} in a separate source file, because @code{yylex}
5029 typically needs to be able to refer to the above-mentioned declarations
5030 and to the token type codes. @xref{Token Values, ,Semantic Values of
5031 Tokens}.
5032
5033 @findex %code requires
5034 @findex %code provides
5035 If you have declared @code{%code requires} or @code{%code provides}, the output
5036 header also contains their code.
5037 @xref{Decl Summary, ,%code}.
5038 @end deffn
5039
5040 @deffn {Directive} %defines @var{defines-file}
5041 Same as above, but save in the file @var{defines-file}.
5042 @end deffn
5043
5044 @deffn {Directive} %destructor
5045 Specify how the parser should reclaim the memory associated to
5046 discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
5047 @end deffn
5048
5049 @deffn {Directive} %file-prefix "@var{prefix}"
5050 Specify a prefix to use for all Bison output file names. The names are
5051 chosen as if the input file were named @file{@var{prefix}.y}.
5052 @end deffn
5053
5054 @deffn {Directive} %language "@var{language}"
5055 Specify the programming language for the generated parser. Currently
5056 supported languages include C, C++, and Java.
5057 @var{language} is case-insensitive.
5058
5059 This directive is experimental and its effect may be modified in future
5060 releases.
5061 @end deffn
5062
5063 @deffn {Directive} %locations
5064 Generate the code processing the locations (@pxref{Action Features,
5065 ,Special Features for Use in Actions}). This mode is enabled as soon as
5066 the grammar uses the special @samp{@@@var{n}} tokens, but if your
5067 grammar does not use it, using @samp{%locations} allows for more
5068 accurate syntax error messages.
5069 @end deffn
5070
5071 @deffn {Directive} %name-prefix "@var{prefix}"
5072 Rename the external symbols used in the parser so that they start with
5073 @var{prefix} instead of @samp{yy}. The precise list of symbols renamed
5074 in C parsers
5075 is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
5076 @code{yylval}, @code{yychar}, @code{yydebug}, and
5077 (if locations are used) @code{yylloc}. If you use a push parser,
5078 @code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5079 @code{yypstate_new} and @code{yypstate_delete} will
5080 also be renamed. For example, if you use @samp{%name-prefix "c_"}, the
5081 names become @code{c_parse}, @code{c_lex}, and so on.
5082 For C++ parsers, see the @code{%define namespace} documentation in this
5083 section.
5084 @xref{Multiple Parsers, ,Multiple Parsers in the Same Program}.
5085 @end deffn
5086
5087 @ifset defaultprec
5088 @deffn {Directive} %no-default-prec
5089 Do not assign a precedence to rules lacking an explicit @code{%prec}
5090 modifier (@pxref{Contextual Precedence, ,Context-Dependent
5091 Precedence}).
5092 @end deffn
5093 @end ifset
5094
5095 @deffn {Directive} %no-lines
5096 Don't generate any @code{#line} preprocessor commands in the parser
5097 file. Ordinarily Bison writes these commands in the parser file so that
5098 the C compiler and debuggers will associate errors and object code with
5099 your source file (the grammar file). This directive causes them to
5100 associate errors with the parser file, treating it an independent source
5101 file in its own right.
5102 @end deffn
5103
5104 @deffn {Directive} %output "@var{file}"
5105 Specify @var{file} for the parser file.
5106 @end deffn
5107
5108 @deffn {Directive} %pure-parser
5109 Deprecated version of @code{%define api.pure} (@pxref{Decl Summary, ,%define}),
5110 for which Bison is more careful to warn about unreasonable usage.
5111 @end deffn
5112
5113 @deffn {Directive} %require "@var{version}"
5114 Require version @var{version} or higher of Bison. @xref{Require Decl, ,
5115 Require a Version of Bison}.
5116 @end deffn
5117
5118 @deffn {Directive} %skeleton "@var{file}"
5119 Specify the skeleton to use.
5120
5121 @c You probably don't need this option unless you are developing Bison.
5122 @c You should use @code{%language} if you want to specify the skeleton for a
5123 @c different language, because it is clearer and because it will always choose t he
5124 @c correct skeleton for non-deterministic or push parsers.
5125
5126 If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
5127 file in the Bison installation directory.
5128 If it does, @var{file} is an absolute file name or a file name relative to the
5129 directory of the grammar file.
5130 This is similar to how most shells resolve commands.
5131 @end deffn
5132
5133 @deffn {Directive} %token-table
5134 Generate an array of token names in the parser file. The name of the
5135 array is @code{yytname}; @code{yytname[@var{i}]} is the name of the
5136 token whose internal Bison token code number is @var{i}. The first
5137 three elements of @code{yytname} correspond to the predefined tokens
5138 @code{"$end"},
5139 @code{"error"}, and @code{"$undefined"}; after these come the symbols
5140 defined in the grammar file.
5141
5142 The name in the table includes all the characters needed to represent
5143 the token in Bison. For single-character literals and literal
5144 strings, this includes the surrounding quoting characters and any
5145 escape sequences. For example, the Bison single-character literal
5146 @code{'+'} corresponds to a three-character name, represented in C as
5147 @code{"'+'"}; and the Bison two-character literal string @code{"\\/"}
5148 corresponds to a five-character name, represented in C as
5149 @code{"\"\\\\/\""}.
5150
5151 When you specify @code{%token-table}, Bison also generates macro
5152 definitions for macros @code{YYNTOKENS}, @code{YYNNTS}, and
5153 @code{YYNRULES}, and @code{YYNSTATES}:
5154
5155 @table @code
5156 @item YYNTOKENS
5157 The highest token number, plus one.
5158 @item YYNNTS
5159 The number of nonterminal symbols.
5160 @item YYNRULES
5161 The number of grammar rules,
5162 @item YYNSTATES
5163 The number of parser states (@pxref{Parser States}).
5164 @end table
5165 @end deffn
5166
5167 @deffn {Directive} %verbose
5168 Write an extra output file containing verbose descriptions of the
5169 parser states and what is done for each type of lookahead token in
5170 that state. @xref{Understanding, , Understanding Your Parser}, for more
5171 information.
5172 @end deffn
5173
5174 @deffn {Directive} %yacc
5175 Pretend the option @option{--yacc} was given, i.e., imitate Yacc,
5176 including its naming conventions. @xref{Bison Options}, for more.
5177 @end deffn
5178
5179
5180 @node Multiple Parsers
5181 @section Multiple Parsers in the Same Program
5182
5183 Most programs that use Bison parse only one language and therefore contain
5184 only one Bison parser. But what if you want to parse more than one
5185 language with the same program? Then you need to avoid a name conflict
5186 between different definitions of @code{yyparse}, @code{yylval}, and so on.
5187
5188 The easy way to do this is to use the option @samp{-p @var{prefix}}
5189 (@pxref{Invocation, ,Invoking Bison}). This renames the interface
5190 functions and variables of the Bison parser to start with @var{prefix}
5191 instead of @samp{yy}. You can use this to give each parser distinct
5192 names that do not conflict.
5193
5194 The precise list of symbols renamed is @code{yyparse}, @code{yylex},
5195 @code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yylloc},
5196 @code{yychar} and @code{yydebug}. If you use a push parser,
5197 @code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
5198 @code{yypstate_new} and @code{yypstate_delete} will also be renamed.
5199 For example, if you use @samp{-p c}, the names become @code{cparse},
5200 @code{clex}, and so on.
5201
5202 @strong{All the other variables and macros associated with Bison are not
5203 renamed.} These others are not global; there is no conflict if the same
5204 name is used in different parsers. For example, @code{YYSTYPE} is not
5205 renamed, but defining this in different ways in different parsers causes
5206 no trouble (@pxref{Value Type, ,Data Types of Semantic Values}).
5207
5208 The @samp{-p} option works by adding macro definitions to the beginning
5209 of the parser source file, defining @code{yyparse} as
5210 @code{@var{prefix}parse}, and so on. This effectively substitutes one
5211 name for the other in the entire parser file.
5212
5213 @node Interface
5214 @chapter Parser C-Language Interface
5215 @cindex C-language interface
5216 @cindex interface
5217
5218 The Bison parser is actually a C function named @code{yyparse}. Here we
5219 describe the interface conventions of @code{yyparse} and the other
5220 functions that it needs to use.
5221
5222 Keep in mind that the parser uses many C identifiers starting with
5223 @samp{yy} and @samp{YY} for internal purposes. If you use such an
5224 identifier (aside from those in this manual) in an action or in epilogue
5225 in the grammar file, you are likely to run into trouble.
5226
5227 @menu
5228 * Parser Function:: How to call @code{yyparse} and what it returns.
5229 * Push Parser Function:: How to call @code{yypush_parse} and what it returns.
5230 * Pull Parser Function:: How to call @code{yypull_parse} and what it returns.
5231 * Parser Create Function:: How to call @code{yypstate_new} and what it returns.
5232 * Parser Delete Function:: How to call @code{yypstate_delete} and what it retur ns.
5233 * Lexical:: You must supply a function @code{yylex}
5234 which reads tokens.
5235 * Error Reporting:: You must supply a function @code{yyerror}.
5236 * Action Features:: Special features for use in actions.
5237 * Internationalization:: How to let the parser speak in the user's
5238 native language.
5239 @end menu
5240
5241 @node Parser Function
5242 @section The Parser Function @code{yyparse}
5243 @findex yyparse
5244
5245 You call the function @code{yyparse} to cause parsing to occur. This
5246 function reads tokens, executes actions, and ultimately returns when it
5247 encounters end-of-input or an unrecoverable syntax error. You can also
5248 write an action which directs @code{yyparse} to return immediately
5249 without reading further.
5250
5251
5252 @deftypefun int yyparse (void)
5253 The value returned by @code{yyparse} is 0 if parsing was successful (return
5254 is due to end-of-input).
5255
5256 The value is 1 if parsing failed because of invalid input, i.e., input
5257 that contains a syntax error or that causes @code{YYABORT} to be
5258 invoked.
5259
5260 The value is 2 if parsing failed due to memory exhaustion.
5261 @end deftypefun
5262
5263 In an action, you can cause immediate return from @code{yyparse} by using
5264 these macros:
5265
5266 @defmac YYACCEPT
5267 @findex YYACCEPT
5268 Return immediately with value 0 (to report success).
5269 @end defmac
5270
5271 @defmac YYABORT
5272 @findex YYABORT
5273 Return immediately with value 1 (to report failure).
5274 @end defmac
5275
5276 If you use a reentrant parser, you can optionally pass additional
5277 parameter information to it in a reentrant way. To do so, use the
5278 declaration @code{%parse-param}:
5279
5280 @deffn {Directive} %parse-param @{@var{argument-declaration}@}
5281 @findex %parse-param
5282 Declare that an argument declared by the braced-code
5283 @var{argument-declaration} is an additional @code{yyparse} argument.
5284 The @var{argument-declaration} is used when declaring
5285 functions or prototypes. The last identifier in
5286 @var{argument-declaration} must be the argument name.
5287 @end deffn
5288
5289 Here's an example. Write this in the parser:
5290
5291 @example
5292 %parse-param @{int *nastiness@}
5293 %parse-param @{int *randomness@}
5294 @end example
5295
5296 @noindent
5297 Then call the parser like this:
5298
5299 @example
5300 @{
5301 int nastiness, randomness;
5302 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
5303 value = yyparse (&nastiness, &randomness);
5304 @dots{}
5305 @}
5306 @end example
5307
5308 @noindent
5309 In the grammar actions, use expressions like this to refer to the data:
5310
5311 @example
5312 exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
5313 @end example
5314
5315 @node Push Parser Function
5316 @section The Push Parser Function @code{yypush_parse}
5317 @findex yypush_parse
5318
5319 (The current push parsing interface is experimental and may evolve.
5320 More user feedback will help to stabilize it.)
5321
5322 You call the function @code{yypush_parse} to parse a single token. This
5323 function is available if either the @code{%define api.push_pull "push"} or
5324 @code{%define api.push_pull "both"} declaration is used.
5325 @xref{Push Decl, ,A Push Parser}.
5326
5327 @deftypefun int yypush_parse (yypstate *yyps)
5328 The value returned by @code{yypush_parse} is the same as for yyparse with the
5329 following exception. @code{yypush_parse} will return YYPUSH_MORE if more input
5330 is required to finish parsing the grammar.
5331 @end deftypefun
5332
5333 @node Pull Parser Function
5334 @section The Pull Parser Function @code{yypull_parse}
5335 @findex yypull_parse
5336
5337 (The current push parsing interface is experimental and may evolve.
5338 More user feedback will help to stabilize it.)
5339
5340 You call the function @code{yypull_parse} to parse the rest of the input
5341 stream. This function is available if the @code{%define api.push_pull "both"}
5342 declaration is used.
5343 @xref{Push Decl, ,A Push Parser}.
5344
5345 @deftypefun int yypull_parse (yypstate *yyps)
5346 The value returned by @code{yypull_parse} is the same as for @code{yyparse}.
5347 @end deftypefun
5348
5349 @node Parser Create Function
5350 @section The Parser Create Function @code{yystate_new}
5351 @findex yypstate_new
5352
5353 (The current push parsing interface is experimental and may evolve.
5354 More user feedback will help to stabilize it.)
5355
5356 You call the function @code{yypstate_new} to create a new parser instance.
5357 This function is available if either the @code{%define api.push_pull "push"} or
5358 @code{%define api.push_pull "both"} declaration is used.
5359 @xref{Push Decl, ,A Push Parser}.
5360
5361 @deftypefun yypstate *yypstate_new (void)
5362 The fuction will return a valid parser instance if there was memory available
5363 or 0 if no memory was available.
5364 In impure mode, it will also return 0 if a parser instance is currently
5365 allocated.
5366 @end deftypefun
5367
5368 @node Parser Delete Function
5369 @section The Parser Delete Function @code{yystate_delete}
5370 @findex yypstate_delete
5371
5372 (The current push parsing interface is experimental and may evolve.
5373 More user feedback will help to stabilize it.)
5374
5375 You call the function @code{yypstate_delete} to delete a parser instance.
5376 function is available if either the @code{%define api.push_pull "push"} or
5377 @code{%define api.push_pull "both"} declaration is used.
5378 @xref{Push Decl, ,A Push Parser}.
5379
5380 @deftypefun void yypstate_delete (yypstate *yyps)
5381 This function will reclaim the memory associated with a parser instance.
5382 After this call, you should no longer attempt to use the parser instance.
5383 @end deftypefun
5384
5385 @node Lexical
5386 @section The Lexical Analyzer Function @code{yylex}
5387 @findex yylex
5388 @cindex lexical analyzer
5389
5390 The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
5391 the input stream and returns them to the parser. Bison does not create
5392 this function automatically; you must write it so that @code{yyparse} can
5393 call it. The function is sometimes referred to as a lexical scanner.
5394
5395 In simple programs, @code{yylex} is often defined at the end of the Bison
5396 grammar file. If @code{yylex} is defined in a separate source file, you
5397 need to arrange for the token-type macro definitions to be available there.
5398 To do this, use the @samp{-d} option when you run Bison, so that it will
5399 write these macro definitions into a separate header file
5400 @file{@var{name}.tab.h} which you can include in the other source files
5401 that need it. @xref{Invocation, ,Invoking Bison}.
5402
5403 @menu
5404 * Calling Convention:: How @code{yyparse} calls @code{yylex}.
5405 * Token Values:: How @code{yylex} must return the semantic value
5406 of the token it has read.
5407 * Token Locations:: How @code{yylex} must return the text location
5408 (line number, etc.) of the token, if the
5409 actions want that.
5410 * Pure Calling:: How the calling convention differs in a pure parser
5411 (@pxref{Pure Decl, ,A Pure (Reentrant) Parser}).
5412 @end menu
5413
5414 @node Calling Convention
5415 @subsection Calling Convention for @code{yylex}
5416
5417 The value that @code{yylex} returns must be the positive numeric code
5418 for the type of token it has just found; a zero or negative value
5419 signifies end-of-input.
5420
5421 When a token is referred to in the grammar rules by a name, that name
5422 in the parser file becomes a C macro whose definition is the proper
5423 numeric code for that token type. So @code{yylex} can use the name
5424 to indicate that type. @xref{Symbols}.
5425
5426 When a token is referred to in the grammar rules by a character literal,
5427 the numeric code for that character is also the code for the token type.
5428 So @code{yylex} can simply return that character code, possibly converted
5429 to @code{unsigned char} to avoid sign-extension. The null character
5430 must not be used this way, because its code is zero and that
5431 signifies end-of-input.
5432
5433 Here is an example showing these things:
5434
5435 @example
5436 int
5437 yylex (void)
5438 @{
5439 @dots{}
5440 if (c == EOF) /* Detect end-of-input. */
5441 return 0;
5442 @dots{}
5443 if (c == '+' || c == '-')
5444 return c; /* Assume token type for `+' is '+'. */
5445 @dots{}
5446 return INT; /* Return the type of the token. */
5447 @dots{}
5448 @}
5449 @end example
5450
5451 @noindent
5452 This interface has been designed so that the output from the @code{lex}
5453 utility can be used without change as the definition of @code{yylex}.
5454
5455 If the grammar uses literal string tokens, there are two ways that
5456 @code{yylex} can determine the token type codes for them:
5457
5458 @itemize @bullet
5459 @item
5460 If the grammar defines symbolic token names as aliases for the
5461 literal string tokens, @code{yylex} can use these symbolic names like
5462 all others. In this case, the use of the literal string tokens in
5463 the grammar file has no effect on @code{yylex}.
5464
5465 @item
5466 @code{yylex} can find the multicharacter token in the @code{yytname}
5467 table. The index of the token in the table is the token type's code.
5468 The name of a multicharacter token is recorded in @code{yytname} with a
5469 double-quote, the token's characters, and another double-quote. The
5470 token's characters are escaped as necessary to be suitable as input
5471 to Bison.
5472
5473 Here's code for looking up a multicharacter token in @code{yytname},
5474 assuming that the characters of the token are stored in
5475 @code{token_buffer}, and assuming that the token does not contain any
5476 characters like @samp{"} that require escaping.
5477
5478 @smallexample
5479 for (i = 0; i < YYNTOKENS; i++)
5480 @{
5481 if (yytname[i] != 0
5482 && yytname[i][0] == '"'
5483 && ! strncmp (yytname[i] + 1, token_buffer,
5484 strlen (token_buffer))
5485 && yytname[i][strlen (token_buffer) + 1] == '"'
5486 && yytname[i][strlen (token_buffer) + 2] == 0)
5487 break;
5488 @}
5489 @end smallexample
5490
5491 The @code{yytname} table is generated only if you use the
5492 @code{%token-table} declaration. @xref{Decl Summary}.
5493 @end itemize
5494
5495 @node Token Values
5496 @subsection Semantic Values of Tokens
5497
5498 @vindex yylval
5499 In an ordinary (nonreentrant) parser, the semantic value of the token must
5500 be stored into the global variable @code{yylval}. When you are using
5501 just one data type for semantic values, @code{yylval} has that type.
5502 Thus, if the type is @code{int} (the default), you might write this in
5503 @code{yylex}:
5504
5505 @example
5506 @group
5507 @dots{}
5508 yylval = value; /* Put value onto Bison stack. */
5509 return INT; /* Return the type of the token. */
5510 @dots{}
5511 @end group
5512 @end example
5513
5514 When you are using multiple data types, @code{yylval}'s type is a union
5515 made from the @code{%union} declaration (@pxref{Union Decl, ,The
5516 Collection of Value Types}). So when you store a token's value, you
5517 must use the proper member of the union. If the @code{%union}
5518 declaration looks like this:
5519
5520 @example
5521 @group
5522 %union @{
5523 int intval;
5524 double val;
5525 symrec *tptr;
5526 @}
5527 @end group
5528 @end example
5529
5530 @noindent
5531 then the code in @code{yylex} might look like this:
5532
5533 @example
5534 @group
5535 @dots{}
5536 yylval.intval = value; /* Put value onto Bison stack. */
5537 return INT; /* Return the type of the token. */
5538 @dots{}
5539 @end group
5540 @end example
5541
5542 @node Token Locations
5543 @subsection Textual Locations of Tokens
5544
5545 @vindex yylloc
5546 If you are using the @samp{@@@var{n}}-feature (@pxref{Locations, ,
5547 Tracking Locations}) in actions to keep track of the textual locations
5548 of tokens and groupings, then you must provide this information in
5549 @code{yylex}. The function @code{yyparse} expects to find the textual
5550 location of a token just parsed in the global variable @code{yylloc}.
5551 So @code{yylex} must store the proper data in that variable.
5552
5553 By default, the value of @code{yylloc} is a structure and you need only
5554 initialize the members that are going to be used by the actions. The
5555 four members are called @code{first_line}, @code{first_column},
5556 @code{last_line} and @code{last_column}. Note that the use of this
5557 feature makes the parser noticeably slower.
5558
5559 @tindex YYLTYPE
5560 The data type of @code{yylloc} has the name @code{YYLTYPE}.
5561
5562 @node Pure Calling
5563 @subsection Calling Conventions for Pure Parsers
5564
5565 When you use the Bison declaration @code{%define api.pure} to request a
5566 pure, reentrant parser, the global communication variables @code{yylval}
5567 and @code{yylloc} cannot be used. (@xref{Pure Decl, ,A Pure (Reentrant)
5568 Parser}.) In such parsers the two global variables are replaced by
5569 pointers passed as arguments to @code{yylex}. You must declare them as
5570 shown here, and pass the information back by storing it through those
5571 pointers.
5572
5573 @example
5574 int
5575 yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
5576 @{
5577 @dots{}
5578 *lvalp = value; /* Put value onto Bison stack. */
5579 return INT; /* Return the type of the token. */
5580 @dots{}
5581 @}
5582 @end example
5583
5584 If the grammar file does not use the @samp{@@} constructs to refer to
5585 textual locations, then the type @code{YYLTYPE} will not be defined. In
5586 this case, omit the second argument; @code{yylex} will be called with
5587 only one argument.
5588
5589
5590 If you wish to pass the additional parameter data to @code{yylex}, use
5591 @code{%lex-param} just like @code{%parse-param} (@pxref{Parser
5592 Function}).
5593
5594 @deffn {Directive} lex-param @{@var{argument-declaration}@}
5595 @findex %lex-param
5596 Declare that the braced-code @var{argument-declaration} is an
5597 additional @code{yylex} argument declaration.
5598 @end deffn
5599
5600 For instance:
5601
5602 @example
5603 %parse-param @{int *nastiness@}
5604 %lex-param @{int *nastiness@}
5605 %parse-param @{int *randomness@}
5606 @end example
5607
5608 @noindent
5609 results in the following signature:
5610
5611 @example
5612 int yylex (int *nastiness);
5613 int yyparse (int *nastiness, int *randomness);
5614 @end example
5615
5616 If @code{%define api.pure} is added:
5617
5618 @example
5619 int yylex (YYSTYPE *lvalp, int *nastiness);
5620 int yyparse (int *nastiness, int *randomness);
5621 @end example
5622
5623 @noindent
5624 and finally, if both @code{%define api.pure} and @code{%locations} are used:
5625
5626 @example
5627 int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
5628 int yyparse (int *nastiness, int *randomness);
5629 @end example
5630
5631 @node Error Reporting
5632 @section The Error Reporting Function @code{yyerror}
5633 @cindex error reporting function
5634 @findex yyerror
5635 @cindex parse error
5636 @cindex syntax error
5637
5638 The Bison parser detects a @dfn{syntax error} or @dfn{parse error}
5639 whenever it reads a token which cannot satisfy any syntax rule. An
5640 action in the grammar can also explicitly proclaim an error, using the
5641 macro @code{YYERROR} (@pxref{Action Features, ,Special Features for Use
5642 in Actions}).
5643
5644 The Bison parser expects to report the error by calling an error
5645 reporting function named @code{yyerror}, which you must supply. It is
5646 called by @code{yyparse} whenever a syntax error is found, and it
5647 receives one argument. For a syntax error, the string is normally
5648 @w{@code{"syntax error"}}.
5649
5650 @findex %error-verbose
5651 If you invoke the directive @code{%error-verbose} in the Bison
5652 declarations section (@pxref{Bison Declarations, ,The Bison Declarations
5653 Section}), then Bison provides a more verbose and specific error message
5654 string instead of just plain @w{@code{"syntax error"}}.
5655
5656 The parser can detect one other kind of error: memory exhaustion. This
5657 can happen when the input contains constructions that are very deeply
5658 nested. It isn't likely you will encounter this, since the Bison
5659 parser normally extends its stack automatically up to a very large limit. But
5660 if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
5661 fashion, except that the argument string is @w{@code{"memory exhausted"}}.
5662
5663 In some cases diagnostics like @w{@code{"syntax error"}} are
5664 translated automatically from English to some other language before
5665 they are passed to @code{yyerror}. @xref{Internationalization}.
5666
5667 The following definition suffices in simple programs:
5668
5669 @example
5670 @group
5671 void
5672 yyerror (char const *s)
5673 @{
5674 @end group
5675 @group
5676 fprintf (stderr, "%s\n", s);
5677 @}
5678 @end group
5679 @end example
5680
5681 After @code{yyerror} returns to @code{yyparse}, the latter will attempt
5682 error recovery if you have written suitable error recovery grammar rules
5683 (@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
5684 immediately return 1.
5685
5686 Obviously, in location tracking pure parsers, @code{yyerror} should have
5687 an access to the current location.
5688 This is indeed the case for the @acronym{GLR}
5689 parsers, but not for the Yacc parser, for historical reasons. I.e., if
5690 @samp{%locations %define api.pure} is passed then the prototypes for
5691 @code{yyerror} are:
5692
5693 @example
5694 void yyerror (char const *msg); /* Yacc parsers. */
5695 void yyerror (YYLTYPE *locp, char const *msg); /* GLR parsers. */
5696 @end example
5697
5698 If @samp{%parse-param @{int *nastiness@}} is used, then:
5699
5700 @example
5701 void yyerror (int *nastiness, char const *msg); /* Yacc parsers. */
5702 void yyerror (int *nastiness, char const *msg); /* GLR parsers. */
5703 @end example
5704
5705 Finally, @acronym{GLR} and Yacc parsers share the same @code{yyerror} calling
5706 convention for absolutely pure parsers, i.e., when the calling
5707 convention of @code{yylex} @emph{and} the calling convention of
5708 @code{%define api.pure} are pure.
5709 I.e.:
5710
5711 @example
5712 /* Location tracking. */
5713 %locations
5714 /* Pure yylex. */
5715 %define api.pure
5716 %lex-param @{int *nastiness@}
5717 /* Pure yyparse. */
5718 %parse-param @{int *nastiness@}
5719 %parse-param @{int *randomness@}
5720 @end example
5721
5722 @noindent
5723 results in the following signatures for all the parser kinds:
5724
5725 @example
5726 int yylex (YYSTYPE *lvalp, YYLTYPE *llocp, int *nastiness);
5727 int yyparse (int *nastiness, int *randomness);
5728 void yyerror (YYLTYPE *locp,
5729 int *nastiness, int *randomness,
5730 char const *msg);
5731 @end example
5732
5733 @noindent
5734 The prototypes are only indications of how the code produced by Bison
5735 uses @code{yyerror}. Bison-generated code always ignores the returned
5736 value, so @code{yyerror} can return any type, including @code{void}.
5737 Also, @code{yyerror} can be a variadic function; that is why the
5738 message is always passed last.
5739
5740 Traditionally @code{yyerror} returns an @code{int} that is always
5741 ignored, but this is purely for historical reasons, and @code{void} is
5742 preferable since it more accurately describes the return type for
5743 @code{yyerror}.
5744
5745 @vindex yynerrs
5746 The variable @code{yynerrs} contains the number of syntax errors
5747 reported so far. Normally this variable is global; but if you
5748 request a pure parser (@pxref{Pure Decl, ,A Pure (Reentrant) Parser})
5749 then it is a local variable which only the actions can access.
5750
5751 @node Action Features
5752 @section Special Features for Use in Actions
5753 @cindex summary, action features
5754 @cindex action features summary
5755
5756 Here is a table of Bison constructs, variables and macros that
5757 are useful in actions.
5758
5759 @deffn {Variable} $$
5760 Acts like a variable that contains the semantic value for the
5761 grouping made by the current rule. @xref{Actions}.
5762 @end deffn
5763
5764 @deffn {Variable} $@var{n}
5765 Acts like a variable that contains the semantic value for the
5766 @var{n}th component of the current rule. @xref{Actions}.
5767 @end deffn
5768
5769 @deffn {Variable} $<@var{typealt}>$
5770 Like @code{$$} but specifies alternative @var{typealt} in the union
5771 specified by the @code{%union} declaration. @xref{Action Types, ,Data
5772 Types of Values in Actions}.
5773 @end deffn
5774
5775 @deffn {Variable} $<@var{typealt}>@var{n}
5776 Like @code{$@var{n}} but specifies alternative @var{typealt} in the
5777 union specified by the @code{%union} declaration.
5778 @xref{Action Types, ,Data Types of Values in Actions}.
5779 @end deffn
5780
5781 @deffn {Macro} YYABORT;
5782 Return immediately from @code{yyparse}, indicating failure.
5783 @xref{Parser Function, ,The Parser Function @code{yyparse}}.
5784 @end deffn
5785
5786 @deffn {Macro} YYACCEPT;
5787 Return immediately from @code{yyparse}, indicating success.
5788 @xref{Parser Function, ,The Parser Function @code{yyparse}}.
5789 @end deffn
5790
5791 @deffn {Macro} YYBACKUP (@var{token}, @var{value});
5792 @findex YYBACKUP
5793 Unshift a token. This macro is allowed only for rules that reduce
5794 a single value, and only when there is no lookahead token.
5795 It is also disallowed in @acronym{GLR} parsers.
5796 It installs a lookahead token with token type @var{token} and
5797 semantic value @var{value}; then it discards the value that was
5798 going to be reduced by this rule.
5799
5800 If the macro is used when it is not valid, such as when there is
5801 a lookahead token already, then it reports a syntax error with
5802 a message @samp{cannot back up} and performs ordinary error
5803 recovery.
5804
5805 In either case, the rest of the action is not executed.
5806 @end deffn
5807
5808 @deffn {Macro} YYEMPTY
5809 @vindex YYEMPTY
5810 Value stored in @code{yychar} when there is no lookahead token.
5811 @end deffn
5812
5813 @deffn {Macro} YYEOF
5814 @vindex YYEOF
5815 Value stored in @code{yychar} when the lookahead is the end of the input
5816 stream.
5817 @end deffn
5818
5819 @deffn {Macro} YYERROR;
5820 @findex YYERROR
5821 Cause an immediate syntax error. This statement initiates error
5822 recovery just as if the parser itself had detected an error; however, it
5823 does not call @code{yyerror}, and does not print any message. If you
5824 want to print an error message, call @code{yyerror} explicitly before
5825 the @samp{YYERROR;} statement. @xref{Error Recovery}.
5826 @end deffn
5827
5828 @deffn {Macro} YYRECOVERING
5829 @findex YYRECOVERING
5830 The expression @code{YYRECOVERING ()} yields 1 when the parser
5831 is recovering from a syntax error, and 0 otherwise.
5832 @xref{Error Recovery}.
5833 @end deffn
5834
5835 @deffn {Variable} yychar
5836 Variable containing either the lookahead token, or @code{YYEOF} when the
5837 lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead
5838 has been performed so the next token is not yet known.
5839 Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
5840 Actions}).
5841 @xref{Lookahead, ,Lookahead Tokens}.
5842 @end deffn
5843
5844 @deffn {Macro} yyclearin;
5845 Discard the current lookahead token. This is useful primarily in
5846 error rules.
5847 Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
5848 Semantic Actions}).
5849 @xref{Error Recovery}.
5850 @end deffn
5851
5852 @deffn {Macro} yyerrok;
5853 Resume generating error messages immediately for subsequent syntax
5854 errors. This is useful primarily in error rules.
5855 @xref{Error Recovery}.
5856 @end deffn
5857
5858 @deffn {Variable} yylloc
5859 Variable containing the lookahead token location when @code{yychar} is not set
5860 to @code{YYEMPTY} or @code{YYEOF}.
5861 Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
5862 Actions}).
5863 @xref{Actions and Locations, ,Actions and Locations}.
5864 @end deffn
5865
5866 @deffn {Variable} yylval
5867 Variable containing the lookahead token semantic value when @code{yychar} is
5868 not set to @code{YYEMPTY} or @code{YYEOF}.
5869 Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
5870 Actions}).
5871 @xref{Actions, ,Actions}.
5872 @end deffn
5873
5874 @deffn {Value} @@$
5875 @findex @@$
5876 Acts like a structure variable containing information on the textual location
5877 of the grouping made by the current rule. @xref{Locations, ,
5878 Tracking Locations}.
5879
5880 @c Check if those paragraphs are still useful or not.
5881
5882 @c @example
5883 @c struct @{
5884 @c int first_line, last_line;
5885 @c int first_column, last_column;
5886 @c @};
5887 @c @end example
5888
5889 @c Thus, to get the starting line number of the third component, you would
5890 @c use @samp{@@3.first_line}.
5891
5892 @c In order for the members of this structure to contain valid information,
5893 @c you must make @code{yylex} supply this information about each token.
5894 @c If you need only certain members, then @code{yylex} need only fill in
5895 @c those members.
5896
5897 @c The use of this feature makes the parser noticeably slower.
5898 @end deffn
5899
5900 @deffn {Value} @@@var{n}
5901 @findex @@@var{n}
5902 Acts like a structure variable containing information on the textual location
5903 of the @var{n}th component of the current rule. @xref{Locations, ,
5904 Tracking Locations}.
5905 @end deffn
5906
5907 @node Internationalization
5908 @section Parser Internationalization
5909 @cindex internationalization
5910 @cindex i18n
5911 @cindex NLS
5912 @cindex gettext
5913 @cindex bison-po
5914
5915 A Bison-generated parser can print diagnostics, including error and
5916 tracing messages. By default, they appear in English. However, Bison
5917 also supports outputting diagnostics in the user's native language. To
5918 make this work, the user should set the usual environment variables.
5919 @xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
5920 For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
5921 set the user's locale to French Canadian using the @acronym{UTF}-8
5922 encoding. The exact set of available locales depends on the user's
5923 installation.
5924
5925 The maintainer of a package that uses a Bison-generated parser enables
5926 the internationalization of the parser's output through the following
5927 steps. Here we assume a package that uses @acronym{GNU} Autoconf and
5928 @acronym{GNU} Automake.
5929
5930 @enumerate
5931 @item
5932 @cindex bison-i18n.m4
5933 Into the directory containing the @acronym{GNU} Autoconf macros used
5934 by the package---often called @file{m4}---copy the
5935 @file{bison-i18n.m4} file installed by Bison under
5936 @samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
5937 For example:
5938
5939 @example
5940 cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
5941 @end example
5942
5943 @item
5944 @findex BISON_I18N
5945 @vindex BISON_LOCALEDIR
5946 @vindex YYENABLE_NLS
5947 In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
5948 invocation, add an invocation of @code{BISON_I18N}. This macro is
5949 defined in the file @file{bison-i18n.m4} that you copied earlier. It
5950 causes @samp{configure} to find the value of the
5951 @code{BISON_LOCALEDIR} variable, and it defines the source-language
5952 symbol @code{YYENABLE_NLS} to enable translations in the
5953 Bison-generated parser.
5954
5955 @item
5956 In the @code{main} function of your program, designate the directory
5957 containing Bison's runtime message catalog, through a call to
5958 @samp{bindtextdomain} with domain name @samp{bison-runtime}.
5959 For example:
5960
5961 @example
5962 bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
5963 @end example
5964
5965 Typically this appears after any other call @code{bindtextdomain
5966 (PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
5967 @samp{BISON_LOCALEDIR} to be defined as a string through the
5968 @file{Makefile}.
5969
5970 @item
5971 In the @file{Makefile.am} that controls the compilation of the @code{main}
5972 function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
5973 either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
5974
5975 @example
5976 DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
5977 @end example
5978
5979 or:
5980
5981 @example
5982 AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
5983 @end example
5984
5985 @item
5986 Finally, invoke the command @command{autoreconf} to generate the build
5987 infrastructure.
5988 @end enumerate
5989
5990
5991 @node Algorithm
5992 @chapter The Bison Parser Algorithm
5993 @cindex Bison parser algorithm
5994 @cindex algorithm of parser
5995 @cindex shifting
5996 @cindex reduction
5997 @cindex parser stack
5998 @cindex stack, parser
5999
6000 As Bison reads tokens, it pushes them onto a stack along with their
6001 semantic values. The stack is called the @dfn{parser stack}. Pushing a
6002 token is traditionally called @dfn{shifting}.
6003
6004 For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
6005 @samp{3} to come. The stack will have four elements, one for each token
6006 that was shifted.
6007
6008 But the stack does not always have an element for each token read. When
6009 the last @var{n} tokens and groupings shifted match the components of a
6010 grammar rule, they can be combined according to that rule. This is called
6011 @dfn{reduction}. Those tokens and groupings are replaced on the stack by a
6012 single grouping whose symbol is the result (left hand side) of that rule.
6013 Running the rule's action is part of the process of reduction, because this
6014 is what computes the semantic value of the resulting grouping.
6015
6016 For example, if the infix calculator's parser stack contains this:
6017
6018 @example
6019 1 + 5 * 3
6020 @end example
6021
6022 @noindent
6023 and the next input token is a newline character, then the last three
6024 elements can be reduced to 15 via the rule:
6025
6026 @example
6027 expr: expr '*' expr;
6028 @end example
6029
6030 @noindent
6031 Then the stack contains just these three elements:
6032
6033 @example
6034 1 + 15
6035 @end example
6036
6037 @noindent
6038 At this point, another reduction can be made, resulting in the single value
6039 16. Then the newline token can be shifted.
6040
6041 The parser tries, by shifts and reductions, to reduce the entire input down
6042 to a single grouping whose symbol is the grammar's start-symbol
6043 (@pxref{Language and Grammar, ,Languages and Context-Free Grammars}).
6044
6045 This kind of parser is known in the literature as a bottom-up parser.
6046
6047 @menu
6048 * Lookahead:: Parser looks one token ahead when deciding what to do.
6049 * Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
6050 * Precedence:: Operator precedence works by resolving conflicts.
6051 * Contextual Precedence:: When an operator's precedence depends on context.
6052 * Parser States:: The parser is a finite-state-machine with stack.
6053 * Reduce/Reduce:: When two rules are applicable in the same situation.
6054 * Mystery Conflicts:: Reduce/reduce conflicts that look unjustified.
6055 * Generalized LR Parsing:: Parsing arbitrary context-free grammars.
6056 * Memory Management:: What happens when memory is exhausted. How to avoid it.
6057 @end menu
6058
6059 @node Lookahead
6060 @section Lookahead Tokens
6061 @cindex lookahead token
6062
6063 The Bison parser does @emph{not} always reduce immediately as soon as the
6064 last @var{n} tokens and groupings match a rule. This is because such a
6065 simple strategy is inadequate to handle most languages. Instead, when a
6066 reduction is possible, the parser sometimes ``looks ahead'' at the next
6067 token in order to decide what to do.
6068
6069 When a token is read, it is not immediately shifted; first it becomes the
6070 @dfn{lookahead token}, which is not on the stack. Now the parser can
6071 perform one or more reductions of tokens and groupings on the stack, while
6072 the lookahead token remains off to the side. When no more reductions
6073 should take place, the lookahead token is shifted onto the stack. This
6074 does not mean that all possible reductions have been done; depending on the
6075 token type of the lookahead token, some rules may choose to delay their
6076 application.
6077
6078 Here is a simple case where lookahead is needed. These three rules define
6079 expressions which contain binary addition operators and postfix unary
6080 factorial operators (@samp{!}), and allow parentheses for grouping.
6081
6082 @example
6083 @group
6084 expr: term '+' expr
6085 | term
6086 ;
6087 @end group
6088
6089 @group
6090 term: '(' expr ')'
6091 | term '!'
6092 | NUMBER
6093 ;
6094 @end group
6095 @end example
6096
6097 Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
6098 should be done? If the following token is @samp{)}, then the first three
6099 tokens must be reduced to form an @code{expr}. This is the only valid
6100 course, because shifting the @samp{)} would produce a sequence of symbols
6101 @w{@code{term ')'}}, and no rule allows this.
6102
6103 If the following token is @samp{!}, then it must be shifted immediately so
6104 that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
6105 parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
6106 @code{expr}. It would then be impossible to shift the @samp{!} because
6107 doing so would produce on the stack the sequence of symbols @code{expr
6108 '!'}. No rule allows that sequence.
6109
6110 @vindex yychar
6111 @vindex yylval
6112 @vindex yylloc
6113 The lookahead token is stored in the variable @code{yychar}.
6114 Its semantic value and location, if any, are stored in the variables
6115 @code{yylval} and @code{yylloc}.
6116 @xref{Action Features, ,Special Features for Use in Actions}.
6117
6118 @node Shift/Reduce
6119 @section Shift/Reduce Conflicts
6120 @cindex conflicts
6121 @cindex shift/reduce conflicts
6122 @cindex dangling @code{else}
6123 @cindex @code{else}, dangling
6124
6125 Suppose we are parsing a language which has if-then and if-then-else
6126 statements, with a pair of rules like this:
6127
6128 @example
6129 @group
6130 if_stmt:
6131 IF expr THEN stmt
6132 | IF expr THEN stmt ELSE stmt
6133 ;
6134 @end group
6135 @end example
6136
6137 @noindent
6138 Here we assume that @code{IF}, @code{THEN} and @code{ELSE} are
6139 terminal symbols for specific keyword tokens.
6140
6141 When the @code{ELSE} token is read and becomes the lookahead token, the
6142 contents of the stack (assuming the input is valid) are just right for
6143 reduction by the first rule. But it is also legitimate to shift the
6144 @code{ELSE}, because that would lead to eventual reduction by the second
6145 rule.
6146
6147 This situation, where either a shift or a reduction would be valid, is
6148 called a @dfn{shift/reduce conflict}. Bison is designed to resolve
6149 these conflicts by choosing to shift, unless otherwise directed by
6150 operator precedence declarations. To see the reason for this, let's
6151 contrast it with the other alternative.
6152
6153 Since the parser prefers to shift the @code{ELSE}, the result is to attach
6154 the else-clause to the innermost if-statement, making these two inputs
6155 equivalent:
6156
6157 @example
6158 if x then if y then win (); else lose;
6159
6160 if x then do; if y then win (); else lose; end;
6161 @end example
6162
6163 But if the parser chose to reduce when possible rather than shift, the
6164 result would be to attach the else-clause to the outermost if-statement,
6165 making these two inputs equivalent:
6166
6167 @example
6168 if x then if y then win (); else lose;
6169
6170 if x then do; if y then win (); end; else lose;
6171 @end example
6172
6173 The conflict exists because the grammar as written is ambiguous: either
6174 parsing of the simple nested if-statement is legitimate. The established
6175 convention is that these ambiguities are resolved by attaching the
6176 else-clause to the innermost if-statement; this is what Bison accomplishes
6177 by choosing to shift rather than reduce. (It would ideally be cleaner to
6178 write an unambiguous grammar, but that is very hard to do in this case.)
6179 This particular ambiguity was first encountered in the specifications of
6180 Algol 60 and is called the ``dangling @code{else}'' ambiguity.
6181
6182 To avoid warnings from Bison about predictable, legitimate shift/reduce
6183 conflicts, use the @code{%expect @var{n}} declaration. There will be no
6184 warning as long as the number of shift/reduce conflicts is exactly @var{n}.
6185 @xref{Expect Decl, ,Suppressing Conflict Warnings}.
6186
6187 The definition of @code{if_stmt} above is solely to blame for the
6188 conflict, but the conflict does not actually appear without additional
6189 rules. Here is a complete Bison input file that actually manifests the
6190 conflict:
6191
6192 @example
6193 @group
6194 %token IF THEN ELSE variable
6195 %%
6196 @end group
6197 @group
6198 stmt: expr
6199 | if_stmt
6200 ;
6201 @end group
6202
6203 @group
6204 if_stmt:
6205 IF expr THEN stmt
6206 | IF expr THEN stmt ELSE stmt
6207 ;
6208 @end group
6209
6210 expr: variable
6211 ;
6212 @end example
6213
6214 @node Precedence
6215 @section Operator Precedence
6216 @cindex operator precedence
6217 @cindex precedence of operators
6218
6219 Another situation where shift/reduce conflicts appear is in arithmetic
6220 expressions. Here shifting is not always the preferred resolution; the
6221 Bison declarations for operator precedence allow you to specify when to
6222 shift and when to reduce.
6223
6224 @menu
6225 * Why Precedence:: An example showing why precedence is needed.
6226 * Using Precedence:: How to specify precedence in Bison grammars.
6227 * Precedence Examples:: How these features are used in the previous example.
6228 * How Precedence:: How they work.
6229 @end menu
6230
6231 @node Why Precedence
6232 @subsection When Precedence is Needed
6233
6234 Consider the following ambiguous grammar fragment (ambiguous because the
6235 input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
6236
6237 @example
6238 @group
6239 expr: expr '-' expr
6240 | expr '*' expr
6241 | expr '<' expr
6242 | '(' expr ')'
6243 @dots{}
6244 ;
6245 @end group
6246 @end example
6247
6248 @noindent
6249 Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
6250 should it reduce them via the rule for the subtraction operator? It
6251 depends on the next token. Of course, if the next token is @samp{)}, we
6252 must reduce; shifting is invalid because no single rule can reduce the
6253 token sequence @w{@samp{- 2 )}} or anything starting with that. But if
6254 the next token is @samp{*} or @samp{<}, we have a choice: either
6255 shifting or reduction would allow the parse to complete, but with
6256 different results.
6257
6258 To decide which one Bison should do, we must consider the results. If
6259 the next operator token @var{op} is shifted, then it must be reduced
6260 first in order to permit another opportunity to reduce the difference.
6261 The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
6262 hand, if the subtraction is reduced before shifting @var{op}, the result
6263 is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
6264 reduce should depend on the relative precedence of the operators
6265 @samp{-} and @var{op}: @samp{*} should be shifted first, but not
6266 @samp{<}.
6267
6268 @cindex associativity
6269 What about input such as @w{@samp{1 - 2 - 5}}; should this be
6270 @w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
6271 operators we prefer the former, which is called @dfn{left association}.
6272 The latter alternative, @dfn{right association}, is desirable for
6273 assignment operators. The choice of left or right association is a
6274 matter of whether the parser chooses to shift or reduce when the stack
6275 contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting
6276 makes right-associativity.
6277
6278 @node Using Precedence
6279 @subsection Specifying Operator Precedence
6280 @findex %left
6281 @findex %right
6282 @findex %nonassoc
6283
6284 Bison allows you to specify these choices with the operator precedence
6285 declarations @code{%left} and @code{%right}. Each such declaration
6286 contains a list of tokens, which are operators whose precedence and
6287 associativity is being declared. The @code{%left} declaration makes all
6288 those operators left-associative and the @code{%right} declaration makes
6289 them right-associative. A third alternative is @code{%nonassoc}, which
6290 declares that it is a syntax error to find the same operator twice ``in a
6291 row''.
6292
6293 The relative precedence of different operators is controlled by the
6294 order in which they are declared. The first @code{%left} or
6295 @code{%right} declaration in the file declares the operators whose
6296 precedence is lowest, the next such declaration declares the operators
6297 whose precedence is a little higher, and so on.
6298
6299 @node Precedence Examples
6300 @subsection Precedence Examples
6301
6302 In our example, we would want the following declarations:
6303
6304 @example
6305 %left '<'
6306 %left '-'
6307 %left '*'
6308 @end example
6309
6310 In a more complete example, which supports other operators as well, we
6311 would declare them in groups of equal precedence. For example, @code{'+'} is
6312 declared with @code{'-'}:
6313
6314 @example
6315 %left '<' '>' '=' NE LE GE
6316 %left '+' '-'
6317 %left '*' '/'
6318 @end example
6319
6320 @noindent
6321 (Here @code{NE} and so on stand for the operators for ``not equal''
6322 and so on. We assume that these tokens are more than one character long
6323 and therefore are represented by names, not character literals.)
6324
6325 @node How Precedence
6326 @subsection How Precedence Works
6327
6328 The first effect of the precedence declarations is to assign precedence
6329 levels to the terminal symbols declared. The second effect is to assign
6330 precedence levels to certain rules: each rule gets its precedence from
6331 the last terminal symbol mentioned in the components. (You can also
6332 specify explicitly the precedence of a rule. @xref{Contextual
6333 Precedence, ,Context-Dependent Precedence}.)
6334
6335 Finally, the resolution of conflicts works by comparing the precedence
6336 of the rule being considered with that of the lookahead token. If the
6337 token's precedence is higher, the choice is to shift. If the rule's
6338 precedence is higher, the choice is to reduce. If they have equal
6339 precedence, the choice is made based on the associativity of that
6340 precedence level. The verbose output file made by @samp{-v}
6341 (@pxref{Invocation, ,Invoking Bison}) says how each conflict was
6342 resolved.
6343
6344 Not all rules and not all tokens have precedence. If either the rule or
6345 the lookahead token has no precedence, then the default is to shift.
6346
6347 @node Contextual Precedence
6348 @section Context-Dependent Precedence
6349 @cindex context-dependent precedence
6350 @cindex unary operator precedence
6351 @cindex precedence, context-dependent
6352 @cindex precedence, unary operator
6353 @findex %prec
6354
6355 Often the precedence of an operator depends on the context. This sounds
6356 outlandish at first, but it is really very common. For example, a minus
6357 sign typically has a very high precedence as a unary operator, and a
6358 somewhat lower precedence (lower than multiplication) as a binary operator.
6359
6360 The Bison precedence declarations, @code{%left}, @code{%right} and
6361 @code{%nonassoc}, can only be used once for a given token; so a token has
6362 only one precedence declared in this way. For context-dependent
6363 precedence, you need to use an additional mechanism: the @code{%prec}
6364 modifier for rules.
6365
6366 The @code{%prec} modifier declares the precedence of a particular rule by
6367 specifying a terminal symbol whose precedence should be used for that rule.
6368 It's not necessary for that symbol to appear otherwise in the rule. The
6369 modifier's syntax is:
6370
6371 @example
6372 %prec @var{terminal-symbol}
6373 @end example
6374
6375 @noindent
6376 and it is written after the components of the rule. Its effect is to
6377 assign the rule the precedence of @var{terminal-symbol}, overriding
6378 the precedence that would be deduced for it in the ordinary way. The
6379 altered rule precedence then affects how conflicts involving that rule
6380 are resolved (@pxref{Precedence, ,Operator Precedence}).
6381
6382 Here is how @code{%prec} solves the problem of unary minus. First, declare
6383 a precedence for a fictitious terminal symbol named @code{UMINUS}. There
6384 are no tokens of this type, but the symbol serves to stand for its
6385 precedence:
6386
6387 @example
6388 @dots{}
6389 %left '+' '-'
6390 %left '*'
6391 %left UMINUS
6392 @end example
6393
6394 Now the precedence of @code{UMINUS} can be used in specific rules:
6395
6396 @example
6397 @group
6398 exp: @dots{}
6399 | exp '-' exp
6400 @dots{}
6401 | '-' exp %prec UMINUS
6402 @end group
6403 @end example
6404
6405 @ifset defaultprec
6406 If you forget to append @code{%prec UMINUS} to the rule for unary
6407 minus, Bison silently assumes that minus has its usual precedence.
6408 This kind of problem can be tricky to debug, since one typically
6409 discovers the mistake only by testing the code.
6410
6411 The @code{%no-default-prec;} declaration makes it easier to discover
6412 this kind of problem systematically. It causes rules that lack a
6413 @code{%prec} modifier to have no precedence, even if the last terminal
6414 symbol mentioned in their components has a declared precedence.
6415
6416 If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
6417 for all rules that participate in precedence conflict resolution.
6418 Then you will see any shift/reduce conflict until you tell Bison how
6419 to resolve it, either by changing your grammar or by adding an
6420 explicit precedence. This will probably add declarations to the
6421 grammar, but it helps to protect against incorrect rule precedences.
6422
6423 The effect of @code{%no-default-prec;} can be reversed by giving
6424 @code{%default-prec;}, which is the default.
6425 @end ifset
6426
6427 @node Parser States
6428 @section Parser States
6429 @cindex finite-state machine
6430 @cindex parser state
6431 @cindex state (of parser)
6432
6433 The function @code{yyparse} is implemented using a finite-state machine.
6434 The values pushed on the parser stack are not simply token type codes; they
6435 represent the entire sequence of terminal and nonterminal symbols at or
6436 near the top of the stack. The current state collects all the information
6437 about previous input which is relevant to deciding what to do next.
6438
6439 Each time a lookahead token is read, the current parser state together
6440 with the type of lookahead token are looked up in a table. This table
6441 entry can say, ``Shift the lookahead token.'' In this case, it also
6442 specifies the new parser state, which is pushed onto the top of the
6443 parser stack. Or it can say, ``Reduce using rule number @var{n}.''
6444 This means that a certain number of tokens or groupings are taken off
6445 the top of the stack, and replaced by one grouping. In other words,
6446 that number of states are popped from the stack, and one new state is
6447 pushed.
6448
6449 There is one other alternative: the table can say that the lookahead token
6450 is erroneous in the current state. This causes error processing to begin
6451 (@pxref{Error Recovery}).
6452
6453 @node Reduce/Reduce
6454 @section Reduce/Reduce Conflicts
6455 @cindex reduce/reduce conflict
6456 @cindex conflicts, reduce/reduce
6457
6458 A reduce/reduce conflict occurs if there are two or more rules that apply
6459 to the same sequence of input. This usually indicates a serious error
6460 in the grammar.
6461
6462 For example, here is an erroneous attempt to define a sequence
6463 of zero or more @code{word} groupings.
6464
6465 @example
6466 sequence: /* empty */
6467 @{ printf ("empty sequence\n"); @}
6468 | maybeword
6469 | sequence word
6470 @{ printf ("added word %s\n", $2); @}
6471 ;
6472
6473 maybeword: /* empty */
6474 @{ printf ("empty maybeword\n"); @}
6475 | word
6476 @{ printf ("single word %s\n", $1); @}
6477 ;
6478 @end example
6479
6480 @noindent
6481 The error is an ambiguity: there is more than one way to parse a single
6482 @code{word} into a @code{sequence}. It could be reduced to a
6483 @code{maybeword} and then into a @code{sequence} via the second rule.
6484 Alternatively, nothing-at-all could be reduced into a @code{sequence}
6485 via the first rule, and this could be combined with the @code{word}
6486 using the third rule for @code{sequence}.
6487
6488 There is also more than one way to reduce nothing-at-all into a
6489 @code{sequence}. This can be done directly via the first rule,
6490 or indirectly via @code{maybeword} and then the second rule.
6491
6492 You might think that this is a distinction without a difference, because it
6493 does not change whether any particular input is valid or not. But it does
6494 affect which actions are run. One parsing order runs the second rule's
6495 action; the other runs the first rule's action and the third rule's action.
6496 In this example, the output of the program changes.
6497
6498 Bison resolves a reduce/reduce conflict by choosing to use the rule that
6499 appears first in the grammar, but it is very risky to rely on this. Every
6500 reduce/reduce conflict must be studied and usually eliminated. Here is the
6501 proper way to define @code{sequence}:
6502
6503 @example
6504 sequence: /* empty */
6505 @{ printf ("empty sequence\n"); @}
6506 | sequence word
6507 @{ printf ("added word %s\n", $2); @}
6508 ;
6509 @end example
6510
6511 Here is another common error that yields a reduce/reduce conflict:
6512
6513 @example
6514 sequence: /* empty */
6515 | sequence words
6516 | sequence redirects
6517 ;
6518
6519 words: /* empty */
6520 | words word
6521 ;
6522
6523 redirects:/* empty */
6524 | redirects redirect
6525 ;
6526 @end example
6527
6528 @noindent
6529 The intention here is to define a sequence which can contain either
6530 @code{word} or @code{redirect} groupings. The individual definitions of
6531 @code{sequence}, @code{words} and @code{redirects} are error-free, but the
6532 three together make a subtle ambiguity: even an empty input can be parsed
6533 in infinitely many ways!
6534
6535 Consider: nothing-at-all could be a @code{words}. Or it could be two
6536 @code{words} in a row, or three, or any number. It could equally well be a
6537 @code{redirects}, or two, or any number. Or it could be a @code{words}
6538 followed by three @code{redirects} and another @code{words}. And so on.
6539
6540 Here are two ways to correct these rules. First, to make it a single level
6541 of sequence:
6542
6543 @example
6544 sequence: /* empty */
6545 | sequence word
6546 | sequence redirect
6547 ;
6548 @end example
6549
6550 Second, to prevent either a @code{words} or a @code{redirects}
6551 from being empty:
6552
6553 @example
6554 sequence: /* empty */
6555 | sequence words
6556 | sequence redirects
6557 ;
6558
6559 words: word
6560 | words word
6561 ;
6562
6563 redirects:redirect
6564 | redirects redirect
6565 ;
6566 @end example
6567
6568 @node Mystery Conflicts
6569 @section Mysterious Reduce/Reduce Conflicts
6570
6571 Sometimes reduce/reduce conflicts can occur that don't look warranted.
6572 Here is an example:
6573
6574 @example
6575 @group
6576 %token ID
6577
6578 %%
6579 def: param_spec return_spec ','
6580 ;
6581 param_spec:
6582 type
6583 | name_list ':' type
6584 ;
6585 @end group
6586 @group
6587 return_spec:
6588 type
6589 | name ':' type
6590 ;
6591 @end group
6592 @group
6593 type: ID
6594 ;
6595 @end group
6596 @group
6597 name: ID
6598 ;
6599 name_list:
6600 name
6601 | name ',' name_list
6602 ;
6603 @end group
6604 @end example
6605
6606 It would seem that this grammar can be parsed with only a single token
6607 of lookahead: when a @code{param_spec} is being read, an @code{ID} is
6608 a @code{name} if a comma or colon follows, or a @code{type} if another
6609 @code{ID} follows. In other words, this grammar is @acronym{LR}(1).
6610
6611 @cindex @acronym{LR}(1)
6612 @cindex @acronym{LALR}(1)
6613 However, Bison, like most parser generators, cannot actually handle all
6614 @acronym{LR}(1) grammars. In this grammar, two contexts, that after
6615 an @code{ID}
6616 at the beginning of a @code{param_spec} and likewise at the beginning of
6617 a @code{return_spec}, are similar enough that Bison assumes they are the
6618 same. They appear similar because the same set of rules would be
6619 active---the rule for reducing to a @code{name} and that for reducing to
6620 a @code{type}. Bison is unable to determine at that stage of processing
6621 that the rules would require different lookahead tokens in the two
6622 contexts, so it makes a single parser state for them both. Combining
6623 the two contexts causes a conflict later. In parser terminology, this
6624 occurrence means that the grammar is not @acronym{LALR}(1).
6625
6626 In general, it is better to fix deficiencies than to document them. But
6627 this particular deficiency is intrinsically hard to fix; parser
6628 generators that can handle @acronym{LR}(1) grammars are hard to write
6629 and tend to
6630 produce parsers that are very large. In practice, Bison is more useful
6631 as it is now.
6632
6633 When the problem arises, you can often fix it by identifying the two
6634 parser states that are being confused, and adding something to make them
6635 look distinct. In the above example, adding one rule to
6636 @code{return_spec} as follows makes the problem go away:
6637
6638 @example
6639 @group
6640 %token BOGUS
6641 @dots{}
6642 %%
6643 @dots{}
6644 return_spec:
6645 type
6646 | name ':' type
6647 /* This rule is never used. */
6648 | ID BOGUS
6649 ;
6650 @end group
6651 @end example
6652
6653 This corrects the problem because it introduces the possibility of an
6654 additional active rule in the context after the @code{ID} at the beginning of
6655 @code{return_spec}. This rule is not active in the corresponding context
6656 in a @code{param_spec}, so the two contexts receive distinct parser states.
6657 As long as the token @code{BOGUS} is never generated by @code{yylex},
6658 the added rule cannot alter the way actual input is parsed.
6659
6660 In this particular example, there is another way to solve the problem:
6661 rewrite the rule for @code{return_spec} to use @code{ID} directly
6662 instead of via @code{name}. This also causes the two confusing
6663 contexts to have different sets of active rules, because the one for
6664 @code{return_spec} activates the altered rule for @code{return_spec}
6665 rather than the one for @code{name}.
6666
6667 @example
6668 param_spec:
6669 type
6670 | name_list ':' type
6671 ;
6672 return_spec:
6673 type
6674 | ID ':' type
6675 ;
6676 @end example
6677
6678 For a more detailed exposition of @acronym{LALR}(1) parsers and parser
6679 generators, please see:
6680 Frank DeRemer and Thomas Pennello, Efficient Computation of
6681 @acronym{LALR}(1) Look-Ahead Sets, @cite{@acronym{ACM} Transactions on
6682 Programming Languages and Systems}, Vol.@: 4, No.@: 4 (October 1982),
6683 pp.@: 615--649 @uref{http://doi.acm.org/10.1145/69622.357187}.
6684
6685 @node Generalized LR Parsing
6686 @section Generalized @acronym{LR} (@acronym{GLR}) Parsing
6687 @cindex @acronym{GLR} parsing
6688 @cindex generalized @acronym{LR} (@acronym{GLR}) parsing
6689 @cindex ambiguous grammars
6690 @cindex nondeterministic parsing
6691
6692 Bison produces @emph{deterministic} parsers that choose uniquely
6693 when to reduce and which reduction to apply
6694 based on a summary of the preceding input and on one extra token of lookahead.
6695 As a result, normal Bison handles a proper subset of the family of
6696 context-free languages.
6697 Ambiguous grammars, since they have strings with more than one possible
6698 sequence of reductions cannot have deterministic parsers in this sense.
6699 The same is true of languages that require more than one symbol of
6700 lookahead, since the parser lacks the information necessary to make a
6701 decision at the point it must be made in a shift-reduce parser.
6702 Finally, as previously mentioned (@pxref{Mystery Conflicts}),
6703 there are languages where Bison's particular choice of how to
6704 summarize the input seen so far loses necessary information.
6705
6706 When you use the @samp{%glr-parser} declaration in your grammar file,
6707 Bison generates a parser that uses a different algorithm, called
6708 Generalized @acronym{LR} (or @acronym{GLR}). A Bison @acronym{GLR}
6709 parser uses the same basic
6710 algorithm for parsing as an ordinary Bison parser, but behaves
6711 differently in cases where there is a shift-reduce conflict that has not
6712 been resolved by precedence rules (@pxref{Precedence}) or a
6713 reduce-reduce conflict. When a @acronym{GLR} parser encounters such a
6714 situation, it
6715 effectively @emph{splits} into a several parsers, one for each possible
6716 shift or reduction. These parsers then proceed as usual, consuming
6717 tokens in lock-step. Some of the stacks may encounter other conflicts
6718 and split further, with the result that instead of a sequence of states,
6719 a Bison @acronym{GLR} parsing stack is what is in effect a tree of states.
6720
6721 In effect, each stack represents a guess as to what the proper parse
6722 is. Additional input may indicate that a guess was wrong, in which case
6723 the appropriate stack silently disappears. Otherwise, the semantics
6724 actions generated in each stack are saved, rather than being executed
6725 immediately. When a stack disappears, its saved semantic actions never
6726 get executed. When a reduction causes two stacks to become equivalent,
6727 their sets of semantic actions are both saved with the state that
6728 results from the reduction. We say that two stacks are equivalent
6729 when they both represent the same sequence of states,
6730 and each pair of corresponding states represents a
6731 grammar symbol that produces the same segment of the input token
6732 stream.
6733
6734 Whenever the parser makes a transition from having multiple
6735 states to having one, it reverts to the normal @acronym{LALR}(1) parsing
6736 algorithm, after resolving and executing the saved-up actions.
6737 At this transition, some of the states on the stack will have semantic
6738 values that are sets (actually multisets) of possible actions. The
6739 parser tries to pick one of the actions by first finding one whose rule
6740 has the highest dynamic precedence, as set by the @samp{%dprec}
6741 declaration. Otherwise, if the alternative actions are not ordered by
6742 precedence, but there the same merging function is declared for both
6743 rules by the @samp{%merge} declaration,
6744 Bison resolves and evaluates both and then calls the merge function on
6745 the result. Otherwise, it reports an ambiguity.
6746
6747 It is possible to use a data structure for the @acronym{GLR} parsing tree that
6748 permits the processing of any @acronym{LALR}(1) grammar in linear time (in the
6749 size of the input), any unambiguous (not necessarily
6750 @acronym{LALR}(1)) grammar in
6751 quadratic worst-case time, and any general (possibly ambiguous)
6752 context-free grammar in cubic worst-case time. However, Bison currently
6753 uses a simpler data structure that requires time proportional to the
6754 length of the input times the maximum number of stacks required for any
6755 prefix of the input. Thus, really ambiguous or nondeterministic
6756 grammars can require exponential time and space to process. Such badly
6757 behaving examples, however, are not generally of practical interest.
6758 Usually, nondeterminism in a grammar is local---the parser is ``in
6759 doubt'' only for a few tokens at a time. Therefore, the current data
6760 structure should generally be adequate. On @acronym{LALR}(1) portions of a
6761 grammar, in particular, it is only slightly slower than with the default
6762 Bison parser.
6763
6764 For a more detailed exposition of @acronym{GLR} parsers, please see: Elizabeth
6765 Scott, Adrian Johnstone and Shamsa Sadaf Hussain, Tomita-Style
6766 Generalised @acronym{LR} Parsers, Royal Holloway, University of
6767 London, Department of Computer Science, TR-00-12,
6768 @uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps },
6769 (2000-12-24).
6770
6771 @node Memory Management
6772 @section Memory Management, and How to Avoid Memory Exhaustion
6773 @cindex memory exhaustion
6774 @cindex memory management
6775 @cindex stack overflow
6776 @cindex parser stack overflow
6777 @cindex overflow of parser stack
6778
6779 The Bison parser stack can run out of memory if too many tokens are shifted and
6780 not reduced. When this happens, the parser function @code{yyparse}
6781 calls @code{yyerror} and then returns 2.
6782
6783 Because Bison parsers have growing stacks, hitting the upper limit
6784 usually results from using a right recursion instead of a left
6785 recursion, @xref{Recursion, ,Recursive Rules}.
6786
6787 @vindex YYMAXDEPTH
6788 By defining the macro @code{YYMAXDEPTH}, you can control how deep the
6789 parser stack can become before memory is exhausted. Define the
6790 macro with a value that is an integer. This value is the maximum number
6791 of tokens that can be shifted (and not reduced) before overflow.
6792
6793 The stack space allowed is not necessarily allocated. If you specify a
6794 large value for @code{YYMAXDEPTH}, the parser normally allocates a small
6795 stack at first, and then makes it bigger by stages as needed. This
6796 increasing allocation happens automatically and silently. Therefore,
6797 you do not need to make @code{YYMAXDEPTH} painfully small merely to save
6798 space for ordinary inputs that do not need much stack.
6799
6800 However, do not allow @code{YYMAXDEPTH} to be a value so large that
6801 arithmetic overflow could occur when calculating the size of the stack
6802 space. Also, do not allow @code{YYMAXDEPTH} to be less than
6803 @code{YYINITDEPTH}.
6804
6805 @cindex default stack limit
6806 The default value of @code{YYMAXDEPTH}, if you do not define it, is
6807 10000.
6808
6809 @vindex YYINITDEPTH
6810 You can control how much stack is allocated initially by defining the
6811 macro @code{YYINITDEPTH} to a positive integer. For the C
6812 @acronym{LALR}(1) parser, this value must be a compile-time constant
6813 unless you are assuming C99 or some other target language or compiler
6814 that allows variable-length arrays. The default is 200.
6815
6816 Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
6817
6818 @c FIXME: C++ output.
6819 Because of semantical differences between C and C++, the
6820 @acronym{LALR}(1) parsers in C produced by Bison cannot grow when compiled
6821 by C++ compilers. In this precise case (compiling a C parser as C++) you are
6822 suggested to grow @code{YYINITDEPTH}. The Bison maintainers hope to fix
6823 this deficiency in a future release.
6824
6825 @node Error Recovery
6826 @chapter Error Recovery
6827 @cindex error recovery
6828 @cindex recovery from errors
6829
6830 It is not usually acceptable to have a program terminate on a syntax
6831 error. For example, a compiler should recover sufficiently to parse the
6832 rest of the input file and check it for errors; a calculator should accept
6833 another expression.
6834
6835 In a simple interactive command parser where each input is one line, it may
6836 be sufficient to allow @code{yyparse} to return 1 on error and have the
6837 caller ignore the rest of the input line when that happens (and then call
6838 @code{yyparse} again). But this is inadequate for a compiler, because it
6839 forgets all the syntactic context leading up to the error. A syntax error
6840 deep within a function in the compiler input should not cause the compiler
6841 to treat the following line like the beginning of a source file.
6842
6843 @findex error
6844 You can define how to recover from a syntax error by writing rules to
6845 recognize the special token @code{error}. This is a terminal symbol that
6846 is always defined (you need not declare it) and reserved for error
6847 handling. The Bison parser generates an @code{error} token whenever a
6848 syntax error happens; if you have provided a rule to recognize this token
6849 in the current context, the parse can continue.
6850
6851 For example:
6852
6853 @example
6854 stmnts: /* empty string */
6855 | stmnts '\n'
6856 | stmnts exp '\n'
6857 | stmnts error '\n'
6858 @end example
6859
6860 The fourth rule in this example says that an error followed by a newline
6861 makes a valid addition to any @code{stmnts}.
6862
6863 What happens if a syntax error occurs in the middle of an @code{exp}? The
6864 error recovery rule, interpreted strictly, applies to the precise sequence
6865 of a @code{stmnts}, an @code{error} and a newline. If an error occurs in
6866 the middle of an @code{exp}, there will probably be some additional tokens
6867 and subexpressions on the stack after the last @code{stmnts}, and there
6868 will be tokens to read before the next newline. So the rule is not
6869 applicable in the ordinary way.
6870
6871 But Bison can force the situation to fit the rule, by discarding part of
6872 the semantic context and part of the input. First it discards states
6873 and objects from the stack until it gets back to a state in which the
6874 @code{error} token is acceptable. (This means that the subexpressions
6875 already parsed are discarded, back to the last complete @code{stmnts}.)
6876 At this point the @code{error} token can be shifted. Then, if the old
6877 lookahead token is not acceptable to be shifted next, the parser reads
6878 tokens and discards them until it finds a token which is acceptable. In
6879 this example, Bison reads and discards input until the next newline so
6880 that the fourth rule can apply. Note that discarded symbols are
6881 possible sources of memory leaks, see @ref{Destructor Decl, , Freeing
6882 Discarded Symbols}, for a means to reclaim this memory.
6883
6884 The choice of error rules in the grammar is a choice of strategies for
6885 error recovery. A simple and useful strategy is simply to skip the rest of
6886 the current input line or current statement if an error is detected:
6887
6888 @example
6889 stmnt: error ';' /* On error, skip until ';' is read. */
6890 @end example
6891
6892 It is also useful to recover to the matching close-delimiter of an
6893 opening-delimiter that has already been parsed. Otherwise the
6894 close-delimiter will probably appear to be unmatched, and generate another,
6895 spurious error message:
6896
6897 @example
6898 primary: '(' expr ')'
6899 | '(' error ')'
6900 @dots{}
6901 ;
6902 @end example
6903
6904 Error recovery strategies are necessarily guesses. When they guess wrong,
6905 one syntax error often leads to another. In the above example, the error
6906 recovery rule guesses that an error is due to bad input within one
6907 @code{stmnt}. Suppose that instead a spurious semicolon is inserted in the
6908 middle of a valid @code{stmnt}. After the error recovery rule recovers
6909 from the first error, another syntax error will be found straightaway,
6910 since the text following the spurious semicolon is also an invalid
6911 @code{stmnt}.
6912
6913 To prevent an outpouring of error messages, the parser will output no error
6914 message for another syntax error that happens shortly after the first; only
6915 after three consecutive input tokens have been successfully shifted will
6916 error messages resume.
6917
6918 Note that rules which accept the @code{error} token may have actions, just
6919 as any other rules can.
6920
6921 @findex yyerrok
6922 You can make error messages resume immediately by using the macro
6923 @code{yyerrok} in an action. If you do this in the error rule's action, no
6924 error messages will be suppressed. This macro requires no arguments;
6925 @samp{yyerrok;} is a valid C statement.
6926
6927 @findex yyclearin
6928 The previous lookahead token is reanalyzed immediately after an error. If
6929 this is unacceptable, then the macro @code{yyclearin} may be used to clear
6930 this token. Write the statement @samp{yyclearin;} in the error rule's
6931 action.
6932 @xref{Action Features, ,Special Features for Use in Actions}.
6933
6934 For example, suppose that on a syntax error, an error handling routine is
6935 called that advances the input stream to some point where parsing should
6936 once again commence. The next symbol returned by the lexical scanner is
6937 probably correct. The previous lookahead token ought to be discarded
6938 with @samp{yyclearin;}.
6939
6940 @vindex YYRECOVERING
6941 The expression @code{YYRECOVERING ()} yields 1 when the parser
6942 is recovering from a syntax error, and 0 otherwise.
6943 Syntax error diagnostics are suppressed while recovering from a syntax
6944 error.
6945
6946 @node Context Dependency
6947 @chapter Handling Context Dependencies
6948
6949 The Bison paradigm is to parse tokens first, then group them into larger
6950 syntactic units. In many languages, the meaning of a token is affected by
6951 its context. Although this violates the Bison paradigm, certain techniques
6952 (known as @dfn{kludges}) may enable you to write Bison parsers for such
6953 languages.
6954
6955 @menu
6956 * Semantic Tokens:: Token parsing can depend on the semantic context.
6957 * Lexical Tie-ins:: Token parsing can depend on the syntactic context.
6958 * Tie-in Recovery:: Lexical tie-ins have implications for how
6959 error recovery rules must be written.
6960 @end menu
6961
6962 (Actually, ``kludge'' means any technique that gets its job done but is
6963 neither clean nor robust.)
6964
6965 @node Semantic Tokens
6966 @section Semantic Info in Token Types
6967
6968 The C language has a context dependency: the way an identifier is used
6969 depends on what its current meaning is. For example, consider this:
6970
6971 @example
6972 foo (x);
6973 @end example
6974
6975 This looks like a function call statement, but if @code{foo} is a typedef
6976 name, then this is actually a declaration of @code{x}. How can a Bison
6977 parser for C decide how to parse this input?
6978
6979 The method used in @acronym{GNU} C is to have two different token types,
6980 @code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
6981 identifier, it looks up the current declaration of the identifier in order
6982 to decide which token type to return: @code{TYPENAME} if the identifier is
6983 declared as a typedef, @code{IDENTIFIER} otherwise.
6984
6985 The grammar rules can then express the context dependency by the choice of
6986 token type to recognize. @code{IDENTIFIER} is accepted as an expression,
6987 but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
6988 @code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
6989 is @emph{not} significant, such as in declarations that can shadow a
6990 typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
6991 accepted---there is one rule for each of the two token types.
6992
6993 This technique is simple to use if the decision of which kinds of
6994 identifiers to allow is made at a place close to where the identifier is
6995 parsed. But in C this is not always so: C allows a declaration to
6996 redeclare a typedef name provided an explicit type has been specified
6997 earlier:
6998
6999 @example
7000 typedef int foo, bar;
7001 int baz (void)
7002 @{
7003 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
7004 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
7005 return foo (bar);
7006 @}
7007 @end example
7008
7009 Unfortunately, the name being declared is separated from the declaration
7010 construct itself by a complicated syntactic structure---the ``declarator''.
7011
7012 As a result, part of the Bison parser for C needs to be duplicated, with
7013 all the nonterminal names changed: once for parsing a declaration in
7014 which a typedef name can be redefined, and once for parsing a
7015 declaration in which that can't be done. Here is a part of the
7016 duplication, with actions omitted for brevity:
7017
7018 @example
7019 initdcl:
7020 declarator maybeasm '='
7021 init
7022 | declarator maybeasm
7023 ;
7024
7025 notype_initdcl:
7026 notype_declarator maybeasm '='
7027 init
7028 | notype_declarator maybeasm
7029 ;
7030 @end example
7031
7032 @noindent
7033 Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
7034 cannot. The distinction between @code{declarator} and
7035 @code{notype_declarator} is the same sort of thing.
7036
7037 There is some similarity between this technique and a lexical tie-in
7038 (described next), in that information which alters the lexical analysis is
7039 changed during parsing by other parts of the program. The difference is
7040 here the information is global, and is used for other purposes in the
7041 program. A true lexical tie-in has a special-purpose flag controlled by
7042 the syntactic context.
7043
7044 @node Lexical Tie-ins
7045 @section Lexical Tie-ins
7046 @cindex lexical tie-in
7047
7048 One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
7049 which is set by Bison actions, whose purpose is to alter the way tokens are
7050 parsed.
7051
7052 For example, suppose we have a language vaguely like C, but with a special
7053 construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
7054 an expression in parentheses in which all integers are hexadecimal. In
7055 particular, the token @samp{a1b} must be treated as an integer rather than
7056 as an identifier if it appears in that context. Here is how you can do it:
7057
7058 @example
7059 @group
7060 %@{
7061 int hexflag;
7062 int yylex (void);
7063 void yyerror (char const *);
7064 %@}
7065 %%
7066 @dots{}
7067 @end group
7068 @group
7069 expr: IDENTIFIER
7070 | constant
7071 | HEX '('
7072 @{ hexflag = 1; @}
7073 expr ')'
7074 @{ hexflag = 0;
7075 $$ = $4; @}
7076 | expr '+' expr
7077 @{ $$ = make_sum ($1, $3); @}
7078 @dots{}
7079 ;
7080 @end group
7081
7082 @group
7083 constant:
7084 INTEGER
7085 | STRING
7086 ;
7087 @end group
7088 @end example
7089
7090 @noindent
7091 Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
7092 it is nonzero, all integers are parsed in hexadecimal, and tokens starting
7093 with letters are parsed as integers if possible.
7094
7095 The declaration of @code{hexflag} shown in the prologue of the parser file
7096 is needed to make it accessible to the actions (@pxref{Prologue, ,The Prologue}) .
7097 You must also write the code in @code{yylex} to obey the flag.
7098
7099 @node Tie-in Recovery
7100 @section Lexical Tie-ins and Error Recovery
7101
7102 Lexical tie-ins make strict demands on any error recovery rules you have.
7103 @xref{Error Recovery}.
7104
7105 The reason for this is that the purpose of an error recovery rule is to
7106 abort the parsing of one construct and resume in some larger construct.
7107 For example, in C-like languages, a typical error recovery rule is to skip
7108 tokens until the next semicolon, and then start a new statement, like this:
7109
7110 @example
7111 stmt: expr ';'
7112 | IF '(' expr ')' stmt @{ @dots{} @}
7113 @dots{}
7114 error ';'
7115 @{ hexflag = 0; @}
7116 ;
7117 @end example
7118
7119 If there is a syntax error in the middle of a @samp{hex (@var{expr})}
7120 construct, this error rule will apply, and then the action for the
7121 completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
7122 remain set for the entire rest of the input, or until the next @code{hex}
7123 keyword, causing identifiers to be misinterpreted as integers.
7124
7125 To avoid this problem the error recovery rule itself clears @code{hexflag}.
7126
7127 There may also be an error recovery rule that works within expressions.
7128 For example, there could be a rule which applies within parentheses
7129 and skips to the close-parenthesis:
7130
7131 @example
7132 @group
7133 expr: @dots{}
7134 | '(' expr ')'
7135 @{ $$ = $2; @}
7136 | '(' error ')'
7137 @dots{}
7138 @end group
7139 @end example
7140
7141 If this rule acts within the @code{hex} construct, it is not going to abort
7142 that construct (since it applies to an inner level of parentheses within
7143 the construct). Therefore, it should not clear the flag: the rest of
7144 the @code{hex} construct should be parsed with the flag still in effect.
7145
7146 What if there is an error recovery rule which might abort out of the
7147 @code{hex} construct or might not, depending on circumstances? There is no
7148 way you can write the action to determine whether a @code{hex} construct is
7149 being aborted or not. So if you are using a lexical tie-in, you had better
7150 make sure your error recovery rules are not of this kind. Each rule must
7151 be such that you can be sure that it always will, or always won't, have to
7152 clear the flag.
7153
7154 @c ================================================== Debugging Your Parser
7155
7156 @node Debugging
7157 @chapter Debugging Your Parser
7158
7159 Developing a parser can be a challenge, especially if you don't
7160 understand the algorithm (@pxref{Algorithm, ,The Bison Parser
7161 Algorithm}). Even so, sometimes a detailed description of the automaton
7162 can help (@pxref{Understanding, , Understanding Your Parser}), or
7163 tracing the execution of the parser can give some insight on why it
7164 behaves improperly (@pxref{Tracing, , Tracing Your Parser}).
7165
7166 @menu
7167 * Understanding:: Understanding the structure of your parser.
7168 * Tracing:: Tracing the execution of your parser.
7169 @end menu
7170
7171 @node Understanding
7172 @section Understanding Your Parser
7173
7174 As documented elsewhere (@pxref{Algorithm, ,The Bison Parser Algorithm})
7175 Bison parsers are @dfn{shift/reduce automata}. In some cases (much more
7176 frequent than one would hope), looking at this automaton is required to
7177 tune or simply fix a parser. Bison provides two different
7178 representation of it, either textually or graphically (as a DOT file).
7179
7180 The textual file is generated when the options @option{--report} or
7181 @option{--verbose} are specified, see @xref{Invocation, , Invoking
7182 Bison}. Its name is made by removing @samp{.tab.c} or @samp{.c} from
7183 the parser output file name, and adding @samp{.output} instead.
7184 Therefore, if the input file is @file{foo.y}, then the parser file is
7185 called @file{foo.tab.c} by default. As a consequence, the verbose
7186 output file is called @file{foo.output}.
7187
7188 The following grammar file, @file{calc.y}, will be used in the sequel:
7189
7190 @example
7191 %token NUM STR
7192 %left '+' '-'
7193 %left '*'
7194 %%
7195 exp: exp '+' exp
7196 | exp '-' exp
7197 | exp '*' exp
7198 | exp '/' exp
7199 | NUM
7200 ;
7201 useless: STR;
7202 %%
7203 @end example
7204
7205 @command{bison} reports:
7206
7207 @example
7208 calc.y: warning: 1 nonterminal and 1 rule useless in grammar
7209 calc.y:11.1-7: warning: nonterminal useless in grammar: useless
7210 calc.y:11.10-12: warning: rule useless in grammar: useless: STR
7211 calc.y: conflicts: 7 shift/reduce
7212 @end example
7213
7214 When given @option{--report=state}, in addition to @file{calc.tab.c}, it
7215 creates a file @file{calc.output} with contents detailed below. The
7216 order of the output and the exact presentation might vary, but the
7217 interpretation is the same.
7218
7219 The first section includes details on conflicts that were solved thanks
7220 to precedence and/or associativity:
7221
7222 @example
7223 Conflict in state 8 between rule 2 and token '+' resolved as reduce.
7224 Conflict in state 8 between rule 2 and token '-' resolved as reduce.
7225 Conflict in state 8 between rule 2 and token '*' resolved as shift.
7226 @exdent @dots{}
7227 @end example
7228
7229 @noindent
7230 The next section lists states that still have conflicts.
7231
7232 @example
7233 State 8 conflicts: 1 shift/reduce
7234 State 9 conflicts: 1 shift/reduce
7235 State 10 conflicts: 1 shift/reduce
7236 State 11 conflicts: 4 shift/reduce
7237 @end example
7238
7239 @noindent
7240 @cindex token, useless
7241 @cindex useless token
7242 @cindex nonterminal, useless
7243 @cindex useless nonterminal
7244 @cindex rule, useless
7245 @cindex useless rule
7246 The next section reports useless tokens, nonterminal and rules. Useless
7247 nonterminals and rules are removed in order to produce a smaller parser,
7248 but useless tokens are preserved, since they might be used by the
7249 scanner (note the difference between ``useless'' and ``unused''
7250 below):
7251
7252 @example
7253 Nonterminals useless in grammar:
7254 useless
7255
7256 Terminals unused in grammar:
7257 STR
7258
7259 Rules useless in grammar:
7260 #6 useless: STR;
7261 @end example
7262
7263 @noindent
7264 The next section reproduces the exact grammar that Bison used:
7265
7266 @example
7267 Grammar
7268
7269 Number, Line, Rule
7270 0 5 $accept -> exp $end
7271 1 5 exp -> exp '+' exp
7272 2 6 exp -> exp '-' exp
7273 3 7 exp -> exp '*' exp
7274 4 8 exp -> exp '/' exp
7275 5 9 exp -> NUM
7276 @end example
7277
7278 @noindent
7279 and reports the uses of the symbols:
7280
7281 @example
7282 Terminals, with rules where they appear
7283
7284 $end (0) 0
7285 '*' (42) 3
7286 '+' (43) 1
7287 '-' (45) 2
7288 '/' (47) 4
7289 error (256)
7290 NUM (258) 5
7291
7292 Nonterminals, with rules where they appear
7293
7294 $accept (8)
7295 on left: 0
7296 exp (9)
7297 on left: 1 2 3 4 5, on right: 0 1 2 3 4
7298 @end example
7299
7300 @noindent
7301 @cindex item
7302 @cindex pointed rule
7303 @cindex rule, pointed
7304 Bison then proceeds onto the automaton itself, describing each state
7305 with it set of @dfn{items}, also known as @dfn{pointed rules}. Each
7306 item is a production rule together with a point (marked by @samp{.})
7307 that the input cursor.
7308
7309 @example
7310 state 0
7311
7312 $accept -> . exp $ (rule 0)
7313
7314 NUM shift, and go to state 1
7315
7316 exp go to state 2
7317 @end example
7318
7319 This reads as follows: ``state 0 corresponds to being at the very
7320 beginning of the parsing, in the initial rule, right before the start
7321 symbol (here, @code{exp}). When the parser returns to this state right
7322 after having reduced a rule that produced an @code{exp}, the control
7323 flow jumps to state 2. If there is no such transition on a nonterminal
7324 symbol, and the lookahead is a @code{NUM}, then this token is shifted on
7325 the parse stack, and the control flow jumps to state 1. Any other
7326 lookahead triggers a syntax error.''
7327
7328 @cindex core, item set
7329 @cindex item set core
7330 @cindex kernel, item set
7331 @cindex item set core
7332 Even though the only active rule in state 0 seems to be rule 0, the
7333 report lists @code{NUM} as a lookahead token because @code{NUM} can be
7334 at the beginning of any rule deriving an @code{exp}. By default Bison
7335 reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
7336 you want to see more detail you can invoke @command{bison} with
7337 @option{--report=itemset} to list all the items, include those that can
7338 be derived:
7339
7340 @example
7341 state 0
7342
7343 $accept -> . exp $ (rule 0)
7344 exp -> . exp '+' exp (rule 1)
7345 exp -> . exp '-' exp (rule 2)
7346 exp -> . exp '*' exp (rule 3)
7347 exp -> . exp '/' exp (rule 4)
7348 exp -> . NUM (rule 5)
7349
7350 NUM shift, and go to state 1
7351
7352 exp go to state 2
7353 @end example
7354
7355 @noindent
7356 In the state 1...
7357
7358 @example
7359 state 1
7360
7361 exp -> NUM . (rule 5)
7362
7363 $default reduce using rule 5 (exp)
7364 @end example
7365
7366 @noindent
7367 the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token
7368 (@samp{$default}), the parser will reduce it. If it was coming from
7369 state 0, then, after this reduction it will return to state 0, and will
7370 jump to state 2 (@samp{exp: go to state 2}).
7371
7372 @example
7373 state 2
7374
7375 $accept -> exp . $ (rule 0)
7376 exp -> exp . '+' exp (rule 1)
7377 exp -> exp . '-' exp (rule 2)
7378 exp -> exp . '*' exp (rule 3)
7379 exp -> exp . '/' exp (rule 4)
7380
7381 $ shift, and go to state 3
7382 '+' shift, and go to state 4
7383 '-' shift, and go to state 5
7384 '*' shift, and go to state 6
7385 '/' shift, and go to state 7
7386 @end example
7387
7388 @noindent
7389 In state 2, the automaton can only shift a symbol. For instance,
7390 because of the item @samp{exp -> exp . '+' exp}, if the lookahead if
7391 @samp{+}, it will be shifted on the parse stack, and the automaton
7392 control will jump to state 4, corresponding to the item @samp{exp -> exp
7393 '+' . exp}. Since there is no default action, any other token than
7394 those listed above will trigger a syntax error.
7395
7396 The state 3 is named the @dfn{final state}, or the @dfn{accepting
7397 state}:
7398
7399 @example
7400 state 3
7401
7402 $accept -> exp $ . (rule 0)
7403
7404 $default accept
7405 @end example
7406
7407 @noindent
7408 the initial rule is completed (the start symbol and the end
7409 of input were read), the parsing exits successfully.
7410
7411 The interpretation of states 4 to 7 is straightforward, and is left to
7412 the reader.
7413
7414 @example
7415 state 4
7416
7417 exp -> exp '+' . exp (rule 1)
7418
7419 NUM shift, and go to state 1
7420
7421 exp go to state 8
7422
7423 state 5
7424
7425 exp -> exp '-' . exp (rule 2)
7426
7427 NUM shift, and go to state 1
7428
7429 exp go to state 9
7430
7431 state 6
7432
7433 exp -> exp '*' . exp (rule 3)
7434
7435 NUM shift, and go to state 1
7436
7437 exp go to state 10
7438
7439 state 7
7440
7441 exp -> exp '/' . exp (rule 4)
7442
7443 NUM shift, and go to state 1
7444
7445 exp go to state 11
7446 @end example
7447
7448 As was announced in beginning of the report, @samp{State 8 conflicts:
7449 1 shift/reduce}:
7450
7451 @example
7452 state 8
7453
7454 exp -> exp . '+' exp (rule 1)
7455 exp -> exp '+' exp . (rule 1)
7456 exp -> exp . '-' exp (rule 2)
7457 exp -> exp . '*' exp (rule 3)
7458 exp -> exp . '/' exp (rule 4)
7459
7460 '*' shift, and go to state 6
7461 '/' shift, and go to state 7
7462
7463 '/' [reduce using rule 1 (exp)]
7464 $default reduce using rule 1 (exp)
7465 @end example
7466
7467 Indeed, there are two actions associated to the lookahead @samp{/}:
7468 either shifting (and going to state 7), or reducing rule 1. The
7469 conflict means that either the grammar is ambiguous, or the parser lacks
7470 information to make the right decision. Indeed the grammar is
7471 ambiguous, as, since we did not specify the precedence of @samp{/}, the
7472 sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
7473 NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
7474 NUM}, which corresponds to reducing rule 1.
7475
7476 Because in @acronym{LALR}(1) parsing a single decision can be made, Bison
7477 arbitrarily chose to disable the reduction, see @ref{Shift/Reduce, ,
7478 Shift/Reduce Conflicts}. Discarded actions are reported in between
7479 square brackets.
7480
7481 Note that all the previous states had a single possible action: either
7482 shifting the next token and going to the corresponding state, or
7483 reducing a single rule. In the other cases, i.e., when shifting
7484 @emph{and} reducing is possible or when @emph{several} reductions are
7485 possible, the lookahead is required to select the action. State 8 is
7486 one such state: if the lookahead is @samp{*} or @samp{/} then the action
7487 is shifting, otherwise the action is reducing rule 1. In other words,
7488 the first two items, corresponding to rule 1, are not eligible when the
7489 lookahead token is @samp{*}, since we specified that @samp{*} has higher
7490 precedence than @samp{+}. More generally, some items are eligible only
7491 with some set of possible lookahead tokens. When run with
7492 @option{--report=lookahead}, Bison specifies these lookahead tokens:
7493
7494 @example
7495 state 8
7496
7497 exp -> exp . '+' exp (rule 1)
7498 exp -> exp '+' exp . [$, '+', '-', '/'] (rule 1)
7499 exp -> exp . '-' exp (rule 2)
7500 exp -> exp . '*' exp (rule 3)
7501 exp -> exp . '/' exp (rule 4)
7502
7503 '*' shift, and go to state 6
7504 '/' shift, and go to state 7
7505
7506 '/' [reduce using rule 1 (exp)]
7507 $default reduce using rule 1 (exp)
7508 @end example
7509
7510 The remaining states are similar:
7511
7512 @example
7513 state 9
7514
7515 exp -> exp . '+' exp (rule 1)
7516 exp -> exp . '-' exp (rule 2)
7517 exp -> exp '-' exp . (rule 2)
7518 exp -> exp . '*' exp (rule 3)
7519 exp -> exp . '/' exp (rule 4)
7520
7521 '*' shift, and go to state 6
7522 '/' shift, and go to state 7
7523
7524 '/' [reduce using rule 2 (exp)]
7525 $default reduce using rule 2 (exp)
7526
7527 state 10
7528
7529 exp -> exp . '+' exp (rule 1)
7530 exp -> exp . '-' exp (rule 2)
7531 exp -> exp . '*' exp (rule 3)
7532 exp -> exp '*' exp . (rule 3)
7533 exp -> exp . '/' exp (rule 4)
7534
7535 '/' shift, and go to state 7
7536
7537 '/' [reduce using rule 3 (exp)]
7538 $default reduce using rule 3 (exp)
7539
7540 state 11
7541
7542 exp -> exp . '+' exp (rule 1)
7543 exp -> exp . '-' exp (rule 2)
7544 exp -> exp . '*' exp (rule 3)
7545 exp -> exp . '/' exp (rule 4)
7546 exp -> exp '/' exp . (rule 4)
7547
7548 '+' shift, and go to state 4
7549 '-' shift, and go to state 5
7550 '*' shift, and go to state 6
7551 '/' shift, and go to state 7
7552
7553 '+' [reduce using rule 4 (exp)]
7554 '-' [reduce using rule 4 (exp)]
7555 '*' [reduce using rule 4 (exp)]
7556 '/' [reduce using rule 4 (exp)]
7557 $default reduce using rule 4 (exp)
7558 @end example
7559
7560 @noindent
7561 Observe that state 11 contains conflicts not only due to the lack of
7562 precedence of @samp{/} with respect to @samp{+}, @samp{-}, and
7563 @samp{*}, but also because the
7564 associativity of @samp{/} is not specified.
7565
7566
7567 @node Tracing
7568 @section Tracing Your Parser
7569 @findex yydebug
7570 @cindex debugging
7571 @cindex tracing the parser
7572
7573 If a Bison grammar compiles properly but doesn't do what you want when it
7574 runs, the @code{yydebug} parser-trace feature can help you figure out why.
7575
7576 There are several means to enable compilation of trace facilities:
7577
7578 @table @asis
7579 @item the macro @code{YYDEBUG}
7580 @findex YYDEBUG
7581 Define the macro @code{YYDEBUG} to a nonzero value when you compile the
7582 parser. This is compliant with @acronym{POSIX} Yacc. You could use
7583 @samp{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
7584 YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue, , The
7585 Prologue}).
7586
7587 @item the option @option{-t}, @option{--debug}
7588 Use the @samp{-t} option when you run Bison (@pxref{Invocation,
7589 ,Invoking Bison}). This is @acronym{POSIX} compliant too.
7590
7591 @item the directive @samp{%debug}
7592 @findex %debug
7593 Add the @code{%debug} directive (@pxref{Decl Summary, ,Bison
7594 Declaration Summary}). This is a Bison extension, which will prove
7595 useful when Bison will output parsers for languages that don't use a
7596 preprocessor. Unless @acronym{POSIX} and Yacc portability matter to
7597 you, this is
7598 the preferred solution.
7599 @end table
7600
7601 We suggest that you always enable the debug option so that debugging is
7602 always possible.
7603
7604 The trace facility outputs messages with macro calls of the form
7605 @code{YYFPRINTF (stderr, @var{format}, @var{args})} where
7606 @var{format} and @var{args} are the usual @code{printf} format and variadic
7607 arguments. If you define @code{YYDEBUG} to a nonzero value but do not
7608 define @code{YYFPRINTF}, @code{<stdio.h>} is automatically included
7609 and @code{YYFPRINTF} is defined to @code{fprintf}.
7610
7611 Once you have compiled the program with trace facilities, the way to
7612 request a trace is to store a nonzero value in the variable @code{yydebug}.
7613 You can do this by making the C code do it (in @code{main}, perhaps), or
7614 you can alter the value with a C debugger.
7615
7616 Each step taken by the parser when @code{yydebug} is nonzero produces a
7617 line or two of trace information, written on @code{stderr}. The trace
7618 messages tell you these things:
7619
7620 @itemize @bullet
7621 @item
7622 Each time the parser calls @code{yylex}, what kind of token was read.
7623
7624 @item
7625 Each time a token is shifted, the depth and complete contents of the
7626 state stack (@pxref{Parser States}).
7627
7628 @item
7629 Each time a rule is reduced, which rule it is, and the complete contents
7630 of the state stack afterward.
7631 @end itemize
7632
7633 To make sense of this information, it helps to refer to the listing file
7634 produced by the Bison @samp{-v} option (@pxref{Invocation, ,Invoking
7635 Bison}). This file shows the meaning of each state in terms of
7636 positions in various rules, and also what each state will do with each
7637 possible input token. As you read the successive trace messages, you
7638 can see that the parser is functioning according to its specification in
7639 the listing file. Eventually you will arrive at the place where
7640 something undesirable happens, and you will see which parts of the
7641 grammar are to blame.
7642
7643 The parser file is a C program and you can use C debuggers on it, but it's
7644 not easy to interpret what it is doing. The parser function is a
7645 finite-state machine interpreter, and aside from the actions it executes
7646 the same code over and over. Only the values of variables show where in
7647 the grammar it is working.
7648
7649 @findex YYPRINT
7650 The debugging information normally gives the token type of each token
7651 read, but not its semantic value. You can optionally define a macro
7652 named @code{YYPRINT} to provide a way to print the value. If you define
7653 @code{YYPRINT}, it should take three arguments. The parser will pass a
7654 standard I/O stream, the numeric code for the token type, and the token
7655 value (from @code{yylval}).
7656
7657 Here is an example of @code{YYPRINT} suitable for the multi-function
7658 calculator (@pxref{Mfcalc Declarations, ,Declarations for @code{mfcalc}}):
7659
7660 @smallexample
7661 %@{
7662 static void print_token_value (FILE *, int, YYSTYPE);
7663 #define YYPRINT(file, type, value) print_token_value (file, type, value)
7664 %@}
7665
7666 @dots{} %% @dots{} %% @dots{}
7667
7668 static void
7669 print_token_value (FILE *file, int type, YYSTYPE value)
7670 @{
7671 if (type == VAR)
7672 fprintf (file, "%s", value.tptr->name);
7673 else if (type == NUM)
7674 fprintf (file, "%d", value.val);
7675 @}
7676 @end smallexample
7677
7678 @c ================================================= Invoking Bison
7679
7680 @node Invocation
7681 @chapter Invoking Bison
7682 @cindex invoking Bison
7683 @cindex Bison invocation
7684 @cindex options for invoking Bison
7685
7686 The usual way to invoke Bison is as follows:
7687
7688 @example
7689 bison @var{infile}
7690 @end example
7691
7692 Here @var{infile} is the grammar file name, which usually ends in
7693 @samp{.y}. The parser file's name is made by replacing the @samp{.y}
7694 with @samp{.tab.c} and removing any leading directory. Thus, the
7695 @samp{bison foo.y} file name yields
7696 @file{foo.tab.c}, and the @samp{bison hack/foo.y} file name yields
7697 @file{foo.tab.c}. It's also possible, in case you are writing
7698 C++ code instead of C in your grammar file, to name it @file{foo.ypp}
7699 or @file{foo.y++}. Then, the output files will take an extension like
7700 the given one as input (respectively @file{foo.tab.cpp} and
7701 @file{foo.tab.c++}).
7702 This feature takes effect with all options that manipulate file names like
7703 @samp{-o} or @samp{-d}.
7704
7705 For example :
7706
7707 @example
7708 bison -d @var{infile.yxx}
7709 @end example
7710 @noindent
7711 will produce @file{infile.tab.cxx} and @file{infile.tab.hxx}, and
7712
7713 @example
7714 bison -d -o @var{output.c++} @var{infile.y}
7715 @end example
7716 @noindent
7717 will produce @file{output.c++} and @file{outfile.h++}.
7718
7719 For compatibility with @acronym{POSIX}, the standard Bison
7720 distribution also contains a shell script called @command{yacc} that
7721 invokes Bison with the @option{-y} option.
7722
7723 @menu
7724 * Bison Options:: All the options described in detail,
7725 in alphabetical order by short options.
7726 * Option Cross Key:: Alphabetical list of long options.
7727 * Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
7728 @end menu
7729
7730 @node Bison Options
7731 @section Bison Options
7732
7733 Bison supports both traditional single-letter options and mnemonic long
7734 option names. Long option names are indicated with @samp{--} instead of
7735 @samp{-}. Abbreviations for option names are allowed as long as they
7736 are unique. When a long option takes an argument, like
7737 @samp{--file-prefix}, connect the option name and the argument with
7738 @samp{=}.
7739
7740 Here is a list of options that can be used with Bison, alphabetized by
7741 short option. It is followed by a cross key alphabetized by long
7742 option.
7743
7744 @c Please, keep this ordered as in `bison --help'.
7745 @noindent
7746 Operations modes:
7747 @table @option
7748 @item -h
7749 @itemx --help
7750 Print a summary of the command-line options to Bison and exit.
7751
7752 @item -V
7753 @itemx --version
7754 Print the version number of Bison and exit.
7755
7756 @item --print-localedir
7757 Print the name of the directory containing locale-dependent data.
7758
7759 @item --print-datadir
7760 Print the name of the directory containing skeletons and XSLT.
7761
7762 @item -y
7763 @itemx --yacc
7764 Act more like the traditional Yacc command. This can cause
7765 different diagnostics to be generated, and may change behavior in
7766 other minor ways. Most importantly, imitate Yacc's output
7767 file name conventions, so that the parser output file is called
7768 @file{y.tab.c}, and the other outputs are called @file{y.output} and
7769 @file{y.tab.h}.
7770 Also, if generating an @acronym{LALR}(1) parser in C, generate @code{#define}
7771 statements in addition to an @code{enum} to associate token numbers with token
7772 names.
7773 Thus, the following shell script can substitute for Yacc, and the Bison
7774 distribution contains such a script for compatibility with @acronym{POSIX}:
7775
7776 @example
7777 #! /bin/sh
7778 bison -y "$@@"
7779 @end example
7780
7781 The @option{-y}/@option{--yacc} option is intended for use with
7782 traditional Yacc grammars. If your grammar uses a Bison extension
7783 like @samp{%glr-parser}, Bison might not be Yacc-compatible even if
7784 this option is specified.
7785
7786 @item -W
7787 @itemx --warnings
7788 Output warnings falling in @var{category}. @var{category} can be one
7789 of:
7790 @table @code
7791 @item midrule-values
7792 Warn about mid-rule values that are set but not used within any of the actions
7793 of the parent rule.
7794 For example, warn about unused @code{$2} in:
7795
7796 @example
7797 exp: '1' @{ $$ = 1; @} '+' exp @{ $$ = $1 + $4; @};
7798 @end example
7799
7800 Also warn about mid-rule values that are used but not set.
7801 For example, warn about unset @code{$$} in the mid-rule action in:
7802
7803 @example
7804 exp: '1' @{ $1 = 1; @} '+' exp @{ $$ = $2 + $4; @};
7805 @end example
7806
7807 These warnings are not enabled by default since they sometimes prove to
7808 be false alarms in existing grammars employing the Yacc constructs
7809 @code{$0} or @code{$-@var{n}} (where @var{n} is some positive integer).
7810
7811
7812 @item yacc
7813 Incompatibilities with @acronym{POSIX} Yacc.
7814
7815 @item all
7816 All the warnings.
7817 @item none
7818 Turn off all the warnings.
7819 @item error
7820 Treat warnings as errors.
7821 @end table
7822
7823 A category can be turned off by prefixing its name with @samp{no-}. For
7824 instance, @option{-Wno-syntax} will hide the warnings about unused
7825 variables.
7826 @end table
7827
7828 @noindent
7829 Tuning the parser:
7830
7831 @table @option
7832 @item -t
7833 @itemx --debug
7834 In the parser file, define the macro @code{YYDEBUG} to 1 if it is not
7835 already defined, so that the debugging facilities are compiled.
7836 @xref{Tracing, ,Tracing Your Parser}.
7837
7838 @item -L @var{language}
7839 @itemx --language=@var{language}
7840 Specify the programming language for the generated parser, as if
7841 @code{%language} was specified (@pxref{Decl Summary, , Bison Declaration
7842 Summary}). Currently supported languages include C, C++, and Java.
7843 @var{language} is case-insensitive.
7844
7845 This option is experimental and its effect may be modified in future
7846 releases.
7847
7848 @item --locations
7849 Pretend that @code{%locations} was specified. @xref{Decl Summary}.
7850
7851 @item -p @var{prefix}
7852 @itemx --name-prefix=@var{prefix}
7853 Pretend that @code{%name-prefix "@var{prefix}"} was specified.
7854 @xref{Decl Summary}.
7855
7856 @item -l
7857 @itemx --no-lines
7858 Don't put any @code{#line} preprocessor commands in the parser file.
7859 Ordinarily Bison puts them in the parser file so that the C compiler
7860 and debuggers will associate errors with your source file, the
7861 grammar file. This option causes them to associate errors with the
7862 parser file, treating it as an independent source file in its own right.
7863
7864 @item -S @var{file}
7865 @itemx --skeleton=@var{file}
7866 Specify the skeleton to use, similar to @code{%skeleton}
7867 (@pxref{Decl Summary, , Bison Declaration Summary}).
7868
7869 @c You probably don't need this option unless you are developing Bison.
7870 @c You should use @option{--language} if you want to specify the skeleton for a
7871 @c different language, because it is clearer and because it will always
7872 @c choose the correct skeleton for non-deterministic or push parsers.
7873
7874 If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
7875 file in the Bison installation directory.
7876 If it does, @var{file} is an absolute file name or a file name relative to the
7877 current working directory.
7878 This is similar to how most shells resolve commands.
7879
7880 @item -k
7881 @itemx --token-table
7882 Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
7883 @end table
7884
7885 @noindent
7886 Adjust the output:
7887
7888 @table @option
7889 @item --defines[=@var{file}]
7890 Pretend that @code{%defines} was specified, i.e., write an extra output
7891 file containing macro definitions for the token type names defined in
7892 the grammar, as well as a few other declarations. @xref{Decl Summary}.
7893
7894 @item -d
7895 This is the same as @code{--defines} except @code{-d} does not accept a
7896 @var{file} argument since POSIX Yacc requires that @code{-d} can be bundled
7897 with other short options.
7898
7899 @item -b @var{file-prefix}
7900 @itemx --file-prefix=@var{prefix}
7901 Pretend that @code{%file-prefix} was specified, i.e., specify prefix to use
7902 for all Bison output file names. @xref{Decl Summary}.
7903
7904 @item -r @var{things}
7905 @itemx --report=@var{things}
7906 Write an extra output file containing verbose description of the comma
7907 separated list of @var{things} among:
7908
7909 @table @code
7910 @item state
7911 Description of the grammar, conflicts (resolved and unresolved), and
7912 @acronym{LALR} automaton.
7913
7914 @item lookahead
7915 Implies @code{state} and augments the description of the automaton with
7916 each rule's lookahead set.
7917
7918 @item itemset
7919 Implies @code{state} and augments the description of the automaton with
7920 the full set of items for each state, instead of its core only.
7921 @end table
7922
7923 @item --report-file=@var{file}
7924 Specify the @var{file} for the verbose description.
7925
7926 @item -v
7927 @itemx --verbose
7928 Pretend that @code{%verbose} was specified, i.e., write an extra output
7929 file containing verbose descriptions of the grammar and
7930 parser. @xref{Decl Summary}.
7931
7932 @item -o @var{file}
7933 @itemx --output=@var{file}
7934 Specify the @var{file} for the parser file.
7935
7936 The other output files' names are constructed from @var{file} as
7937 described under the @samp{-v} and @samp{-d} options.
7938
7939 @item -g[@var{file}]
7940 @itemx --graph[=@var{file}]
7941 Output a graphical representation of the @acronym{LALR}(1) grammar
7942 automaton computed by Bison, in @uref{http://www.graphviz.org/, Graphviz}
7943 @uref{http://www.graphviz.org/doc/info/lang.html, @acronym{DOT}} format.
7944 @code{@var{file}} is optional.
7945 If omitted and the grammar file is @file{foo.y}, the output file will be
7946 @file{foo.dot}.
7947
7948 @item -x[@var{file}]
7949 @itemx --xml[=@var{file}]
7950 Output an XML report of the @acronym{LALR}(1) automaton computed by Bison.
7951 @code{@var{file}} is optional.
7952 If omitted and the grammar file is @file{foo.y}, the output file will be
7953 @file{foo.xml}.
7954 (The current XML schema is experimental and may evolve.
7955 More user feedback will help to stabilize it.)
7956 @end table
7957
7958 @node Option Cross Key
7959 @section Option Cross Key
7960
7961 @c FIXME: How about putting the directives too?
7962 Here is a list of options, alphabetized by long option, to help you find
7963 the corresponding short option.
7964
7965 @multitable {@option{--defines=@var{defines-file}}} {@option{-b @var{file-prefix }XXX}}
7966 @headitem Long Option @tab Short Option
7967 @include cross-options.texi
7968 @end multitable
7969
7970 @node Yacc Library
7971 @section Yacc Library
7972
7973 The Yacc library contains default implementations of the
7974 @code{yyerror} and @code{main} functions. These default
7975 implementations are normally not useful, but @acronym{POSIX} requires
7976 them. To use the Yacc library, link your program with the
7977 @option{-ly} option. Note that Bison's implementation of the Yacc
7978 library is distributed under the terms of the @acronym{GNU} General
7979 Public License (@pxref{Copying}).
7980
7981 If you use the Yacc library's @code{yyerror} function, you should
7982 declare @code{yyerror} as follows:
7983
7984 @example
7985 int yyerror (char const *);
7986 @end example
7987
7988 Bison ignores the @code{int} value returned by this @code{yyerror}.
7989 If you use the Yacc library's @code{main} function, your
7990 @code{yyparse} function should have the following type signature:
7991
7992 @example
7993 int yyparse (void);
7994 @end example
7995
7996 @c ================================================= C++ Bison
7997
7998 @node Other Languages
7999 @chapter Parsers Written In Other Languages
8000
8001 @menu
8002 * C++ Parsers:: The interface to generate C++ parser classes
8003 * Java Parsers:: The interface to generate Java parser classes
8004 @end menu
8005
8006 @node C++ Parsers
8007 @section C++ Parsers
8008
8009 @menu
8010 * C++ Bison Interface:: Asking for C++ parser generation
8011 * C++ Semantic Values:: %union vs. C++
8012 * C++ Location Values:: The position and location classes
8013 * C++ Parser Interface:: Instantiating and running the parser
8014 * C++ Scanner Interface:: Exchanges between yylex and parse
8015 * A Complete C++ Example:: Demonstrating their use
8016 @end menu
8017
8018 @node C++ Bison Interface
8019 @subsection C++ Bison Interface
8020 @c - %skeleton "lalr1.cc"
8021 @c - Always pure
8022 @c - initial action
8023
8024 The C++ @acronym{LALR}(1) parser is selected using the skeleton directive,
8025 @samp{%skeleton "lalr1.c"}, or the synonymous command-line option
8026 @option{--skeleton=lalr1.c}.
8027 @xref{Decl Summary}.
8028
8029 When run, @command{bison} will create several entities in the @samp{yy}
8030 namespace.
8031 @findex %define namespace
8032 Use the @samp{%define namespace} directive to change the namespace name, see
8033 @ref{Decl Summary}.
8034 The various classes are generated in the following files:
8035
8036 @table @file
8037 @item position.hh
8038 @itemx location.hh
8039 The definition of the classes @code{position} and @code{location},
8040 used for location tracking. @xref{C++ Location Values}.
8041
8042 @item stack.hh
8043 An auxiliary class @code{stack} used by the parser.
8044
8045 @item @var{file}.hh
8046 @itemx @var{file}.cc
8047 (Assuming the extension of the input file was @samp{.yy}.) The
8048 declaration and implementation of the C++ parser class. The basename
8049 and extension of these two files follow the same rules as with regular C
8050 parsers (@pxref{Invocation}).
8051
8052 The header is @emph{mandatory}; you must either pass
8053 @option{-d}/@option{--defines} to @command{bison}, or use the
8054 @samp{%defines} directive.
8055 @end table
8056
8057 All these files are documented using Doxygen; run @command{doxygen}
8058 for a complete and accurate documentation.
8059
8060 @node C++ Semantic Values
8061 @subsection C++ Semantic Values
8062 @c - No objects in unions
8063 @c - YYSTYPE
8064 @c - Printer and destructor
8065
8066 The @code{%union} directive works as for C, see @ref{Union Decl, ,The
8067 Collection of Value Types}. In particular it produces a genuine
8068 @code{union}@footnote{In the future techniques to allow complex types
8069 within pseudo-unions (similar to Boost variants) might be implemented to
8070 alleviate these issues.}, which have a few specific features in C++.
8071 @itemize @minus
8072 @item
8073 The type @code{YYSTYPE} is defined but its use is discouraged: rather
8074 you should refer to the parser's encapsulated type
8075 @code{yy::parser::semantic_type}.
8076 @item
8077 Non POD (Plain Old Data) types cannot be used. C++ forbids any
8078 instance of classes with constructors in unions: only @emph{pointers}
8079 to such objects are allowed.
8080 @end itemize
8081
8082 Because objects have to be stored via pointers, memory is not
8083 reclaimed automatically: using the @code{%destructor} directive is the
8084 only means to avoid leaks. @xref{Destructor Decl, , Freeing Discarded
8085 Symbols}.
8086
8087
8088 @node C++ Location Values
8089 @subsection C++ Location Values
8090 @c - %locations
8091 @c - class Position
8092 @c - class Location
8093 @c - %define filename_type "const symbol::Symbol"
8094
8095 When the directive @code{%locations} is used, the C++ parser supports
8096 location tracking, see @ref{Locations, , Locations Overview}. Two
8097 auxiliary classes define a @code{position}, a single point in a file,
8098 and a @code{location}, a range composed of a pair of
8099 @code{position}s (possibly spanning several files).
8100
8101 @deftypemethod {position} {std::string*} file
8102 The name of the file. It will always be handled as a pointer, the
8103 parser will never duplicate nor deallocate it. As an experimental
8104 feature you may change it to @samp{@var{type}*} using @samp{%define
8105 filename_type "@var{type}"}.
8106 @end deftypemethod
8107
8108 @deftypemethod {position} {unsigned int} line
8109 The line, starting at 1.
8110 @end deftypemethod
8111
8112 @deftypemethod {position} {unsigned int} lines (int @var{height} = 1)
8113 Advance by @var{height} lines, resetting the column number.
8114 @end deftypemethod
8115
8116 @deftypemethod {position} {unsigned int} column
8117 The column, starting at 0.
8118 @end deftypemethod
8119
8120 @deftypemethod {position} {unsigned int} columns (int @var{width} = 1)
8121 Advance by @var{width} columns, without changing the line number.
8122 @end deftypemethod
8123
8124 @deftypemethod {position} {position&} operator+= (position& @var{pos}, int @var{ width})
8125 @deftypemethodx {position} {position} operator+ (const position& @var{pos}, int @var{width})
8126 @deftypemethodx {position} {position&} operator-= (const position& @var{pos}, in t @var{width})
8127 @deftypemethodx {position} {position} operator- (position& @var{pos}, int @var{w idth})
8128 Various forms of syntactic sugar for @code{columns}.
8129 @end deftypemethod
8130
8131 @deftypemethod {position} {position} operator<< (std::ostream @var{o}, const pos ition& @var{p})
8132 Report @var{p} on @var{o} like this:
8133 @samp{@var{file}:@var{line}.@var{column}}, or
8134 @samp{@var{line}.@var{column}} if @var{file} is null.
8135 @end deftypemethod
8136
8137 @deftypemethod {location} {position} begin
8138 @deftypemethodx {location} {position} end
8139 The first, inclusive, position of the range, and the first beyond.
8140 @end deftypemethod
8141
8142 @deftypemethod {location} {unsigned int} columns (int @var{width} = 1)
8143 @deftypemethodx {location} {unsigned int} lines (int @var{height} = 1)
8144 Advance the @code{end} position.
8145 @end deftypemethod
8146
8147 @deftypemethod {location} {location} operator+ (const location& @var{begin}, con st location& @var{end})
8148 @deftypemethodx {location} {location} operator+ (const location& @var{begin}, in t @var{width})
8149 @deftypemethodx {location} {location} operator+= (const location& @var{loc}, int @var{width})
8150 Various forms of syntactic sugar.
8151 @end deftypemethod
8152
8153 @deftypemethod {location} {void} step ()
8154 Move @code{begin} onto @code{end}.
8155 @end deftypemethod
8156
8157
8158 @node C++ Parser Interface
8159 @subsection C++ Parser Interface
8160 @c - define parser_class_name
8161 @c - Ctor
8162 @c - parse, error, set_debug_level, debug_level, set_debug_stream,
8163 @c debug_stream.
8164 @c - Reporting errors
8165
8166 The output files @file{@var{output}.hh} and @file{@var{output}.cc}
8167 declare and define the parser class in the namespace @code{yy}. The
8168 class name defaults to @code{parser}, but may be changed using
8169 @samp{%define parser_class_name "@var{name}"}. The interface of
8170 this class is detailed below. It can be extended using the
8171 @code{%parse-param} feature: its semantics is slightly changed since
8172 it describes an additional member of the parser class, and an
8173 additional argument for its constructor.
8174
8175 @defcv {Type} {parser} {semantic_value_type}
8176 @defcvx {Type} {parser} {location_value_type}
8177 The types for semantics value and locations.
8178 @end defcv
8179
8180 @deftypemethod {parser} {} parser (@var{type1} @var{arg1}, ...)
8181 Build a new parser object. There are no arguments by default, unless
8182 @samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
8183 @end deftypemethod
8184
8185 @deftypemethod {parser} {int} parse ()
8186 Run the syntactic analysis, and return 0 on success, 1 otherwise.
8187 @end deftypemethod
8188
8189 @deftypemethod {parser} {std::ostream&} debug_stream ()
8190 @deftypemethodx {parser} {void} set_debug_stream (std::ostream& @var{o})
8191 Get or set the stream used for tracing the parsing. It defaults to
8192 @code{std::cerr}.
8193 @end deftypemethod
8194
8195 @deftypemethod {parser} {debug_level_type} debug_level ()
8196 @deftypemethodx {parser} {void} set_debug_level (debug_level @var{l})
8197 Get or set the tracing level. Currently its value is either 0, no trace,
8198 or nonzero, full tracing.
8199 @end deftypemethod
8200
8201 @deftypemethod {parser} {void} error (const location_type& @var{l}, const std::s tring& @var{m})
8202 The definition for this member function must be supplied by the user:
8203 the parser uses it to report a parser error occurring at @var{l},
8204 described by @var{m}.
8205 @end deftypemethod
8206
8207
8208 @node C++ Scanner Interface
8209 @subsection C++ Scanner Interface
8210 @c - prefix for yylex.
8211 @c - Pure interface to yylex
8212 @c - %lex-param
8213
8214 The parser invokes the scanner by calling @code{yylex}. Contrary to C
8215 parsers, C++ parsers are always pure: there is no point in using the
8216 @code{%define api.pure} directive. Therefore the interface is as follows.
8217
8218 @deftypemethod {parser} {int} yylex (semantic_value_type& @var{yylval}, location _type& @var{yylloc}, @var{type1} @var{arg1}, ...)
8219 Return the next token. Its type is the return value, its semantic
8220 value and location being @var{yylval} and @var{yylloc}. Invocations of
8221 @samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
8222 @end deftypemethod
8223
8224
8225 @node A Complete C++ Example
8226 @subsection A Complete C++ Example
8227
8228 This section demonstrates the use of a C++ parser with a simple but
8229 complete example. This example should be available on your system,
8230 ready to compile, in the directory @dfn{../bison/examples/calc++}. It
8231 focuses on the use of Bison, therefore the design of the various C++
8232 classes is very naive: no accessors, no encapsulation of members etc.
8233 We will use a Lex scanner, and more precisely, a Flex scanner, to
8234 demonstrate the various interaction. A hand written scanner is
8235 actually easier to interface with.
8236
8237 @menu
8238 * Calc++ --- C++ Calculator:: The specifications
8239 * Calc++ Parsing Driver:: An active parsing context
8240 * Calc++ Parser:: A parser class
8241 * Calc++ Scanner:: A pure C++ Flex scanner
8242 * Calc++ Top Level:: Conducting the band
8243 @end menu
8244
8245 @node Calc++ --- C++ Calculator
8246 @subsubsection Calc++ --- C++ Calculator
8247
8248 Of course the grammar is dedicated to arithmetics, a single
8249 expression, possibly preceded by variable assignments. An
8250 environment containing possibly predefined variables such as
8251 @code{one} and @code{two}, is exchanged with the parser. An example
8252 of valid input follows.
8253
8254 @example
8255 three := 3
8256 seven := one + two * three
8257 seven * seven
8258 @end example
8259
8260 @node Calc++ Parsing Driver
8261 @subsubsection Calc++ Parsing Driver
8262 @c - An env
8263 @c - A place to store error messages
8264 @c - A place for the result
8265
8266 To support a pure interface with the parser (and the scanner) the
8267 technique of the ``parsing context'' is convenient: a structure
8268 containing all the data to exchange. Since, in addition to simply
8269 launch the parsing, there are several auxiliary tasks to execute (open
8270 the file for parsing, instantiate the parser etc.), we recommend
8271 transforming the simple parsing context structure into a fully blown
8272 @dfn{parsing driver} class.
8273
8274 The declaration of this driver class, @file{calc++-driver.hh}, is as
8275 follows. The first part includes the CPP guard and imports the
8276 required standard library components, and the declaration of the parser
8277 class.
8278
8279 @comment file: calc++-driver.hh
8280 @example
8281 #ifndef CALCXX_DRIVER_HH
8282 # define CALCXX_DRIVER_HH
8283 # include <string>
8284 # include <map>
8285 # include "calc++-parser.hh"
8286 @end example
8287
8288
8289 @noindent
8290 Then comes the declaration of the scanning function. Flex expects
8291 the signature of @code{yylex} to be defined in the macro
8292 @code{YY_DECL}, and the C++ parser expects it to be declared. We can
8293 factor both as follows.
8294
8295 @comment file: calc++-driver.hh
8296 @example
8297 // Tell Flex the lexer's prototype ...
8298 # define YY_DECL \
8299 yy::calcxx_parser::token_type \
8300 yylex (yy::calcxx_parser::semantic_type* yylval, \
8301 yy::calcxx_parser::location_type* yylloc, \
8302 calcxx_driver& driver)
8303 // ... and declare it for the parser's sake.
8304 YY_DECL;
8305 @end example
8306
8307 @noindent
8308 The @code{calcxx_driver} class is then declared with its most obvious
8309 members.
8310
8311 @comment file: calc++-driver.hh
8312 @example
8313 // Conducting the whole scanning and parsing of Calc++.
8314 class calcxx_driver
8315 @{
8316 public:
8317 calcxx_driver ();
8318 virtual ~calcxx_driver ();
8319
8320 std::map<std::string, int> variables;
8321
8322 int result;
8323 @end example
8324
8325 @noindent
8326 To encapsulate the coordination with the Flex scanner, it is useful to
8327 have two members function to open and close the scanning phase.
8328
8329 @comment file: calc++-driver.hh
8330 @example
8331 // Handling the scanner.
8332 void scan_begin ();
8333 void scan_end ();
8334 bool trace_scanning;
8335 @end example
8336
8337 @noindent
8338 Similarly for the parser itself.
8339
8340 @comment file: calc++-driver.hh
8341 @example
8342 // Run the parser. Return 0 on success.
8343 int parse (const std::string& f);
8344 std::string file;
8345 bool trace_parsing;
8346 @end example
8347
8348 @noindent
8349 To demonstrate pure handling of parse errors, instead of simply
8350 dumping them on the standard error output, we will pass them to the
8351 compiler driver using the following two member functions. Finally, we
8352 close the class declaration and CPP guard.
8353
8354 @comment file: calc++-driver.hh
8355 @example
8356 // Error handling.
8357 void error (const yy::location& l, const std::string& m);
8358 void error (const std::string& m);
8359 @};
8360 #endif // ! CALCXX_DRIVER_HH
8361 @end example
8362
8363 The implementation of the driver is straightforward. The @code{parse}
8364 member function deserves some attention. The @code{error} functions
8365 are simple stubs, they should actually register the located error
8366 messages and set error state.
8367
8368 @comment file: calc++-driver.cc
8369 @example
8370 #include "calc++-driver.hh"
8371 #include "calc++-parser.hh"
8372
8373 calcxx_driver::calcxx_driver ()
8374 : trace_scanning (false), trace_parsing (false)
8375 @{
8376 variables["one"] = 1;
8377 variables["two"] = 2;
8378 @}
8379
8380 calcxx_driver::~calcxx_driver ()
8381 @{
8382 @}
8383
8384 int
8385 calcxx_driver::parse (const std::string &f)
8386 @{
8387 file = f;
8388 scan_begin ();
8389 yy::calcxx_parser parser (*this);
8390 parser.set_debug_level (trace_parsing);
8391 int res = parser.parse ();
8392 scan_end ();
8393 return res;
8394 @}
8395
8396 void
8397 calcxx_driver::error (const yy::location& l, const std::string& m)
8398 @{
8399 std::cerr << l << ": " << m << std::endl;
8400 @}
8401
8402 void
8403 calcxx_driver::error (const std::string& m)
8404 @{
8405 std::cerr << m << std::endl;
8406 @}
8407 @end example
8408
8409 @node Calc++ Parser
8410 @subsubsection Calc++ Parser
8411
8412 The parser definition file @file{calc++-parser.yy} starts by asking for
8413 the C++ LALR(1) skeleton, the creation of the parser header file, and
8414 specifies the name of the parser class. Because the C++ skeleton
8415 changed several times, it is safer to require the version you designed
8416 the grammar for.
8417
8418 @comment file: calc++-parser.yy
8419 @example
8420 %skeleton "lalr1.cc" /* -*- C++ -*- */
8421 %require "@value{VERSION}"
8422 %defines
8423 %define parser_class_name "calcxx_parser"
8424 @end example
8425
8426 @noindent
8427 @findex %code requires
8428 Then come the declarations/inclusions needed to define the
8429 @code{%union}. Because the parser uses the parsing driver and
8430 reciprocally, both cannot include the header of the other. Because the
8431 driver's header needs detailed knowledge about the parser class (in
8432 particular its inner types), it is the parser's header which will simply
8433 use a forward declaration of the driver.
8434 @xref{Decl Summary, ,%code}.
8435
8436 @comment file: calc++-parser.yy
8437 @example
8438 %code requires @{
8439 # include <string>
8440 class calcxx_driver;
8441 @}
8442 @end example
8443
8444 @noindent
8445 The driver is passed by reference to the parser and to the scanner.
8446 This provides a simple but effective pure interface, not relying on
8447 global variables.
8448
8449 @comment file: calc++-parser.yy
8450 @example
8451 // The parsing context.
8452 %parse-param @{ calcxx_driver& driver @}
8453 %lex-param @{ calcxx_driver& driver @}
8454 @end example
8455
8456 @noindent
8457 Then we request the location tracking feature, and initialize the
8458 first location's file name. Afterwards new locations are computed
8459 relatively to the previous locations: the file name will be
8460 automatically propagated.
8461
8462 @comment file: calc++-parser.yy
8463 @example
8464 %locations
8465 %initial-action
8466 @{
8467 // Initialize the initial location.
8468 @@$.begin.filename = @@$.end.filename = &driver.file;
8469 @};
8470 @end example
8471
8472 @noindent
8473 Use the two following directives to enable parser tracing and verbose
8474 error messages.
8475
8476 @comment file: calc++-parser.yy
8477 @example
8478 %debug
8479 %error-verbose
8480 @end example
8481
8482 @noindent
8483 Semantic values cannot use ``real'' objects, but only pointers to
8484 them.
8485
8486 @comment file: calc++-parser.yy
8487 @example
8488 // Symbols.
8489 %union
8490 @{
8491 int ival;
8492 std::string *sval;
8493 @};
8494 @end example
8495
8496 @noindent
8497 @findex %code
8498 The code between @samp{%code @{} and @samp{@}} is output in the
8499 @file{*.cc} file; it needs detailed knowledge about the driver.
8500
8501 @comment file: calc++-parser.yy
8502 @example
8503 %code @{
8504 # include "calc++-driver.hh"
8505 @}
8506 @end example
8507
8508
8509 @noindent
8510 The token numbered as 0 corresponds to end of file; the following line
8511 allows for nicer error messages referring to ``end of file'' instead
8512 of ``$end''. Similarly user friendly named are provided for each
8513 symbol. Note that the tokens names are prefixed by @code{TOKEN_} to
8514 avoid name clashes.
8515
8516 @comment file: calc++-parser.yy
8517 @example
8518 %token END 0 "end of file"
8519 %token ASSIGN ":="
8520 %token <sval> IDENTIFIER "identifier"
8521 %token <ival> NUMBER "number"
8522 %type <ival> exp
8523 @end example
8524
8525 @noindent
8526 To enable memory deallocation during error recovery, use
8527 @code{%destructor}.
8528
8529 @c FIXME: Document %printer, and mention that it takes a braced-code operand.
8530 @comment file: calc++-parser.yy
8531 @example
8532 %printer @{ debug_stream () << *$$; @} "identifier"
8533 %destructor @{ delete $$; @} "identifier"
8534
8535 %printer @{ debug_stream () << $$; @} <ival>
8536 @end example
8537
8538 @noindent
8539 The grammar itself is straightforward.
8540
8541 @comment file: calc++-parser.yy
8542 @example
8543 %%
8544 %start unit;
8545 unit: assignments exp @{ driver.result = $2; @};
8546
8547 assignments: assignments assignment @{@}
8548 | /* Nothing. */ @{@};
8549
8550 assignment:
8551 "identifier" ":=" exp
8552 @{ driver.variables[*$1] = $3; delete $1; @};
8553
8554 %left '+' '-';
8555 %left '*' '/';
8556 exp: exp '+' exp @{ $$ = $1 + $3; @}
8557 | exp '-' exp @{ $$ = $1 - $3; @}
8558 | exp '*' exp @{ $$ = $1 * $3; @}
8559 | exp '/' exp @{ $$ = $1 / $3; @}
8560 | "identifier" @{ $$ = driver.variables[*$1]; delete $1; @}
8561 | "number" @{ $$ = $1; @};
8562 %%
8563 @end example
8564
8565 @noindent
8566 Finally the @code{error} member function registers the errors to the
8567 driver.
8568
8569 @comment file: calc++-parser.yy
8570 @example
8571 void
8572 yy::calcxx_parser::error (const yy::calcxx_parser::location_type& l,
8573 const std::string& m)
8574 @{
8575 driver.error (l, m);
8576 @}
8577 @end example
8578
8579 @node Calc++ Scanner
8580 @subsubsection Calc++ Scanner
8581
8582 The Flex scanner first includes the driver declaration, then the
8583 parser's to get the set of defined tokens.
8584
8585 @comment file: calc++-scanner.ll
8586 @example
8587 %@{ /* -*- C++ -*- */
8588 # include <cstdlib>
8589 # include <errno.h>
8590 # include <limits.h>
8591 # include <string>
8592 # include "calc++-driver.hh"
8593 # include "calc++-parser.hh"
8594
8595 /* Work around an incompatibility in flex (at least versions
8596 2.5.31 through 2.5.33): it generates code that does
8597 not conform to C89. See Debian bug 333231
8598 <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>. */
8599 # undef yywrap
8600 # define yywrap() 1
8601
8602 /* By default yylex returns int, we use token_type.
8603 Unfortunately yyterminate by default returns 0, which is
8604 not of token_type. */
8605 #define yyterminate() return token::END
8606 %@}
8607 @end example
8608
8609 @noindent
8610 Because there is no @code{#include}-like feature we don't need
8611 @code{yywrap}, we don't need @code{unput} either, and we parse an
8612 actual file, this is not an interactive session with the user.
8613 Finally we enable the scanner tracing features.
8614
8615 @comment file: calc++-scanner.ll
8616 @example
8617 %option noyywrap nounput batch debug
8618 @end example
8619
8620 @noindent
8621 Abbreviations allow for more readable rules.
8622
8623 @comment file: calc++-scanner.ll
8624 @example
8625 id [a-zA-Z][a-zA-Z_0-9]*
8626 int [0-9]+
8627 blank [ \t]
8628 @end example
8629
8630 @noindent
8631 The following paragraph suffices to track locations accurately. Each
8632 time @code{yylex} is invoked, the begin position is moved onto the end
8633 position. Then when a pattern is matched, the end position is
8634 advanced of its width. In case it matched ends of lines, the end
8635 cursor is adjusted, and each time blanks are matched, the begin cursor
8636 is moved onto the end cursor to effectively ignore the blanks
8637 preceding tokens. Comments would be treated equally.
8638
8639 @comment file: calc++-scanner.ll
8640 @example
8641 %@{
8642 # define YY_USER_ACTION yylloc->columns (yyleng);
8643 %@}
8644 %%
8645 %@{
8646 yylloc->step ();
8647 %@}
8648 @{blank@}+ yylloc->step ();
8649 [\n]+ yylloc->lines (yyleng); yylloc->step ();
8650 @end example
8651
8652 @noindent
8653 The rules are simple, just note the use of the driver to report errors.
8654 It is convenient to use a typedef to shorten
8655 @code{yy::calcxx_parser::token::identifier} into
8656 @code{token::identifier} for instance.
8657
8658 @comment file: calc++-scanner.ll
8659 @example
8660 %@{
8661 typedef yy::calcxx_parser::token token;
8662 %@}
8663 /* Convert ints to the actual type of tokens. */
8664 [-+*/] return yy::calcxx_parser::token_type (yytext[0]);
8665 ":=" return token::ASSIGN;
8666 @{int@} @{
8667 errno = 0;
8668 long n = strtol (yytext, NULL, 10);
8669 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
8670 driver.error (*yylloc, "integer is out of range");
8671 yylval->ival = n;
8672 return token::NUMBER;
8673 @}
8674 @{id@} yylval->sval = new std::string (yytext); return token::IDENTIFIER;
8675 . driver.error (*yylloc, "invalid character");
8676 %%
8677 @end example
8678
8679 @noindent
8680 Finally, because the scanner related driver's member function depend
8681 on the scanner's data, it is simpler to implement them in this file.
8682
8683 @comment file: calc++-scanner.ll
8684 @example
8685 void
8686 calcxx_driver::scan_begin ()
8687 @{
8688 yy_flex_debug = trace_scanning;
8689 if (file == "-")
8690 yyin = stdin;
8691 else if (!(yyin = fopen (file.c_str (), "r")))
8692 @{
8693 error (std::string ("cannot open ") + file);
8694 exit (1);
8695 @}
8696 @}
8697
8698 void
8699 calcxx_driver::scan_end ()
8700 @{
8701 fclose (yyin);
8702 @}
8703 @end example
8704
8705 @node Calc++ Top Level
8706 @subsubsection Calc++ Top Level
8707
8708 The top level file, @file{calc++.cc}, poses no problem.
8709
8710 @comment file: calc++.cc
8711 @example
8712 #include <iostream>
8713 #include "calc++-driver.hh"
8714
8715 int
8716 main (int argc, char *argv[])
8717 @{
8718 calcxx_driver driver;
8719 for (++argv; argv[0]; ++argv)
8720 if (*argv == std::string ("-p"))
8721 driver.trace_parsing = true;
8722 else if (*argv == std::string ("-s"))
8723 driver.trace_scanning = true;
8724 else if (!driver.parse (*argv))
8725 std::cout << driver.result << std::endl;
8726 @}
8727 @end example
8728
8729 @node Java Parsers
8730 @section Java Parsers
8731
8732 @menu
8733 * Java Bison Interface:: Asking for Java parser generation
8734 * Java Semantic Values:: %type and %token vs. Java
8735 * Java Location Values:: The position and location classes
8736 * Java Parser Interface:: Instantiating and running the parser
8737 * Java Scanner Interface:: Specifying the scanner for the parser
8738 * Java Action Features:: Special features for use in actions
8739 * Java Differences:: Differences between C/C++ and Java Grammars
8740 * Java Declarations Summary:: List of Bison declarations used with Java
8741 @end menu
8742
8743 @node Java Bison Interface
8744 @subsection Java Bison Interface
8745 @c - %language "Java"
8746
8747 (The current Java interface is experimental and may evolve.
8748 More user feedback will help to stabilize it.)
8749
8750 The Java parser skeletons are selected using the @code{%language "Java"}
8751 directive or the @option{-L java}/@option{--language=java} option.
8752
8753 @c FIXME: Documented bug.
8754 When generating a Java parser, @code{bison @var{basename}.y} will create
8755 a single Java source file named @file{@var{basename}.java}. Using an
8756 input file without a @file{.y} suffix is currently broken. The basename
8757 of the output file can be changed by the @code{%file-prefix} directive
8758 or the @option{-p}/@option{--name-prefix} option. The entire output file
8759 name can be changed by the @code{%output} directive or the
8760 @option{-o}/@option{--output} option. The output file contains a single
8761 class for the parser.
8762
8763 You can create documentation for generated parsers using Javadoc.
8764
8765 Contrary to C parsers, Java parsers do not use global variables; the
8766 state of the parser is always local to an instance of the parser class.
8767 Therefore, all Java parsers are ``pure'', and the @code{%pure-parser}
8768 and @code{%define api.pure} directives does not do anything when used in
8769 Java.
8770
8771 Push parsers are currently unsupported in Java and @code{%define
8772 api.push_pull} have no effect.
8773
8774 @acronym{GLR} parsers are currently unsupported in Java. Do not use the
8775 @code{glr-parser} directive.
8776
8777 No header file can be generated for Java parsers. Do not use the
8778 @code{%defines} directive or the @option{-d}/@option{--defines} options.
8779
8780 @c FIXME: Possible code change.
8781 Currently, support for debugging and verbose errors are always compiled
8782 in. Thus the @code{%debug} and @code{%token-table} directives and the
8783 @option{-t}/@option{--debug} and @option{-k}/@option{--token-table}
8784 options have no effect. This may change in the future to eliminate
8785 unused code in the generated parser, so use @code{%debug} and
8786 @code{%verbose-error} explicitly if needed. Also, in the future the
8787 @code{%token-table} directive might enable a public interface to
8788 access the token names and codes.
8789
8790 @node Java Semantic Values
8791 @subsection Java Semantic Values
8792 @c - No %union, specify type in %type/%token.
8793 @c - YYSTYPE
8794 @c - Printer and destructor
8795
8796 There is no @code{%union} directive in Java parsers. Instead, the
8797 semantic values' types (class names) should be specified in the
8798 @code{%type} or @code{%token} directive:
8799
8800 @example
8801 %type <Expression> expr assignment_expr term factor
8802 %type <Integer> number
8803 @end example
8804
8805 By default, the semantic stack is declared to have @code{Object} members,
8806 which means that the class types you specify can be of any class.
8807 To improve the type safety of the parser, you can declare the common
8808 superclass of all the semantic values using the @code{%define stype}
8809 directive. For example, after the following declaration:
8810
8811 @example
8812 %define stype "ASTNode"
8813 @end example
8814
8815 @noindent
8816 any @code{%type} or @code{%token} specifying a semantic type which
8817 is not a subclass of ASTNode, will cause a compile-time error.
8818
8819 @c FIXME: Documented bug.
8820 Types used in the directives may be qualified with a package name.
8821 Primitive data types are accepted for Java version 1.5 or later. Note
8822 that in this case the autoboxing feature of Java 1.5 will be used.
8823 Generic types may not be used; this is due to a limitation in the
8824 implementation of Bison, and may change in future releases.
8825
8826 Java parsers do not support @code{%destructor}, since the language
8827 adopts garbage collection. The parser will try to hold references
8828 to semantic values for as little time as needed.
8829
8830 Java parsers do not support @code{%printer}, as @code{toString()}
8831 can be used to print the semantic values. This however may change
8832 (in a backwards-compatible way) in future versions of Bison.
8833
8834
8835 @node Java Location Values
8836 @subsection Java Location Values
8837 @c - %locations
8838 @c - class Position
8839 @c - class Location
8840
8841 When the directive @code{%locations} is used, the Java parser
8842 supports location tracking, see @ref{Locations, , Locations Overview}.
8843 An auxiliary user-defined class defines a @dfn{position}, a single point
8844 in a file; Bison itself defines a class representing a @dfn{location},
8845 a range composed of a pair of positions (possibly spanning several
8846 files). The location class is an inner class of the parser; the name
8847 is @code{Location} by default, and may also be renamed using
8848 @code{%define location_type "@var{class-name}}.
8849
8850 The location class treats the position as a completely opaque value.
8851 By default, the class name is @code{Position}, but this can be changed
8852 with @code{%define position_type "@var{class-name}"}. This class must
8853 be supplied by the user.
8854
8855
8856 @deftypeivar {Location} {Position} begin
8857 @deftypeivarx {Location} {Position} end
8858 The first, inclusive, position of the range, and the first beyond.
8859 @end deftypeivar
8860
8861 @deftypeop {Constructor} {Location} {} Location (Position @var{loc})
8862 Create a @code{Location} denoting an empty range located at a given point.
8863 @end deftypeop
8864
8865 @deftypeop {Constructor} {Location} {} Location (Position @var{begin}, Position @var{end})
8866 Create a @code{Location} from the endpoints of the range.
8867 @end deftypeop
8868
8869 @deftypemethod {Location} {String} toString ()
8870 Prints the range represented by the location. For this to work
8871 properly, the position class should override the @code{equals} and
8872 @code{toString} methods appropriately.
8873 @end deftypemethod
8874
8875
8876 @node Java Parser Interface
8877 @subsection Java Parser Interface
8878 @c - define parser_class_name
8879 @c - Ctor
8880 @c - parse, error, set_debug_level, debug_level, set_debug_stream,
8881 @c debug_stream.
8882 @c - Reporting errors
8883
8884 The name of the generated parser class defaults to @code{YYParser}. The
8885 @code{YY} prefix may be changed using the @code{%name-prefix} directive
8886 or the @option{-p}/@option{--name-prefix} option. Alternatively, use
8887 @code{%define parser_class_name "@var{name}"} to give a custom name to
8888 the class. The interface of this class is detailed below.
8889
8890 By default, the parser class has package visibility. A declaration
8891 @code{%define public} will change to public visibility. Remember that,
8892 according to the Java language specification, the name of the @file{.java}
8893 file should match the name of the class in this case. Similarly, you can
8894 use @code{abstract}, @code{final} and @code{strictfp} with the
8895 @code{%define} declaration to add other modifiers to the parser class.
8896
8897 The Java package name of the parser class can be specified using the
8898 @code{%define package} directive. The superclass and the implemented
8899 interfaces of the parser class can be specified with the @code{%define
8900 extends} and @code{%define implements} directives.
8901
8902 The parser class defines an inner class, @code{Location}, that is used
8903 for location tracking (see @ref{Java Location Values}), and a inner
8904 interface, @code{Lexer} (see @ref{Java Scanner Interface}). Other than
8905 these inner class/interface, and the members described in the interface
8906 below, all the other members and fields are preceded with a @code{yy} or
8907 @code{YY} prefix to avoid clashes with user code.
8908
8909 @c FIXME: The following constants and variables are still undocumented:
8910 @c @code{bisonVersion}, @code{bisonSkeleton} and @code{errorVerbose}.
8911
8912 The parser class can be extended using the @code{%parse-param}
8913 directive. Each occurrence of the directive will add a @code{protected
8914 final} field to the parser class, and an argument to its constructor,
8915 which initialize them automatically.
8916
8917 Token names defined by @code{%token} and the predefined @code{EOF} token
8918 name are added as constant fields to the parser class.
8919
8920 @deftypeop {Constructor} {YYParser} {} YYParser (@var{lex_param}, @dots{}, @var{ parse_param}, @dots{})
8921 Build a new parser object with embedded @code{%code lexer}. There are
8922 no parameters, unless @code{%parse-param}s and/or @code{%lex-param}s are
8923 used.
8924 @end deftypeop
8925
8926 @deftypeop {Constructor} {YYParser} {} YYParser (Lexer @var{lexer}, @var{parse_p aram}, @dots{})
8927 Build a new parser object using the specified scanner. There are no
8928 additional parameters unless @code{%parse-param}s are used.
8929
8930 If the scanner is defined by @code{%code lexer}, this constructor is
8931 declared @code{protected} and is called automatically with a scanner
8932 created with the correct @code{%lex-param}s.
8933 @end deftypeop
8934
8935 @deftypemethod {YYParser} {boolean} parse ()
8936 Run the syntactic analysis, and return @code{true} on success,
8937 @code{false} otherwise.
8938 @end deftypemethod
8939
8940 @deftypemethod {YYParser} {boolean} recovering ()
8941 During the syntactic analysis, return @code{true} if recovering
8942 from a syntax error.
8943 @xref{Error Recovery}.
8944 @end deftypemethod
8945
8946 @deftypemethod {YYParser} {java.io.PrintStream} getDebugStream ()
8947 @deftypemethodx {YYParser} {void} setDebugStream (java.io.printStream @var{o})
8948 Get or set the stream used for tracing the parsing. It defaults to
8949 @code{System.err}.
8950 @end deftypemethod
8951
8952 @deftypemethod {YYParser} {int} getDebugLevel ()
8953 @deftypemethodx {YYParser} {void} setDebugLevel (int @var{l})
8954 Get or set the tracing level. Currently its value is either 0, no trace,
8955 or nonzero, full tracing.
8956 @end deftypemethod
8957
8958
8959 @node Java Scanner Interface
8960 @subsection Java Scanner Interface
8961 @c - %code lexer
8962 @c - %lex-param
8963 @c - Lexer interface
8964
8965 There are two possible ways to interface a Bison-generated Java parser
8966 with a scanner: the scanner may be defined by @code{%code lexer}, or
8967 defined elsewhere. In either case, the scanner has to implement the
8968 @code{Lexer} inner interface of the parser class.
8969
8970 In the first case, the body of the scanner class is placed in
8971 @code{%code lexer} blocks. If you want to pass parameters from the
8972 parser constructor to the scanner constructor, specify them with
8973 @code{%lex-param}; they are passed before @code{%parse-param}s to the
8974 constructor.
8975
8976 In the second case, the scanner has to implement the @code{Lexer} interface,
8977 which is defined within the parser class (e.g., @code{YYParser.Lexer}).
8978 The constructor of the parser object will then accept an object
8979 implementing the interface; @code{%lex-param} is not used in this
8980 case.
8981
8982 In both cases, the scanner has to implement the following methods.
8983
8984 @deftypemethod {Lexer} {void} yyerror (Location @var{loc}, String @var{msg})
8985 This method is defined by the user to emit an error message. The first
8986 parameter is omitted if location tracking is not active. Its type can be
8987 changed using @code{%define location_type "@var{class-name}".}
8988 @end deftypemethod
8989
8990 @deftypemethod {Lexer} {int} yylex ()
8991 Return the next token. Its type is the return value, its semantic
8992 value and location are saved and returned by the ther methods in the
8993 interface.
8994
8995 Use @code{%define lex_throws} to specify any uncaught exceptions.
8996 Default is @code{java.io.IOException}.
8997 @end deftypemethod
8998
8999 @deftypemethod {Lexer} {Position} getStartPos ()
9000 @deftypemethodx {Lexer} {Position} getEndPos ()
9001 Return respectively the first position of the last token that
9002 @code{yylex} returned, and the first position beyond it. These
9003 methods are not needed unless location tracking is active.
9004
9005 The return type can be changed using @code{%define position_type
9006 "@var{class-name}".}
9007 @end deftypemethod
9008
9009 @deftypemethod {Lexer} {Object} getLVal ()
9010 Return the semantical value of the last token that yylex returned.
9011
9012 The return type can be changed using @code{%define stype
9013 "@var{class-name}".}
9014 @end deftypemethod
9015
9016
9017 @node Java Action Features
9018 @subsection Special Features for Use in Java Actions
9019
9020 The following special constructs can be uses in Java actions.
9021 Other analogous C action features are currently unavailable for Java.
9022
9023 Use @code{%define throws} to specify any uncaught exceptions from parser
9024 actions, and initial actions specified by @code{%initial-action}.
9025
9026 @defvar $@var{n}
9027 The semantic value for the @var{n}th component of the current rule.
9028 This may not be assigned to.
9029 @xref{Java Semantic Values}.
9030 @end defvar
9031
9032 @defvar $<@var{typealt}>@var{n}
9033 Like @code{$@var{n}} but specifies a alternative type @var{typealt}.
9034 @xref{Java Semantic Values}.
9035 @end defvar
9036
9037 @defvar $$
9038 The semantic value for the grouping made by the current rule. As a
9039 value, this is in the base type (@code{Object} or as specified by
9040 @code{%define stype}) as in not cast to the declared subtype because
9041 casts are not allowed on the left-hand side of Java assignments.
9042 Use an explicit Java cast if the correct subtype is needed.
9043 @xref{Java Semantic Values}.
9044 @end defvar
9045
9046 @defvar $<@var{typealt}>$
9047 Same as @code{$$} since Java always allow assigning to the base type.
9048 Perhaps we should use this and @code{$<>$} for the value and @code{$$}
9049 for setting the value but there is currently no easy way to distinguish
9050 these constructs.
9051 @xref{Java Semantic Values}.
9052 @end defvar
9053
9054 @defvar @@@var{n}
9055 The location information of the @var{n}th component of the current rule.
9056 This may not be assigned to.
9057 @xref{Java Location Values}.
9058 @end defvar
9059
9060 @defvar @@$
9061 The location information of the grouping made by the current rule.
9062 @xref{Java Location Values}.
9063 @end defvar
9064
9065 @deffn {Statement} {return YYABORT;}
9066 Return immediately from the parser, indicating failure.
9067 @xref{Java Parser Interface}.
9068 @end deffn
9069
9070 @deffn {Statement} {return YYACCEPT;}
9071 Return immediately from the parser, indicating success.
9072 @xref{Java Parser Interface}.
9073 @end deffn
9074
9075 @deffn {Statement} {return YYERROR;}
9076 Start error recovery without printing an error message.
9077 @xref{Error Recovery}.
9078 @end deffn
9079
9080 @deffn {Statement} {return YYFAIL;}
9081 Print an error message and start error recovery.
9082 @xref{Error Recovery}.
9083 @end deffn
9084
9085 @deftypefn {Function} {boolean} recovering ()
9086 Return whether error recovery is being done. In this state, the parser
9087 reads token until it reaches a known state, and then restarts normal
9088 operation.
9089 @xref{Error Recovery}.
9090 @end deftypefn
9091
9092 @deftypefn {Function} {protected void} yyerror (String msg)
9093 @deftypefnx {Function} {protected void} yyerror (Position pos, String msg)
9094 @deftypefnx {Function} {protected void} yyerror (Location loc, String msg)
9095 Print an error message using the @code{yyerror} method of the scanner
9096 instance in use.
9097 @end deftypefn
9098
9099
9100 @node Java Differences
9101 @subsection Differences between C/C++ and Java Grammars
9102
9103 The different structure of the Java language forces several differences
9104 between C/C++ grammars, and grammars designed for Java parsers. This
9105 section summarizes these differences.
9106
9107 @itemize
9108 @item
9109 Java lacks a preprocessor, so the @code{YYERROR}, @code{YYACCEPT},
9110 @code{YYABORT} symbols (@pxref{Table of Symbols}) cannot obviously be
9111 macros. Instead, they should be preceded by @code{return} when they
9112 appear in an action. The actual definition of these symbols is
9113 opaque to the Bison grammar, and it might change in the future. The
9114 only meaningful operation that you can do, is to return them.
9115 See @pxref{Java Action Features}.
9116
9117 Note that of these three symbols, only @code{YYACCEPT} and
9118 @code{YYABORT} will cause a return from the @code{yyparse}
9119 method@footnote{Java parsers include the actions in a separate
9120 method than @code{yyparse} in order to have an intuitive syntax that
9121 corresponds to these C macros.}.
9122
9123 @item
9124 Java lacks unions, so @code{%union} has no effect. Instead, semantic
9125 values have a common base type: @code{Object} or as specified by
9126 @code{%define stype}. Angle backets on @code{%token}, @code{type},
9127 @code{$@var{n}} and @code{$$} specify subtypes rather than fields of
9128 an union. The type of @code{$$}, even with angle brackets, is the base
9129 type since Java casts are not allow on the left-hand side of assignments.
9130 Also, @code{$@var{n}} and @code{@@@var{n}} are not allowed on the
9131 left-hand side of assignments. See @pxref{Java Semantic Values} and
9132 @pxref{Java Action Features}.
9133
9134 @item
9135 The prolog declarations have a different meaning than in C/C++ code.
9136 @table @asis
9137 @item @code{%code imports}
9138 blocks are placed at the beginning of the Java source code. They may
9139 include copyright notices. For a @code{package} declarations, it is
9140 suggested to use @code{%define package} instead.
9141
9142 @item unqualified @code{%code}
9143 blocks are placed inside the parser class.
9144
9145 @item @code{%code lexer}
9146 blocks, if specified, should include the implementation of the
9147 scanner. If there is no such block, the scanner can be any class
9148 that implements the appropriate interface (see @pxref{Java Scanner
9149 Interface}).
9150 @end table
9151
9152 Other @code{%code} blocks are not supported in Java parsers.
9153 In particular, @code{%@{ @dots{} %@}} blocks should not be used
9154 and may give an error in future versions of Bison.
9155
9156 The epilogue has the same meaning as in C/C++ code and it can
9157 be used to define other classes used by the parser @emph{outside}
9158 the parser class.
9159 @end itemize
9160
9161
9162 @node Java Declarations Summary
9163 @subsection Java Declarations Summary
9164
9165 This summary only include declarations specific to Java or have special
9166 meaning when used in a Java parser.
9167
9168 @deffn {Directive} {%language "Java"}
9169 Generate a Java class for the parser.
9170 @end deffn
9171
9172 @deffn {Directive} %lex-param @{@var{type} @var{name}@}
9173 A parameter for the lexer class defined by @code{%code lexer}
9174 @emph{only}, added as parameters to the lexer constructor and the parser
9175 constructor that @emph{creates} a lexer. Default is none.
9176 @xref{Java Scanner Interface}.
9177 @end deffn
9178
9179 @deffn {Directive} %name-prefix "@var{prefix}"
9180 The prefix of the parser class name @code{@var{prefix}Parser} if
9181 @code{%define parser_class_name} is not used. Default is @code{YY}.
9182 @xref{Java Bison Interface}.
9183 @end deffn
9184
9185 @deffn {Directive} %parse-param @{@var{type} @var{name}@}
9186 A parameter for the parser class added as parameters to constructor(s)
9187 and as fields initialized by the constructor(s). Default is none.
9188 @xref{Java Parser Interface}.
9189 @end deffn
9190
9191 @deffn {Directive} %token <@var{type}> @var{token} @dots{}
9192 Declare tokens. Note that the angle brackets enclose a Java @emph{type}.
9193 @xref{Java Semantic Values}.
9194 @end deffn
9195
9196 @deffn {Directive} %type <@var{type}> @var{nonterminal} @dots{}
9197 Declare the type of nonterminals. Note that the angle brackets enclose
9198 a Java @emph{type}.
9199 @xref{Java Semantic Values}.
9200 @end deffn
9201
9202 @deffn {Directive} %code @{ @var{code} @dots{} @}
9203 Code appended to the inside of the parser class.
9204 @xref{Java Differences}.
9205 @end deffn
9206
9207 @deffn {Directive} {%code imports} @{ @var{code} @dots{} @}
9208 Code inserted just after the @code{package} declaration.
9209 @xref{Java Differences}.
9210 @end deffn
9211
9212 @deffn {Directive} {%code lexer} @{ @var{code} @dots{} @}
9213 Code added to the body of a inner lexer class within the parser class.
9214 @xref{Java Scanner Interface}.
9215 @end deffn
9216
9217 @deffn {Directive} %% @var{code} @dots{}
9218 Code (after the second @code{%%}) appended to the end of the file,
9219 @emph{outside} the parser class.
9220 @xref{Java Differences}.
9221 @end deffn
9222
9223 @deffn {Directive} %@{ @var{code} @dots{} %@}
9224 Not supported. Use @code{%code import} instead.
9225 @xref{Java Differences}.
9226 @end deffn
9227
9228 @deffn {Directive} {%define abstract}
9229 Whether the parser class is declared @code{abstract}. Default is false.
9230 @xref{Java Bison Interface}.
9231 @end deffn
9232
9233 @deffn {Directive} {%define extends} "@var{superclass}"
9234 The superclass of the parser class. Default is none.
9235 @xref{Java Bison Interface}.
9236 @end deffn
9237
9238 @deffn {Directive} {%define final}
9239 Whether the parser class is declared @code{final}. Default is false.
9240 @xref{Java Bison Interface}.
9241 @end deffn
9242
9243 @deffn {Directive} {%define implements} "@var{interfaces}"
9244 The implemented interfaces of the parser class, a comma-separated list.
9245 Default is none.
9246 @xref{Java Bison Interface}.
9247 @end deffn
9248
9249 @deffn {Directive} {%define lex_throws} "@var{exceptions}"
9250 The exceptions thrown by the @code{yylex} method of the lexer, a
9251 comma-separated list. Default is @code{java.io.IOException}.
9252 @xref{Java Scanner Interface}.
9253 @end deffn
9254
9255 @deffn {Directive} {%define location_type} "@var{class}"
9256 The name of the class used for locations (a range between two
9257 positions). This class is generated as an inner class of the parser
9258 class by @command{bison}. Default is @code{Location}.
9259 @xref{Java Location Values}.
9260 @end deffn
9261
9262 @deffn {Directive} {%define package} "@var{package}"
9263 The package to put the parser class in. Default is none.
9264 @xref{Java Bison Interface}.
9265 @end deffn
9266
9267 @deffn {Directive} {%define parser_class_name} "@var{name}"
9268 The name of the parser class. Default is @code{YYParser} or
9269 @code{@var{name-prefix}Parser}.
9270 @xref{Java Bison Interface}.
9271 @end deffn
9272
9273 @deffn {Directive} {%define position_type} "@var{class}"
9274 The name of the class used for positions. This class must be supplied by
9275 the user. Default is @code{Position}.
9276 @xref{Java Location Values}.
9277 @end deffn
9278
9279 @deffn {Directive} {%define public}
9280 Whether the parser class is declared @code{public}. Default is false.
9281 @xref{Java Bison Interface}.
9282 @end deffn
9283
9284 @deffn {Directive} {%define stype} "@var{class}"
9285 The base type of semantic values. Default is @code{Object}.
9286 @xref{Java Semantic Values}.
9287 @end deffn
9288
9289 @deffn {Directive} {%define strictfp}
9290 Whether the parser class is declared @code{strictfp}. Default is false.
9291 @xref{Java Bison Interface}.
9292 @end deffn
9293
9294 @deffn {Directive} {%define throws} "@var{exceptions}"
9295 The exceptions thrown by user-supplied parser actions and
9296 @code{%initial-action}, a comma-separated list. Default is none.
9297 @xref{Java Parser Interface}.
9298 @end deffn
9299
9300
9301 @c ================================================= FAQ
9302
9303 @node FAQ
9304 @chapter Frequently Asked Questions
9305 @cindex frequently asked questions
9306 @cindex questions
9307
9308 Several questions about Bison come up occasionally. Here some of them
9309 are addressed.
9310
9311 @menu
9312 * Memory Exhausted:: Breaking the Stack Limits
9313 * How Can I Reset the Parser:: @code{yyparse} Keeps some State
9314 * Strings are Destroyed:: @code{yylval} Loses Track of Strings
9315 * Implementing Gotos/Loops:: Control Flow in the Calculator
9316 * Multiple start-symbols:: Factoring closely related grammars
9317 * Secure? Conform?:: Is Bison @acronym{POSIX} safe?
9318 * I can't build Bison:: Troubleshooting
9319 * Where can I find help?:: Troubleshouting
9320 * Bug Reports:: Troublereporting
9321 * More Languages:: Parsers in C++, Java, and so on
9322 * Beta Testing:: Experimenting development versions
9323 * Mailing Lists:: Meeting other Bison users
9324 @end menu
9325
9326 @node Memory Exhausted
9327 @section Memory Exhausted
9328
9329 @display
9330 My parser returns with error with a @samp{memory exhausted}
9331 message. What can I do?
9332 @end display
9333
9334 This question is already addressed elsewhere, @xref{Recursion,
9335 ,Recursive Rules}.
9336
9337 @node How Can I Reset the Parser
9338 @section How Can I Reset the Parser
9339
9340 The following phenomenon has several symptoms, resulting in the
9341 following typical questions:
9342
9343 @display
9344 I invoke @code{yyparse} several times, and on correct input it works
9345 properly; but when a parse error is found, all the other calls fail
9346 too. How can I reset the error flag of @code{yyparse}?
9347 @end display
9348
9349 @noindent
9350 or
9351
9352 @display
9353 My parser includes support for an @samp{#include}-like feature, in
9354 which case I run @code{yyparse} from @code{yyparse}. This fails
9355 although I did specify @code{%define api.pure}.
9356 @end display
9357
9358 These problems typically come not from Bison itself, but from
9359 Lex-generated scanners. Because these scanners use large buffers for
9360 speed, they might not notice a change of input file. As a
9361 demonstration, consider the following source file,
9362 @file{first-line.l}:
9363
9364 @verbatim
9365 %{
9366 #include <stdio.h>
9367 #include <stdlib.h>
9368 %}
9369 %%
9370 .*\n ECHO; return 1;
9371 %%
9372 int
9373 yyparse (char const *file)
9374 {
9375 yyin = fopen (file, "r");
9376 if (!yyin)
9377 exit (2);
9378 /* One token only. */
9379 yylex ();
9380 if (fclose (yyin) != 0)
9381 exit (3);
9382 return 0;
9383 }
9384
9385 int
9386 main (void)
9387 {
9388 yyparse ("input");
9389 yyparse ("input");
9390 return 0;
9391 }
9392 @end verbatim
9393
9394 @noindent
9395 If the file @file{input} contains
9396
9397 @verbatim
9398 input:1: Hello,
9399 input:2: World!
9400 @end verbatim
9401
9402 @noindent
9403 then instead of getting the first line twice, you get:
9404
9405 @example
9406 $ @kbd{flex -ofirst-line.c first-line.l}
9407 $ @kbd{gcc -ofirst-line first-line.c -ll}
9408 $ @kbd{./first-line}
9409 input:1: Hello,
9410 input:2: World!
9411 @end example
9412
9413 Therefore, whenever you change @code{yyin}, you must tell the
9414 Lex-generated scanner to discard its current buffer and switch to the
9415 new one. This depends upon your implementation of Lex; see its
9416 documentation for more. For Flex, it suffices to call
9417 @samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
9418 Flex-generated scanner needs to read from several input streams to
9419 handle features like include files, you might consider using Flex
9420 functions like @samp{yy_switch_to_buffer} that manipulate multiple
9421 input buffers.
9422
9423 If your Flex-generated scanner uses start conditions (@pxref{Start
9424 conditions, , Start conditions, flex, The Flex Manual}), you might
9425 also want to reset the scanner's state, i.e., go back to the initial
9426 start condition, through a call to @samp{BEGIN (0)}.
9427
9428 @node Strings are Destroyed
9429 @section Strings are Destroyed
9430
9431 @display
9432 My parser seems to destroy old strings, or maybe it loses track of
9433 them. Instead of reporting @samp{"foo", "bar"}, it reports
9434 @samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
9435 @end display
9436
9437 This error is probably the single most frequent ``bug report'' sent to
9438 Bison lists, but is only concerned with a misunderstanding of the role
9439 of the scanner. Consider the following Lex code:
9440
9441 @verbatim
9442 %{
9443 #include <stdio.h>
9444 char *yylval = NULL;
9445 %}
9446 %%
9447 .* yylval = yytext; return 1;
9448 \n /* IGNORE */
9449 %%
9450 int
9451 main ()
9452 {
9453 /* Similar to using $1, $2 in a Bison action. */
9454 char *fst = (yylex (), yylval);
9455 char *snd = (yylex (), yylval);
9456 printf ("\"%s\", \"%s\"\n", fst, snd);
9457 return 0;
9458 }
9459 @end verbatim
9460
9461 If you compile and run this code, you get:
9462
9463 @example
9464 $ @kbd{flex -osplit-lines.c split-lines.l}
9465 $ @kbd{gcc -osplit-lines split-lines.c -ll}
9466 $ @kbd{printf 'one\ntwo\n' | ./split-lines}
9467 "one
9468 two", "two"
9469 @end example
9470
9471 @noindent
9472 this is because @code{yytext} is a buffer provided for @emph{reading}
9473 in the action, but if you want to keep it, you have to duplicate it
9474 (e.g., using @code{strdup}). Note that the output may depend on how
9475 your implementation of Lex handles @code{yytext}. For instance, when
9476 given the Lex compatibility option @option{-l} (which triggers the
9477 option @samp{%array}) Flex generates a different behavior:
9478
9479 @example
9480 $ @kbd{flex -l -osplit-lines.c split-lines.l}
9481 $ @kbd{gcc -osplit-lines split-lines.c -ll}
9482 $ @kbd{printf 'one\ntwo\n' | ./split-lines}
9483 "two", "two"
9484 @end example
9485
9486
9487 @node Implementing Gotos/Loops
9488 @section Implementing Gotos/Loops
9489
9490 @display
9491 My simple calculator supports variables, assignments, and functions,
9492 but how can I implement gotos, or loops?
9493 @end display
9494
9495 Although very pedagogical, the examples included in the document blur
9496 the distinction to make between the parser---whose job is to recover
9497 the structure of a text and to transmit it to subsequent modules of
9498 the program---and the processing (such as the execution) of this
9499 structure. This works well with so called straight line programs,
9500 i.e., precisely those that have a straightforward execution model:
9501 execute simple instructions one after the others.
9502
9503 @cindex abstract syntax tree
9504 @cindex @acronym{AST}
9505 If you want a richer model, you will probably need to use the parser
9506 to construct a tree that does represent the structure it has
9507 recovered; this tree is usually called the @dfn{abstract syntax tree},
9508 or @dfn{@acronym{AST}} for short. Then, walking through this tree,
9509 traversing it in various ways, will enable treatments such as its
9510 execution or its translation, which will result in an interpreter or a
9511 compiler.
9512
9513 This topic is way beyond the scope of this manual, and the reader is
9514 invited to consult the dedicated literature.
9515
9516
9517 @node Multiple start-symbols
9518 @section Multiple start-symbols
9519
9520 @display
9521 I have several closely related grammars, and I would like to share their
9522 implementations. In fact, I could use a single grammar but with
9523 multiple entry points.
9524 @end display
9525
9526 Bison does not support multiple start-symbols, but there is a very
9527 simple means to simulate them. If @code{foo} and @code{bar} are the two
9528 pseudo start-symbols, then introduce two new tokens, say
9529 @code{START_FOO} and @code{START_BAR}, and use them as switches from the
9530 real start-symbol:
9531
9532 @example
9533 %token START_FOO START_BAR;
9534 %start start;
9535 start: START_FOO foo
9536 | START_BAR bar;
9537 @end example
9538
9539 These tokens prevents the introduction of new conflicts. As far as the
9540 parser goes, that is all that is needed.
9541
9542 Now the difficult part is ensuring that the scanner will send these
9543 tokens first. If your scanner is hand-written, that should be
9544 straightforward. If your scanner is generated by Lex, them there is
9545 simple means to do it: recall that anything between @samp{%@{ ... %@}}
9546 after the first @code{%%} is copied verbatim in the top of the generated
9547 @code{yylex} function. Make sure a variable @code{start_token} is
9548 available in the scanner (e.g., a global variable or using
9549 @code{%lex-param} etc.), and use the following:
9550
9551 @example
9552 /* @r{Prologue.} */
9553 %%
9554 %@{
9555 if (start_token)
9556 @{
9557 int t = start_token;
9558 start_token = 0;
9559 return t;
9560 @}
9561 %@}
9562 /* @r{The rules.} */
9563 @end example
9564
9565
9566 @node Secure? Conform?
9567 @section Secure? Conform?
9568
9569 @display
9570 Is Bison secure? Does it conform to POSIX?
9571 @end display
9572
9573 If you're looking for a guarantee or certification, we don't provide it.
9574 However, Bison is intended to be a reliable program that conforms to the
9575 @acronym{POSIX} specification for Yacc. If you run into problems,
9576 please send us a bug report.
9577
9578 @node I can't build Bison
9579 @section I can't build Bison
9580
9581 @display
9582 I can't build Bison because @command{make} complains that
9583 @code{msgfmt} is not found.
9584 What should I do?
9585 @end display
9586
9587 Like most GNU packages with internationalization support, that feature
9588 is turned on by default. If you have problems building in the @file{po}
9589 subdirectory, it indicates that your system's internationalization
9590 support is lacking. You can re-configure Bison with
9591 @option{--disable-nls} to turn off this support, or you can install GNU
9592 gettext from @url{ftp://ftp.gnu.org/gnu/gettext/} and re-configure
9593 Bison. See the file @file{ABOUT-NLS} for more information.
9594
9595
9596 @node Where can I find help?
9597 @section Where can I find help?
9598
9599 @display
9600 I'm having trouble using Bison. Where can I find help?
9601 @end display
9602
9603 First, read this fine manual. Beyond that, you can send mail to
9604 @email{help-bison@@gnu.org}. This mailing list is intended to be
9605 populated with people who are willing to answer questions about using
9606 and installing Bison. Please keep in mind that (most of) the people on
9607 the list have aspects of their lives which are not related to Bison (!),
9608 so you may not receive an answer to your question right away. This can
9609 be frustrating, but please try not to honk them off; remember that any
9610 help they provide is purely voluntary and out of the kindness of their
9611 hearts.
9612
9613 @node Bug Reports
9614 @section Bug Reports
9615
9616 @display
9617 I found a bug. What should I include in the bug report?
9618 @end display
9619
9620 Before you send a bug report, make sure you are using the latest
9621 version. Check @url{ftp://ftp.gnu.org/pub/gnu/bison/} or one of its
9622 mirrors. Be sure to include the version number in your bug report. If
9623 the bug is present in the latest version but not in a previous version,
9624 try to determine the most recent version which did not contain the bug.
9625
9626 If the bug is parser-related, you should include the smallest grammar
9627 you can which demonstrates the bug. The grammar file should also be
9628 complete (i.e., I should be able to run it through Bison without having
9629 to edit or add anything). The smaller and simpler the grammar, the
9630 easier it will be to fix the bug.
9631
9632 Include information about your compilation environment, including your
9633 operating system's name and version and your compiler's name and
9634 version. If you have trouble compiling, you should also include a
9635 transcript of the build session, starting with the invocation of
9636 `configure'. Depending on the nature of the bug, you may be asked to
9637 send additional files as well (such as `config.h' or `config.cache').
9638
9639 Patches are most welcome, but not required. That is, do not hesitate to
9640 send a bug report just because you can not provide a fix.
9641
9642 Send bug reports to @email{bug-bison@@gnu.org}.
9643
9644 @node More Languages
9645 @section More Languages
9646
9647 @display
9648 Will Bison ever have C++ and Java support? How about @var{insert your
9649 favorite language here}?
9650 @end display
9651
9652 C++ and Java support is there now, and is documented. We'd love to add other
9653 languages; contributions are welcome.
9654
9655 @node Beta Testing
9656 @section Beta Testing
9657
9658 @display
9659 What is involved in being a beta tester?
9660 @end display
9661
9662 It's not terribly involved. Basically, you would download a test
9663 release, compile it, and use it to build and run a parser or two. After
9664 that, you would submit either a bug report or a message saying that
9665 everything is okay. It is important to report successes as well as
9666 failures because test releases eventually become mainstream releases,
9667 but only if they are adequately tested. If no one tests, development is
9668 essentially halted.
9669
9670 Beta testers are particularly needed for operating systems to which the
9671 developers do not have easy access. They currently have easy access to
9672 recent GNU/Linux and Solaris versions. Reports about other operating
9673 systems are especially welcome.
9674
9675 @node Mailing Lists
9676 @section Mailing Lists
9677
9678 @display
9679 How do I join the help-bison and bug-bison mailing lists?
9680 @end display
9681
9682 See @url{http://lists.gnu.org/}.
9683
9684 @c ================================================= Table of Symbols
9685
9686 @node Table of Symbols
9687 @appendix Bison Symbols
9688 @cindex Bison symbols, table of
9689 @cindex symbols in Bison, table of
9690
9691 @deffn {Variable} @@$
9692 In an action, the location of the left-hand side of the rule.
9693 @xref{Locations, , Locations Overview}.
9694 @end deffn
9695
9696 @deffn {Variable} @@@var{n}
9697 In an action, the location of the @var{n}-th symbol of the right-hand
9698 side of the rule. @xref{Locations, , Locations Overview}.
9699 @end deffn
9700
9701 @deffn {Variable} $$
9702 In an action, the semantic value of the left-hand side of the rule.
9703 @xref{Actions}.
9704 @end deffn
9705
9706 @deffn {Variable} $@var{n}
9707 In an action, the semantic value of the @var{n}-th symbol of the
9708 right-hand side of the rule. @xref{Actions}.
9709 @end deffn
9710
9711 @deffn {Delimiter} %%
9712 Delimiter used to separate the grammar rule section from the
9713 Bison declarations section or the epilogue.
9714 @xref{Grammar Layout, ,The Overall Layout of a Bison Grammar}.
9715 @end deffn
9716
9717 @c Don't insert spaces, or check the DVI output.
9718 @deffn {Delimiter} %@{@var{code}%@}
9719 All code listed between @samp{%@{} and @samp{%@}} is copied directly to
9720 the output file uninterpreted. Such code forms the prologue of the input
9721 file. @xref{Grammar Outline, ,Outline of a Bison
9722 Grammar}.
9723 @end deffn
9724
9725 @deffn {Construct} /*@dots{}*/
9726 Comment delimiters, as in C.
9727 @end deffn
9728
9729 @deffn {Delimiter} :
9730 Separates a rule's result from its components. @xref{Rules, ,Syntax of
9731 Grammar Rules}.
9732 @end deffn
9733
9734 @deffn {Delimiter} ;
9735 Terminates a rule. @xref{Rules, ,Syntax of Grammar Rules}.
9736 @end deffn
9737
9738 @deffn {Delimiter} |
9739 Separates alternate rules for the same result nonterminal.
9740 @xref{Rules, ,Syntax of Grammar Rules}.
9741 @end deffn
9742
9743 @deffn {Directive} <*>
9744 Used to define a default tagged @code{%destructor} or default tagged
9745 @code{%printer}.
9746
9747 This feature is experimental.
9748 More user feedback will help to determine whether it should become a permanent
9749 feature.
9750
9751 @xref{Destructor Decl, , Freeing Discarded Symbols}.
9752 @end deffn
9753
9754 @deffn {Directive} <>
9755 Used to define a default tagless @code{%destructor} or default tagless
9756 @code{%printer}.
9757
9758 This feature is experimental.
9759 More user feedback will help to determine whether it should become a permanent
9760 feature.
9761
9762 @xref{Destructor Decl, , Freeing Discarded Symbols}.
9763 @end deffn
9764
9765 @deffn {Symbol} $accept
9766 The predefined nonterminal whose only rule is @samp{$accept: @var{start}
9767 $end}, where @var{start} is the start symbol. @xref{Start Decl, , The
9768 Start-Symbol}. It cannot be used in the grammar.
9769 @end deffn
9770
9771 @deffn {Directive} %code @{@var{code}@}
9772 @deffnx {Directive} %code @var{qualifier} @{@var{code}@}
9773 Insert @var{code} verbatim into output parser source.
9774 @xref{Decl Summary,,%code}.
9775 @end deffn
9776
9777 @deffn {Directive} %debug
9778 Equip the parser for debugging. @xref{Decl Summary}.
9779 @end deffn
9780
9781 @deffn {Directive} %debug
9782 Equip the parser for debugging. @xref{Decl Summary}.
9783 @end deffn
9784
9785 @ifset defaultprec
9786 @deffn {Directive} %default-prec
9787 Assign a precedence to rules that lack an explicit @samp{%prec}
9788 modifier. @xref{Contextual Precedence, ,Context-Dependent
9789 Precedence}.
9790 @end deffn
9791 @end ifset
9792
9793 @deffn {Directive} %define @var{define-variable}
9794 @deffnx {Directive} %define @var{define-variable} @var{value}
9795 Define a variable to adjust Bison's behavior.
9796 @xref{Decl Summary,,%define}.
9797 @end deffn
9798
9799 @deffn {Directive} %defines
9800 Bison declaration to create a header file meant for the scanner.
9801 @xref{Decl Summary}.
9802 @end deffn
9803
9804 @deffn {Directive} %defines @var{defines-file}
9805 Same as above, but save in the file @var{defines-file}.
9806 @xref{Decl Summary}.
9807 @end deffn
9808
9809 @deffn {Directive} %destructor
9810 Specify how the parser should reclaim the memory associated to
9811 discarded symbols. @xref{Destructor Decl, , Freeing Discarded Symbols}.
9812 @end deffn
9813
9814 @deffn {Directive} %dprec
9815 Bison declaration to assign a precedence to a rule that is used at parse
9816 time to resolve reduce/reduce conflicts. @xref{GLR Parsers, ,Writing
9817 @acronym{GLR} Parsers}.
9818 @end deffn
9819
9820 @deffn {Symbol} $end
9821 The predefined token marking the end of the token stream. It cannot be
9822 used in the grammar.
9823 @end deffn
9824
9825 @deffn {Symbol} error
9826 A token name reserved for error recovery. This token may be used in
9827 grammar rules so as to allow the Bison parser to recognize an error in
9828 the grammar without halting the process. In effect, a sentence
9829 containing an error may be recognized as valid. On a syntax error, the
9830 token @code{error} becomes the current lookahead token. Actions
9831 corresponding to @code{error} are then executed, and the lookahead
9832 token is reset to the token that originally caused the violation.
9833 @xref{Error Recovery}.
9834 @end deffn
9835
9836 @deffn {Directive} %error-verbose
9837 Bison declaration to request verbose, specific error message strings
9838 when @code{yyerror} is called.
9839 @end deffn
9840
9841 @deffn {Directive} %file-prefix "@var{prefix}"
9842 Bison declaration to set the prefix of the output files. @xref{Decl
9843 Summary}.
9844 @end deffn
9845
9846 @deffn {Directive} %glr-parser
9847 Bison declaration to produce a @acronym{GLR} parser. @xref{GLR
9848 Parsers, ,Writing @acronym{GLR} Parsers}.
9849 @end deffn
9850
9851 @deffn {Directive} %initial-action
9852 Run user code before parsing. @xref{Initial Action Decl, , Performing Actions b efore Parsing}.
9853 @end deffn
9854
9855 @deffn {Directive} %language
9856 Specify the programming language for the generated parser.
9857 @xref{Decl Summary}.
9858 @end deffn
9859
9860 @deffn {Directive} %left
9861 Bison declaration to assign left associativity to token(s).
9862 @xref{Precedence Decl, ,Operator Precedence}.
9863 @end deffn
9864
9865 @deffn {Directive} %lex-param @{@var{argument-declaration}@}
9866 Bison declaration to specifying an additional parameter that
9867 @code{yylex} should accept. @xref{Pure Calling,, Calling Conventions
9868 for Pure Parsers}.
9869 @end deffn
9870
9871 @deffn {Directive} %merge
9872 Bison declaration to assign a merging function to a rule. If there is a
9873 reduce/reduce conflict with a rule having the same merging function, the
9874 function is applied to the two semantic values to get a single result.
9875 @xref{GLR Parsers, ,Writing @acronym{GLR} Parsers}.
9876 @end deffn
9877
9878 @deffn {Directive} %name-prefix "@var{prefix}"
9879 Bison declaration to rename the external symbols. @xref{Decl Summary}.
9880 @end deffn
9881
9882 @ifset defaultprec
9883 @deffn {Directive} %no-default-prec
9884 Do not assign a precedence to rules that lack an explicit @samp{%prec}
9885 modifier. @xref{Contextual Precedence, ,Context-Dependent
9886 Precedence}.
9887 @end deffn
9888 @end ifset
9889
9890 @deffn {Directive} %no-lines
9891 Bison declaration to avoid generating @code{#line} directives in the
9892 parser file. @xref{Decl Summary}.
9893 @end deffn
9894
9895 @deffn {Directive} %nonassoc
9896 Bison declaration to assign nonassociativity to token(s).
9897 @xref{Precedence Decl, ,Operator Precedence}.
9898 @end deffn
9899
9900 @deffn {Directive} %output "@var{file}"
9901 Bison declaration to set the name of the parser file. @xref{Decl
9902 Summary}.
9903 @end deffn
9904
9905 @deffn {Directive} %parse-param @{@var{argument-declaration}@}
9906 Bison declaration to specifying an additional parameter that
9907 @code{yyparse} should accept. @xref{Parser Function,, The Parser
9908 Function @code{yyparse}}.
9909 @end deffn
9910
9911 @deffn {Directive} %prec
9912 Bison declaration to assign a precedence to a specific rule.
9913 @xref{Contextual Precedence, ,Context-Dependent Precedence}.
9914 @end deffn
9915
9916 @deffn {Directive} %pure-parser
9917 Deprecated version of @code{%define api.pure} (@pxref{Decl Summary, ,%define}),
9918 for which Bison is more careful to warn about unreasonable usage.
9919 @end deffn
9920
9921 @deffn {Directive} %require "@var{version}"
9922 Require version @var{version} or higher of Bison. @xref{Require Decl, ,
9923 Require a Version of Bison}.
9924 @end deffn
9925
9926 @deffn {Directive} %right
9927 Bison declaration to assign right associativity to token(s).
9928 @xref{Precedence Decl, ,Operator Precedence}.
9929 @end deffn
9930
9931 @deffn {Directive} %skeleton
9932 Specify the skeleton to use; usually for development.
9933 @xref{Decl Summary}.
9934 @end deffn
9935
9936 @deffn {Directive} %start
9937 Bison declaration to specify the start symbol. @xref{Start Decl, ,The
9938 Start-Symbol}.
9939 @end deffn
9940
9941 @deffn {Directive} %token
9942 Bison declaration to declare token(s) without specifying precedence.
9943 @xref{Token Decl, ,Token Type Names}.
9944 @end deffn
9945
9946 @deffn {Directive} %token-table
9947 Bison declaration to include a token name table in the parser file.
9948 @xref{Decl Summary}.
9949 @end deffn
9950
9951 @deffn {Directive} %type
9952 Bison declaration to declare nonterminals. @xref{Type Decl,
9953 ,Nonterminal Symbols}.
9954 @end deffn
9955
9956 @deffn {Symbol} $undefined
9957 The predefined token onto which all undefined values returned by
9958 @code{yylex} are mapped. It cannot be used in the grammar, rather, use
9959 @code{error}.
9960 @end deffn
9961
9962 @deffn {Directive} %union
9963 Bison declaration to specify several possible data types for semantic
9964 values. @xref{Union Decl, ,The Collection of Value Types}.
9965 @end deffn
9966
9967 @deffn {Macro} YYABORT
9968 Macro to pretend that an unrecoverable syntax error has occurred, by
9969 making @code{yyparse} return 1 immediately. The error reporting
9970 function @code{yyerror} is not called. @xref{Parser Function, ,The
9971 Parser Function @code{yyparse}}.
9972
9973 For Java parsers, this functionality is invoked using @code{return YYABORT;}
9974 instead.
9975 @end deffn
9976
9977 @deffn {Macro} YYACCEPT
9978 Macro to pretend that a complete utterance of the language has been
9979 read, by making @code{yyparse} return 0 immediately.
9980 @xref{Parser Function, ,The Parser Function @code{yyparse}}.
9981
9982 For Java parsers, this functionality is invoked using @code{return YYACCEPT;}
9983 instead.
9984 @end deffn
9985
9986 @deffn {Macro} YYBACKUP
9987 Macro to discard a value from the parser stack and fake a lookahead
9988 token. @xref{Action Features, ,Special Features for Use in Actions}.
9989 @end deffn
9990
9991 @deffn {Variable} yychar
9992 External integer variable that contains the integer value of the
9993 lookahead token. (In a pure parser, it is a local variable within
9994 @code{yyparse}.) Error-recovery rule actions may examine this variable.
9995 @xref{Action Features, ,Special Features for Use in Actions}.
9996 @end deffn
9997
9998 @deffn {Variable} yyclearin
9999 Macro used in error-recovery rule actions. It clears the previous
10000 lookahead token. @xref{Error Recovery}.
10001 @end deffn
10002
10003 @deffn {Macro} YYDEBUG
10004 Macro to define to equip the parser with tracing code. @xref{Tracing,
10005 ,Tracing Your Parser}.
10006 @end deffn
10007
10008 @deffn {Variable} yydebug
10009 External integer variable set to zero by default. If @code{yydebug}
10010 is given a nonzero value, the parser will output information on input
10011 symbols and parser action. @xref{Tracing, ,Tracing Your Parser}.
10012 @end deffn
10013
10014 @deffn {Macro} yyerrok
10015 Macro to cause parser to recover immediately to its normal mode
10016 after a syntax error. @xref{Error Recovery}.
10017 @end deffn
10018
10019 @deffn {Macro} YYERROR
10020 Macro to pretend that a syntax error has just been detected: call
10021 @code{yyerror} and then perform normal error recovery if possible
10022 (@pxref{Error Recovery}), or (if recovery is impossible) make
10023 @code{yyparse} return 1. @xref{Error Recovery}.
10024
10025 For Java parsers, this functionality is invoked using @code{return YYERROR;}
10026 instead.
10027 @end deffn
10028
10029 @deffn {Function} yyerror
10030 User-supplied function to be called by @code{yyparse} on error.
10031 @xref{Error Reporting, ,The Error
10032 Reporting Function @code{yyerror}}.
10033 @end deffn
10034
10035 @deffn {Macro} YYERROR_VERBOSE
10036 An obsolete macro that you define with @code{#define} in the prologue
10037 to request verbose, specific error message strings
10038 when @code{yyerror} is called. It doesn't matter what definition you
10039 use for @code{YYERROR_VERBOSE}, just whether you define it. Using
10040 @code{%error-verbose} is preferred.
10041 @end deffn
10042
10043 @deffn {Macro} YYINITDEPTH
10044 Macro for specifying the initial size of the parser stack.
10045 @xref{Memory Management}.
10046 @end deffn
10047
10048 @deffn {Function} yylex
10049 User-supplied lexical analyzer function, called with no arguments to get
10050 the next token. @xref{Lexical, ,The Lexical Analyzer Function
10051 @code{yylex}}.
10052 @end deffn
10053
10054 @deffn {Macro} YYLEX_PARAM
10055 An obsolete macro for specifying an extra argument (or list of extra
10056 arguments) for @code{yyparse} to pass to @code{yylex}. The use of this
10057 macro is deprecated, and is supported only for Yacc like parsers.
10058 @xref{Pure Calling,, Calling Conventions for Pure Parsers}.
10059 @end deffn
10060
10061 @deffn {Variable} yylloc
10062 External variable in which @code{yylex} should place the line and column
10063 numbers associated with a token. (In a pure parser, it is a local
10064 variable within @code{yyparse}, and its address is passed to
10065 @code{yylex}.)
10066 You can ignore this variable if you don't use the @samp{@@} feature in the
10067 grammar actions.
10068 @xref{Token Locations, ,Textual Locations of Tokens}.
10069 In semantic actions, it stores the location of the lookahead token.
10070 @xref{Actions and Locations, ,Actions and Locations}.
10071 @end deffn
10072
10073 @deffn {Type} YYLTYPE
10074 Data type of @code{yylloc}; by default, a structure with four
10075 members. @xref{Location Type, , Data Types of Locations}.
10076 @end deffn
10077
10078 @deffn {Variable} yylval
10079 External variable in which @code{yylex} should place the semantic
10080 value associated with a token. (In a pure parser, it is a local
10081 variable within @code{yyparse}, and its address is passed to
10082 @code{yylex}.)
10083 @xref{Token Values, ,Semantic Values of Tokens}.
10084 In semantic actions, it stores the semantic value of the lookahead token.
10085 @xref{Actions, ,Actions}.
10086 @end deffn
10087
10088 @deffn {Macro} YYMAXDEPTH
10089 Macro for specifying the maximum size of the parser stack. @xref{Memory
10090 Management}.
10091 @end deffn
10092
10093 @deffn {Variable} yynerrs
10094 Global variable which Bison increments each time it reports a syntax error.
10095 (In a pure parser, it is a local variable within @code{yyparse}. In a
10096 pure push parser, it is a member of yypstate.)
10097 @xref{Error Reporting, ,The Error Reporting Function @code{yyerror}}.
10098 @end deffn
10099
10100 @deffn {Function} yyparse
10101 The parser function produced by Bison; call this function to start
10102 parsing. @xref{Parser Function, ,The Parser Function @code{yyparse}}.
10103 @end deffn
10104
10105 @deffn {Function} yypstate_delete
10106 The function to delete a parser instance, produced by Bison in push mode;
10107 call this function to delete the memory associated with a parser.
10108 @xref{Parser Delete Function, ,The Parser Delete Function
10109 @code{yypstate_delete}}.
10110 (The current push parsing interface is experimental and may evolve.
10111 More user feedback will help to stabilize it.)
10112 @end deffn
10113
10114 @deffn {Function} yypstate_new
10115 The function to create a parser instance, produced by Bison in push mode;
10116 call this function to create a new parser.
10117 @xref{Parser Create Function, ,The Parser Create Function
10118 @code{yypstate_new}}.
10119 (The current push parsing interface is experimental and may evolve.
10120 More user feedback will help to stabilize it.)
10121 @end deffn
10122
10123 @deffn {Function} yypull_parse
10124 The parser function produced by Bison in push mode; call this function to
10125 parse the rest of the input stream.
10126 @xref{Pull Parser Function, ,The Pull Parser Function
10127 @code{yypull_parse}}.
10128 (The current push parsing interface is experimental and may evolve.
10129 More user feedback will help to stabilize it.)
10130 @end deffn
10131
10132 @deffn {Function} yypush_parse
10133 The parser function produced by Bison in push mode; call this function to
10134 parse a single token. @xref{Push Parser Function, ,The Push Parser Function
10135 @code{yypush_parse}}.
10136 (The current push parsing interface is experimental and may evolve.
10137 More user feedback will help to stabilize it.)
10138 @end deffn
10139
10140 @deffn {Macro} YYPARSE_PARAM
10141 An obsolete macro for specifying the name of a parameter that
10142 @code{yyparse} should accept. The use of this macro is deprecated, and
10143 is supported only for Yacc like parsers. @xref{Pure Calling,, Calling
10144 Conventions for Pure Parsers}.
10145 @end deffn
10146
10147 @deffn {Macro} YYRECOVERING
10148 The expression @code{YYRECOVERING ()} yields 1 when the parser
10149 is recovering from a syntax error, and 0 otherwise.
10150 @xref{Action Features, ,Special Features for Use in Actions}.
10151 @end deffn
10152
10153 @deffn {Macro} YYSTACK_USE_ALLOCA
10154 Macro used to control the use of @code{alloca} when the C
10155 @acronym{LALR}(1) parser needs to extend its stacks. If defined to 0,
10156 the parser will use @code{malloc} to extend its stacks. If defined to
10157 1, the parser will use @code{alloca}. Values other than 0 and 1 are
10158 reserved for future Bison extensions. If not defined,
10159 @code{YYSTACK_USE_ALLOCA} defaults to 0.
10160
10161 In the all-too-common case where your code may run on a host with a
10162 limited stack and with unreliable stack-overflow checking, you should
10163 set @code{YYMAXDEPTH} to a value that cannot possibly result in
10164 unchecked stack overflow on any of your target hosts when
10165 @code{alloca} is called. You can inspect the code that Bison
10166 generates in order to determine the proper numeric values. This will
10167 require some expertise in low-level implementation details.
10168 @end deffn
10169
10170 @deffn {Type} YYSTYPE
10171 Data type of semantic values; @code{int} by default.
10172 @xref{Value Type, ,Data Types of Semantic Values}.
10173 @end deffn
10174
10175 @node Glossary
10176 @appendix Glossary
10177 @cindex glossary
10178
10179 @table @asis
10180 @item Backus-Naur Form (@acronym{BNF}; also called ``Backus Normal Form'')
10181 Formal method of specifying context-free grammars originally proposed
10182 by John Backus, and slightly improved by Peter Naur in his 1960-01-02
10183 committee document contributing to what became the Algol 60 report.
10184 @xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10185
10186 @item Context-free grammars
10187 Grammars specified as rules that can be applied regardless of context.
10188 Thus, if there is a rule which says that an integer can be used as an
10189 expression, integers are allowed @emph{anywhere} an expression is
10190 permitted. @xref{Language and Grammar, ,Languages and Context-Free
10191 Grammars}.
10192
10193 @item Dynamic allocation
10194 Allocation of memory that occurs during execution, rather than at
10195 compile time or on entry to a function.
10196
10197 @item Empty string
10198 Analogous to the empty set in set theory, the empty string is a
10199 character string of length zero.
10200
10201 @item Finite-state stack machine
10202 A ``machine'' that has discrete states in which it is said to exist at
10203 each instant in time. As input to the machine is processed, the
10204 machine moves from state to state as specified by the logic of the
10205 machine. In the case of the parser, the input is the language being
10206 parsed, and the states correspond to various stages in the grammar
10207 rules. @xref{Algorithm, ,The Bison Parser Algorithm}.
10208
10209 @item Generalized @acronym{LR} (@acronym{GLR})
10210 A parsing algorithm that can handle all context-free grammars, including those
10211 that are not @acronym{LALR}(1). It resolves situations that Bison's
10212 usual @acronym{LALR}(1)
10213 algorithm cannot by effectively splitting off multiple parsers, trying all
10214 possible parsers, and discarding those that fail in the light of additional
10215 right context. @xref{Generalized LR Parsing, ,Generalized
10216 @acronym{LR} Parsing}.
10217
10218 @item Grouping
10219 A language construct that is (in general) grammatically divisible;
10220 for example, `expression' or `declaration' in C@.
10221 @xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10222
10223 @item Infix operator
10224 An arithmetic operator that is placed between the operands on which it
10225 performs some operation.
10226
10227 @item Input stream
10228 A continuous flow of data between devices or programs.
10229
10230 @item Language construct
10231 One of the typical usage schemas of the language. For example, one of
10232 the constructs of the C language is the @code{if} statement.
10233 @xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10234
10235 @item Left associativity
10236 Operators having left associativity are analyzed from left to right:
10237 @samp{a+b+c} first computes @samp{a+b} and then combines with
10238 @samp{c}. @xref{Precedence, ,Operator Precedence}.
10239
10240 @item Left recursion
10241 A rule whose result symbol is also its first component symbol; for
10242 example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion, ,Recursive
10243 Rules}.
10244
10245 @item Left-to-right parsing
10246 Parsing a sentence of a language by analyzing it token by token from
10247 left to right. @xref{Algorithm, ,The Bison Parser Algorithm}.
10248
10249 @item Lexical analyzer (scanner)
10250 A function that reads an input stream and returns tokens one by one.
10251 @xref{Lexical, ,The Lexical Analyzer Function @code{yylex}}.
10252
10253 @item Lexical tie-in
10254 A flag, set by actions in the grammar rules, which alters the way
10255 tokens are parsed. @xref{Lexical Tie-ins}.
10256
10257 @item Literal string token
10258 A token which consists of two or more fixed characters. @xref{Symbols}.
10259
10260 @item Lookahead token
10261 A token already read but not yet shifted. @xref{Lookahead, ,Lookahead
10262 Tokens}.
10263
10264 @item @acronym{LALR}(1)
10265 The class of context-free grammars that Bison (like most other parser
10266 generators) can handle; a subset of @acronym{LR}(1). @xref{Mystery
10267 Conflicts, ,Mysterious Reduce/Reduce Conflicts}.
10268
10269 @item @acronym{LR}(1)
10270 The class of context-free grammars in which at most one token of
10271 lookahead is needed to disambiguate the parsing of any piece of input.
10272
10273 @item Nonterminal symbol
10274 A grammar symbol standing for a grammatical construct that can
10275 be expressed through rules in terms of smaller constructs; in other
10276 words, a construct that is not a token. @xref{Symbols}.
10277
10278 @item Parser
10279 A function that recognizes valid sentences of a language by analyzing
10280 the syntax structure of a set of tokens passed to it from a lexical
10281 analyzer.
10282
10283 @item Postfix operator
10284 An arithmetic operator that is placed after the operands upon which it
10285 performs some operation.
10286
10287 @item Reduction
10288 Replacing a string of nonterminals and/or terminals with a single
10289 nonterminal, according to a grammar rule. @xref{Algorithm, ,The Bison
10290 Parser Algorithm}.
10291
10292 @item Reentrant
10293 A reentrant subprogram is a subprogram which can be in invoked any
10294 number of times in parallel, without interference between the various
10295 invocations. @xref{Pure Decl, ,A Pure (Reentrant) Parser}.
10296
10297 @item Reverse polish notation
10298 A language in which all operators are postfix operators.
10299
10300 @item Right recursion
10301 A rule whose result symbol is also its last component symbol; for
10302 example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion, ,Recursive
10303 Rules}.
10304
10305 @item Semantics
10306 In computer languages, the semantics are specified by the actions
10307 taken for each instance of the language, i.e., the meaning of
10308 each statement. @xref{Semantics, ,Defining Language Semantics}.
10309
10310 @item Shift
10311 A parser is said to shift when it makes the choice of analyzing
10312 further input from the stream rather than reducing immediately some
10313 already-recognized rule. @xref{Algorithm, ,The Bison Parser Algorithm}.
10314
10315 @item Single-character literal
10316 A single character that is recognized and interpreted as is.
10317 @xref{Grammar in Bison, ,From Formal Rules to Bison Input}.
10318
10319 @item Start symbol
10320 The nonterminal symbol that stands for a complete valid utterance in
10321 the language being parsed. The start symbol is usually listed as the
10322 first nonterminal symbol in a language specification.
10323 @xref{Start Decl, ,The Start-Symbol}.
10324
10325 @item Symbol table
10326 A data structure where symbol names and associated data are stored
10327 during parsing to allow for recognition and use of existing
10328 information in repeated uses of a symbol. @xref{Multi-function Calc}.
10329
10330 @item Syntax error
10331 An error encountered during parsing of an input stream due to invalid
10332 syntax. @xref{Error Recovery}.
10333
10334 @item Token
10335 A basic, grammatically indivisible unit of a language. The symbol
10336 that describes a token in the grammar is a terminal symbol.
10337 The input of the Bison parser is a stream of tokens which comes from
10338 the lexical analyzer. @xref{Symbols}.
10339
10340 @item Terminal symbol
10341 A grammar symbol that has no rules in the grammar and therefore is
10342 grammatically indivisible. The piece of text it represents is a token.
10343 @xref{Language and Grammar, ,Languages and Context-Free Grammars}.
10344 @end table
10345
10346 @node Copying This Manual
10347 @appendix Copying This Manual
10348 @include fdl.texi
10349
10350 @node Index
10351 @unnumbered Index
10352
10353 @printindex cp
10354
10355 @bye
10356
10357 @c LocalWords: texinfo setfilename settitle setchapternewpage finalout
10358 @c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex
10359 @c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry
10360 @c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa
10361 @c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc
10362 @c LocalWords: rpcalc Lexer Expr ltcalc mfcalc yylex
10363 @c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref
10364 @c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex
10365 @c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge
10366 @c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG
10367 @c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit
10368 @c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok
10369 @c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln
10370 @c LocalWords: smallexample symrec val tptr FNCT fnctptr func struct sym
10371 @c LocalWords: fnct putsym getsym fname arith fncts atan ptr malloc sizeof
10372 @c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum
10373 @c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype
10374 @c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs
10375 @c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES
10376 @c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param
10377 @c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP
10378 @c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword
10379 @c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH
10380 @c LocalWords: YYINITDEPTH stmnts ref stmnt initdcl maybeasm notype
10381 @c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args
10382 @c LocalWords: infile ypp yxx outfile itemx tex leaderfill
10383 @c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll
10384 @c LocalWords: nbar yytext fst snd osplit ntwo strdup AST
10385 @c LocalWords: YYSTACK DVI fdl printindex
OLDNEW
« no previous file with comments | « bison/src/bison/2.4.1/bison-2.4.1-src/doc/bison.info ('k') | bison/src/bison/2.4.1/bison-2.4.1-src/doc/bison.x » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698