Interpreters from functions and grammars

Comput Lang Vol ll. No 2, pp 85--104, 1986 0096-0551 86 5300+000 Pnnted m Great Britain Pergamon Journal~ Ltd INTERPRETERS FROM FUNCTIONS A N D G R A M M A R S GREG MICHAELSON Department of Computer Science, Heriot-Watt Um~ersity, 79 Grassmarket, Edinburgh EHI 2HJ. U K, (Received 23 September 1985) Abstract--The addition of context free grammar rules to a functional language simplifies the construction of interpreters from denotationaI semantic language definitions. Functional abstraction over grammar rules enables the specificationand processingof context sensitive language syntax aspects in a functional style. DenotatLonaI semantics Language design Language implementation Functional language Grammar rules Interpreters Contextsensitive processing 1. I N T R O D U C T I O N 1.1 Denotational semantics Denotational semantics [1] is a formalism for defining programming languages. A denotational semantic definition consists of semantic domains, abstract syntax and semantic functions. The semantic domains define the universe of objects in which the language is given meaning, the abstract syntax defines meaningful syntactic constructs and the meaning of each abstract syntax construct is defined by a semantic function over the serriantic domains. Abstract syntax is distinguished from concrete syntax which describes the physical representation of programs and contains details which are not significant from a semantic point of view [2]. Approaches based on denotational semantics are used to provide succinct and theoretically rigorous programming language standards. These then inform language implementations and program manipulation. Such approaches are also used for programming language design [3] but there are few software tools which are appropriate for experimenting with denotational semantic based designs. 1.2 Compiler generators Compiler generators [4], which are used to automate the construction of production compilers, are based on very different methodologies to denotational semantics and place a far greater emphasis on concrete syntax. A typical compiler generator, like YACC [5], inputs a formalised definition of the context free [6] concrete syntax of a language along with semantic actions written in an imperative language. These implement context sensitive concrete syntax checking and the generation of the target language. When compiler generators are used with denotational semantic designs a basic problem is the lack of correspondence between the functional meta-language used to define semantic equations and the imperative semantic action language. Translating semantic equations into semantic actions loses clarity and is a potential source of errors. When the design changes so must the translation for the compiler generator. One alternative is to use an existing compiler generator with a new semantic action language which has a well defined correspondence with the functional meta-language [7]. Another is to develop a system which is oriented specifically for use with denotational semantics. 1.3 SIS SIS [8] is a language implementation system which enables direct experimentation with denotational semantic language designs. It takes in a full denotational semantic definition and uses it to generate what is in effect the code for an abstract machine. The code is then interpreted. The strict adherence to denotational semantics, however, with the formal separation of concrete syntax, abstract syntax and semantics, appears to restrict the ease with which SIS may be used. CL H :~: 85  86 GREGMICHAZLSOY In particular, the denotational semantic methodology treats concrete syntax considerations as of secondary importance with context sensitive syntax aspects often bring ignored or pushed into the semantics. Concrete syntax is a major part of a language design as it determines the physical appearance of programs and influences the ease with which they are understood and manipulated. SIS provides grammar rules for specifying a context free concrete syntax and associating it with the abstract syntax but there is no provision for context sensitivity other than incorporation in semantic equations. SIS is essentially a batch system and definitions are frozen by compiler-compilation. Thus, there is no facility for moving easily from definition to test without using the host filing system and associated tools, and re-compiling the whole definition for each change. There is, of course, no reason why SIS should not be implemented within an interactive environment but the need for separate but related sub-definitions may restrict flexibility and integration. 1.4 Navel Navel*, which is described in the rest of this paper, was conceived of as an interpreter-interpreter language which would enable the interactive testing and modification of language designs. It is a functional language, heavily influenced by the original 'let... i n . . . ' SASL [9], and has grammar rules fully integrated as functional objects. Navel interpreters are written in a denotational semantic style using functional semantic equations but, unlike SIS, denotational semantics is not directly implemented. The main differences are the loss of the traditional distinction between the syntactic and semantic meta-languages, the lack of need for abstract syntax to mediate concrete syntax and semantics, and the absence of semantic domain specifications. These differences lose much of the rigour of SIS but simplify interpretive implementation from a formal definition. Context sensitive aspects of concrete syntax may be specified in a functional style through higher order grammar rules. Subsequent sections describe the functional language, the construction of an interpreter for a small language from a denotational semantic definition, rule generalisation through functional abstraction and the use of higher order rules for checking assignment before use, declarations and type consistency. 2. NAVEL 2.1 Introduction Navel [10] is a weakly typed functional language. It is implemented within an interactive environment which provides file access to the host system and local editing facilities. The main features of the language are described through examples. Each example may pre-suppose preceding examples. In examples, the prompt 'ok' heralds the input and execution of an expression or command ending with a ';'. 2.2 Objects and definitions The Navel objects are integers, characters which coerce to integers, booleans, lists, fields, functions and rules. Global definition commands are used to establish name/object associations which are retained for use by subsequent definitions and expressions: ok def Value_Added_Tax = 15; ok Value__Added_Tax; 15 ok def pie_beans_and_chips = 135; ok pie-beans_and_chips.Value_Added_Tax/! 00; 2O *Not A Very Exciting Language.  Interpreters from functions and grammars 87 2.3 Ltsts and strmgs The concatenation operator ':' is used to construct lists. The empty list is '()'. The operators "hd' and "tl" are used to extract the head and tail from lists: ok def squares = 1:4:9:16:25:( ); ok hd tl squares; 4 List elements may also be accessed through indexing. The empty list is normally omitted from the end of lists: ok squares; 1:4:9:16:25: Strings are lists of characters. They are usually denoted by sequences of characters within ....s: ok def orange = 'o':'r':'a':'n':'g':'e':; ok orange; "'orange" Lists are lazy evaluated [l l], i.e. function calls in lists are not evaluated until they are selected. Lazy infinite lists may be constructed using recursiye functions. 2.4 Fields A field is an object tagged with a name and is denoted by the name followed by the object within '[' and ']'. The object is selected from a field with the o p e r a t o r ' ' " ok def plant = [tree "larch"]; ok p l a n t " tree; "larch" The field selection operator is overloaded to enable field object selection from lists: ok def plants = [tree "larch"]:[bush "privet" ]:[tree "fir"]:; ok plants ^ bush; "'privet" When there are several fields in a list with the same name, the index operator '@' is used to identify the requisite instance: ok plants A tree @ 2; "'fir" 2.5 Functions Functions are based on lambda calculus [1]. They may be defined with a single formal parameter: ok def cube = lam x.x,x,x; ok cube 3; 27 88 GREG MICHAEL.SON or a list of formal parameters: ok def sum_squares = lam x:y.x,x + y,y; ok sum_squares 3:4; 25 For function definitions, the 'lam' may be dropped and the formal parameters moved to the left of the ' = 's: ok def volume length:depth;height = length,depth,height; Function calls are evaluated in applicative order. Formal parameter lists may be arbitrarily complicated and used for structure match selection with actual parameter lists: ok def tl_hd (hh:ht):t = ht; ok tl_hd ("bangers":"mash"):("fish":"chips"):; ~'mash" 2.6 Control expressions Navel provides 'if" and 'case' control expressions which enable the construction of recursive functions: ok def power x:n = if n = 0 then 1 else x,(power x:n-l); ok def fib n = case n of o-) ~, 1-) 1, (fib n - l ) + (fib n-2); The 'case' expression ends with a default. 2.7 Local definitions Local definitions are used to introduce temporary name/object associations within expressions: ok def sum_squares x:y-- let sq n = n , n in (sq x ) + (sq y); They are evaluated in applicative order and are directly equivalent to function calls. Local definitions may be mutually recursive. For global definitions, as in POP2 [12], a forward reference to an undefined name in a function definition results in the creation of a dummy definition for the name which may be fully defined later, 2.8 Partial application Functions may return functions through partial application: ok def add x = lain y.x + y ;  Interpreters from functions and grammars 89 ok def increment = add 1; ok increment 99; 100 Partial application is used in subsequent sections to construct update functions which, in effect, extend functions. For example, an array may be modelled as a function from addresses to the values held at those addresses. Initially, the array is empty and returns "fail" for any address: ok def array_0 addr = "fail"; When a value is assigned to an array address, a new function is constructed which returns that value if passed the corresponding address or calls the old function to find the value: ok def new_array old_array:address:value = lain addr. (if addr = address then value else old_array addr); Now, for example, '33' may be assigned to address '3': ok def array_l = new_array array_0:3:33; and "55' may be assigned to address '5': ok def array_2 = new_array array_l:5:55; ok a r r a y ~ 3; 33 ok array__2 6; "fail" 2.9 Input and output Output is effected with the 'write' expression which returns the value of its argument and also sends it to the standard output. Characters and strings are output without quotes: ok def writeln 1-~ let line = write 1 in let nl = write '\n' in ( ); ok writeln "festive greetings!"; festive greetings! The 'readln' object returns the next line from the standard input as a string: ok readln; testing testing 1 2 3 "testing testing 1 2 3" Amusing effects result from its use in function calls in lazy evaluated lists! Other facilities include character input, lazy file input and file output.  90 GROG MICHAELSON 2.10 Grammar rules Navel grammar rules are based on context free productions. The notation used is yet another variant of B.N.F. [13]. As in B.N.F., productions with the same left hand side name are combined into a single rule with a number of options. Unlike B.N.F., right hand sides of productions are not necessarily bound to particular left hand side non-terminals but are objects in their own right. For Navel rules, the terminal symbols are strings and the non-terminal symbols are names associated with strings or rules. A rule consists of one or more options separated by Ts within '{' and '}'. At simplest, an option is a sequence of one or more strings and names. For example, the B.N.F. for binary numbers: (binary) :: = (digit) (binary) I (digit) (digit) :: = 0 [ 1 might be rendered as: ok def binary = {digit binary I digit}; ok def digit = {"0" I " i " } ; 2.11 Parsing with rules A rule is used like a function to parse a string to return a list containing the list representation of the parse tree, or the empty list if the parse fails, and the rest of the string. Parsing is left to right. For a rule, each option is tried in turrr until one succeeds. If an option begins with a string and the argument string starts with that string then the rest of the option is applied to the rest of the argument string and the resulting parse tree list starts with the string starting the option. Otherwise the option fails: ok {"l"} "101"; ("1":):"01" ok {"i .... 0"} "lOl"; ("l":"O":):"l" Similarly, if an option begins with a name then its associated object is found and used and a field consisting of the sub-tree tagged with the name forms the start of the parse tree list: ok {digit} "10"; ([digit "1":]:):"0" ok {binary} "10"; ([binary [digit "l":]:[binary [digit "0":]:]:]:): ok {binary} "0123"; ([binary [digit "O":]:[binary [digit "l":]:]:]:):"2Y' No attempt is made to automatically factor rules or to remove left recursion because, as is discussed in Section 4, rules may be abstracted over and may contain any expressions that return strings or rules. Thus, it is not possible in general to check statically a rule for left recursion or options with common initial symbol sequences. With some rules the parser may fail to recognise valid strings because there is no backtracking within options. Option order may be important. If two options have a common initial symbol sequence and the shorter option precedes the longer then the shorter may lead to successful  Interpreters from funct,ons and grammars 91 recognition of a sub-string: ok def binary = {digit ] digit binary}: ok {binary} "111"; ([binary [digit "'1":]:]:):"I1" 2.12 Finding a rule that produced a tree The 'rule' expression is similar to a 'case' expression and is used to identify a rule (not necessarily unique) that produced a parse tree. For example, to interpret binary numbers: ok def mdigit d = rule d of {"0"}-) 0, {"1"}-) 1, "fail"; ok def mbinary v b = rule b of {digit binary}-) mbinary 2,v + (mdigit b A digit) b ~ binary, {digit}-) 2,v + (mdigit b ^ digit), "fail"; ok def m string = let tree:rest = binary string in if tree = () I r e s t ( ) ( ) then "syntax error" else mbinary 0 tree; ok m "101"; 5 2.13 Simplifying rules Rules may contain sub-rules. A successful sub-rule option is treated as if it were in-line. The empty rule '{ }' matches anything and returns the empty list '( )'. This may be used to factor rules: ok def binary = {digit {binary I { }}}; The Navel rule 'number' matches a sequence of decimal digits and the rule "word' matches a sequence of letters: ok word "praxis"; "praxis": ok {number} "2001"; ([number "2001"]:): The rule 'fail' always fails. This is used in subsequent sections to initialise rules which are to be extended. Rules may be used for the generation of strings but this is not discussed here. 92 GREG MICHAELSON 2.14 Implementation The present Navel implementation is written in C and runs under UNIX*. Navel objects are represented by linked lists of tagged two-field cells held in a heap. All name/object associations are made and accessed via a stack. Programs are compiled to parse trees which contain stack frame relative addresses for names. These are found by symbolic evaluation at compile time. At run time, when a function is encountered the positions of objects for free variables are known from the compile time symbolic evaluation and the objects are picked up directly to make a closure. When a function is entere d, the actual parameter and free variable objects are pushed onto the stack. When the object associated with a name is required it is accessed directly through the stack. No text is held for programs. A pretty printer is used to reconstruct text from parse trees for function editing and saving global name/object associations as definitions in the hostess filing system. Preliminary timing tests suggest that for worst case insertion sort and for Ackermann's function. Navel runs at more than twice the speed of the 1979 SASL [14], the same speed as interpreted F R A N Z LISP [15] and half the speed of CProlog [16]. 3. FROM D E N O T A T I O N A L S E M A N T I C S TO A N A V E L I N T E R P R E T E R 3.1 Introduction The following sections describe a denotational semantics based on that for T I N Y [2] and discuss the construction of a Navel interpreter from it. TINY is an imperative language with assignment, integer arithmetic and I/O. Here. a subset of TINY is used without conditional and iterative commands to simplify the presentation. For example, to output double-the value of an input: a: = read b:=a+a output b 3.2 Denotational semantics 3.2.1 Semantic domains and abstract syntax. The semantic domains for T I N Y are: State = M e m o r y , I n p u t , O u t p u t Memory = I d e - ) [ N u m + {unbound}] Input = N u m , Output = N u m , Ide = identifiers Num = numbers A memory is a function mapping names onto values and returns 'unbound' for a name without an associated value. Inputs and outputs are lists of numbers. The abstract syntax is: E ::=011 [ read ] I I E l + E 2 where "E' ranges over the domain 'Exp' of expressions, and C ::=I:=El output E I C1 C2 where 'C' ranges over the domain 'Corn' of commands. 3.2.2 Semantic notation and auxiliary definitions. In the semantic notation, a list is denoted as a sequence of elements separated by ",'s within '(' and ")'. Alternatively, '.' is used as the concatenation operator. 'hd' and 'tl' select the head and tail of lists respectively. *UNIX is a trademark of Bell Laboratones.  Interpreters from functions and grammars 93 The conditional e x p r e s s i o n - - ' a - ) b , c ' - - m e a n s that if "a" is true then "b" is returned: otherwise "c" is returned. This m a y also be used to match a list of values with a list of variables and bind the ,,ariables, as, for example, in the second definition for function sequencing below. The o p e r a t o r "," is used for function sequencing. There are cases for sequencing a monadic function with m o n a d m and dyadic functions: a) f: D 1 - ) [ D 2 + {errorl] g: D 2 - ) [ D 3 + {error}] f , ,,--c"D I - ) [ D 3 + {error's] f • g = larnbda x.(f x = e r r o r ) - ) e r r o r , g ( f x) b) f: D I - ) [ [ D 2 • D3] + {error's] g: D 2 - ) D 3 - ) [ D 4 + {error~j] f • g: D I - ) [ D 4 + {error~j] f • g = lambda x.(f x = e r r o r ) - ) e r r o r , ( f x = (d2,d3))-)g d2 d3 It also traps errors. For function updating. "f[rv/lv]" means that a new function is constructed which when called returns 'rv' for an a r g u m e n t with value "Iv'; otherwise "f" is called with the argument: f[rv/lv] 1 = 1 = l v - ) r v , f t This is used to extend the m e m o r y function with a name and value from an assignment. The following auxiliary function is used in semantic definitions: result: N u m - ) S t a t e - ) [ [ N u m • State] + [errorl] result = lambda v s.(~.s) 3.2.3 Semantic functions. The semantic function for expressions is: E: E x p - ) S t a t e - ) [ [ N u m • State] + {error}] E[0] = result 0 E[I] = result 1 E[read] = lambda (ma.o).null i - ) e r r o r , (hd i,(m,tl i,o)) E[I] = l a m b d a (m,i,o).m I = u n b o u n d - ) e r r o r , (m I,(m,i,o)) E[EI + E2] = E [ E 1 ] , l a m b d a v l . E [ E 2 ] , l a m b d a v2.result (vl + v2) Note that for an identifier T in an expression, the m e m o r y function ' m ' returns "unbound" if the identifier has not been assigned to. Thus the context sensitive syntax requirement that names be assigned before use has been dealt with in the semantics. In Section 5, a syntactic specification for this using higher order g r a m m a r rules is discussed. The semantic function for c o m m a n d s is: C: C o m - ) S t a t e - ) [ S t a t e + {error}] C[I: = E] = E[E] • l a m b d a v (m,i,o).(m[v/I),i,o) C[output E] = E[E] • lambda v (m,i,o).(m,i,v.o) c [ c 1 c2] = C [ C 1 ] • C[C2] 3.3 Navel interpreter 3.3.1 Concrete syntax. To construct a Navel interpreter from this definition, it is first necessary to contrive a concrete syntax for the language. Left recursion is removed through the introduction of auxiliary rules: ok def b a s e = { " 0 " I "1'" I "'read'" [ word}; ok def exp = {base " ' + ' " exp [ base}: ok def c o m m = {word "': = ' " exp I "'output" exp}; ok 94 GREG MICHAELSON def comms = {comm comms [ comm}; ok {comms} "a: = 0 b: = a"; ([comms [comm [word "a"]:": = ":[exp [base "0":]:]: ]: [comms [comm [word "b"]:": = ":[exp [base [word "a"]:]:]: ]: ]: ]: ): 3.3.2 Semantic notation and auxiliary definitions in Navel. Many aspects of the semantic notation have direct Navel equivalents but function sequencing must be modelled explicitly: ok def stara f g x = i f (f x ) = e r r o r then error else g (f x); ok def starb f g x = i f (f x ) = error " then error else let d2:d3 = f x in g d2 d3; Function updating is modelled by the functional: ok defu frv lv= lain arg. (if arg = lv then rv else f arg); 'unbound' and 'error' are modelled as strings: ok def unbound = "unbound"; ok def error = "error"; The auxiliary function 'result' translates to: ok def result v s = v:s; 3.3.3 Semantic functions. The semantic functions in Navel are: ok def mbase b = rule b of {"0"}-)result 0, { " l " } - ) r e s u i t 1, {"read"}-)lam m:i:o.(if i = () then error else (hd i):(m:(tl i):o)),  Interpreters from functions and grammars 95 { w o r d } - ) l a m m:i:o.(if (m b " word) = u n b o u n d then error else ( m b word):(m:i:o)), "error-base":b; ok def mexp e = rule e of {base}-)mbase e ^ base, {base " + " e x p } - ) s t a r b (mbase e " base) lam vl.(starb (mexp e exp) lam v2.(result vl + v2)), "error-exp":e ok def m c o m m c = rule c o f {word ": = " e x p } - ) s t a r b (mexp c ^ exp) lam v m:i:o.(u m v c " word):i:o, { " o u t p u t " e x p } - ) s t a r b (mexp c ^ exp) lam v m:i:o.m:i:(v:o), "error--comm":c; ok def m c o m m s c = rule c o f {comm}-)mcomm c ^ comm, ^ ^ {comm c o m m s } - ) s t a r a ( m c o m m c c o m m ) ( m c o m m s c comms), "error-comms":c; - 3.3.4 Interpreter. To run the semantic functions as an interpreter, a function is constructed which takes the string for a T I N Y p r o g r a m with the input list, parses the p r o g r a m and calls the "mcomms' semantic function with the tree, an initial m e m o r y which returns ' u n b o u n d ' for all names, the input and an empty output list: ok def interpret text input = let tree:rest = c o m m s text in if tree = () [ r e s t ( ) ( ) then "syntax error":rest else let s = m c o m m s tree (lain name.unbound):input: in if s = error then error else let m:i:o = s in o; ok interpret "a: = read output a + a" 1:; 2: 3.3.5 Interactive testing. F o r interactive testing, 'interpret' m a y be called repeatedly by a recursive function that p r o m p t s for and inputs the p r o g r a m text and input lists: ok def p r o m p t _ r e a d p r o m p t = let p = write p r o m p t in readln; 96 GREG MICHAELSON ok def run n = let text = p r o m p L r e a d "'program)" in let data = convert (prompt_read "'data)") in let output = writeln (interpret text data) in run n; ok run ( ); program) output read + read data) 1 0 1: program) . .. The function 'convert" should transform a string into a list of numbers. One way is to parse the string and strip out the fields from the tree: ok def convert string = let numb = {"0" I "1"} and n u m b s = (numb numbs I numb} and value n = if n = "0": then 0 else 1 . . . . and strip n = rule n of {numb}-) (value n ^ numb):, {numb numbs}-) (value n ^ numb):(strip n " numbs), "'input error":n in strip (hd (numbs string)); Thus, rules may also be used to define and process the inputs for a programming language. Full checks that the inputs are correct are not included here. 3.4 Denotational semantics and Navel A denotational semantics consists of three separate sections with distinct description languages. In Navel, there are no semantic domain specifications and syntax and semantics are described in a unitary language. The domain specifications are used to describe the functionality of the semantic equations. This would correspond to specifying explicitly the types of formal parameters and results of functions in a strongly typed language and would enable checks for type consistency at compile time. Navel is weakly typed with run-time type checking. It is thus less rigorous than denotational semantics and has a relatively inefficient implementation. In denotational semantics, abstract syntax mediates concrete syntax and semantic equations. In Navel, there is no necessity to introduce abstract syntax. In principle, its absence will make the interpreter less general: the semantic functions must be changed if the concrete syntax is changed whereas for a denotational semantics it would only be necessary to change the concrete/abstract syntax association specification. In practise, however, it is not clear that language design and development proceed cleanly with changes in one area isolated from other areas. The omission of abstract syntax may lead to extra semantic functions, as in this example, but reduces the number of different stages in a design. There is one denotational semantic equation for each abstract syntax construct with sub- equations for each sub-construct. Sub-constructs and sub-equations are associated through implicit pattern matching on the sub-construct abstracted implicitly from a sub-tree. Sub-trees are then selected implicitly by mention of the associated sub-construct name.  Interpreters from functions and grammars 97 Part of the motivation for Navel's development was to investigate the association of syntax and semantics in language definitions. This is highlighted through explicit operations on tree represent- ations for rule identification and sub-tree selection. Here, Navel interpreters are more akin to operational semantics [17]. 4. G E N E R A L I S I N G RULES Rules have the same civil rights as other Navel objects. In particular they may be abstracted over which enables the specification of rule schemas. For example, consider the following rules for sequences of numbers and words: ok def numbers = {number {"," numbers l[ }}}: ok def w o r d s = {word ['1:" words I {}}}; These have a common structure which may be expressed by: ok def sequence one separator m a n y = {one {separator many I~ ~fi,~'~" and they may now be defined by: ok def numbers = sequence number "," {numbers}; ok -- def words = sequence word ":" {words}; When rules are formed in this way, the parse tree list fields will contain the names from the schema: ok numbers "11,22"; ([one [number "1 I"]]: [separator ",":]: [many [numbers [one [number "22"]]:]:]:): Unlike LISP[18] literals are not objects and so here there is no literal replacement. As with functions, the values of the arguments are used. Navel allows rules to contain bracketed expressions returning rules or strings. These are then treated as if they were in-line. Bracketing a name results in its value being used without a field being constructed for the resulting sub-tree. This may be used to make rule schemas which give appropriate names to fields: ok def sequence one separator many = {(one) {(separator) (many) [ {)}I: ok def numbers = sequence number "," {numbers}; ok numbers "11,22"; ([number "'11"]: [numbers [number "'22"]:]:): Such abstraction over rules enables the equivalent of consistent substitution in two-level grammars [19]. Two-level grammars are based on the generation of rules from rule schemas through what is in effect the macro expansion of rule names. This contrasts with Navel where names are fixed and rules are generated from schemas through parameter substitution. 98 GILEG MtCrtAELSON For example: ok def match n symb = ifn=l then {(symb)} else {(symb) (match n-1 symb)}; will generate a rule to recognise a specified length sequence for a given symbol. This may be used to recognise: a.b.c ~ for any given 'n'; ok def anbncn a b c n = {(match n a) (match n b) (match n c)}; ok def xnynzn-- let x = "x" and y = " y " and z = "z" in anbncn {x} {y} {z}; ok xnynzn 2 "xxyyzz"; ([x "'x"]:[x "'x"]:[y "y"]:[y "y"]:[z "z"]:[z "z"]:): Recognising: anbnc n for an unknown 'n' is more tricky. It might appear that: ok def anbncnstar a b c n = {(anbncn a b c n) I (anbncnstar a b c n + 1)}; would do the trick. This is, alas, only a partial solution. If the string being checked is not of the form: anbnc n then there is no value of 'n' for which the first rule option can succeed and the rule will recurse indefinitely. Such context sensitive checking requires the passing of information about the parse from one part of a rule to another. Context free rules have no mechanism for information passing other than that an earlier rule part must have succeeded for a later rule part to be reached. This may be overcome within a functional framework by making the application of each rule part explicit and then generating and passing information between them through intermediate processing. In effect, information from the start of the parse is used to select or construct appropriate rules from the rest of the parse. For example, to return to: a,bOc. where 'n' is unknown until the sequence has been examined, processing the 'a's may be used to guide the construction of a new rule to recognise the corresponding number of 'b's and ~c's. For each 'a', a new 'b' is added to the rule for 'b's and a new 'c' is added to the rule for "c's. At the same time, the tree for the 'a's is accumulated. When all the 'a's have been found, a rule  Interpreters from functions and grammars 99 is built to recognise the same number of "b's and "c's. The trees for the 'a's and for the "b's and 'c's are joined to form the final tree. ok def an a b c atree bn cn text = let tl:sl = a text in if t l ( )( ) then an a b c (hd tl):atree {(b) (bn)} {(c) (cn)} sl else let t2:s2 = {(bn) (cn)} sl in (append atree t2):s2; ok def anbncn a b c = a n a b c () {} {}; These approaches are used in the next two sections for parsing context sensitive aspects of programming languages. 5. C H E C K I N G V A R I A B L E A S S I G N M E N T B E F O R E USE 5.1 Assignment checking Considering the language T I N Y from section 4 with the additional requirement that an identifier in an expression must have appeared on the left had side of a preceding assignment. This may be specified in Navel in a manner similar to dynamic syntax [20, 21] by constructing rules from assignments for use in checking subsequent commands. Initially, a base may only be a digit ~3r a 'read': ok def b a s e = {"0" I "1" [ " r e a d " } ; ok defexp = {base " + " e x p l base}; ok def comm = {word ": = " exp I " o u t p u t " exp}; and there are no assigned variables: ok def assigned = fail; The rule for assigned variables is extended by the function: ok def new id old = {(id) [ (old)}; which when passed an identifier and the previous rule for assigned variables returns a rule which tries to recognise that identifier before using the previous assigned variable rule. When an assignment has been recognised, the identifier is checked to see if it is recognised by the rule for assigned identifiers. If it is then it has been assigned already and the current rules for assigned identifiers and commands are returned. Otherwise, the identifier is added to the rule for assigned identifiers which is then used to build a new rule for recognising commands: ok def check id:assigned:comm = let t:r = assigned id in if t( )( ) then assigned:comm else let newassigned = new id assigned in newassigned:(makecomm newassigned); 100 GREG MICHAELSON ok def m a k e c o m m assigned = let b a s e = {assigned I '0"" I "1"" I "'read"] and exp = {base " ' + "" exp I basel and c o m m - - [ w o r d "': = "" exp I "'output" exp l in {comm}; 5.2 Command sequence and program checking C o m m a n d sequences are recognized by a rule that picks up the first command. If it is the last c o m m a n d in the sequence then the tree for it is returned. Otherwise, if it is an assignment then new c o m m a n d and assigned identifier rules are constructed. The commands rule is called recursively to recognise the rest of the sequence and a final tree is constructed and returned: ok def comms assigned:comm text = let tl:rl = c o m m text in if tl = ( ) then ( ):text else if r l - - ( ) then ([comms tl]:): else let t'~:r2 = rule t l " comm of {word "': = " e x p } - ) comms (check (id tl):assigned:comm) rl, comms assigned:comm rl in if t 2 = ( ) then ( ):text else ([comms (hd tl):t2]:):r2; ok def id c = c ^ c o m m ^ word; The rule for a program is constructed from the rule for commands with the initial assigned identifier and c o m m a n d rules: ok def program = comms assigned:{comm}; ok program "'a: = 0 b: = a"; ([comms [comm [word "a"l:":---":[exp [base "0":]:]: 1: [comms [comm [word "b"]:": = ":[exp [base [assigned "a":]:]:]: ]: ]: 1: ): The interpreter can now be simplified with the removal of checks for errors from unassigned identifiers.  Interpreters from functions and grammars I01 5.3 Checking in syntax and semantics In the TINY semantics in Section 4, the update function: def u f r v I v = lam arg.(if arg = lv then rv else f arg); was used to extend the memory function. Here: def new id old = {(id) I (old)}; is used in the same way to extend rules. "Iv' and 'f' in the update function correspond to "id' and 'old" in the rule extender, The return o f ' r v ' for 'Iv' in the update function corresponds to the return of a tree for the successful recognition of 'id'. There is no equivalent for "arg' because rules have implicit formal parameters. Here it may be made explicit by: def new id old = lain arg.({(id) L (old)} arg); The use of the update and rule extender functions are further analogous. In TINY, assignments both introduce and update name/value associations For a name in an expression, the memory function is used to find the corresponding value. If the required name had not been introduced by a previous assignment then the memory function would return 'unbound'. Assignment before use is a syntactic requirement and is irrelevant for semantics [2] but the use of a memory function implicitly encompasses the checking of this requirement. 6. D E C L A R A T I O N AND TYPE C H E C K I N G 6.1 Syntax for declared and typed variables The following rules describe partially the syntax of a language fragTnent with declarations. assignment and integer arithmetic: ok def program = {declarations statements}; ok def declarations = {declaration {declarations I }}}; ok def declaration= {{"long" I "short"} word}; ok def statements = {statement {statements I {I}}: ok def statement = {word ": = " expression}: ok def expression = {base { " + " base I { }}}; ok def base = {word [ number [ "(" expression ")"}; It is required that no identifier may appear in more than one declaration and that an identifier must appear in a declaration before it appears in a statement. It is intended that the language should enable 'long' and 'short' precision arithmetic, that variables be typed according to the precision of the values they may hold and that 'long' values may not be coerced to 'short'. Thus if the identifier on the left of a statement appears in a declaration preceded by "short" then all identifiers in the expression on the right of the statement must also appear in declarations preceded by "short". 6.2 Denotational semantics with declarations For a denotational semantic definition of an imperative language with declarations it is sometimes convenient to model name/value associations in two stages, mediated by addresses, CL II 2--D 102 GREG MICHAELSON through an environment function which records name/address associations and a store function which records address/value associations. When a declaration is encountered, a new name/address association is added to the environment. When a name is encountered in a command, the environment is used to find the associated address. If the name is in an expression then the address is used to find the corresponding value from the store. If the name is that of a variable which is being assigned to then a new address/value association is added to the store. As with the memory function for TINY, environment use may encompass name checking. If a name in a command has not appeared in a preceding declaration then the environment will not have an address for it. Similarly, if a name in a declaration has appeared in a preceding declaration then the environment will already have an associated address for it. The environment function may also be extended to enable the recording and checking of type information for names. 6.3 Declaration checking Name checking may be transferred to the syntax with the construction of appropriate rules from declarations. For the above language, separate rules will be used to check whether or not identifiers have been declared as "'short" or "'long". Initially, nothing has been declared and so these rules should not match anything: ok def init_short = fail; ok def init_long = fail; As before, rules are extended with the function: ok def new old name =-{(name) ( o l d ) } ;[- When a declaration is found, the rules are used to check the identifier for re-declaration and the appropriate rule is then extended with the identifier: ok def decls string:long:short = let tree:rest = {{"short" I "long"} word} string in if tree = ( ) then long:short:string else if hd(long tree " w o r d ) ( ) ( ) I hd(short tree ^ w o r d ) ( ) ( ) then long:short:string else rule tree of {"long" word}-) decls rest:(new long tree ^ word):short, {"short" word}-) decls rest:long:(new short tree ^ word), "never going to reach h e r e . . . " ; 6.4 Assignment and program checking Separate rules are required to recognise assignment to variables declared as "long' and 'short'. They are built from a schema which defines local rules for recognising expressions: ok def makeassign nametype basetype = let base = {(basetype) J number f "(" expression ")"} and expression = {base { " + " base I { }}} in {(nametype) "': = " expression}; Programs are parsed by a function which uses 'decls' to parse the declarations and construct the rules for 'long's and "short's. They are then used with the assignment schema to construct the  Interpreters from functions and grammars 103 assignment rules which are incorporated into local rules for recognising statements: ok def program string = let long:short:rest = decls string:init_long:init_short in let lassign = makeassign {long} {long ! short} and sassign = makeassign {short} {short} and statement = {lassign I sassign} and statements = {statement {statements [ { }}} in statements rest; ok program "'short a long b short c c: = a + a b: = c + b"; ([statement [sassign [short "c":]: [exp [base [short "a":]:]:" +":[base [short "a":]:]:]: ]: ]: [statements [statement [lassign [long "b":]: [exp [base [short "c":]:]:" +":[base [long "b":]:]:]: ]~ ]: 1: ): To simplify the example there are no diagnostics and no tree is built for declarations: these could easily be added. 7. C O N C L U S I O N S The approaches described here could be used with any language with general pattern recognition facilities. For example, string scanning in Icon [22] is ideal for context free parsing and may be combined naturally with list construction for tree production. Prolog [23] provides context free grammar rules as syntactic sugaring of the underlying pattern recognition aspects of unification. Navel's novelty lies in the integration of grammar rules as first class objects within a functional language. Work continues on the use of higher order grammar rules in interpretive language imple- mentations, for example for handling forward references and to associate abstract and concrete syntax. Another area of interest is the more general use of grammar rules for the direct construction of data structures from data specifications. 8. S U M M A R Y This paper discusses Navel, a functional language which is intended for use as an interpreter- interpreter to simplify the construction of interactive tests intepreters from formal language definitions. Denotational semantics provides a rigorous and versatile methodology for defining pro- gramming languages. However, current production compiler generators are not suitable for direct experimentation with such language definitions because their semantic definition facilities are based on imperative languages which do not correspond closely to denotational semantics semantic [04 GREG M[CHAELSON equations. The compiler generator SIS is designed for use with denotational semantic definitions but is based on the separation of lexicaL syntactic and semantic formalisms which may restrict its flexibility. Pure functional languages are very similar to the semantic equations of denotational semantics but lack facilities for syntactic specification. Navel is a functional language with fully integrated grammar rules, implemented in an interactive environment. Its use enables the unitary expression of the syntactic as well as the semantic aspects of a denotational language definition. Although there is some loss of rigour, interpreter construction is simplified greatly. Here, Navel is illustrated through examples and used to construct an interpreter from the denotational semantic definition of a simple assignment language. Denotational semantics does not address the context sensitive aspects of programming language syntax. Navel provides a means to handle context sensitivity through functional abstraction over grammar rules to construct rule schemas, which are analogous to higher order functions. Here, this approach is presented through discussion of the classic 'a"b~c"' problem and is used to process assignment and declaration before used, and simple type matching. Acknowledgements--This work was started in the Department of Computing Science, University of Glasgow and is continued in the Department of Computer Science, Heriot-Watt University. The author wishes to thank Bdl Findlay (Glasgow). David Watt (Glasgov,). Paul Chisholm (Heriot-Watt) and Stuart Anderson (Heriot-Watt) for much fruitful d~sputation, Zdrav Podolski (Glasgow,,) and Chris Miller (Heriot-Watt) for making UNIX intelligible and David Watt, Paul Ch,sholm, Stuart Anderson and Hov~ard Williams (Heriot-Watt) for chewing through previous drafts of this paper. REFERENCES I. Stoy J Denotational Semantics: The Scott-Strachey Approach to Programming Language Theoo'. MIT Press. Cambridge. Mass. (1977). 2. Gordon M. The Denotational Description of Programm#tg Languages. Springer, New York (1979). 3. Tennent R. Language design methods based on semantic principles. Acta Informatica 8, 97-112 (1977). 4. Gries D. Compiler Constructton for Digttal Computers. Wdey, New York (1971). 5 Johnson S, Yacc: yet another compiler compiler No 32, Computing Science Technical Report, Bell Laboratories. Murray Hill, NJ 07974 (1975). 6. Hopcroft J. and Ullman J. Formal Languages and their Relation to Automata. Addison-Wesley, Reading, Mass. (1969). 7. Sethi R. Control flow aspects of semantic-directed compiling. ACM Trans. Programming Lang. Systems 5(4), 55'4-595 (1983). 8. Mosses P. SIS--Semantics lmplementatton System Reference Manual and User Guide. DAIMI MD-30, Computer Science Department, Aarhus University, Denmark (1979). 9. Turner D. SASL Language Manual. Department of Computational Science, University of St Andrews, St Andrews, Scotland (1975). 10. M,chaelson G. User Gutde to NAVEL. Department of Computer Science, Heriot-Watt University, Endmburgh, Scotland (1984) 11. Friedman D. and Wise D. CONS should not evaluate its arguments. In Automata, Languages and Programming (Edited by Milner R. and Michaelson S.), pp. 257-284. Endinburgh University Press, Edinburgh, Scotland (1976). 12. Burstall R., Collins J. and Popplestone R. Programming in POP2. Edinburgh University Press, Edinburgh, Scotland (1977). 13. Naur P. (Ed.), Revised report on the algorithmic language ALGOL 60. Commun. ACM 6(1). 1-17 (1963) 14. Turner D. SASL Language Manual. CS/79/3, Department of Computational Science, University of St Andrews, St Andrews, Scotland ([979). 15. Foderaro J. and Sklowar K. The FRANZ LISP Manual. University of California at Berkeley (1983). 16. Pereira F. CProlog User's Manual Verston 1.1 EdCAAD, Department of Architecture, University of Edinburgh, Edinburgh. Scotland (1982). 17. Wegner P. The Vienna defimtion language. ACM Comput. Surv. 4(I), 5-63 (1972). 18. McCarthy J. The LISP 1.5 Programmers" Manual. MIT Press, Cambridge, Mass. (1969). 19. Cleaveland J. and Uzgahs R. Grammars for Programming Languages, 4. Elsevier Computer Science Library (Programming Languages Series), Amsterdam (1977). 20. Hanford K. and Jones C. Dynamic syntax: A concept for the definition of the syntax of programming languages. TR 12.090, IBM Tech. Rep. (1971). 21. Ginsburg S. and Rounds E. Dynamic syntax specification using grammar forms. IEEE Trans. Software Engng SE..4I 1). 44-55 (1978). 22. Griswold R. E. and Griswold M. T. The Icon Programming Language. Prentice-Hall, Englewood Chffs, N.J. (1983). 23. Clocksin W. F. and Mellish C. S. Programming in Prolog. Springer, Berlin (1981). About the Author--GgEGOR'~ JoH'q MICHAELSONhas a B A. in Computer Science from the Umvers~ty of Essex (1973) and an M.Sc. in Computational Science from the University of St Andrews (1982). He has taught Computer Studies at Napier College, Edinburgh. Computing Science at the University of Glasgow, and is at present a Lecturer in Computer Science at the Heriot-Watt University, Edinburgh. His current research is into the use of formal definmons in programming language implementations.