softwaredesign/raw/book/설계원칙-201-286.md

# **Pattern Matching**

*Pattern matching* is a technology that supports the creation of domain-specific languages and other systems that should have an additive character.

Pattern matching is a generalization of equality testing. In equality testing we compare two objects to determine that they have the same structure and contents. In pattern matching, we generalize equality testing to allow some parts of the structure and contents to be unspecified. A *pattern* specifies certain parts of the data exactly, but it has "holes" (*pattern variables*) that match the unspecified parts of the data. We may impose constraints on what a pattern variable can match, and we may require that multiple instances of the same pattern variable match the same thing.

A pattern can be matched to a part of a larger datum; the context of the match is unspecified. The ability to work with partial information means that only the specified parts of the pattern are assumptions about the data matched; there are few or no assumptions about the unspecified parts.

This property of pattern matching enables the construction of very flexible rule-based systems. In a rule-based system one can add new rules to add new capabilities, though there are difficulties associated with how rules are defined and how they interact with one another. For example, if more than one rule is applicable, the results may depend on the ordering of application. We already encountered problems with the interaction of rules, in the boardgame rule interpreter. (See the critique on page 63.)

Besides the use of patterns to match data that meets a partial specification, patterns can themselves represent partially known information. Merging such patterns (*unification*) can generate more specific information than the individual patterns contribute.

Another use of pattern matching is as a generalization of generic procedures. Generic procedures allow us to do miraculous things by modulating the meanings of the free variables in a program. We may think of the program that employs a generic procedure, such as that bound to +, as advertising for a handler that can do the job of "+ing" the arguments supplied. A handler attached to a generic procedure is applicable if the arguments supplied satisfy the predicates provided when it was attached. But the language for advertising jobs that need to be done is rather limited—all we have is a single symbol, in this case +. If we instead use patterns to advertise jobs to be done and other patterns to advertise procedures that might do those jobs, we have a much richer language: *patterndirected invocation*.

## **4.1 Patterns**

The elementary laws of algebra can be expressed as equivalence of patterns:

$$a \times (b+c) \iff a \times b + a \times c$$

This is the distributive law of multiplication over addition. It says that we can replace one side with the other without changing the value of the expression. Each side of the law is a pattern, with pattern variables *a*, *b*, and *c*, and pattern constants × and +. What the law says is that if we find an algebraic expression that is the product of something and a sum of terms, we can replace it with a sum of two products, and vice versa.

Let's see how to organize programs based on pattern matching. A key idea here will be compilation of patterns into combinators that are the pieces of a matcher. In section 4.2 we will demonstrate this in a term-rewriting system for elementary algebra.

#### **A Language of Patterns**

The first job is to make up our language of patterns. We will start with something simple. We will make our patterns out of Lisp (Scheme) lists. Unlike the mathematical example above, we will not have reserved symbols, such as and +, so we will have to distinguish pattern variables from pattern constants. A pattern variable can be represented by a list beginning with the query symbol, ?. This is a traditional choice. So in this language the patterns that make up the distributive law may be represented as follows, assuming that we are manipulating Lisp prefix mathematical expressions:

```
(* (? a) (+ (? b) (? c)))
(+ (* (? a) (? b)) (* (? a) (? c)))
```

You might complain that we could have used distinguished symbols, such as ?a instead of the long-winded (? a). That would be fine, but that choice would make it a bit harder to extend, say if we want a variable that is restricted to match only numbers. Of course, we can add syntax later if it is helpful, but remember Alan Perlis's maxim: "Syntactic sugar causes cancer of the semicolon." With our simple list representation we are able to restrict the pattern variable (? a) to match only numbers by adding the predicate describing the restriction to its list representation: (? a ,number?).

One constraint on our design of the matcher is that the second pattern above should match

```
(+ (* (cos x) (exp y)) (* (cos x) (sin z)))
where a=(cos x), b=(exp y), and c=(sin z).
  But it should not match
```

```
(+ (* (cos x) (exp y)) (* (cos (+ x y)) (sin z)))
```

<span id="page-3-0"></span>because there is no consistent assignment possible for (? a), unless, somehow, x=(+ x y). [1](#page-82-0) We will learn about that sort of situation when we study unification matching in section 4.4; here we will decide that there is no match possible.

Another constraint on the matcher, which will have an important influence on its structure, is the need to match an unknown number of consecutive elements in a list. For example, suppose we want to make a rule to replace a sum of squares of a sine and a cosine with 1, even if they are not consecutive in the sum:

```
(+ ... (expt (sin theta) 2) ... (expt (cos theta) 2) ...)
(+ 1 ... ... ...)
```

The ... here may stand for many terms. We will use a *segment variable*, written with the prefix ??, to match many terms. So the pattern we will write is:

```
(+ (?? t1)
   (expt (sin (? x)) 2)
   (?? t2)
   (expt (cos (? x)) 2)
   (?? t3))
```

We needed three segment variables here. The segment variable (?? t1) will match the terms before the sine term, (?? t2) will match the terms between the sine term and the cosine term, and (?? t3) will match the terms after the cosine term.

Segment variables have a profound effect, because we don't know how long a segment is until we find the next part that matches, and we may be able to match the same data item many ways. For example, there may be squares of both sines and cosines of two different angles in the same sum. Even simpler, the pattern

```
(a (?? x) (?? y) (?? x) c)
```

can match the datum

```
(a b b b b b b c)
```

in four different ways. (Notice that the segment variable x must eat up the same number of bs in the two places it appears in the pattern.) So the matcher must do a search over the space of possible assignments to the segment variables.

## **4.2 Term rewriting**

Term-rewriting systems are powerful tools for creating domainspecific languages for manipulating expression-like information. If we have a syntactic system of expressions, where we may need to replace some subexpressions with "equivalent" subexpressions, we can often use a rule-based term-rewriting system to describe those transformations. For example, many compiler optimizations can be expressed as local rewrites of program fragments in a larger context. The essential features of a term-rewriting system are the use of a pattern matcher to identify the information to be transformed, and a template system for instantiating the replacement. There has been extensive research [72] into the problem of constructing convergent term-rewriting systems from equational theories (systems of "equivalent" expressions), but we won't get into that here. Also, there is a superficial similarity between the patterns matched against and the templates for instantiation, which may suggest the possibility of making bidirectional rules; we won't look at that either. First we develop a simple unidirectional system, in which patterns are used to recognize inputs and templates are used to make outputs.

Here is an approximation to some algebraic simplification rules:

```
(define algebra-1
  (rule-simplifier
    (list
    ;; Associative law of addition
    (rule '(+ (? a) (+ (? b) (? c)))
          '(+ (+ ,a ,b) ,c))
    ;; Commutative law of multiplication
    (rule '(* (? b) (? a))
          (and (expr<? a b)
```

```
'(* ,a ,b)))
;; Distributive law of multiplication over addition
(rule '(* (? a) (+ (? b) (? c)))
      '(+ (* ,a ,b) (* ,a ,c))))))
```

There are three rules in algebra-1. The first rule implements the associative law of addition, the second implements the commutative law of multiplication, and the third implements the distributive law of multiplication over addition.

Each rule is composed of two parts: a pattern to match a subexpression, and a *consequent* expression. If the pattern matches, the consequent is evaluated. If the value of the consequent is #f the rule is not applicable. Otherwise the result of the evaluation replaces the matched subexpression. Notice that we use the backquote mechanism described on page 391 to simplify writing the consequent expression.

The rules are gathered in a list for the rule-simplifier procedure. The result is a simplifier procedure that can be applied to an algebraic expression.

```
(algebra-1 '(* (+ y (+ z w)) x))
(+ (+ (* x y) (* x z)) (* w x))
```

Notice the restriction predicate, expr<?, in the consequent of the rule for the commutative law:

```
(rule '(* (? b) (? a))
      (and (expr<? a b)
           '(* ,a ,b)))
```

If the consequent expression returns #f, that match is considered to have failed. The system backtracks into the matcher to look for an alternative match; if none are forthcoming, the rule is not applicable. In the commutative law the restriction predicate expr<? imposes an ordering on algebraic expressions. The reason for this restriction is left as exercise 4.1.

## **Exercise 4.1: Guard expressions**

Why is the (expr<? a b) restriction necessary in the commutative law? What would go wrong if there were no restriction?

#### **4.2.1 Segment variables in algebra**

The algebra-2 rule system is far more interesting. It is built with the assumption that addition and multiplication are *n*-ary operations. We need segment variables to make this work. We also use the number? predicate in variable restrictions to support rules for numerical simplification.

```
(define algebra-2
 (rule-simplifier
  (list
   ;; Sums
   (rule '(+ (? a))
         a) ; unary + is identity
   (rule '(+ (?? a) (+ (?? b)) (?? c))
         '(+ ,@a ,@b ,@c)) ; associative: use n-ary +
   (rule '(+ (?? a) (? y) (? x) (?? b))
         (and (expr<? x y) ; commutative
              '(+ ,@a ,x ,y ,@b)))
   ;; Products
   (rule '(* (? a))
         a) ; unary * is identity
   (rule '(* (?? a) (* (?? b)) (?? c))
         '(* ,@a ,@b ,@c)) ; associative: use n-ary *
   (rule '(* (?? a) (? y) (? x) (?? b))
         (and (expr<? x y) ; commutative
              '(* ,@a ,x ,y ,@b)))
   ;; Distributive law
   (rule '(* (?? a) (+ (?? b)) (?? c))
         '(+ ,@(map (lambda (x) '(* ,@a ,x ,@c)) b)))
   ;; Numerical simplifications
  (rule '(+ 0 (?? x))
         '(+ ,@x))
   (rule '(+ (? x ,number?) (? y ,number?) (?? z))
         '(+ ,(+ x y) ,@z))
   (rule '(* 0 (?? x))
         0)
   (rule '(* 1 (?? x))
         '(* ,@x))
```

```
(rule '(* (? x ,number?) (? y ,number?) (?? z))
      '(* ,(* x y) ,@z))
)))
```

With algebra-2 we implement some numerical simplifications, in addition to dealing with multiple arguments to sums and products. Notice that we used the backquote mechanism to build patterns that include the predicate number? as restrictions on the variables, in addition to its use for constructing the consequent expressions. For further understanding of the simplifier, see exercise 4.2.

Now we can get these results:

```
(algebra-2 '(* (+ y (+ z w)) x))
(+ (* w x) (* x y) (* x z))
(algebra-2 '(+ (* 3 (+ x 1)) -3))
(* 3 x)
```

At this point we can see how it may be possible to extend such a system to simplify large classes of algebraic expressions.

### **Exercise 4.2: Term ordering**

According to the predicate expr<?, an expression that is explicitly a number is less than any expression that is not explicitly a number.

- **a.** In algebra-2 how does the ordering on expressions imposed by the commutative laws make the numerical simplification rules effective?
- **b.** Suppose that the commutative laws did not force an ordering. How would we have to express the numerical simplification rules? Explain why numerical simplification would become very expensive.

## **Exercise 4.3: Sorting efficiency**

The ordering in the commutative laws evolves an order *n* <sup>2</sup> bubble sort on the terms of a sum and the factors of a product. This can get pretty bad if there are many terms, as in a serious algebra problem. Is there some way in this system to make a more efficient sort? If not, why not? If so, how would you arrange it?

## **Exercise 4.4: Collecting terms**

The system we have described does not collect like terms. For example:

```
(algebra-2 '(+ (* 4 x) (* 3 x)))
(+ (* 3 x) (* 4 x))
```

Make a new system algebra-3 that includes rules that cause the collection of like terms, leaving the result as a sum of terms. Demonstrate your solution. Your solution must be able to handle problems like

```
(algebra-3
  '(+ y (* x -2 w) (* x 4 y) (* w x) z (* 5 z) (* x w) (* x y
3)))
(+ y (* 6 z) (* 7 x y))
```

#### **4.2.2 Implementation of rule systems**

Now that we have some experience with the use of a pattern-based rule system, let's dive in to see how it works.

We implement a rule as a procedure that matches the rule's pattern against a given expression. If the match succeeds, the rule evaluates its consequent in an environment in which the pattern variables are bound to their matched data. Rule procedures take success and failure continuations that can be used to backtrack into the consequent or pattern-match part of a rule. [2](#page-82-1)

<span id="page-8-0"></span>The rule-simplifier procedure used above is a constructor for a simple recursive simplifier. It produces simplify-expression, a procedure that takes an expression and uses the rules to simplify the expression. It recursively simplifies all the subexpressions of an

expression and then applies the rules to simplify the resulting expression. It does this repeatedly until the process converges. Thus the expression returned is a fixed point of the simplification process.

```
(define (rule-simplifier the-rules)
 (define (simplify-expression expression)
   (let ((subexpressions-simplified
          (if (list? expression)
              (map simplify-expression expression)
              expression)))
     (try-rules subexpressions-simplified the-rules
       (lambda (result fail) ; A: success continuation
         (simplify-expression result))
       (lambda () ; B: failure continuation
         subexpressions-simplified))))
 simplify-expression)
```

The procedure try-rules just scans the list of rules, sequencing the scan by means of the succeed and fail continuations.

```
(define (try-rules data rules succeed fail)
 (let per-rule ((rules rules))
   (if (null? rules)
       (fail) ; out of rules: go to B above
       (try-rule data
                 (car rules)
                 succeed ; if rule succeeds go to A above
                 (lambda () ; if rule fails try other rules
                   (per-rule (cdr rules)))))))
(define (try-rule data rule succeed fail)
 (rule data succeed fail))
```

Rule construction is implemented by the procedure make-rule, which takes a rule pattern and a handler that implements the consequent expression. For example, the commutative law rule on page 161 can be made directly with make-rule:

```
(make-rule '(* (? b) (? a))
  (lambda (b a)
    (and (expr<? a b)
         '(* ,a ,b))))
```

The handler (lambda (b a) ...) needs to get arguments that are the values of the pattern variables named a and b from the dictionary produced by the matcher procedure. The rule applies the handler to a list of these values in the order in which they appear in the pattern. Thus the handler must be written with its parameters in that order.

The rule constructor make-rule compiles the pattern into a match procedure. The rule it returns is a procedure that uses that match procedure to match the data. If the match succeeds, the rule applies the handler to values of the pattern variables resulting from the match.

We will learn how a pattern is compiled into a match procedure in section 4.3; all we need to know here is that the match procedure can be run using run-matcher and that if the match succeeds, the third argument to run-matcher is called with a *dictionary*. The dictionary dict is a mapping of pattern variables to the subexpressions that they were matched against. If the match fails, run-matcher returns #f, and the rule fails.

```
(define (make-rule pattern handler)
  (let ((match-procedure (match:compile-pattern pattern)))
    (define (the-rule data succeed fail)
      (or (run-matcher match-procedure data
            (lambda (dict)
              (let ((result
                     (apply handler
                            (match:all-values dict))))
                (and result
                     (succeed result
                              (lambda () #f))))))
          (fail)))
    the-rule))
```

The procedure match:all-values returns the values of the pattern variables in the order in which they appear in the pattern.

#### **4.2.3 Aside: Magic macrology**

Compare the rule definition given on page 161:

```
(rule '(* (? b) (? a))
      (and (expr<? a b)
           '(* ,a ,b)))
```

with what make-rule needs for its arguments:

```
(make-rule '(* (? b) (? a))
  (lambda (b a)
    (and (expr<? a b)
         '(* ,a ,b))))
```

The names a and b are repeated: they occur both in the pattern and in the parameter list of the handler, in the same order. This is both obnoxious to write and error-prone, because we must remember to repeat the names, and we can make a mistake if we repeat them incorrectly or in the wrong order.

This is a case for syntactic abstraction, otherwise known as a *macro*. The following rather magical code fragment transforms the rule definition into the desired call to make-rule:

```
(define-syntax rule
  (er-macro-transformer
   (lambda (form rename compare)
     (let ((pattern (cadr form))
          (handler-body (caddr form))
          (r-make-rule (rename 'make-rule))
          (r-lambda (rename 'lambda)))
      '(,r-make-rule ,pattern
                     (,r-lambda
                      ,(match:pattern-names pattern)
                      ,handler-body))))))
```

We can at least partially check this macro with the following magic incantation that expands the macros that appear in an expression:

```
(pp (syntax '(rule '(* (? b) (? a))
                   (and (expr<? a b)
                        '(* ,a ,b)))
            (the-environment)))
(make-rule '(* (? b) (? a))
           (lambda (b a)
             (and (expr<? a b)
                  (list '* a b))))
```

We see that the rule expands into a call to make-rule with the pattern and its handler procedure. This is the expression that is evaluated to make the rule. In more conventional languages, a macro, such as rule, expands directly into code that is substituted for the macro call. However, this process is not referentially transparent, because the macro expansion may use symbols that conflict with the user's symbols. In Scheme we try to avoid this problem, allowing a user to write *hygienic macros* that cannot cause conflicts. But this is more complicated than just substituting one expression for another. We will not try to explain the problems or the solutions here, but we will just use the solutions described in the MIT/GNU Scheme reference manual [51].

#### **4.2.4 Pattern-directed invocation**

The rule executive, try-rules, can also be used to implement procedures that use patterns for dispatching on input properties. The arguments to a pattern operator are matched against the pattern operator's rule pattern. The consequent of the matching rule computes the value to be returned.

For example, we could write the traditional factorial procedure, distributing the conditional as follows:

```
(define factorial
  (make-pattern-operator
   (rule '(0) 1)
   (rule '((? n ,positive?))
         (* n (factorial (- n 1))))))
(factorial 10)
3628800
```

We could also use this mechanism to build procedures whose behavior depends on the number of arguments supplied. For example, the Lisp - operator is negation when applied to one argument and subtraction when applied to multiple arguments:

```
(define -
  (make-pattern-operator
```

```
(rule '((? x)) (n:- 0 x))
(rule '((? x) (?? y)) (n:- x (apply n:+ y)))))
```

We can allow a pattern operator to be extended with additional rules dynamically. Such pattern operators are analogous to generic procedures, allowing the programmer to distribute the rule definitions nonlocally. For example, in a peephole optimizer we may want to group various optimizations with different parts of the code generator of a compiler.

```
(define peephole (make-pattern-operator))
(attach-rule! peephole
  (rule '((push (? reg1))
          (pop (? reg2)))
        (if (eqv? reg1 reg2)
            '()
            '((move ,reg1 ,reg2)))))
(attach-rule! peephole
  (rule '((or (? reg) (? const1 ,unsigned-integer?))
          (or (? reg) (? const2 ,unsigned-integer?)))
        '((or ,reg
              ,(bitwise-or const1 const2)))))
```

The first rule could be in the control-structure part of the optimizer, and the second rule could be located with the logical arithmetic part of the optimizer.

Here is one way to implement pattern operators. The last rule passed to make-pattern-operator is the *default* rule. It is always tried last, no matter what other rules may be added later.

```
(define (make-pattern-operator . rules)
  (let ((rules
         (cons 'rules
               (if (pair? rules)
                   (except-last-pair rules)
                   '())))
        (default-rule
         (and (pair? rules)
              (last rules))))
    (define (the-operator . data)
      (define (succeed value fail) value)
      (define (fail)
```

```
(error "No applicable operations:" data))
  (try-rules data
             (cdr rules)
             succeed
             (if default-rule
                 (lambda ()
                   (try-rule data
                              default-rule
                              succeed
                              fail))
                 fail)))
(set-pattern-metadata! the-operator rules)
the-operator))
```

We use set-pattern-metadata! to attach the rule list as a "sticky note" to an operator, and we use pattern-metadata to retrieve it in the code below. We have procedures to add a rule to the front (override-rule!) or to the end (attach-rule!) of an operator's rule list:

```
(define (attach-rule! operator rule)
  (let ((metadata (pattern-metadata operator)))
    (set-cdr! metadata
              (append (cdr metadata)
                      (list rule)))))
(define (override-rule! operator rule)
  (let ((metadata (pattern-metadata operator)))
    (set-cdr! metadata
              (cons rule (cdr metadata)))))
```

### **4.3 Design of the matcher**

<span id="page-14-0"></span>Now that we have seen some of the power of pattern matching, we will explore how it works. Our matcher is constructed from a family of match procedures (or *matchers*), and some combinators that combine them to produce compound matchers. [3](#page-82-2) Each primitive element of the pattern is represented by a primitive matcher, and the only combination, list, is represented by a combinator that combines matchers for the list elements to make a compound one.

All match procedures take three arguments: a list containing data to be matched, a dictionary of bindings of pattern variables, and a continuation procedure (succeed) to be called if the match is successful. The arguments to succeed must be the new dictionary resulting from the match and the number of items that were consumed from the input list. This number will be used for determining where to proceed after a segment match returns. A match procedure returns #f if the match is unsuccessful.

There are three primitive match procedures and one combinator. Let's go through them. We will also need a small procedure that we can pass to a match procedure as its succeed argument to report the result:

```
(define (result-receiver dict n-eaten)
  '(success ,(match:bindings dict) ,n-eaten))
```

#### **Pattern constants**

The procedure match:eqv takes a pattern constant, such as x, and produces a match procedure, eqv-match, that succeeds if and only if the first data item is equal (using eqv?) to the pattern constant. It does not add to the dictionary. The second argument to succeed is the number of items matched, which is 1 for this match procedure.

```
(define (match:eqv pattern-constant)
  (define (eqv-match data dictionary succeed)
    (and (pair? data)
         (eqv? (car data) pattern-constant)
         (succeed dictionary 1)))
  eqv-match)
```

### For example:

```
(define x-matcher (match:eqv 'x))
(x-matcher '(x) (match:new-dict) result-receiver)
(success () 1)
(x-matcher '(y) (match:new-dict) result-receiver)
#f
```

#### **Element variables**

The procedure match:element is used to make a match procedure, element-match, for a pattern variable, such as (? x), that is intended to match a single item.

When matching an *element variable* there are two possibilities: either the element variable already has a value or it does not yet have a value. If the variable has a value, it is bound in the dictionary. In that case the match succeeds if and only if the binding's value is equal (using equal?) to the first data item. If the variable does not have a value, it is not bound. In that case the match succeeds, extending the dictionary by adding a binding of the variable to the first data item. In either case of success, we indicate that the number of items consumed is 1.

```
(define (match:element variable)
  (define (element-match data dictionary succeed)
    (and (pair? data)
         (let ((binding (match:lookup variable dictionary)))
           (if binding
               (and (equal? (match:binding-value binding)
                            (car data))
                    (succeed dictionary 1))
               (succeed (match:extend-dict variable
                                            (car data)
                                            dictionary)
                        1)))))
  element-match)
```

Here are some examples. A match binding is a list: the first element is the variable name; the second element is the value; and the third element is the variable's type (here all are ? element variables).

```
((match:element '(? x))
'(a) (match:new-dict) result-receiver)
(success ((x a ?)) 1)
((match:element '(? x))
'(a b) (match:new-dict) result-receiver)
(success ((x a ?)) 1)
```

```
((match:element '(? x))
'((a b) c) (match:new-dict) result-receiver)
(success ((x (a b) ?)) 1)
```

#### **Segment variables**

The procedure match:segment is used to make a match procedure, segment-match, for a pattern variable, such as (?? x), that is intended to match a sequence of items. A segment-variable matcher is more complicated than an element-variable matcher because it can consume an unknown number of data items. Thus a segment matcher must inform its caller not only of the new dictionary, but also of how many items from the data were eaten.

When matching a segment variable there are two possibilities: either the segment variable already has a value or it does not yet have a value. If the segment variable has a value, the value must match the data; this is checked by match:segment-equal? on page 174. If the segment variable does not yet have a value, it must be given one.

The matcher, segment-match, returned by match:segment, succeeds with some initial sublist of the data, (list-head data i), as a possible assignment to the segment variable. (It starts with i=0, assuming that the segment will eat no items from the data.) However, if that success leads to a later failure in the match, the segment matcher tries to eat one more element than it had already tried (by executing (lp (+ i 1))). If the segment matcher runs out of data items, it fails to match. This is the key to the backtracking search that is needed when there are segment variables.

```
(define (match:segment variable)
  (define (segment-match data dictionary succeed)
    (and (list? data)
         (let ((binding (match:lookup variable dictionary)))
           (if binding
               (match:segment-equal?
                data
                (match:binding-value binding)
                (lambda (n) (succeed dictionary n)))
```

```
(let ((n (length data)))
                 (let lp ((i 0))
                   (and (<= i n)
                        (or (succeed (match:extend-dict
                                      variable
                                      (list-head data i)
                                      dictionary)
                                     i)
                            (lp (+ i 1))))))))))
  segment-match)
  For example:
((match:segment '(?? a))
 '(z z z) (match:new-dict) result-receiver)
(success ((a () ??)) 0)
```

Of course, a zero-length segment is okay.

If we want to see all of the possible matches, we change the result receiver to return #f after printing the successful result. This causes the matcher procedure to come up with an alternative value, if possible.

```
(define (print-all-results dict n-eaten)
  (pp '(success ,(match:bindings dict) ,n-eaten))
  ;; by returning #f we force backtracking.
  #f)
((match:segment '(?? a))
 '(z z z) (match:new-dict) print-all-results)
(success ((a () ??)) 0)
(success ((a (z) ??)) 1)
(success ((a (z z) ??)) 2)
(success ((a (z z z) ??)) 3)
#f
```

Now, returning to the case of a segment variable that already has a value, we need to make sure that the value matches an initial segment of the data. This is handled by match:segment-equal?. It compares the elements of the value against the data. If that works, it returns by calling ok (the procedure passed as its third argument) with the number of elements consumed from the data (which must be the length of the value); otherwise it returns #f.

```
(define (match:segment-equal? data value ok)
  (let lp ((data data) (value value) (n 0))
    (cond ((pair? value)
           (if (and (pair? data)
                    (equal? (car data) (car value)))
               (lp (cdr data) (cdr value) (+ n 1))
               #f))
           ((null? value) (ok n))
           (else #f))))
```

#### **Matching lists**

Finally, there is the match:list combinator, which takes a list of match procedures and makes a match procedure that matches a data list if and only if the given matchers eat up all of the elements in the data list. It applies the matchers in succession. Each matcher tells the list combinator how many items to jump over before passing the remaining data to the next matcher.

```
(define (match:list matchers)
  (define (list-match data dictionary succeed)
    (and (pair? data)
         (let lp ((data-list (car data))
                  (matchers matchers)
                 (dictionary dictionary))
           (cond ((pair? matchers)
                  ((car matchers)
                   data-list
                   dictionary
                   (lambda (new-dictionary n)
                     ;; The essence of list matching:
                     (lp (list-tail data-list n)
                         (cdr matchers)
                         new-dictionary))))
                 ((pair? data-list) #f) ;unmatched data
                 ((null? data-list) (succeed dictionary 1))
                 (else #f)))))
  list-match)
```

Notice that the matcher, list-match, returned by the match:list combinator has exactly the same interface as the other matchers, allowing it to be incorporated into a combination. The fact that all of

the basic match procedures have exactly the same interface makes this a system of combinators.

Now we can make a matcher that matches a list of any number of elements, starting with the symbol a, ending with the symbol b, and with a segment variable (?? x) between them, by the combination:

```
((match:list (list (match:eqv 'a)
                   (match:segment '(?? x))
                   (match:eqv 'b)))
 '((a 1 2 b))
 (match:new-dict)
 result-receiver)
(success ((x (1 2) ??)) 1)
```

This was a successful match. The dictionary returned has exactly one entry: x=(1 2), and the match ate precisely one element (the list (a 1 2 b)) from the list supplied.

```
((match:list (list (match:eqv 'a)
                   (match:segment '(?? x))
                   (match:eqv 'b)))
 '((a 1 2 b 3))
 (match:new-dict)
 result-receiver)
#f
```

This was a failure, because there was nothing to match the 3 after the b in the input data.

### **The dictionary**

The dictionary we will use is just a headed list of bindings. Each binding is a list of the variable's name, its value, and its type.

```
(define (match:new-dict)
  (list 'dict))
(define (match:bindings dict)
  (cdr dict))
(define (match:new-bindings dict bindings)
  (cons 'dict bindings))
```

```
(define (match:extend-dict var value dict)
  (match:new-bindings dict
                      (cons (match:make-binding var value)
                            (match:bindings dict))))
(define (match:lookup var dict)
  (let ((name
         (if (symbol? var)
             var
             (match:var-name var))))
    (find (lambda (binding)
            (eq? name (match:binding-name binding)))
          (match:bindings dict))))
(define (match:make-binding var value)
  (list (match:var-name var)
        value
        (match:var-type var)))
(define match:binding-name car)
(define match:binding-type caddr)
(define match:binding-value
  (simple-generic-procedure 'match:binding-value 1 cadr))
```

The accessor match:binding-value is just cadr, but is made generic to allow future extensions. This will be needed in the code supporting section 4.5.

#### **4.3.1 Compiling patterns**

We can automate the construction of pattern matchers from patterns with an elementary compiler. The compiler produces as its value a match procedure appropriate for the pattern it is given, which has exactly the same interface as the elementary matchers given above.

The match-procedure interface is convenient for composing matchers, but not very friendly to humans. For playing with the matcher it is more convenient to use:

```
(define (run-matcher match-procedure datum succeed)
  (match-procedure (list datum)
                   (match:new-dict)
                   (lambda (dict n)
```

```
(and (= n 1)
     (succeed dict)))))
```

With this interface we are hiding several details about match procedures: we wrap the incoming datum in a list; we check that exactly one element of that list (the datum) has been matched; and we provide the initial dictionary.

Some simple examples are:

```
(run-matcher
 (match:compile-pattern '(a ((? b) 2 3) (? b) c))
 '(a (1 2 3) 2 c)
 match:bindings)
#f
(run-matcher
 (match:compile-pattern '(a ((? b) 2 3) (? b) c))
 '(a (1 2 3) 1 c)
 match:bindings)
((b 1 ?))
```

As we saw before, some patterns involving segment variables may match in many ways, and we can elicit all of the matches by failing back into the matcher to select the next one, until they are all exhausted:

```
(run-matcher
 (match:compile-pattern '(a (?? x) (?? y) (?? x) c))
 '(a b b b b b b c)
 print-all-matches)
((y (b b b b b b) ??) (x () ??))
((y (b b b b) ??) (x (b) ??))
((y (b b) ??) (x (b b) ??))
((y () ??) (x (b b b) ??))
#f
```

The possible matches require both instances of (?? x) to match the same data.

The procedure print-all-matches prints the bindings and forces a failure.

```
(define (print-all-matches dict)
  (pp (match:bindings dict))
  #f)
```

To make this compiler we need to define the syntax of pattern variables. For now we have a very simple syntax: a pattern variable is a list of the type (? or ??) and the name.

```
(define (match:var-type var)
  (car var))
(define (match:var-type? object)
  (memq object match:var-types))
(define match:var-types '(? ??))
(define (match:named-var? object)
  (and (pair? object)
       (match:var-type? (car object))
       (n:>= (length object) 2)
       (symbol? (cadr object))))
(define (match:element-var? object)
  (and (match:var? object)
       (eq? '? (match:var-type object))))
(define (match:segment-var? object)
  (and (match:var? object)
       (eq? '?? (match:var-type object))))
```

This code is more complex than one might expect, because we will extend the variable syntax in section 4.5 and some of the exercises.

```
(define match:var-name
  (simple-generic-procedure 'match:var-name 1
    (constant-generic-procedure-handler #f)))
(define-generic-procedure-handler match:var-name
  (match-args match:named-var?)
  cadr)
```

The default handler is a procedure that always returns false, and at this time there is only one substantive handler, which retrieves the name of a named variable.

We also make the predicate that determines if its argument is a match variable generic, although at this time the only objects that satisfy match:var? are named variables.

```
(define match:var?
  (simple-generic-procedure 'match:var? 1
    (constant-generic-procedure-handler #f)))
(define-generic-procedure-handler match:var?
  (match-args match:named-var?)
  (constant-generic-procedure-handler #t))
```

The compiler maps a pattern to the corresponding matcher:

```
(define (match:compile-pattern pattern)
  (cond ((match:var? pattern)
         (case (match:var-type pattern)
           ((?) (match:element pattern))
           ((??) (match:segment pattern))
           (else (error "Unknown var type:" pattern))))
        ((list? pattern)
         (match:list (map match:compile-pattern pattern)))
        (else ; constant
         (match:eqv pattern))))
```

By varying this compiler, we can change the syntax of patterns any way we like.

```
(run-matcher
 (match:compile-pattern '(a ((? b) 2 3) (? b) c))
 '(a (1 2 3) 1 c)
 match:bindings)
((b 1 ?))
```

## **Exercise 4.5: Backtracking**

In the example on page 177 we got multiple matches, by returning #f from the success procedure print-all-matches. This is probably pretty mysterious. How does it work? Explain, in a short but clear paragraph, how the sequence of matches is generated.

#### **4.3.2 Match-variable restrictions**

Often we want to restrict the kind of object that can be matched by a pattern variable. For example, we may want to make a pattern in which a variable can match only a positive integer. One way to do

this, which we used in our term-rewriting system in section 4.2.1, is to allow a variable to carry a predicate for testing the datum for acceptability. For example, we may be interested in finding positive integer powers of sine functions. We could write the pattern we want as follows:

```
'(expt (sin (? x)) (? n ,exact-positive-integer?))
```

We cannot use simple quotation for such a pattern, because the predicate expression (here exact-positive-integer?) must be evaluated before being included in the pattern. As in term rewriting (section 4.2), we use the backquote mechanism to do this.

To make a matcher that can check that data satisfies such a predicate, we add a single line to match:element:

```
(define (match:element variable)
  (define (element-match data dictionary succeed)
    (and (pair? data)
         (match:satisfies-restriction? variable (car data))
         (let ((binding (match:lookup variable dictionary)))
           (if binding
               (and (equal? (match:binding-value binding)
                            (car data))
                    (succeed dictionary 1))
               (succeed (match:extend-dict variable
                                            (car data)
                                            dictionary)
                        1)))))
  element-match)
(define (match:satisfies-restriction? var value)
  (or (not (match:var-has-restriction? var))
      ((match:var-restriction var) value)))
```

## **Exercise 4.6: Pattern alternatives: choice is good**

An interesting way to extend our pattern language is to introduce a choice operator:

```
(?:choice pattern ...)
```

This should compile into a matcher that tries to match each of the given *pattern*s in order from left to right, returning the first successful match, or #f if none match. (This should remind you of regular expression "alternation" (see page 40), but the name "choice" is more traditional in pattern matching.)

#### For example:

```
(run-matcher
 (match:compile-pattern '(?:choice a b (? x) c))
 'z
 match:bindings)
((x z ?))
(run-matcher
 (match:compile-pattern
 '((? y) (?:choice a b (? x ,string?) (? y ,symbol?) c)))
 '(z z)
 match:bindings)
((y z ?))
(run-matcher
 (match:compile-pattern '(?:choice b (? x ,symbol?)))
  'b
  print-all-matches)
()
((x b ?))
#f
```

**To do:** Implement a new matcher procedure, match:choice, for this new pattern schema. Augment the pattern compiler appropriately.

## **Exercise 4.7: Naming in patterns**

Another extension is to provide named patterns, analogous to Scheme's letrec.

Naming allows shorter, more modular patterns while also supporting recursive subpatterns, including mutually recursive subpatterns.

For instance, the pattern:

```
(?:pletrec ((odd-even-etc (?:choice () (1 (?:ref even-odd-
etc))))
            (even-odd-etc (?:choice () (2 (?:ref odd-even-
etc)))))
    (?:ref odd-even-etc))
```

should match all lists of the following form (including the empty list):

```
(1 (2 (1 (2 (1 ...)))))
```

Here, ?:pletrec introduces a block of mutually recursive pattern definitions while ?:ref substitutes a defined pattern in place (in order to distinguish such references from literal symbols like a and from pattern variables like (? x)).

**To do:** Implement these new ?:pletrec and ?:ref pattern schemas. One approach is to implement new matcher procedures, match:pletrec and match:ref, then augment the pattern compiler appropriately. Other approaches may also work. Explain your approach briefly if it is subtle or non-obvious.

**To think (before you do!):** In a proper environment-based letrec-like implementation, nested ?:pletrec instances would introduce distinct contour lines for scoping. But the control structure of our pattern matcher does not make this easy.

The matcher procedures traverse the pattern and data in left-toright depth-first order, binding the first textual appearance of each distinct pattern variable (like (? x)) to its corresponding datum and then treating each subsequent textual appearance in the pattern as a constraining instance. This is achieved by threading the dictionary through the depth-first control path. Pay particular attention to the appearance of new-dictionary in the body of match:list. This control structure, in essence, decrees that the leftmost, deepest instance of each unique pattern variable is a defining instance in an implicit flat global namespace, with all subsequent downstream appearances being constraining instances.

So let's not worry about that scoping complexity in this exercise. Specifically, just as pattern variables all share a common global namespace, so too can your pattern definitions.

Of course, if you are really ambitious, you can implement lexical scoping by rewriting all the existing matcher interfaces to accept an extra pattern-environment parameter. But that is a job for another time and place (in exercise 4.9).

## **Exercise 4.8: Hoist by our own petard**

On the surface it may appear that it would be easy to extend this matcher system to allow vector patterns and vector data. But we made a strong assumption in the design of this matcher—that it is a matcher of list patterns on list data.

- **a.** What would it take to liberate this code from that assumption so that you could make a matcher that encompassed both kinds of compound data? Or arbitrary sequences? What sorts of changes are required? Do you need to change the interface to a match procedure?
- **b.** Make it so! (This is lots of work.)

## **Exercise 4.9: General pattern language**

Even with the addition of ?:pletrec and ?:ref in exercise 4.7, the pattern matcher we have is not a complete language, in that it does not support namespace scoping and parametric patterns. For example, we cannot write the following pattern, which is intended to match only lists of symbols that are palindromes.

```
(?:pletrec ((palindrome
            (?:pnew (x)
              (?:choice ()
                        ((? x ,symbol?)
                         (?:ref palindrome)
                         (? x))))))
   (?:ref palindrome))
```

For this to work in any reasonable way, ?:pnew creates fresh lexically scoped pattern variables that can be referred to only in the body of the ?:pnew.

A fully worked-out pattern language is a wonderful subsystem to have, but it is not easy to build.

**To do:** Flesh out this idea to produce a full pattern language. Not for the faint of heart!

## **4.4 Unification match**

Pattern matching is, as we have said, a kind of equality testing between structured data items. It is a generalization of equality testing because we allow some parts of the data to be unspecified, by allowing pattern variables that match the unspecified parts of the data. But we require that every occurrence of the same pattern variable must match equivalent data.

So far our matchers have been one-sided: we match a pattern with variables against data that contains no variables, producing a dictionary—a map from each variable in the pattern to a matching piece of the data. If we substitute the matching value for each variable into the original pattern we make a *substitution instance* of the pattern. The resulting instance is always equivalent to the original data.

We are about to remove the limitation that the data has no variables: we will allow variables on both sides of the match. This powerful kind of match is called *unification*. The result of a successful unification is also a dictionary, but the values for variables may contain variables, and the dictionary may not give values for every variable in the patterns. If we substitute the values associated with variables in the dictionary for variables that appear in either of the two given patterns, we obtain a substitution instance of both initial patterns. This substitution instance, which may contain variables, is called the *unifier* of the patterns. The unifier is the most general common substitution instance of the two patterns:

any other common substitution instance of the two patterns is a substitution instance of the unifier. The unifier is unique, up to renaming of the variables, so unification is well defined. [4](#page-82-3)

<span id="page-30-0"></span>Unification was first described by J. A. Robinson in his famous invention of the resolution procedure for theorem proving [104]. [5](#page-82-4)

<span id="page-30-1"></span>For a simple example, suppose we have several sources of partial information about Ben Franklin's dates of birth and death:

```
(define a
  '(((? gn) franklin) (? bdate) ((? dmo) (? dday) 1790)))
(define b
  '((ben franklin) ((? bmo) 6 1705) (apr 17 (? dyear))))
(define c
  '((ben (? fn)) (jan (? bday) 1705) (apr 17 (? dyear))))
```

The unification of two expressions will give us a dictionary of the values of the variables that are derived from the match. We can use that dictionary to construct the unifier of the two patterns:

```
(unifier a b)
((ben franklin) ((? bmo) 6 1705) (apr 17 1790))
(unifier a c)
((ben franklin) (jan (? bday) 1705) (apr 17 1790))
(unifier b c)
((ben franklin) (jan 6 1705) (apr 17 (? dyear)))
```

Each of these results is more fully specified than the partial information provided by any source. We can further combine all three sources to get a full picture:

```
(unifier a (unifier b c))
((ben franklin) (jan 6 1705) (apr 17 1790))
(unifier b (unifier a c))
((ben franklin) (jan 6 1705) (apr 17 1790))
(unifier c (unifier a b))
((ben franklin) (jan 6 1705) (apr 17 1790))
```

Often there are constraints among the variables in an expression. For example, there may be multiple instances of the same variable, which must remain consistent, as in the following derivation:

```
(define addition-commutativity
  '(= (+ (? u) (? v)) (+ (? v) (? u))))
(unifier '(= (+ (cos (? a)) (exp (? b))) (? c))
         addition-commutativity)
(= (+ (cos (? a)) (exp (? b))) (+ (exp (? b)) (cos (? a))))
```

#### **4.4.1 How unification works**

We can think of unification as a kind of equation solving. If we think of the patterns as structured partial information, unifying them is testing the proposition that the two patterns are partial information about the same thing. For the patterns to unify, the corresponding parts must unify. So unification is setting up equations among the corresponding parts and solving for the unknowns (the pieces of information left unspecified in each pattern).

The process is analogous to the way we solve numerical equations. The goal is to eliminate as many variables in the equations as we can. The result is a list of substitutions of expressions for the eliminated variables. No substitution expression may refer to any eliminated variable. We scan the equations for one that can be solved for one of the variables in it. Solving for a variable isolates it, finding an equivalent expression that does not contain the variable. We eliminate the variable by replacing all occurrences of it with its the newly discovered value (the equivalent expression), both in the remaining equations and in the values associated with previously eliminated variables in the substitution list. We then add the new substitution to the substitution list. We repeat until there are no more equations to be solved or we encounter a contradiction. The result is either a successful substitution list or a report of the contradiction.

In unification the "equations" are the matches of the corresponding parts of the two input patterns, and the "substitution

list" is the dictionary. One way to implement unification is to walk the common structure of the input patterns. As in one-sided matching, the patterns are represented by list structure. If a variable is encountered on either side of the match, it is bound, in the dictionary, to the data on the other side.

If there are multiple occurrences of a variable in the original patterns, each subsequent occurrence must match the value bound by the first occurrence. This is ensured by the fact that every time a variable is encountered, it is looked up in the dictionary, and if there is a binding for the variable, its value is used instead of the variable. Also, every time a new binding is made and entered into the dictionary, the new binding is used to replace all instances of the newly eliminated variable in the values for other variables in the dictionary.

To obtain the unifier of two patterns we unify them to get the dictionary of substitutions, if they can be unified. If not, unify returns #f, indicating a failure. The dictionary is then used to instantiate one of the patterns; it doesn't matter which is chosen. The procedure match:dict-substitution replaces every instance of a variable in the pattern expression pattern1 that has a binding in the dictionary dict with its value in the dictionary.

```
(define (unifier pattern1 pattern2)
  (let ((dict (unify pattern1 pattern2)))
    (and dict
         ((match:dict-substitution dict) pattern1))))
```

The main interface to the unifier is unify, which returns the dictionary result of a successful match, or #f if the match was unsuccessful.

```
(define (unify pattern1 pattern2)
  (unify:internal pattern1 pattern2
                  (match:new-dict)
                  (lambda (dict) dict)))
```

The unify:internal entry point gives more control of the matching process. It takes the two patterns to be unified, a dictionary that may have some bindings specified, and a success

continuation (succeed) that will be called if the match succeeds. The success continuation provided by unify, above, just returns the dictionary. In section 4.4.4, when we add code to experiment with segment variables in the patterns, we will be able to extract multiple matches by returning #f from succeed, indicating that the result was not the one wanted. The ability to backtrack into the matcher also simplifies other interesting semantic extensions, such as incorporating algebraic expressions and equation solving into the match process. [6](#page-82-5)

<span id="page-33-0"></span>In unify:internal the patterns to be unified, pattern1 and pattern2 are wrapped in lists. The unifier program will compare these lists term by term, building a dictionary that makes corresponding terms equal. The match succeeds if both lists of terms are simultaneously exhausted. At the top level, in unify:internal, the term lists just contain the two given patterns, but the central unification procedure, unify:dispatch, will recursively descend into the pattern, comparing subpatterns of the patterns as lists of terms. [7](#page-83-0)

```
(define (unify:internal pattern1 pattern2 dict succeed)
  ((unify:dispatch (list pattern1) (list pattern2))
  dict
  (lambda (dict fail rest1 rest2)
    (or (and (null? rest1) (null? rest2)
             (succeed dict))
        (fail)))
  (lambda () #f)))
```

The procedure unify:dispatch, which takes two input lists of terms, is the core of the recursive descent matcher. The match process depends on the contents of the term lists. For example, if both term lists begin with a constant, as in (ben franklin) and (ben (? fn)), the constants must be compared, and the match can proceed only if the constants are equal. If one term list begins with a variable, the other can begin with any term, and the variable must be bound to the term it matches. (If both are variables, one of the variables will be eliminated in favor of the other.) So, if one term list is ((? bmo) 6 1705) and the other is (jan (? bday) 1705),

then the variable (? bmo) must be bound to the value jan for the match to proceed. If both term lists start with a list that is not a variable, the match must recursively match the corresponding sublists before proceeding to match the rest of the given term lists. For example, in unifying b and c in the Ben Franklin example, after the first terms are matched the dictionary contains a binding of fn to franklin and the remaining termlists are (((? bmo) 6 1705) (apr 17 (? dyear))) and ((jan (? bday) 1705) (apr 17 (? dyear))). Both of these term lists begin with a list, so the matching must recurse to compare the sublists ((? bmo) 6 1705) and (jan (? bday) 1705).

The procedure unify-dispatcher returned by unify:dispatch takes three arguments: a dictionary, a success continuation, and a failure continuation. If both term lists are exhausted, the match succeeds. If there are more terms to be matched, the generic procedure unify:gdispatch calls an appropriate match procedure that depends on the contents of the two term lists. If the match succeeds, it means that the initial terms of the two term lists could be unified relative to the given dictionary. So the success continuation is called with the new dictionary dict\*, a new failure continuation fail\*, and the unmatched tails, rest1 and rest2, of the input lists. These tails are then matched by a recursive call to unify:dispatch.

```
(define (unify:dispatch terms1 terms2)
  (define (unify-dispatcher dict succeed fail)
    (if (and (null? terms1) (null? terms2))
        (succeed dict fail terms1 terms2)
        ((unify:gdispatch terms1 terms2)
         dict
         (lambda (dict* fail* rest1 rest2)
           ((unify:dispatch rest1 rest2) dict* succeed
fail*))
         fail)))
  unify-dispatcher)
```

The generic procedure unify:gdispatch has handlers for the cases described above: matching two constants, matching two term lists, and matching a variable to something. (Because it is generic, it

can be extended for new kinds of matching.) The default handler, for cases such as matching a constant to a term list, is unify:fail:

```
(define (unify:fail terms1 terms2)
  (define (unify-fail dict succeed fail)
    (fail))
  unify-fail)
(define unify:gdispatch
  (simple-generic-procedure 'unify 2 unify:fail))
```

In this unifier the term lists are matched term by term, so the job of a handler is to match the first terms of the two term lists. Thus the applicability of a handler depends only on the first term of each term list. To simplify the applicability specification we introduce car-satisfies, which takes a predicate and produces a new predicate ensuring that there is a first term in the list and that term satisfies the argument predicate.

```
(define (car-satisfies pred)
  (lambda (terms)
    (and (pair? terms)
         (pred (car terms)))))
```

Any term that is not a match variable or a list is a constant. Constants match only when they are equal:

```
(define (unify:constant-terms terms1 terms2)
  (let ((first1 (car terms1)) (rest1 (cdr terms1))
        (first2 (car terms2)) (rest2 (cdr terms2)))
    (define (unify-constants dict succeed fail)
      (if (eqv? first1 first2)
          (succeed dict fail rest1 rest2)
          (fail)))
    unify-constants))
(define (constant-term? term)
  (and (not (match:var? term))
       (not (list? term))))
(define-generic-procedure-handler unify:gdispatch
  (match-args (car-satisfies constant-term?)
              (car-satisfies constant-term?))
  unify:constant-terms)
```

The handler unify:list-terms is where the recursive descent actually happens. Because the first term of each term list is itself a list, the matcher must recurse to match those sublists. If the match of the sublists succeeds, the match must continue with the rest of the input termlists. (Note that the recursive match will succeed only if all of the terms of the two sublists match; so the unmatched sublist tails passed to the success continuation will be empty and are ignored.)

```
(define (unify:list-terms terms1 terms2)
  (let ((first1 (car terms1)) (rest1 (cdr terms1))
        (first2 (car terms2)) (rest2 (cdr terms2)))
    (define (unify-lists dict succeed fail)
      ((unify:dispatch first1 first2)
       dict
       (lambda (dict* fail* null1 null2)
         (succeed dict* fail* rest1 rest2))
       fail))
    unify-lists))
(define (list-term? term)
  (and (not (match:var? term))
       (list? term)))
(define-generic-procedure-handler unify:gdispatch
  (match-args (car-satisfies list-term?)
              (car-satisfies list-term?))
  unify:list-terms)
```

So far our code implements recursive descent and the matching of constants. Obvious contradictions like matching two different symbols or a symbol to a list produce a failure. In order to solve interesting equations we must be able to match terms with variables. When we find variables, we may add new bindings to the dictionary. The part of the equation solver that deals with variables is the procedure maybe-substitute.

The procedure maybe-substitute gets one term list var-first that starts with a variable. It matches that variable with the first term of the second term list, terms.

If a variable is matched against itself we have a tautology, and the match succeeds with an unchanged dictionary. If the variable

already has a value, we replace the variable with its value and match the resulting list against the term list terms. Finally, if the variable does not have a value, we can eliminate it using do-substitute, which is responsible for adding a binding from var to term when possible.

```
(define (maybe-substitute var-first terms)
  (define (unify-substitute dict succeed fail)
    (let ((var (car var-first)) (rest1 (cdr var-first))
          (term (car terms)) (rest2 (cdr terms)))
      (cond ((and (match:element-var? term)
                  (match:vars-equal? var term))
             (succeed dict fail rest1 rest2))
            ((match:has-binding? var dict)
             ((unify:dispatch
                 (cons (match:get-value var dict) rest1)
                 terms)
              dict succeed fail))
            (else
             (let ((dict* (do-substitute var term dict)))
               (if dict*
                   (succeed dict* fail rest1 rest2)
                   (fail)))))))
  unify-substitute)
```

In do-substitute we first use the old dictionary to clean the incoming term by replacing any previously eliminated variables with their values. Then we check if there are any restrictions on what objects var may match. Finally we check for any occurrences of var in the cleaned term (term\*). If the cleaned term contains a reference to var, the match cannot proceed. [8](#page-83-1) If our match passes all these tests, we make a new dictionary that includes the new binding of var to the cleaned term. The binding values in the new dictionary must also be cleaned of references to var.

```
(define (do-substitute var term dict)
  (let ((term* ((match:dict-substitution dict) term)))
    (and (match:satisfies-restriction? var term*)
         (or (and (match:var? term*)
                  (match:vars-equal? var term*))
         (not (match:occurs-in? var term*)))
     (match:extend-dict var term*
       (match:map-dict-values
```

```
(match:single-substitution var term*)
dict)))))
```

Now that we know how to handle variables we just have to install this handler in our generic dispatcher procedure. The only subtlety here is that the variable to be eliminated may appear in either term list. We must guarantee that the term list containing the variable is the first argument to maybe-substitute.

```
(define (element? term)
  (any-object? term))
(define-generic-procedure-handler unify:gdispatch
  (match-args (car-satisfies match:element-var?)
              (car-satisfies element?))
  (lambda (var-first terms)
    (maybe-substitute var-first terms)))
(define-generic-procedure-handler unify:gdispatch
  (match-args (car-satisfies element?)
              (car-satisfies match:element-var?))
  (lambda (terms var-first)
    (maybe-substitute var-first terms)))
```

<span id="page-38-0"></span>At this point we have a complete, correct, and competent traditional unifier. [9](#page-83-2) It is written with generic procedures, so it can easily be extended to work with other kinds of data. And with only a small amount of work we can add semantic attachments, such as the commutativity of lists beginning with the symbols + and \*. As we shall see, we can also add support for new kinds of syntactic variables, such as segment variables. But before we do that we will look at a real application: type inference.

## **Exercise 4.10: Unifying vectors**

This unifier can be extended to handle data and patterns that are made of other data types, such as vectors. Make handlers for vectors without transforming the vectors to lists (the easy way out!).

### **Exercise 4.11: Unifying strings**

Extend the unifier to allow unification of strings. This could be fun, but you need to invent a syntactic mechanism for representing string variables inside a string. This is pretty delicate, because you may have to represent a string variable with a string expression. This gets into quotation problems—please try not to invent a baroque mechanism. Also, make sure that you don't make assumptions that will prevent you from introducing string segment variables later. (See exercise 4.21 on page 209.)

## **Exercise 4.12: Variable restrictions**

We added support for restrictions on variables as we did in the onesided matcher: we just put a clause for that into the main conditional in the procedure do-substitute. But there are subtle problems.

- What should happen if a restricted variable is matched against another restricted variable?
- When the variable is first bound to its target, after passing the restriction, it is uniformly eliminated. But the restriction is then lost, preventing it from killing an unsuitable later part of the match.

Your task is to understand these problems and determine how to ameliorate them. How important could this be for any applications you might consider? Is there a solution that fits nicely into our implementation strategy?

## **Exercise 4.13: Unifying with pattern combinators?**

Unlike our earlier one-sided pattern matcher, our unification matcher does not compile the patterns into match procedures that combine to build a match procedure for the pattern. But a system of match procedures is potentially more efficient, because it avoids the syntactic analysis of the patterns while matching. Can the unification matcher be broken up in a similar way? If not, why not? Is it a good idea to do so? If not, why not? If so, do it! (This is hard!)

#### **4.4.2 Application: Type inference**

One classic application of unification matching is type inference: given a program and some type information about parts of the program, deduce type information about other parts of the program. For example, if we know that < is a procedure that takes two numerical arguments and produces a boolean value, then if we analyze the expression (g (< x (f y))), we can deduce that f and g are unary procedures; g accepts a boolean argument; f returns a numerical value; and x has a numerical value. If this information is used to deduce properties of the program that the expression is embedded in, we can learn a great deal about the program. Here is an analysis of the expression:

```
(pp (infer-program-types '(g (< x (f y)))))
(t (? type:17)
   ((t (type:procedure ((boolean-type)) (? type:17)) g)
    (t (boolean-type)
       ((t (type:procedure ((numeric-type) (numeric-type))
                           (boolean-type))
           <)
        (t (numeric-type) x)
        (t (numeric-type)
           ((t (type:procedure ((? y:12)) (numeric-type)) f)
            (t (? y:12) y)))))))
```

This is the abstract tree of the given expression, annotated with types. Each subexpression *x* has been expanded into a *typed expression* of the form (t *type x*). For example, the reference to g has the type:

```
(type:procedure ((boolean-type)) (? type:17))
```

As expected, g is a procedure that accepts a boolean argument, but we have no information about its value. The unknown value type is represented by the pattern variable (? type:17).

Let's consider a more substantial example:

```
(define foo
  (infer-program-types
   '(define fact
      (lambda (n)
        (begin
          (define iter
            (lambda (product counter)
              (if (> counter n)
                  product
                  (iter (* product counter)
                         (+ counter 1)))))
          (iter 1 1))))))
```

The result in foo is rather verbose. So we look at it with a simplifier that puts it into "human readable" form.

```
(pp (simplify-annotated-program foo))
(begin
  (define fact
    (lambda (n)
      (declare-type n (numeric-type))
      (define iter
        (lambda (product counter)
          (declare-type product (numeric-type))
          (declare-type counter (numeric-type))
          (if (> counter n)
              product
              (iter (* product counter)
                    (+ counter 1)))))
      (declare-type iter
        (type:procedure ((numeric-type) (numeric-type))
                        (numeric-type)))
      (iter 1 1)))
  (declare-type fact
    (type:procedure ((numeric-type)) (numeric-type))))
```

Here we see that the type inference program was able to determine the complete type of the factorial program—that it is a procedure that takes one numerical input and produces a numerical output. This is reported in a declaration:

```
(declare-type fact
  (type:procedure ((numeric-type)) (numeric-type)))
```

The type of the internal definition iter has also been determined: it takes two numerical arguments and produces a numerical result.

```
(declare-type iter
  (type:procedure ((numeric-type) (numeric-type))
                  (numeric-type)))
```

Also, the type of each internal variable has been determined, and an appropriate declaration has been posted:

```
(declare-type n (numeric-type))
(declare-type product (numeric-type))
(declare-type counter (numeric-type))
```

#### **4.4.3 How type inference works**

The process of type inference has four phases.

- 1. The given program is annotated with type variables for all subexpressions of the program.
- 2. Constraints on the type variables are formulated based on the semantic structure of the program.
- 3. The constraints are unified to eliminate as many of the variables as possible.
- 4. The annotated program is specialized using the dictionary produced by unification of the constraints to make a new annotated program whose type annotations incorporate the constraints.

The plan is implemented in this procedure:

```
(define (infer-program-types expr)
  (let ((texpr (annotate-program expr)))
    (let ((constraints (program-constraints texpr)))
      (let ((dict (unify-constraints constraints)))
        (if dict
            ((match:dict-substitution dict) texpr)
            '***type-error***)))))
```

This procedure complains if the program expression cannot be consistently typed. However, it gives no explanation about why it failed; this could be accomplished by passing information back in the failure continuations.

#### **Annotation**

The annotate-program procedure is implemented in terms of a generic procedure, annotate-expr, to allow easy extension for new language features.

```
(define (annotate-program expr)
  (annotate-expr expr (top-level-env)))
(define annotate-expr
  (simple-generic-procedure 'annotate-expr 2 #f))
```

The annotate-expr procedure takes an environment for bindings of the type variables. It is initialized with the top-level environment created below.

There are simple handlers for annotating simple kinds of expressions. If an explicit number appears as a subexpression it is given a constant type, constructed by (numeric-type):

```
(define-generic-procedure-handler annotate-expr
  (match-args number? any-object?)
  (lambda (expr env)
    (make-texpr (numeric-type) expr)))
```

The procedure make-texpr constructs a typed expression from a type and an expression. Its parts can be selected with texpr-type and texpr-expr.

However, we may not a priori know a type for an identifier, which is represented by a symbol. The procedure get-var-type tries to find the identifier's type in the environment, and failing that, creates a unique type variable for the type annotation of all occurrences of the identifier in that lexical context:

```
(define-generic-procedure-handler annotate-expr
  (match-args symbol? any-object?)
```

```
(lambda (expr env)
  (make-texpr (get-var-type expr env) expr)))
```

We may know types for some identifiers, such as primitive procedures of the language. These are provided in the top-level environment. The primitive procedures shown here have procedure types with type constants (e.g., (numeric-type)) for their arguments and values:

```
(define (top-level-env)
  (list (make-top-level-env-frame)))
(define (make-top-level-env-frame)
  (let ((binary-numerical
         (let ((v (numeric-type)))
           (procedure-type (list v v) v)))
        (binary-comparator
         (let ((v (numeric-type)))
           (procedure-type (list v v) (boolean-type)))))
    (list (cons '+ binary-numerical)
          ...
          (cons '= binary-comparator)
          (cons '< binary-comparator)
          ...)))
```

For a conditional expression, a type variable is created for the value of the conditional expression, and each subexpression is recursively annotated:

```
(define-generic-procedure-handler annotate-expr
  (match-args if-expr? any-object?)
  (lambda (expr env)
    (make-texpr (type-variable)
        (make-if-expr
         (annotate-expr (if-predicate expr) env)
         (annotate-expr (if-consequent expr) env)
         (annotate-expr (if-alternative expr) env)))))
```

There are annotation handlers for every kind of expression. We will not show all of them, but the annotation of a lambda expression is interesting:

```
(define-generic-procedure-handler annotate-expr
  (match-args lambda-expr? any-object?)
  (lambda (expr env)
```

```
(let ((env* (new-frame (lambda-bvl expr) env)))
  (make-texpr
   (procedure-type (map (lambda (name)
                          (get-var-type name env*))
                        (lambda-bvl expr))
                   (type-variable))
   (make-lambda-expr (lambda-bvl expr)
                     (annotate-expr) (lambda-body expr)
                                      env*))))))
```

Just as in an interpreter or compiler, the annotation of a lambda expression makes a new environment frame to hold information about the bound variables; in this case we create one type variable for each bound variable. We create a procedure type for the value of the lambda expression, with the type variables we just created for the bound variables and a type variable for the value, and we recursively annotate the body.

#### **Constraints**

The program-constraints procedure formulates constraints on the type variables based on the semantic structure of the program. It is also implemented using a generic procedure with handlers for each expression type.

```
(define (program-constraints texpr)
  (program-constraints-1 (texpr-type texpr)
                         (texpr-expr texpr)))
(define program-constraints-1
  (simple-generic-procedure 'program-constraints-1 2 #f))
```

This generic procedure takes two arguments: the type of an expression and the expression itself. It returns a list of constraints on the types that it discovers in its study of the expression. It walks the expression tree, discovering and formulating type constraints where they may be found.

Here is the handler for conditionals:

```
(define-generic-procedure-handler program-constraints-1
  (match-args type-expression? if-expr?)
```

```
(lambda (type expr)
  (append
    (list (constrain (boolean-type)
                     (texpr-type (if-predicate expr)))
          (constrain type
                     (texpr-type (if-consequent expr)))
          (constrain type
                     (texpr-type (if-alternative expr))))
    (program-constraints (if-predicate expr))
    (program-constraints (if-consequent expr))
    (program-constraints (if-alternative expr)))))
```

This handler formulates three type constraints that it adds to the constraints it recursively formulates in the three subexpressions of the conditional. The first constraint is that the value of the predicate expression is a boolean. The second and third constraints are that the type of the value of the conditional is the same as the types of the value of the consequent expression and the value of the alternative expression.

The constraints are represented as equations:

```
(define (constrain lhs rhs)
  '(= ,lhs ,rhs))
```

(The identifiers lhs and rhs are mnemonic for "left-hand side" and "right-hand side" respectively.)

The type constraint for a lambda expression is that the type of the value returned by the lambda expression's procedure is the same as the type of the value of its body. This is combined with the constraints formulated for the body.

```
(define-generic-procedure-handler program-constraints-1
  (match-args type-expression? lambda-expr?)
  (lambda (type expr)
    (cons (constrain (procedure-type-codomain type)
                     (texpr-type (lambda-body expr)))
          (program-constraints (lambda-body expr)))))
```

The constraints for a procedure call are that the operator's type is a procedure type, the types of the operand expressions match the argument types of the procedure, and the type of the value returned by the procedure is the type of the call.

```
(define-generic-procedure-handler program-constraints-1
  (match-args type-expression? combination-expr?)
  (lambda (type expr)
    (cons (constrain (texpr-type (combination-operator expr))
                     (procedure-type
                      (map texpr-type
                           (combination-operands expr))
                      type))
          (append
            (program-constraints (combination-operator expr))
            (append-map program-constraints
                        (combination-operands expr))))))
```

#### **Unification**

Each constraint discovered is an equation of two type expressions. So we now have a set of equations to solve. This is accomplished by unifying the left-hand side (lhs) and right-hand side (rhs) of each equation. All these unifications must be done in the same variablebinding context so as to solve them simultaneously. Since unification of two lists unifies corresponding elements of the lists, we can just combine the constraints into one giant unification:

```
(define (unify-constraints constraints)
  (unify (map constraint-lhs constraints)
         (map constraint-rhs constraints)))
```

The dictionary returned by unify-constraints is then used by infer-program-types (on page 196) to instantiate the typed program.

#### **Critique**

Although this small type-inference system is a nice demonstration of unification, it is not very good at type inference: it doesn't really handle procedures very well. For example, consider the simple case:

```
(pp (infer-program-types
     '(begin (define id (lambda (x) x))
             (id 2))))
```

This apparently works correctly, returning

```
(t (numeric-type)
   (begin
     (t (type:procedure ((numeric-type)) (numeric-type))
        (define id
          (t (type:procedure ((numeric-type)) (numeric-type))
             (lambda (x) (t (numeric-type) x)))))
     (t (numeric-type)
        ((t (type:procedure ((numeric-type)) (numeric-type))
            id)
         (t (numeric-type) 2)))))
```

But notice that the identity procedure has been typed as having a numeric argument and a numeric value, because the procedure was used with a numeric argument. However, the correct type for the identity procedure should not require any specific kind of argument. More generally, the type of a procedure should not depend on its use in an example. This confusion causes a failure to type a perfectly reasonable piece of code:

```
(infer-program-types
  '(begin (define id (lambda (x) x))
          (id 2)
          (id #t)))
***type-error***
```

## **Exercise 4.14: Procedures**

The specific problem shown in the critique above is not very hard to fix, but the general case is complicated. How can we handle procedures passed as arguments and returned as values? Remember that there may be free variables in a procedure that are lexically bound where the procedure was defined.

Work out a fix for this problem, and make it as general as you can.

## **Exercise 4.15: Parametric types**

This exercise examines what it takes to extend this type-inference system to work with parametric types. For example, the Scheme map procedure operates on lists of objects of any type.

- **a.** What must be done to extend the system to support parametric types? Does this extension require us to modify the unifier? If so, explain why it is necessary. If not, explain why it is not necessary.
- **b.** Implement the changes required to allow parametric types to be used.

## **Exercise 4.16: Union types**

The type-inference system we have outlined here does not have any support for union types. For example, the addition operator + is defined for numeric arithmetic. But what if we want + to be both addition of numbers and concatenation of strings?

- **a.** What must be done to extend the system to support union types? Does this extension require us to modify the unifier? If so, explain why it is necessary. If not, explain why it is not necessary.
- **b.** Implement the changes required to allow union types to be used. Note: This is not easy.

## **Exercise 4.17: Side effects**

The type-inference system we have outlined here is adequate for pure functional programs. Can it be elegantly extended to work for programs with assignment? If you think so, explain and demonstrate your design. If you think not, explain your reasons.

This is not easy. It may be a nice term project to understand this and make it work.

### **Exercise 4.18: Is this practical?**

Is our implementation of type inference practical?

- **a.** Estimate the orders of growth of time and space of the annotation and constraints phases with the size of the program being analyzed.
- **b.** Estimate the order of growth of time and space of the giant unification phase, using the algorithm shown. What about the best known algorithms for unification? (This will take some library research.)
- **c.** Is there a way to break up the giant unification phase into parts so that the whole has better asymptotic behavior?

#### **4.4.4 Adding segment variables—an experiment!**

<span id="page-50-0"></span>Adding segment variables to the unifier is exciting: we are not sure exactly what we will get. [10](#page-83-3) But our careful use of generic procedures will ensure that no program that depends on the behavior of the unifier without the addition of segment variables (such as the type inference example) will produce wrong answers as a consequence of this experiment. Indeed, the organization of the unifier in terms of generic procedures makes such experiments relatively unproblematic. [11](#page-84-0)

<span id="page-50-1"></span>The use of predicates for the dispatch controls the interaction between element variables and segment variables. For example, a segment variable may incorporate an element variable into a segment it accumulates, but an element variable may not have a segment variable as its value. Thus, we must change our definition of the predicate element? (on page 192) to exclude segment variables:

```
(define (element? term)
  (not (match:segment-var? term)))
```

We need generic handlers for the segment variable cases. The unify:gdispatch handler for term lists that start with a segment variable is maybe-grab-segment, which is installed as follows. The list known to contain the segment variable is always passed as the first argument to maybe-grab-segment (as we did with maybesubstitute). [12](#page-84-1)

```
(define-generic-procedure-handler unify:gdispatch
  (match-args (car-satisfies match:segment-var?)
              (complement (car-satisfies match:segment-
var?)))
  (lambda (var-first terms)
    (maybe-grab-segment var-first terms)))
(define-generic-procedure-handler unify:gdispatch
  (match-args (complement (car-satisfies match:segment-var?))
              (car-satisfies match:segment-var?))
  (lambda (terms var-first)
    (maybe-grab-segment var-first terms)))
```

In matching two termlists that each start with a segment variable, there is a special case to be handled: if both lists start with the same segment variable, we can dismiss the tautology without further work. Otherwise it is possible that we will get a match starting with either variable. But there are cases where a good match is obtained with one variable but not the other, depending on the further occurrences of the variables in the patterns. To make sure we don't miss a match starting with one of the variables, we make the match symmetrical by trying the other order if the first one fails.

```
(define (unify:segment-var-var var-first1 var-first2)
  (define (unify-seg-var-var dict succeed fail)
    (if (match:vars-equal? (car var-first1) (car var-first2))
        (succeed dict fail (cdr var-first1) (cdr var-first2))
        ((maybe-grab-segment var-first1 var-first2)
         dict
         succeed
         (lambda ()
           ((maybe-grab-segment var-first2 var-first1)
            dict
            succeed
```

```
fail)))))
  unify-seg-var-var)
(define-generic-procedure-handler unify:gdispatch
  (match-args (car-satisfies match:segment-var?)
              (car-satisfies match:segment-var?))
  unify:segment-var-var)
```

The procedure maybe-grab-segment is analogous to the procedure maybe-substitute (on page 191) used for element variables. The case of matching a segment variable against itself was handled by unify:segment-var-var; so maybe-grab-segment starts by checking if the segment variable at the start of var-first has a value. If so, we replace the variable with its value and match the resulting list against the term list terms. Because the binding of a segment variable is the list of elements that it gobbles, we use append to replace the segment variable with its value. The more complex job of matching an unbound segment variable is passed on to grab-segment.

```
(define (maybe-grab-segment var-first terms)
  (define (maybe-grab dict succeed fail)
    (let ((var (car var-first)))
      (if (match:has-binding? var dict)
          ((unify:dispatch
            (append (match:get-value var dict)
                    (cdr var-first))
            terms)
           dict succeed fail)
          ((grab-segment var-first terms)
           dict succeed fail))))
  maybe-grab)
```

The procedure grab-segment is where the segment matching and backtracking actually happen. The term list is broken into two parts: an initial segment and the rest of the terms (terms\*). The initial segment (initial) starts out empty and terms\* is the whole term list. The match tries to proceed with the segment variable bound to initial. If that fails, the failure continuation tries to match with an element moved from terms\* to initial. This is

repeated until either a match succeeds or the match of the entire term list fails:

```
(define (grab-segment var-first terms)
  (define (grab dict succeed fail)
    (let ((var (car var-first)))
      (let slp ((initial '()) (terms* terms))
        (define (continue)
          (if (null? terms*)
              (fail)
              (slp (append initial (list (car terms*)))
                   (cdr terms*))))
        (let ((dict* (do-substitute var initial dict)))
            (if dict*
                (succeed dict* continue (cdr var-first)
terms*)
                (continue))))))
  grab)
```

This appears to be all that is required to make a unifier with an experimental extension for segment variables. With segment variables we must expect to get multiple matches. We can reject a match, forcing the program to backtrack, to get an alternative. Note that each dictionary entry is a list containing the name of the variable, the value, and the type of the variable.

We can use the unifier as a one-sided matcher. For example, there are exactly two ways to match the distributive law pattern to the given algebraic expression, as we see: [13](#page-84-2)

```
(let ((pattern '(* (?? a) (+ (?? b)) (?? c)))
      (expression '(* x y (+ z w) m (+ n o) p)))
  (unify:internal pattern expression (match:new-dict)
    (lambda (dict)
      (pp (match:bindings dict))
      #f)))
((c (m (+ n o) p) ??) (b (z w) ??) (a (x y) ??))
((c (p) ??) (b (n o) ??) (a (x y (+ z w) m) ??))
#f
```

Both of these dictionaries produce the same substitution instance:

```
(* x y (+ z w) m (+ n o) p)
```

But in an algebraic manipulator we really want both dictionaries, because each of them represents a different application of the distributive law.

Things get more complicated and much less clear when segment variables match against lists containing segment variables:

```
(let ((p1 '(a (?? x) (?? y) (?? x) c))
      (p2 '(a b b b (?? w) b b b c)))
  (unify:internal p1 p2 (match:new-dict)
    (lambda (dict)
      (pp (match:bindings dict))
      #f)))
((y (b b b (?? w) b b b) ??) (x () ??))
((y (b b (?? w) b b) ??) (x (b) ??))
((y (b (?? w) b) ??) (x (b b) ??))
((w () ??) (y () ??) (x (b b b) ??))
((w () ??) (y () ??) (x (b b b) ??))
((y ((?? w)) ??) (x (b b b) ??))
((y () ??) (w () ??) (x (b b b) ??))
((w ((?? y)) ??) (x (b b b) ??))
((w () ??) (y () ??) (x (b b b) ??))
#f
```

Apparently, there are many ways to make this match. But many of the dictionaries are really different ways of constructing the same substitution instance. To see this clearly, we construct the substitution instance in each case:

```
(let ((p1 '(a (?? x) (?? y) (?? x) c))
      (p2 '(a b b b (?? w) b b b c)))
  (unify:internal p1 p2 (match:new-dict)
    (lambda (dict)
      (and dict
           (let ((subst (match:dict-substitution dict)))
             (let ((p1* (subst p1)) (p2* (subst p2)))
               (if (not (equal? p1* p2*))
                   (error "Bad dictionary"))
               (pp p1*))))
      #f)))
(a b b b (?? w) b b b c)
(a b b b (?? w) b b b c)
(a b b b (?? w) b b b c) (a b b b b b b c)
(a b b b b b b c)
```

```
(a b b b (?? w) b b b c) (a b b b b b b c)
(a b b b (?? y) b b b c) (a b b b b b b c)
#f
```

So we see that each "solution" is a valid solution to the problem of finding values for the variables that when substituted back into the given patterns make the patterns the same. In this case five of the solutions are equivalent. These five are the most general unifiers, and they are unique up to renaming of the variables. The other four are not as general as possible. But unification is supposed to produce the unique most general common substitution instance of the two input patterns, up to renaming of variables. So with segments, this very useful pattern matcher is not really a unifier.

Actually, the problem is a bit worse. There are perfectly good matches that are not found by this program. Here is an example:

```
;;; A missing match!
(unify:internal '(((?? x) 3) ((?? x)))
                '((4 (?? y)) (4 5))
                (match:new-dict)
                (lambda (dict)
                  (pp (match:bindings dict))
                  #f))
#f
```

But these expressions do match, with the following bindings:

```
((x (4 5) ??) (y (5 3) ??))
```

How sad! But there is a moral to this story. Using generic procedures, we can make possibly problematic extensions to a correct algorithm without undermining its correctness for uses of the unextended algorithm. The extensions may be useful for some purposes, even without satisfying the correctness requirements of the unextended algorithm.

### **Exercise 4.19: Can we fix these problems?**

We have a problem with unifying patterns containing segment variables. We may miss some matches; we may generate multiple copies of the same solution; and some of the solutions, although valid solutions of the problem to make the input patterns equal, are not maximally general. Let's think about fixing this.

- **a.** Write a wrapper for unify:internal that collects all of the solutions. This is not hard if you use assignments, but it might be more fun to look for a functional solution—but don't try too hard!
- **b.** Now that you have all of the solutions, it is easy to eliminate duplicates. Create the result of substituting from each solution into the inputs. You can check that the results of the two substitutions are equal—this is a check that the algorithm for solving is correct. Now save the pair of one substitution and one result for each distinct result. Be careful: The name of a variable doesn't matter, so two resulting dictionaries represent the same solution if you can get one by uniformly renaming variables in the other.
- **c.** If any result in the collection is a substitution instance of another result, it is not a most general common specialization of the two inputs.

Write the predicate substitution-instance? to filter those out. You are now left with the collection of the most general common specializations that this algorithm will generate. Return these.

**d.** Figure out a way to avoid missing matches like the "A missing match!" shown above. Is there a simple extension of the code shown that can handle this kind of match? Note: This is an extremely difficult problem.

### **Exercise 4.20: More general matches**

Beyond the nasty problems shown above, there is an interesting subtlety that is not addressed by the unifier. Consider the following problem:

```
(unifier '((?? x) 3) '(4 (?? y)))
(4 3)
```

Here we see a perfectly good match, but it is not the most general one possible. The problem is that there can be any number of things between the 4 and the 3. A better answer would be:

```
(4 (?? z) 3)
```

Figure out how to get this answer. This requires a significant extension to the unifier.

## **Exercise 4.21: Strings with segments**

If you did not do exercise 4.11 (page 193), do it now. But here we want you to add string segment variables. This can be useful in matching segments of DNA!

## **4.5 Pattern matching on graphs**

The pattern matching we have developed so far is for matching list structures. Such structures are excellent for representing expressions, such as algebraic expressions or the abstract syntax trees of computer-language expressions. However, pattern matching can be used to make systems that operate on a much wider range of data. If the structure of interest can be characterized by an accessibility relation, it may be appropriate to describe the structure as a graph of *nodes* representing "places" and *edges* representing "path elements" describing how the places are interconnected. An electrical circuit is an example of such a structure, where the circuit parts and circuit nodes are places and the accessibility relation is

just the description of the interconnect. A board game like chess or checkers is another, where the board squares may be represented by nodes in a graph and the adjacency of squares may be represented by edges in the graph.

We will implement a graph as a collection of nodes and edges. Our graphs are immutable in the sense that once a node or an edge is added, it cannot be modified; the graph can be changed only by adding more nodes and edges. This will have consequences that we will see in section 4.5.4.

A node contains a collection of edges, and an edge is a combination of a *label* and a *value*. The label of an edge is an object that is unique under eqv?, usually a symbol or a number. The value of an edge is a Scheme object, often another node.

This implementation will work with concrete graphs (where all of the nodes and edges are available at the time we build the graph) and lazy graphs (where the graph is extended, as necessary, at the time of access). In the simpler world of linear sequences, a list is a concrete graph and a stream is a lazy graph that is generated when referenced.

We will first look at a simple example to see how graphs work. We will then use an extended example, a chess referee, to explore more complex uses of graphs and pattern matching on graphs.

#### **4.5.1 Lists as graphs**

We start with the simple but familiar world of lists. The cons cells are the nodes, which will be made with g:cons and whose car and cdr will be implemented as edges labeled with car and cdr and accessed by g:car and g:cdr:

```
(define (g:cons car cdr)
  (let ((pair (make-graph-node 'pair)))
    (pair 'connect! 'car car)
    (pair 'connect! 'cdr cdr)
    pair))
(define (g:car pair) (pair 'edge-value 'car))
(define (g:cdr pair) (pair 'edge-value 'cdr))
```

To represent lists as graphs we need a special end marker for lists:

```
(define nil (make-graph-node 'nil))
(define (g:null) nil)
(define (g:null? object) (eqv? object nil))
```

The conversion of a list to a list graph is:

```
(define (list->graph list)
  (if (pair? list)
      (g:cons (car list) (list->graph (cdr list)))
      (g:null)))
```

and a simple example works as expected:

```
(define g (list->graph '(a b c)))
(and (eqv? 'a (g:car g))
     (eqv? 'b (g:car (g:cdr g)))
     (eqv? 'c (g:car (g:cdr (g:cdr g))))
     (g:null? (g:cdr (g:cdr (g:cdr g)))))
#t
```

We can modify the list-graph constructor to allow lazy graphs, with nodes that are created as edges are traversed:

```
(define (list->lazy-graph list)
  (if (pair? list)
      (g:cons (delay (car list))
              (delay (list->lazy-graph (cdr list))))
      (g:null)))
```

Here we used the Scheme [109] delay to construct a *promise* that will evaluate the delayed (postponed) expression when the promise is forced. Streams [13] (lazy lists) are usually constructed using delay and force.

#### **4.5.2 Implementing graphs**

We have to be able to make *graph nodes* and connect them to other nodes by *edges*. We will represent a graph node as a *bundle*

<span id="page-60-0"></span>*procedure*: a collection of delegate procedures that can be called by name. [14](#page-84-3)

```
(define (make-graph-node name)
  (let ((edges '()))
    (define (get-name) name)
    (define (all-edges) (list-copy edges))
    (define (%find-edge label)
      (find (lambda (edge)
              (eqv? label (edge 'get-label)))
            edges))
    (define (has-edge? label)
      (and (%find-edge label) #t)) ; boolean value
    (define (get-edge label)
      (let ((edge (%find-edge label)))
        (if (not edge)
            (error "No edge with this label:" label))
        edge))
    (define (edge-value label)
      ((get-edge label) 'get-value))
    (define (connect! label value)
      (if (has-edge? label)
          (error "Two edges with same label:" label))
      (set! edges
            (cons (make-graph-edge label value) edges)))
    (define (maybe-connect! label value)
      (if (not (default-object? value))
          (connect! label value)))
    (bundle graph-node? get-name all-edges has-edge?
            get-edge edge-value connect! maybe-connect!)))
```

The argument to make-graph-node is the name of the new node; this is shown when printing a node object. The first argument to the bundle macro is a predicate that the generated bundle will satisfy. In this case, it is defined as

```
(define graph-node? (make-bundle-predicate 'graph-node))
```

We will not show the definitions of other bundle predicates since they are similar.

Edges are also represented as bundle procedures. An edge may have a concrete value or the value may be a promise (constructed by delay) to produce the value when asked. The latter provides for lazy graph structures.

```
(define (make-graph-edge label value)
  (define (get-label) label)
  (define (get-value)
    (if (promise? value)
        (force value)
        value))
  (bundle graph-edge? get-label get-value))
```

## **Exercise 4.22: More lazy graphs**

We have shown how to make concrete lists and lazy lists. How about some more interesting structures?

Perhaps it would be nice to have a dynamically extensible tree. For example, a game tree could be usefully built this way: we may want to elaborate the tree both in breadth and depth as resources become available. Make an example of such a tree that can be extended as more plausible moves are considered at each level, and as more levels are added for consideration.

#### **4.5.3 Matching on graphs**

We might want to search a graph for interesting features. One way to do this is to try to match patterns to the graph. A pattern for a graph could specify an alternating sequence of nodes and edges: a *path*. Such a pattern can be matched by starting at a node and trying to follow the path specified by the pattern.

Imagine, for example, that we have a chessboard and chess pieces. The board squares are nodes of a graph. The nodes representing adjacent squares are connected to a given node by edges. We can label the edges, as seen by the player playing White, by compass directions: north, south, east, west, northeast, southeast, northwest, southwest. Going north is toward the Black side of the board, and going south is toward the White side of the board.

Given such an arrangement, we can specify a move where a knight may move north-north-east as:

```
(define basic-knight-move
  '((? source-node ,(occupied-by 'knight))
    north (?)
    north (?)
    east (? target-node ,maybe-opponent)))
```

This pattern shares several characteristics with those we've looked at in previous sections: element variables are introduced by the ? character; they can have names (e.g., source-node); and they can have restrictions (e.g., (occupied-by 'knight)). We introduce the syntax (?) to indicate an anonymous element variable.

The pattern match starts with source-node, traverses two edges labeled north—with nodes that we don't care about—and finally travels east to reach target-node. We call this kind of pattern a *path pattern*, or in the context of chess, a *move pattern*.

Of course, this is only one possible knight move. But we can generate all possible knight moves by symmetries: we can reflect the knight move east-west, we can rotate it clockwise by 90 degrees, and we can rotate it by 180 degrees:

```
(define all-knight-moves
  (symmetrize-move basic-knight-move
                   reflect-ew rotate-90 rotate-180))
```

The symmetrize-move procedure applies all possible combinations of these three symmetries to produce eight moves. The order in which the symmetry transformations are applied doesn't matter for the transformations we use.

```
(define (symmetrize-move move . transformations)
  (let loop ((xforms transformations) (moves (list move)))
    (if (null? xforms)
        moves
        (loop (cdr xforms)
              (append moves
                      (map (rewrite-path-edges (car xforms))
                           moves))))))
```

where rewrite-path-edges applies its argument to each edge label in a move, producing a new move with substituted edge labels.

One example of such a symmetry transformation is

```
(define (reflect-ew label)
  (case label
    ((east) 'west)
    ((northeast) 'northwest)
    ((northwest) 'northeast)
    ((southeast) 'southwest)
    ((southwest) 'southeast)
    ((west) 'east)
    (else label)))
```

and the others are similar remappings of the compass directions. The resulting list of all knight moves is

```
((source north (?) north (?) east target ) (source north (?)
north (?) west target)
 (source east (?) east (?) south target)
 (source east (?) east (?) north target)
 (source south (?) south (?) west target)
 (source south (?) south (?) east target)
 (source west (?) west (?) north target)
 (source west (?) west (?) south target))
```

where we have simplified the printing by replacing the restricted source and target node variables with *source* and *target*.

Knight moves are special in chess, in that a knight can move over squares occupied by either a friend or an opponent to get to the target square. Rooks, bishops, and queens may not pass through an occupied square, but they may pass through many unoccupied squares on their way to a target square. We need a way to specify such a repeated traversal. We use (\* ...) to specify a repeated traversal:

```
(define basic-queen-move
  '((? source-node ,(occupied-by 'queen))
    (* north (?* ,unoccupied))
    north (? target-node ,maybe-opponent)))
```

The queen may move north through any number of unoccupied squares to the target square. The notation (?\* ...) is a new kind of pattern variable that can be used only inside a (\* ...) pattern. Like a simple pattern variable, it matches one element, but instead

of saving just a single matched value, it collects a list of all elements matched in the repeat. All of the queen's possible moves are then:

```
(define all-queen-moves
  (symmetrize-move basic-queen-move
                   rotate-45 rotate-90 rotate-180))
```

<span id="page-64-0"></span>Pawns have more complicated rules. A pawn is (almost) the only piece whose possible moves depend on its position or the position of a neighboring opponent. [15](#page-84-4) A pawn may go north one or two steps from its initial position, but it may go only one step north if not in its initial position. A pawn may take one step northeast or northwest if and only if that move takes an opponent piece. Finally, a pawn in the penultimate row may move into the last row and be promoted into any piece, usually a queen. [16](#page-84-5)

## <span id="page-64-1"></span>**Exercise 4.23: Filling out chess moves**

We have shown how to make patterns for knight moves and queen moves, but we have not made patterns for moves for all chess pieces.

- **a.** Rook moves and bishop moves are similar to queen moves, but more restricted: a rook cannot move diagonally, and a bishop can move *only* diagonally. Make patterns for all bishop moves and all simple rook moves.
- **b.** Pawn moves are much more complicated. Make a set of patterns for all possible pawn moves (except en passant captures).
- **c.** Make a set of patterns for the king's very limited ways to move. Don't worry about castling or the rule that a king cannot be moved into check.
- **d.** Castling is the final special case. It involves both the king and a rook. Make a set of castling patterns. (See footnote 15.)

### **4.5.4 Chessboards and alternate graph views**

The chessboard, as a graph, incorporates an exciting idea. We want the same patterns to work for both players. But the edges describing directions are different: north for White is south for Black and east for White is west for Black! This makes little difference for the major pieces (the rooks, knights, bishops, kings, queens) with symmetrical move patterns, but White pawns can move only north and Black pawns can move only south. In any case, it would be pleasant to make the move descriptions the same for both players.

We want the two players to have different views of the board graph: we want the meanings of the edge labels to be relative to the player. If the player playing White sees a north edge from (the node representing) square A to square B we want the player playing Black to see a north edge from square B to square A.

To make this work we introduce *graph views*. A graph view is a reversible mapping from one edge label to another. When a graph view is applied to a node, it returns a copy of that node in which the edges are renamed.

In the case of chess the relevant view is with the board rotated 180 degrees:

```
(define rotate-180-view
  (make-graph-view 'inverse rotate-180 rotate-180))
```

where make-graph-view makes a graph view. The procedure graph-node-view applies a view to a node:

```
(graph-node-view node view )
```

White will see a node directly and Black will see the same node projected through the rotate-180-view. Given this map, all operations look the same to both White and Black.

Using a graph view takes care of *relative* addressing, where we are looking at neighbors of a given node. But we also need to do *absolute* addressing, where the node to find is specified by a row and column. Each color wants to see similar addressing, where the home row is 0, and the opponent's home row is 7; likewise each

<span id="page-66-0"></span>color sees the leftmost column as 0 and the rightmost as 7. [17](#page-84-6) White's addresses are the default, and Black's are inverted with the procedure invert-address.

Let's make a board. The following code is specific to chess, since we're not focusing on making an abstract domain. We make an 8 × 8 array of nodes representing squares, each with an address. We iterate through all possible square addresses, connecting each node to each of its neighbors by an edge with the appropriate label. Then we populate the sides with pieces.

```
(define chess-board-size 8)
(define chess-board-indices (iota chess-board-size))
(define chess-board-last-index (last chess-board-indices))
(define (make-chess-board)
  (let ((board (make-chess-board-internal)))
    (for-each (lambda (address)
                (connect-up-square address board))
              board-addresses)
    (populate-sides board)
    board))
```

The possible addresses for chess-board squares are all pairs of integers from 0 to 7:

```
(define board-addresses
  (append-map (lambda (y)
                (map (lambda (x)
                       (make-address x y))
                     chess-board-indices))
              chess-board-indices))
```

The procedure make-chess-board-internal makes the array of nodes for squares as a list of rows, each of which is a list of columns for that row. It returns a bundle procedure with a handful of delegates to manipulate the board.

```
(define (make-chess-board-internal)
  (let ((nodes
         (map (lambda (x)
                (map (lambda (y)
                       (make-graph-node (string x "," y)))
                     chess-board-indices))
```

```
chess-board-indices)))
(let loop ((turn 0))
  See below for the delegate definitions.
  (bundle #f node-at piece-at piece-in address-of
          set-piece-at color next-turn))))
```

The turn variable is the current turn, starting with zero. Even turns are White, and odd turns are Black, as shown by the delegate procedure color:

```
(define (color) (if (white-move?) 'white 'black))
(define (white-move?) (even? turn))
```

The delegate procedure node-at gets the node at a given address. If this is a Black turn, it translates the address and applies the node view.

```
(define (node-at address)
  (define (get-node address)
    (list-ref (list-ref nodes (address-x address))
              (address-y address)))
  (if (white-move?)
      (get-node address)
      (graph-node-view (get-node (invert-address address))
                       rotate-180-view)))
```

The inverse of node-at is the delegate procedure address-of. Each node has an edge, labeled with address, with its address as the value. As with node-at, if this is a Black move the returned address must be translated.

```
(define (address-of node)
  (let ((address (node 'edge-value 'address)))
    (if (white-move?)
        address
        (invert-address address))))
```

The delegate procedure next-turn advances the board after a move is made:

```
(define (next-turn) (loop (+ turn 1)))
```

Connecting the squares to their neighbors does address arithmetic to handle (literal) edge cases, creating a labeled edge

between each square and each of its neighbors. It also creates the address edge for each node.

```
(define (connect-up-square address board)
  (let ((node (board 'node-at address)))
    (node 'connect! 'address address)
    (for-each-direction
     (lambda (label x-delta y-delta)
       (let ((x+ (+ (address-x address) x-delta))
             (y+ (+ (address-y address) y-delta)))
         (if (and (<= 0 x+ chess-board-last-index)
                  (<= 0 y+ chess-board-last-index))
             (node 'connect! label
                   (board 'node-at
                          (make-address x+ y+)))))))))
(define (for-each-direction procedure)
  (procedure 'north 0 1)
  (procedure 'northeast 1 1)
  (procedure 'east 1 0)
  (procedure 'southeast 1 -1)
  (procedure 'south 0 -1)
  (procedure 'southwest -1 -1)
  (procedure 'west -1 0)
  (procedure 'northwest -1 1))
```

An address is represented as a list of column and row number:

```
(define (make-address x y) (list x y))
(define (address-x address) (car address))
(define (address-y address) (cadr address))
(define (address= a b)
  (and (= (address-x a) (address-x b))
       (= (address-y a) (address-y b))))
(define (invert-address address)
  (make-address (- chess-board-last-index
                   (address-x address))
                (- chess-board-last-index
                   (address-y address))))
```

A piece is represented by data incorporating its piece type and its color. Our convention is that at the *n*th turn, each piece on the board will be connected to the node representing the square it occupies by an edge from that node with the label *n*. This is a

consequence of graph immutability; otherwise we could just use a side effect to modify the edge. To populate the board, we connect each piece to the node for its initial square with an edge labeled 0.

```
(define (populate-sides board)
  (define (populate-side color home-row pawn-row)
    (define (do-column col type)
      (add-piece col home-row type)
      (add-piece col pawn-row 'pawn))
    (define (add-piece col row type)
      ((board 'node-at (make-address col row))
        'connect! 0 (make-piece type color)))
    (do-column 0 'rook)
    (do-column 1 'knight)
    (do-column 2 'bishop)
    (do-column 3 'queen)
    (do-column 4 'king)
    (do-column 5 'bishop)
    (do-column 6 'knight)
    (do-column 7 'rook))
  (populate-side 'white 0 1)
  (populate-side 'black 7 6))
  We can now start a game:
(define the-board)
(define (start-chess-game)
  (set! the-board (make-chess-board))
  (print-chess-board the-board))
```

And we get this nice chessboard image:

#### **4.5.5 Chess moves**

Now that we have a chessboard, populated with pieces, we need a way to move those pieces around. If a piece is in a particular square at a particular turn, the node representing that square has an edge, with the turn as its label, whose value is the piece. The following delegate procedures in make-chess-board-internal on page 218 are relevant here:

```
(define (piece-at address)
  (piece-in (node-at address)))
(define (piece-in node)
  (and (node 'has-edge? turn)
       (node 'edge-value turn)))
(define (set-piece-at address piece)
  ((node-at address) 'connect! (+ turn 1) piece))
```

We use piece-at to obtain a piece that we expect to move, given its address. Of course, it is always a good idea to check for obvious errors.

```
(define (get-piece-to-move board from)
  (let ((my-piece (board 'piece-at from)))
    (if (not my-piece)
        (error "No piece in this square:" from))
    (if (not (eq? (board 'color) (piece-color my-piece)))
        (error "Can move only one's own pieces:"
               my-piece from))
    my-piece))
```

To actually make a move we pick the piece up and set it down in the target square. However, this move is allowed only if the target square is empty or if it is occupied by an opponent piece to be captured.

```
(define (simple-move board from to)
  (let ((my-piece (get-piece-to-move board from)))
    (let ((captured (board 'piece-at to)))
      (if (not (no-piece-or-opponent? captured my-piece))
          (error "Can't capture piece of same color:"
                 captured)))
    ;; The move looks good; make it so:
    (board 'set-piece-at to my-piece)
    ;; Now update all the unaffected pieces to
    ;; the next state of the board:
    (for-each (lambda (address)
                (if (not (or (address= from address)
                             (address= to address)))
                    (let ((p (board 'piece-at address)))
                      (if p
                          (board 'set-piece-at address p)))))
              board-addresses)
    (board 'next-turn)))
```

Notice that we didn't put in a check that the piece we want to move is able to make that move. Our only descriptions of the legal moves available to each kind of piece are in the graph patterns that we built in section 4.5.3. In exercise 4.24 on page 225 we will fix this problem.

But first, let's use the matcher to determine whether a move described by such a path pattern is a capture:

```
(define (capture? board from path)
  (let* ((my-piece (get-piece-to-move board from))
         (dict
```

```
(graph-match path
                   (match:extend-dict chess-board:var ;**
                                       board
                                       (match:new-dict))
                   (board 'node-at from))))
(and dict
     (let* ((target (match:get-value 'target-node dict))
            (captured (board 'piece-in target)))
       (and captured
            '(capture ,my-piece
                      ,captured
                      ,(board 'address-of target)))))))
```

The line marked by ;\*\* adds a special binding in the initial dictionary, which is used by some pattern restrictions that need to interrogate the board.

For convenience, chess-move updates the board with a move, then prints the board for the player who will move next.

```
(define (chess-move from to)
  (set! the-board (simple-move the-board from to))
  (print-chess-board the-board))
```

To demonstrate this code we can make an interesting position:

```
(define (giuoco-piano-opening)
 (start-chess-game)
 (chess-move '(4 1) '(4 3)) ;W: P-K4
 (chess-move '(3 1) '(3 3)) ;B: P-K4
 (chess-move '(6 0) '(5 2)) ;W: N-KB3
 (chess-move '(6 0) '(5 2)) ;B: N-QB3
 (chess-move '(5 0) '(2 3)) ;W: B-QB4
 (chess-move '(2 0) '(5 3))) ;B: B-QB4
(giuoco-piano-opening)
```

After lots of printout, we obtain the following board position:

At this point the White Knight at King Bishop 3 is attacking the Black Pawn at King 5. It is not a good idea to take that piece because it is defended by the Black Knight at Queen-Bishop 6, and one should not exchange a knight for a pawn. However, we can use a graph pattern from the knight moves to check that this is a possible capture:

```
(capture? the-board
          (make-address 5 2)
          '((? source-node ,(occupied-by 'knight))
            north (?) north (?)
            west (? target-node ,maybe-opponent)))
(capture (knight white) (pawn black) (4 4))
```

#### Indeed, it is the only possible capture for this knight:

```
(filter-map (lambda (path)
              (capture? the-board
                        (make-address 5 2)
                        path))
            all-knight-moves)
((capture (knight white) (pawn black) (4 4)))
```

## **Exercise 4.24: Legal chess moves**

In exercise 4.23 on page 216 we made a library of patterns for all legal chess moves. Modify the simple-move program (page 222) to check that the piece being moved is allowed to move in the way requested.

#### **4.5.6 Implementing graph matching**

The entry point for using a graph pattern is:

```
(define (graph-match path dict object)
  ((gmatch:compile-path path) object dict
   (lambda (object* dict*)
     dict*)))
```

We compile the path pattern into a match procedure that takes the graph object (a node) to start from, an initial dictionary, and a success continuation. If the pattern successfully matches a sequence of edges starting with that node, it calls the success continuation, which takes the node (object\*) at the end of the matched path and a dictionary of bindings accumulated in the match, as described in section 4.3. [18](#page-84-7) If the pattern fails to match the given object the match procedure returns #f.

<span id="page-74-0"></span>The patterns that we are using for matching against graphs are *expressions* of a small language that we want to compile into match procedures. The syntax of graph-pattern expressions can be described in BNF. Here a postfix \* indicates 0 or more occurrences, postfix + indicates 1 or more, and postfix ? indicates 0 or 1 occurrence. An infix | indicates alternatives. Items surrounded in " are literal strings. For example, a pattern variable to match a single element starts with (?, has an optional name and optional predicate, and ends with ).

```
<edge> = <edge-label> <target>
<edge-label> = <symbol>
<target> = <node-var> | <object-var> | <constant>
<node-var> = <single-var>
```

```
<object-var> = <single-var> | <sequence-var>
<single-var> = "(?" <var-name>? <unary-predicate>? ")"
<sequence-var> = "(?*" <var-name>? <unary-predicate>? ")"
<var-name> = <symbol>
<path> = <node-var> <path-elements>
<path-elements> = <path-element>*
<path-element> =
    <edge>
  | "(*" <path-elements> ")" ; repeat any number of times
  | "(+" <path-elements> ")" ; repeat at least once
  | "(opt" <path-elements> ")" ; one or zero instances
  | "(or" <ppath-elements>+ ")"
  | "(and" <ppath-elements>+ ")"
<ppath-elements> = "(" <path-elements> ")"
```

In our graph-matching language every path in a graph starts with a node variable. A node variable is a single-element variable, which satisfies the predicate match:element-var?. We compile a path as follows:

```
(define (gmatch:compile-path path)
  (if (and (pair? path) (match:element-var? (car path)))
      (gmatch:finish-compile-path (cdr path)
        (gmatch:compile-var (car path)))
      (error "Ill-formed path:" path)))
```

Here we check that the first element of path is an element variable; if so, we compile it into a variable matcher. The remainder of the path, if any, is compiled by finish-compile-path: [19](#page-85-0)

```
(define (gmatch:finish-compile-path rest-elts matcher)
  (if (null? rest-elts)
      matcher
      (gmatch:seq2 matcher
                   (gmatch:compile-path-elts rest-elts))))
```

where seq2 produces a match procedure that sequentially matches its match-procedure arguments:

```
(define (gmatch:seq2 match-first match-rest)
  (define (match-seq object dict succeed)
    (match-first object dict
                 (lambda (object* dict*)
```

```
(match-rest object* dict* succeed))))
match-seq)
```

The variable matcher match-first, produced by compile-var, will match the initial node of the path, and the resulting dictionary dict\* is then used by the result of compile-path-elts (matchrest) to match the remainder of the path, starting with the edge object\*.

There are only a few cases for compiling path-element patterns. Either the path starts with an edge label and a target node, or it starts with a special match form (\*, +, opt, or, and):

```
(define (gmatch:compile-path-elts elts)
  (let ((elt (car elts))
        (rest (cdr elts)))
    (cond ((and (symbol? elt) (pair? rest))
           (gmatch:finish-compile-path (cdr rest)
             (gmatch:compile-edge elt (car rest))))
          ((pair? elt)
           (gmatch:finish-compile-path rest
             (gmatch:compile-path-elt elt)))
          (else
           (error "Ill-formed path elements:" elts)))))
```

An edge may be labeled by any symbol that is not one of the special symbols (\*, +, opt, or, and) used by graph-matcher patterns. The matcher for a simple labeled edge is then compiled by:

```
(define (gmatch:compile-edge label target)
  (let ((match-target (gmatch:compile-target target)))
    (define (match-edge object dict succeed)
      (and (graph-node? object)
           (object 'has-edge? label)
           (match-target (object 'edge-value label)
                         dict succeed)))
    match-edge))
```

The edge matcher, match-edge, checks that the object is a graph node, that there is an edge with the given label emanating from that object, and that the target of the edge (the edge-value) will match (using match-target) the pattern for the target in the graph-match

pattern. The match procedure match-target used in match-edge is made by the compiler compile-target.

There are only two possibilities when compiling a target: a variable or a constant.

```
(define (gmatch:compile-target elt)
  (if (match:var? elt)
      (gmatch:compile-var elt)
      (let ()
        (define (match-constant object dict succeed)
          (and (eqv? elt object)
               (succeed object dict)))
        match-constant)))
```

The special match forms are handled by compile-path-elt:

```
(define (gmatch:compile-path-elt elt)
  (let ((keyword (car elt))
        (args (cdr elt)))
    (case keyword
      ((*) (gmatch:compile-* args))
      ((+) (gmatch:compile-+ args))
      ((opt) (gmatch:compile-opt args))
      ((or) (gmatch:compile-or args))
      ((and) (gmatch:compile-and args))
      (else (error "Ill-formed path element:" elt)))))
```

Compiling a pattern with optional path elements works as follows: There is a recursive call to compile-path-elts with the path element patterns for the optional sequence of path elements, to obtain matcher, the matcher for the elements that are optionally present in the path. When match-opt is applied to a graph node object, the matcher for those path elements is applied; but if it fails, returning #f, the match succeeds with the original object and the original dictionary.

```
(define (gmatch:compile-opt elts)
  (let ((matcher (gmatch:compile-path-elts elts)))
    (define (match-opt object dict succeed)
      (or (matcher object dict succeed)
          (succeed object dict)))
    match-opt))
```

A pattern with repeated path elements, for example the pattern (\* north (?\* ,unoccupied)) in basic-queen-moves on page 215, is compiled like this:

```
(define (gmatch:compile-* elts)
  (gmatch:* (gmatch:compile-path-elts elts)))
```

As for a pattern requiring an optional sequence of path elements, the compiler is called recursively to obtain a matcher for the potentially repeated sequence, which is then passed to gmatch:\*:

```
(define (gmatch:* matcher)
  (define (match-* object dict succeed)
    (or (matcher object dict
                 (lambda (object* dict*)
                   (match-* object* dict* succeed)))
        (succeed object dict)))
  match-*)
```

The graph-pattern matcher match-\* tries to use the matcher passed to it on the graph node object supplied. If it succeeds, match-\* calls itself recursively to try the part of the graph where the last match left off. Eventually it will fail to progress, succeeding with the graph object that matcher failed on.

Compiling patterns requiring at least one, but possibly many, repetitions of a sequence of path elements, indicated with +, is similar to \*. It uses gmatch:\* as above, but requires at least one matching element first:

```
(define (gmatch:compile-+ elts)
  (let ((matcher (gmatch:compile-path-elts elts)))
    (gmatch:seq2 matcher (gmatch:* matcher))))
```

The remaining special path patterns are and and or, each of which contains a number of subpath patterns. An and element must match all of the subpath patterns starting at the current node. An or element must match at least one of the subpath patterns starting at the current node.

```
(define (gmatch:compile-and elt-lists)
  (gmatch:and (map gmatch:compile-path-elts elt-lists)))
```

```
(define (gmatch:compile-or elt-lists)
  (gmatch:or (map gmatch:compile-path-elts elt-lists)))
```

### The procedures and and or are where the real work happens:

```
(define (gmatch:and matchers)
  (lambda (object dict succeed)
    (if (null? matchers)
        (succeed object dict)
        (let loop ((matchers matchers) (dict dict))
          ((car matchers) object dict
           (if (null? (cdr matchers))
               succeed
               (lambda (object* dict*)
                 (loop (cdr matchers) dict*))))))))
(define (gmatch:or matchers)
  (lambda (object dict succeed)
    (let loop ((matchers matchers))
      (if (pair? matchers)
          (or ((car matchers) object dict succeed)
              (loop (cdr matchers)))
          #f))))
```

The procedure compile-var compiles a pattern variable. It is called from compile-path and compile-target, and has four mutually exclusive cases to handle variables with or without the optional name and predicate:

```
(define (gmatch:compile-var var)
  (cond ((match-list? var gmatch:var-type?)
         (gmatch:var-matcher (car var) #f #f))
        ((match-list? var gmatch:var-type? symbol?)
         (gmatch:var-matcher (car var) (cadr var) #f))
        ((match-list? var gmatch:var-type? symbol?
procedure?)
         (gmatch:var-matcher (car var) (cadr var) (caddr
var)))
        ((match-list? var gmatch:var-type? procedure?)
         (gmatch:var-matcher (car var) #f (cadr var)))
        (else
         (error "Ill-formed variable:" var))))
```

The procedure var-type? matches the type symbol of a pattern variable: ? or ?\*. To recognize the four cases of variables, compilevar uses a utility procedure match-list?, which is true if its first argument is a list and each element of the list satisfies the corresponding predicate argument.

```
(define (match-list? datum . preds)
  (let loop ((preds preds) (datum datum))
    (if (pair? preds)
        (and (pair? datum)
             ((car preds) (car datum))
             (loop (cdr preds) (cdr datum)))
        (null? datum))))
```

The procedure var-matcher is the matcher for variables, now that we have decoded their syntax.

```
(define (gmatch:var-matcher var-type var-name restriction)
  (define (match-var object dict succeed)
    (and (or (not restriction)
             (restriction object dict))
         (if var-name
             (let ((dict*
                    (gmatch:bind var-type var-name object
                                 dict)))
               (and dict*
                    (succeed object dict*)))
             (succeed object dict))))
  match-var)
```

Here bind adds a binding for var-name with value object, returning a new dictionary. If the dictionary already has such a binding, and its value is different from object, bind returns #f to indicate a match failure.

And with this we have finished the graph matcher.

## **Exercise 4.25: Graph matching**

The graph matcher described here is very useful, but there are problems for which it isn't well suited. What is an interesting problem that requires extension(s) to the matcher? Find such a problem, define and implement the extension(s), and demonstrate its use on some examples.

## **4.6 Summary**

Patterns are fun, but they are also a very useful way to organize parts of a system for additivity. In this chapter we have seen how to build a term-rewriting system. A rule-based term-rewriting system makes it easy to write programs that do successive replacements of parts of an expression with "equivalent" parts, terminating when no more rules are applicable. Such systems are important components of larger systems that do symbolic manipulation. Algebraic expression simplification is one application, but compilers do huge amounts of this kind of manipulation, to compute optimizations and sometimes to generate code.

We also saw a flexible way to construct a pattern matcher, by "compiling" a pattern into a combination of simple matchers that all have the same interface structure. This makes it easy to add new features and to make such a system very efficient. When we add segment variables, which match an unspecified number of elements, to such a matcher we find that we have to implement a backtracking system, because there may be multiple possible matches to any particular data if the pattern has more than one segment variable. This complicates matters significantly. Besides the intrinsic complexity of backtracking, the backtracking in the pattern matcher must be interfaced to the backtracking system in the rule executive that uses the patterns. We will examine more general ways of dealing with backtracking in section 5.4. We will investigate even more powerful backtracking strategies in section 7.5.2.

If we model partially specified data as patterns with holes (represented by pattern variables), then we find that we need to match patterns against each other to collect the constraints on the data so that we can sharpen the specification. We explored *unification*: the process of merging partial information structures of this kind. This is essentially a way of setting up and solving symbolic equations for the missing parts of the data. Unification is

very powerful, and we showed how to make a simple type-inference engine using unification in this way.

We found that the ideas of pattern matching can be extended to operate on general graphs rather than just hierarchical expressions. This made it easy to work with such complex graphs as chess boards, where we used patterns to specify legal chess moves. Patterns and pattern matching can be a way to express computational thought, and on some problems can be more revealing than other programming methods. But be careful: pattern matching is not the answer to all of the world's problems, so let's not become addicted to it.

- <span id="page-82-0"></span>[1](#page-3-0) Of course, a very clever matcher could deduce that y=0, under the assumption that we are dealing with numbers.
- <span id="page-82-1"></span>[2](#page-8-0) See section 5.4.2 on page 273 for more examples and explanation of this success/failure pattern.
- <span id="page-82-2"></span>[3](#page-14-0) This strategy for building pattern matchers was first described by Carl Hewitt in his PhD thesis [56].
- <span id="page-82-3"></span>[4](#page-30-0) The unifier is unique for patterns with only element variables. This is a theorem; we will not prove it here. In section 4.4.4 we will extend our unifier to include segment variables. However, when the patterns have segment variables, unification will generally yield multiple matches.
- <span id="page-82-4"></span>[5](#page-30-1) For an extensive survey of unification see [6].
- <span id="page-82-5"></span>[6](#page-33-0) In the guts of this unifier it is convenient for a failure to make an explicit call to a failure continuation. But in unify:internal we transition to a different convention for indicating a failure: returning #f from a success continuation. This is to make the convention for use of the unifier the same as the convention for use of the matcher of section 4.3.This is an interesting transition.

In the rule system in section 4.2.2 we used explicit success and failure continuations, so to use the matcher in the rule system we had to make the reverse transition: the matcher used the #f convention, so make-rule (on page 166) had to implement the transition.The choice of convention for implementing failure in a backtracking system is usually a matter of style, but the use of an explicit failure continuation is often easier to extend. Luckily, it is easy to interface these disparate ways of implementing backtracking.

- <span id="page-83-0"></span>[7](#page-33-1) As in the pattern matching system described in section 4.3, the unification matcher is organized around lists of terms to allow later extension to segment variables.
- <span id="page-83-1"></span>[8](#page-37-0) In the unification literature this is called the "occurs check." The occurs check is used to prevent trying to obtain a solution to an equation like *x* = *f*(*x*). Such a fixed-point equation may be solvable, in some cases, if we know more about the function *f*, but this unifier is a *syntactic* matcher. One could put in a hook at this point to ask for a more powerful equation solver to help, but we are not doing that. Most Prolog systems avoid implementing the occurs check for efficiency reasons.
- <span id="page-83-2"></span>[9](#page-38-0) Because unification is so important, there has been a great deal of work developing efficient algorithms. Memoization can be used to make large improvements. For an extensive exposition of unification algorithms see [6].
- <span id="page-83-3"></span>[10](#page-50-0) Others have added segment variables to pattern matchers or unifiers [5], with some success. Apparently there are versions of Prolog that have segment variables [34]. A detailed theoretical treatment of an algorithm that includes sequence variables (another name for segment variables) in a unifier can be found in Kutsia's PhD thesis [79]. However, here we are not trying to build a complete and correct segment unifier. We are just trying to show how easy it is to add some useful new behavior to the elementary unification procedure already built.

- <span id="page-84-0"></span>[11](#page-50-1) The extension to segments is very subtle. We thank Kenny Chen, Will Byrd, and Michael Ballantyne for helping us think about this experiment.
- <span id="page-84-1"></span>[12](#page-51-0) The procedure complement is a combinator for predicates: complement makes a new predicate that is the negation of its argument.
- <span id="page-84-2"></span>[13](#page-53-0) This is a one-sided match that could also be done with the earlier matcher, but this ability to match expressions with variables on both sides of the match is useful.
- <span id="page-84-3"></span>[14](#page-60-0) For an example of how a graph node is used, see g:cons on page 210. For a more complete description of bundle procedures see page 395.
- <span id="page-84-4"></span>[15](#page-64-0) Castling is another special case. Castling is allowed under restricted circumstances: when the king and the rook are in their initial positions, the squares between the king and rook are unoccupied, and the king is not in check and will not have to traverse or land in a square where it would be in check.
- <span id="page-84-5"></span>[16](#page-64-1) There is also a pawn move, en passant capture, that depends on the opponent's previous move.
- <span id="page-84-6"></span>[17](#page-66-0) We use zero-based indexing, unlike the traditional chess conventions, but this is not important except for input and output to players.
- <span id="page-84-7"></span>[18](#page-74-0) But notice that the success continuation of the graph-matcher procedure is different from the success continuation of the expression-matcher procedure. The expression-matcher success continuation takes a dictionary and a number of elements eaten by the matcher (to make segments work), whereas the graphmatcher success continuation takes the final node and the dictionary resulting from matching the matched part of the graph.

<span id="page-85-0"></span>[19](#page-75-0) Although the actual name of the procedure is gmatch:finishcompile-path, we abbreviate such names to elide the gmatch: prefix in text explanations.