44e26d6972
- CLAUDE.md 생성 (볼트 운영 규칙, Karpathy LLM Wiki 10가지 규칙) - 나의 핵심 맥락.md 생성 (아키텍트 프로필, 세컨드 브레인 목적, 핵심 소스) - raw/ 구조 정립 (book/기존 설계원칙 보존, articles/repos/notes/ 추가) - wiki/ 초기화 (index.md, log.md, concepts/sources/patterns/ 폴더) - output/ 초기화 - LLMWiki/ 기존 프롬프트 패턴 파일 보존 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
29 KiB
(extend-arithmetic
vector-extender
(extend-arithmetic function-extender combined-arithmetic)))
How does the ordering of extensions affect the properties of the resulting arithmetic? The following procedure makes points on the unit circle:
(define (unit-circle x)
(vector (sin x) (cos x)))
If we execute each of the following expressions in environments resulting from installing vec-before-func and func-before-vec:
((magnitude unit-circle) 'a)
((magnitude (vector sin cos)) 'a)
The result (unsimplified) should be:
(sqrt (+ (* (sin a) (sin a)) (* (cos a) (cos a))))
However, each of these expressions fails with one of the two orderings of the extensions. Is it possible to make an arithmetic for which both evaluate correctly? Explain.
3.2 Extensible generic procedures
Systems built by combinators, as in section 3.1, result in beautiful diamond-like systems. This is sometimes the right idea, and we will see it arise again, but it is very hard to add to a diamond. If a system is built as a ball of mud, it is easy to add more mud. 11
One organization for a ball of mud is a system erected on a substrate of extensible generic procedures. Modern dynamically typed programming languages, such as Lisp, Scheme, and Python, usually have built-in arithmetic that is generic over a variety of types of numerical quantities, such as integers, floats, rationals, and complex numbers [115, 64, 105]. But systems built on these languages are usually not easily extensible after the fact.
The problems we indicated in section 3.1.5 are the result of using the combinator add-arithmetics. To solve these problems we will abandon that combinator. However, the arithmetic package abstraction is still useful, as is the idea of an extender. We will build an arithmetic package in which the operations use generic procedures that can be dynamically augmented with new behavior. We can then extend the generic arithmetic and add the extensions to the generic arithmetic. 12
We will start by implementing generic procedures, which are procedures that can be dynamically extended by adding handlers after the generic procedures are defined. A generic procedure is a dispatcher combined with a set of rules, each of which describes a handler that is appropriate for a given set of arguments. Such a rule combines a handler with its applicability.
Let's examine how this might work, by defining a generic procedure named plus that works like addition with numeric and symbolic quantities:
(define plus (simple-generic-procedure 'plus 2 #f))
(define-generic-procedure-handler plus
(all-args 2 number?)
(lambda (a b) (+ a b)))
(define-generic-procedure-handler plus
(any-arg 2 symbolic? number?)
(lambda (a b) (list '+ a b)))
(plus 1 2)
3
(plus 1 'a)
(+ 1 a)
(plus 'a 2)
(+ a 2)
(plus 'a 'b)
(+ a b)
The procedure simple-generic-procedure takes three arguments: The first is an arbitrary name to identify the procedure when debugging; the second is the procedure's arity. The third argument is used to provide a default handler; if none is supplied (indicated by #f), then if no specific handler is applicable an error is signaled. Here plus is bound to the new generic procedure returned by simple-generic-procedure. It is a Scheme procedure that can be called with the specified number of arguments.
The procedure define-generic-procedure-handler adds a rule to an existing generic procedure. Its first argument is the generic procedure to be extended; the second argument is an applicability specification (as on page 73) for the rule being added; and the third argument is the handler for arguments that satisfy that specification.
(define-generic-procedure-handler generic-procedure
applicability
handler-procedure)
It is often necessary to specify a rule in which different arguments are of different types. For example, to make a vector arithmetic package we need to specify the interpretation of the * operator. If both arguments are vectors, the appropriate handler computes the dot product. If one argument is a scalar and the other is a vector, then the appropriate handler scales the vector elements by the scalar. The applicability argument is the means by which this is accomplished.
The simple-generic-procedure constructor we used above to make the generic procedure plus is created with the procedure generic-procedure-constructor
(define simple-generic-procedure
(generic-procedure-constructor make-simple-dispatch-store))
where make-simple-dispatch-store is a procedure that encapsulates a strategy for saving, retrieving, and choosing a handler.
The generic-procedure-constructor takes a dispatch-store constructor and produces a generic-procedure constructor that itself takes three arguments—a name that is useful in debugging, an arity, and a default handler to be used if there are no applicable handlers. If the default handler argument is #f, the default handler signals an error:
((generic-procedure-constructor dispatch-store-constructor)
name
arity
default-handler)
The reason why generic procedures are made in this way is that we will need families of generic procedures that differ in the choice of dispatch store.
In section 3.2.3, we will see one way to implement this mechanism. But first let's see how to use it.
3.2.1 Generic arithmetic
We can use this new generic-procedure mechanism to build arithmetic packages in which the operators map to operations that are implemented as generic procedures. This will allow us to make self-referential structures. For example, we might want to make a generic arithmetic that includes vector arithmetic where both the vectors and the components of a vector are manipulated by the same generic procedures. We could not build such a structure using just add-arithmetics introduced earlier.
(define (make-generic-arithmetic dispatch-store-maker)
(make-arithmetic 'generic any-object? '()
constant-union
(let ((make-generic-procedure
(generic-procedure-constructor
dispatch-store-maker)))
(lambda (operator)
(simple-operation operator
any-object?
(make-generic-procedure
operator
(operator-arity operator)
#f))))))
The make-generic-arithmetic procedure creates a new arithmetic. For each arithmetic operator, it constructs an operation that is applicable to any arguments and is implemented by a generic procedure. (The predicate any-object? is true of anything.) We can install this arithmetic in the usual way.
But first, let's define some handlers for the generic procedures. It's pretty simple to do now that we have the generic arithmetic object. For example, we can grab the operations and constants from any already-constructed arithmetic.
(define (add-to-generic-arithmetic! generic-arithmetic
arithmetic)
(add-generic-arith-constants! generic-arithmetic
arithmetic)
(add-generic-arith-operations! generic-arithmetic
arithmetic))
This takes a generic arithmetic package and an ordinary arithmetic package with the same operators. It merges constants into the generic arithmetic using constant-union. And for each operator of the given arithmetic it adds a handler to the corresponding generic procedure.
Adding a handler for a particular operator uses the standard generic procedure mechanism, extracting the necessary applicability and procedure from the arithmetic's operation.
(define (add-generic-arith-operations! generic-arithmetic
arithmetic)
(for-each
(lambda (operator)
(let ((generic-procedure
(simple-operation-procedure
(arithmetic-operation operator
generic-arithmetic)))
(operation
(arithmetic-operation operator arithmetic)))
(define-generic-procedure-handler
generic-procedure
(operation-applicability operation)
(operation-procedure operation))))
(arithmetic-operators arithmetic)))
The add-generic-arith-operations! procedure finds, for each operator in the given arithmetic, the generic procedure that must be augmented. It then defines a handler for that generic procedure that is the handler for that operator in the given arithmetic, using the applicability for that handler in the given arithmetic.
The code for adding the constants from an arithmetic to the generic arithmetic is similar. For each constant name in the generic arithmetic it finds the entry in the association of names to constant values in the generic arithmetic. It then replaces the constant value with the constant-union of the existing constant and the constant it got for that same name from the given arithmetic.
(define (add-generic-arith-constants! generic-arithmetic
arithmetic)
(for-each
(lambda (name)
(let ((binding
(arithmetic-constant-binding name
generic-arithmetic))
(element
(find-arithmetic-constant name arithmetic)))
(set-cdr! binding
(constant-union name
(cdr binding)
element))))
(arithmetic-constant-names generic-arithmetic)))
Fun with generic arithmetics
We can add many arithmetics to a generic arithmetic to give it interesting behavior:
(let ((g
(make-generic-arithmetic make-simple-dispatch-store)))
(add-to-generic-arithmetic! g numeric-arithmetic)
(add-to-generic-arithmetic! g
(function-extender numeric-arithmetic))
(add-to-generic-arithmetic! g
(symbolic-extender numeric-arithmetic))
(install-arithmetic! g))
This produces a generic arithmetic that combines numeric arithmetic with symbolic arithmetic over numeric arithmetic and function arithmetic over numeric arithmetic:
(+ 1 3 'a 'b)
(+ (+ 4 a) b)
And we can even run some more complex problems, as on page 79:
(pp (x 0 ((evolver F 'h stormer-2) numeric-s0 1)))
(+ 9.999833334166664e-3
(* (/ (expt h 2) 12)
-9.999750002487318e-7))
As before, we can mix symbols and functions:
(* 'b ((+ cos sin) 3))
(* b -.8488724885405782)
but the following will signal an error, trying to add the symbolic quantities (cos a) and (sin a) as numbers:
(* 'b ((+ cos sin) 'a))
We get this error because cos and sin are numeric operators, like +. Since we have symbolic arithmetic over numeric arithmetic, these operators are extended so that for symbolic input, here a, they produce symbolic outputs, (cos a) and (sin a). We also added function arithmetic over numeric arithmetic, so if functions are numerically combined (here by +) their outputs may be combined only if the outputs are numbers. But the symbolic results cannot be added numerically. This is a consequence of the way we built the arithmetic g.
But there is magic in generic arithmetic. It can be closed: all extensions to the generic arithmetic can be made over the generic arithmetic!
(let ((g
(make-generic-arithmetic make-simple-dispatch-store)))
(add-to-generic-arithmetic! g numeric-arithmetic)
(extend-generic-arithmetic! g symbolic-extender)
(extend-generic-arithmetic! g function-extender)
(install-arithmetic! g))
Here we use a new procedure extend-generic-arithmetic! that captures a common pattern.
(define (extend-generic-arithmetic! generic-arithmetic
extender)
(add-to-generic-arithmetic! generic-arithmetic
(extender generic-arithmetic)))
Now we can use complex mixed expressions, because the functions are defined over the generic arithmetic:
(* 'b ((+ 'c cos sin) (+ 3 'a)))
(* b (+ (+ c (cos (+ 3 a))) (sin (+ 3 a))))
We can even use functions that return functions:
(((+ (lambda (x) (lambda (y) (cons x y)))
(lambda (x) (lambda (y) (cons y x))))
3)
4)
(+ (3 . 4) (4 . 3))
So perhaps we have achieved nirvana?
3.2.2 Construction depends on order!
Unfortunately, there is a severe dependence on the order in which rules are added to the generic procedures. This is not surprising, because the construction of the generic procedure system is by assignment. We can see this by changing the order of construction:
(let ((g
(make-generic-arithmetic make-simple-dispatch-store)))
(add-to-generic-arithmetic! g numeric-arithmetic)
(extend-generic-arithmetic! g function-extender) ;*
(extend-generic-arithmetic! g symbolic-extender) ;*
(install-arithmetic! g))
and then we will find that the example
(* 'b ((+ 'c cos sin) (+ 3 'a)))
which worked in the previous arithmetic, fails because the symbolic arithmetic captures (+ 'c cos sin) to produce a symbolic expression, which is not a function that can be applied to (+ 3 a). The problem is that the applicability of the symbolic operation for + accepts arguments with at least one symbolic argument and other arguments from the domain predicate of the base. But the symbolic arithmetic was created over the generic arithmetic as a base, and the domain predicate of a generic arithmetic accepts anything! There is also a function operation for + that is applicable to the same arguments, but it has not been chosen because of the accidental ordering of the extensions. Unfortunately, the choice of rule is ambiguous. It would be better to not have more than one applicable operation.
One way to resolve this problem is to restrict the symbolic quantities to represent numbers. We can do this by building our generic arithmetic so that the symbolic arithmetic is over the numeric arithmetic, as we did on page 92, rather than over the entire generic arithmetic:
(let ((g
(make-generic-arithmetic make-simple-dispatch-store)))
(add-to-generic-arithmetic! g numeric-arithmetic)
(extend-generic-arithmetic! g function-extender)
(add-to-generic-arithmetic! g
(symbolic-extender numeric-arithmetic))
(install-arithmetic! g))
This works, independent of the ordering, because there is no ambiguity in the choice of rules. So now the 'c will be interpreted as a constant to be coerced to a constant function by the function extender.
(* 'b ((+ 'c cos sin) (+ 3 'a)))
(* b (+ (+ c (cos (+ 3 a))) (sin (+ 3 a))))
Unfortunately, we may want to have symbolic expressions over other quantities besides numbers. We cannot yet implement a general solution to this problem. But if we really want a literal function named c, we can use literal-function as we did earlier:
(* 'b ((+ (literal-function 'c) cos sin) (+ 3 'a)))
(* b (+ (+ (c (+ 3 a)) (cos (+ 3 a))) (sin (+ 3 a))))
This will work independent of the order of construction of the generic arithmetic.
With this mechanism we are now in a position to evaluate the Stormer integrator with a literal function:
(pp (x 0 ((evolver (literal-function 'F) 'h stormer-2)
(make-initial-history 't 'h 'xt 'xt-h 'xt-2h)
1))
(+ (+ (* 2 xt) (* -1 xt-h))
(* (/ (expt h 2) 12)
(+ (+ (* 13 (f t xt))
(* -2 (f (- t h) xt-h)))
(f (- t (* 2 h)) xt-2h))))
This is pretty ugly, and it would be worse if we looked at the output of two integration steps. But it is interesting to look at the result of simplifying a two-step integration. Using a magic symbolicexpression simplifier we get a pretty readable expression. This can be useful for debugging a numerical process.
(+ (* 2 (expt h 2) (f t xt))
(* -1/4 (expt h 2) (f (+ (* -1 h) t) xt-h))
(* 1/6 (expt h 2) (f (+ (* -2 h) t) xt-2h))
(* 13/12
(expt h 2)
(f (+ h t)
(+ (* 13/12 (expt h 2) (f t xt))
(* -1/6 (expt h 2) (f (+ (* -1 h) t) xt-h))
(* 1/12 (expt h 2) (f (+ (* -2 h) t) xt-2h))
(* 2 xt)
(* -1 xt-h))))
(* 3 xt)
(* -2 xt-h))
For example, notice that there are only four distinct top-level calls to the acceleration function f. The second argument to the fourth top-level call uses three calls to f that have already been computed. If we eliminate common subexpressions we get:
(let* ((G84 (expt h 2)) (G85 (f t xt)) (G87 (* -1 h))
(G88 (+ G87 t)) (G89 (f G88 xt-h)) (G91 (* -2 h))
(G92 (+ G91 t)) (G93 (f G92 xt-2h)))
(+ (* 2 G84 G85)
(* -1/4 G84 G89)
(* 1/6 G84 G93)
(* 13/12 G84
(f (+ h t)
(+ (* 13/12 G84 G85)
(* -1/6 G84 G89)
(* 1/12 G84 G93)
(* 2 xt)
(* -1 xt-h))))
(* 3 xt)
(* -2 xt-h)))
Here we clearly see that there are only four distinct calls to f. Though each integration step in the basic integrator makes three calls to f, the two steps overlap on two intermediate calls. While this is obvious for such a simple example, we see how symbolic evaluation might help in understanding a numerical computation.
3.2.3 Implementing generic procedures
We have used generic procedures to do amazing things. But how do we make such a thing work?
Making constructors for generic procedures
On page 89 we made a simple generic procedure constructor:
(define simple-generic-procedure
(generic-procedure-constructor make-simple-dispatch-store))
The procedure generic-procedure-constructor is given a "dispatch strategy" procedure; it returns a generic-procedure constructor that takes a name, an arity, and a default-handler specification. When this procedure is called with these three arguments it returns a generic procedure that it associates with a newly constructed metadata store for that procedure, which holds the name, the arity, an instance of the dispatch strategy, and the
default handler, if any. The dispatch-strategy instance will maintain the handlers, their applicabilities, and the mechanism for deciding which handler to choose for given arguments to the generic procedure.
The code that implements generic-procedure-constructor is:
(define (generic-procedure-constructor dispatch-store-maker)
(lambda (name arity default-handler)
(let ((metadata
(make-generic-metadata
name arity (dispatch-store-maker)
(or default-handler
(error-generic-procedure-handler name)))))
(define (the-generic-procedure . args)
(generic-procedure-dispatch metadata args))
(set-generic-procedure-metadata! the-generic-procedure
metadata)
the-generic-procedure)))
This implementation uses the-generic-procedure, an ordinary Scheme procedure, to represent the generic procedure, and a metadata store (for rules, etc.) that determines the procedure's behavior. This store is associated with the generic procedure using a "sticky note" (as on page 28) and can later be obtained by calling generic-procedure-metadata. This allows procedures such as define-generic-procedure-handler to modify the metadata of a given generic procedure.
The argument to generic-procedure-constructor is a procedure that creates a dispatch store for saving and retrieving handlers. The dispatch store encapsulates the strategy for choosing a handler.
Here is the simple dispatch-store constructor we have used so far. The dispatch store is implemented as a message-accepting procedure:
(define (make-simple-dispatch-store)
(let ((rules '()) (default-handler #f))
(define (get-handler args)
;; body will be shown in text below.
...)
(define (add-handler! applicability handler)
;; body will be shown in text below.
...)
(define (get-default-handler) default-handler)
(define (set-default-handler! handler)
(set! default-handler handler))
(lambda (message) ; the simple dispatch store
(case message
((get-handler) get-handler)
((add-handler!) add-handler!)
((get-default-handler) get-default-handler)
((set-default-handler!) set-default-handler!)
((get-rules) (lambda () rules))
(else (error "Unknown message:" message))))))
The simple dispatch store just maintains a list of the rules, each of which pairs an applicability with a handler. When the gethandler internal procedure is called with arguments for the generic procedure, it scans the list sequentially for a handler whose applicability is satisfied by the arguments tendered; it returns the handler, or #f if it doesn't find one:
(define (get-handler args)
(let ((rule
(find (lambda (rule)
(predicates-match? (car rule) args))
rules)))
(and rule (cdr rule))))
There are many possible strategies for choosing handlers to run. The above code returns the first applicable handler in the list. Another strategy is to return all applicable handlers. If more than one handler is applicable, perhaps all should be tried (in parallel?) and the results compared! Passing a dispatch-store constructor as an argument to generic-procedure-constructor allows the strategy to be chosen when the generic-procedure constructor is created, rather than being hard-coded into the implementation.
Adding handlers to generic procedures
The handler definition procedure (see below) adds new rules by calling the internal procedure add-handler of the dispatch store.
For make-simple-dispatch-store above, add-handler adds the new rule to the front of the list of rules. (But if there was already a rule for handling that applicability, it just replaces the handler.)
(define (add-handler! applicability handler)
(for-each (lambda (predicates)
(let ((p (assoc predicates rules)))
(if p
(set-cdr! p handler)
(set! rules
(cons (cons predicates handler)
rules)))))
applicability))
The define-generic-procedure-handler procedure uses the metadata table to get the metadata record for the generic procedure. It asks the dispatch store for the add-handler! procedure and uses that procedure to add a rule to the metadata that associates the applicability with the handler. The dispatch-store instance is retrieved from the metadata of the generic procedure by genericmetadata-dispatch-store.
(define (define-generic-procedure-handler generic-procedure
applicability
handler)
(((generic-metadata-dispatch-store
(generic-procedure-metadata generic-procedure))
'add-handler!)
applicability
handler))
Finally, the heart of the mechanism is the dispatch, called by a generic procedure (the-generic-procedure on page 97), which finds an appropriate handler and applies it. The default handler, as supplied during construction of the generic procedure, is called if there is no applicable handler. 13
(define (generic-procedure-dispatch metadata args)
(let ((handler
(get-generic-procedure-handler metadata args)))
(apply handler args)))
(define (get-generic-procedure-handler metadata args)
(or ((generic-metadata-getter metadata) args)
((generic-metadata-default-getter metadata))))
The power of extensible generics
Construction of a system on a substrate of extensible generic procedures is a powerful idea. In our example it is possible to define what is meant by addition, multiplication, etc., for new data types unimagined by the language designer. For example, if the arithmetic operators of a system are implemented as extensible generics, a user may extend them to support arithmetic on quaternions, vectors, matrices, integers modulo a prime, functions, tensors, differential forms, This is not just making new capabilities possible; it also extends old programs, so a program that was written to manipulate simple numerical quantities may become useful for manipulating scalar-valued functions.
We have seen that there are potential problems associated with this use of extensible generic procedures. On the other hand, some "mutations" will be extremely valuable. For example, it is possible to extend arithmetic to symbolic quantities. The simplest way to do this is to make a generic extension to all of the operators to take symbolic quantities as arguments and return a data structure representing the indicated operation on the arguments. With the addition of a simplifier of algebraic expressions we suddenly have a symbolic manipulator. This is useful in debugging purely numerical calculations, because if we give them symbolic arguments we can examine the resulting symbolic expressions to make sure that the program is calculating what we intend it to. It is also the basis of a partial evaluator for optimization of numerical programs. And functional differentiation can be viewed as a generic extension of arithmetic to a compound data type (see section 3.3). The scmutils system we use to teach classical mechanics [121] implements differentiation in exactly this way.
Exercise 3.4: Functional values
The generic arithmetic structure allows us to close the system so that functions that return functions can work, as in the example
(((* 3
(lambda (x) (lambda (y) (+ x y)))
(lambda (x) (lambda (y) (vector y x))))
'a)
4)
(* (* 3 (+ a 4)) #(4 a))
- a. How hard is it to arrange for this to work in the purely combinator-based arithmetic introduced in section 3.1? Why?
- b. Exercise 3.3 on page 86 asked about the implications of ordering of vector and functional extensions. Is the generic system able to support both expressions discussed there (and copied below)? Explain.
((magnitude unit-circle) 'a)
((magnitude (vector sin cos)) 'a)
c. Is there any good way to make the following work at all?
((vector cos sin) 3)
#(-.9899924966004454 .1411200080598672)
Show code that makes this work or explain the difficulties.
Exercise 3.5: A weird bug
Consider the +-like ("plus-like") procedure in arith.scm, shown below, which implements n-ary procedures + and * as part of installing an arithmetic. It returns a pair of a name and a procedure; the installer will bind the name to the procedure.
It seems that it is written to execute the get-identity procedure that computes the identity every time the operation is
called with no arguments.
(define (+-like operator identity-name)
(lambda (arithmetic)
(let ((binary-operation
(find-arithmetic-operation operator arithmetic)))
(and binary-operation
(let ((binary
(operation-procedure binary-operation))
(get-identity
(identity-name->getter identity-name
arithmetic)))
(cons operator
(lambda args
(case (length args)
((0) (get-identity))
((1) (car args))
(else (pairwise binary args))))))))))
Perhaps the identity for an operator should be computed only once, not every time the handler is called. As a consequence, it is proposed that the code should be modified as follows:
(define (+-like operator identity-name)
(lambda (arithmetic)
(let ((binary-operation
(find-arithmetic-operation operator arithmetic)))
(and binary-operation
(let ((binary
(operation-procedure binary-operation))
(identity
((identity-name->getter identity-name
arithmetic))))
(cons operator
(lambda args
(case (length args)
((0) identity)
((1) (car args))
(else (pairwise binary args))))))))))
However, this has a subtle bug! Can you elicit the bug? Can you explain it?
Exercise 3.6: Matrices
Matrices are ubiquitous in scientific and technical computing.
a. Make and install an arithmetic package for matrices of numbers, with operations +, -, negate, and *. This arithmetic needs to be able to know the number of rows and the number of columns in a matrix, since matrix multiplication is defined only if the number of columns in the first matrix is equal to the number of rows in the second one.
Make sure that your multiplier can multiply a matrix with a scalar or with a vector. For matrices to play well with vectors you probably need to distinguish row vectors and column vectors. How does this affect the design of the vector package? (See exercise 3.2 on page 85.)
You may assume that the vectors and matrices are of small dimension, so you do not need to deal with sparse representations. A reasonable representation of a matrix is a Scheme vector in which each element is a Scheme vector representing a row.
- b. Vectors and matrices may contain symbolic numerical quantities. Make this work.
- c. Matrix inversion is appropriate for your arithmetic. If a symbolic matrix is dense, the inverse may take space that is factorial in the dimension. Why?
Note: We are not asking you to implement matrix inversion.
Exercise 3.7: Literal vectors and matrices
It is also possible to have arithmetic on literal matrices and literal vectors with an algebra of symbolic expressions of vectors and matrices. Can you make symbolic algebra of these compound