www

Unnamed repository; edit this file 'description' to name the repository.
Log | Files | Refs | README

index.rkt (53006B)


      1 #lang scribble/manual
      2 
      3 @(require racket/sandbox
      4           scribble/eval
      5           scribble/racket
      6           racket/date
      7           (for-syntax racket/base)
      8           (for-label racket)
      9           (for-label racket/stxparam)
     10           (for-label syntax/parse)
     11           (for-label racket/splicing)
     12           (for-label racket/syntax))
     13 @(define evaluator
     14    (parameterize ([sandbox-output 'string]
     15                   [sandbox-error-output 'string])
     16      (make-evaluator 'racket)))
     17 
     18 @(define typed/evaluator
     19    (parameterize ([sandbox-output 'string]
     20                   [sandbox-error-output 'string])
     21      (make-evaluator 'typed/racket)))
     22 
     23 @(define-syntax-rule (i body ...)
     24    (interaction #:eval evaluator body ...))
     25 
     26 @title[#:version ""]{Fear of Macros}
     27 @author[@hyperlink["http://www.greghendershott.com"
     28                    "Greg Hendershott"]]
     29 @image["fear-of-macros.jpg"]
     30 @para[@smaller{Copyright (c) 2012-2014 by Greg Hendershott. All rights reserved.}]
     31 @para[@smaller["Last updated "
     32                (parameterize ([date-display-format 'iso-8601])
     33                  (date->string (current-date) #t))]]
     34 @para{Feedback and corrections are @hyperlink["https://github.com/greghendershott/fear-of-macros/issues" "welcome here"].}
     35 
     36 Contents:
     37 
     38 @table-of-contents{}
     39 
     40 @; ----------------------------------------------------------------------------
     41 
     42 @section{Preface}
     43 
     44 I learned Racket after 25 years of mostly using C and C++.
     45 
     46 Some psychic whiplash resulted.
     47 
     48 "All the parentheses" was actually not a big deal. Instead, the first
     49 mind warp was functional programming. Before long I wrapped my brain
     50 around it, and went on to become comfortable and effective with many
     51 other aspects and features of Racket.
     52 
     53 But two final frontiers remained: Macros and continuations.
     54 
     55 I found that simple macros were easy and understandable, plus there
     56 were many good tutorials available. But the moment I stepped past
     57 routine pattern-matching, I kind of fell off a cliff into a
     58 terminology soup. I marinaded myself in material, hoping it would
     59 eventually sink in after enough re-readings. I even found myself using
     60 trial and error, rather than having a clear mental model what was
     61 going on. Gah.
     62 
     63 I'm starting to write this at the point where the shapes are slowly
     64 emerging from the fog.
     65 
     66 @margin-note{If you have any corrections, criticisms, complaints, or whatever,
     67 @hyperlink["https://github.com/greghendershott/fear-of-macros/issues" "please
     68 let me know"].}
     69 
     70 My primary motive is selfish. Explaining something forces me to learn
     71 it more thoroughly. Plus if I write something with mistakes, other
     72 people will be eager to point them out and correct me. Is that a
     73 social-engineering variation of meta-programming? Next question,
     74 please. :)
     75 
     76 Finally I do hope it may help other people who have a similar
     77 background and/or learning style as me.
     78 
     79 I want to show how Racket macro features have evolved as solutions to
     80 problems or annoyances. I learn more quickly and deeply when I
     81 discover the answer to a question I already have, or find the solution
     82 to a problem whose pain I already feel. Therefore I'll give you the
     83 questions and problems first, so that you can better appreciate and
     84 understand the answers and solutions.
     85 
     86 @; ----------------------------------------------------------------------------
     87 
     88 @section{Our plan of attack}
     89 
     90 The macro system you will mostly want to use for production-quality
     91 macros is called @racket[syntax-parse]. And don't worry, we'll get to
     92 that soon.
     93 
     94 But if we start there, you're likely to feel overwhelmed by concepts
     95 and terminology, and get very confused. I did.
     96 
     97 1. Instead let's start with the basics: A syntax object and a function
     98 to change it---a "transformer". We'll work at that level for awhile to
     99 get comfortable and to de-mythologize this whole macro business.
    100 
    101 2. Soon we'll realize that pattern-matching would make life
    102 easier. We'll learn about @racket[syntax-case] and its shorthand
    103 cousin, @racket[define-syntax-rule]. We'll discover we can get
    104 confused if we want to munge pattern variables before sticking them
    105 back in the template, and learn how to do that.
    106 
    107 3. At this point we'll be able to write many useful macros. But, what
    108 if we want to write the ever-popular anaphoric if, with a "magic
    109 variable"?  It turns out we've been protected from making certain kind
    110 of mistakes. When we want to do this kind of thing on purpose, we use
    111 a syntax parameter. [There are other, older ways to do this. We won't
    112 look at them. We also won't spend a lot of time
    113 advocating "hygiene"---we'll just stipulate that it's good.]
    114 
    115 4. Finally, we'll realize that our macros could be smarter when
    116 they're used in error. Normal Racket functions optionally can have
    117 contracts and types. These catch usage mistakes and provide clear,
    118 useful error messages. It would be great if there were something
    119 similar for macro. There is. One of the more-recent Racket macro
    120 enhancements is @racket[syntax-parse].
    121 
    122 
    123 @; ----------------------------------------------------------------------------
    124 @; ----------------------------------------------------------------------------
    125 
    126 @section{Transform!}
    127 
    128 @verbatim[#:indent 2]{
    129 YOU ARE INSIDE A ROOM.
    130 THERE ARE KEYS ON THE GROUND.
    131 THERE IS A SHINY BRASS LAMP NEARBY.
    132 
    133 IF YOU GO THE WRONG WAY, YOU WILL BECOME
    134 HOPELESSLY LOST AND CONFUSED.
    135 
    136 > pick up the keys
    137 
    138 YOU HAVE A SYNTAX TRANSFORMER
    139 }
    140 
    141 
    142 @subsection{What is a syntax transformer?}
    143 
    144 A syntax transformer is not one of the トランスフォーマ
    145 @hyperlink["http://en.wikipedia.org/wiki/Transformers" "transformers"].
    146 
    147 Instead, it is simply a function. The function takes syntax and
    148 returns syntax. It transforms syntax.
    149 
    150 Here's a transformer function that ignores its input syntax, and
    151 always outputs syntax for a string literal:
    152 
    153 @(let-syntax([syntax (make-element-id-transformer
    154                       (lambda (stx)
    155                         #'@racket[syntax]))]) ;print as syntax not #'
    156 @i[
    157 (define-syntax foo
    158   (lambda (stx)
    159     (syntax "I am foo")))
    160 ]
    161 )
    162 
    163 Using it:
    164 
    165 @i[
    166 (foo)
    167 ]
    168 
    169 When we use @racket[define-syntax], we're making a transformer
    170 @italic{binding}. This tells the Racket compiler, "Whenever you
    171 encounter a chunk of syntax starting with @racket[foo], please give it
    172 to my transformer function, and replace it with the syntax I give back
    173 to you." So Racket will give anything that looks like @racket[(foo
    174 ...)] to our function, and we can return new syntax to use
    175 instead. Much like a search-and-replace.
    176 
    177 Maybe you know that the usual way to define a function in Racket:
    178 
    179 @racketblock[(define (f x) ...)]
    180 
    181 is shorthand for:
    182 
    183 @racketblock[(define f (lambda (x) ...))]
    184 
    185 That shorthand lets you avoid typing @racket[lambda] and some parentheses.
    186 
    187 Well there is a similar shorthand for @racket[define-syntax]:
    188 
    189 @(let-syntax([syntax (make-element-id-transformer
    190                       (lambda (stx)
    191                         #'@racket[syntax]))]) ;print as syntax not #'
    192 @i[
    193 (define-syntax (also-foo stx)
    194   (syntax "I am also foo"))
    195 (also-foo)
    196 ]
    197 )
    198 
    199 What we want to remember is that this is simply shorthand. We are
    200 still defining a transformer function, which takes syntax and returns
    201 syntax. Everything we do with macros, will be built on top of this
    202 basic idea. It's not magic.
    203 
    204 Speaking of shorthand, there is also a shorthand for @racket[syntax],
    205 which is @tt{#'}:
    206 
    207 @margin-note{@tt{#'} is short for  @racket[syntax] much like
    208 @tt{'} is short for @racket[quote].}
    209 
    210 @i[
    211 (define-syntax (quoted-foo stx)
    212   #'"I am also foo, using #' instead of syntax")
    213 (quoted-foo)
    214 ]
    215 
    216 We'll use the @tt{#'} shorthand from now on.
    217 
    218 Of course, we can emit syntax that is more interesting than a
    219 string literal. How about returning @racket[(displayln "hi")]?
    220 
    221 @i[
    222 (define-syntax (say-hi stx)
    223   #'(displayln "hi"))
    224 (say-hi)
    225 ]
    226 
    227 When Racket expands our program, it sees the occurrence of
    228 @racket[(say-hi)], and sees it has a transformer function for that. It
    229 calls our function with the old syntax, and we return the new syntax,
    230 which is used to evaluate and run our program.
    231 
    232 @; ----------------------------------------------------------------------------
    233 
    234 @subsection{What's the input?}
    235 
    236 Our examples so far have ignored the input syntax and output some
    237 fixed syntax. But typically we will want to transform the input syntax
    238 into something else.
    239 
    240 Let's start by looking closely at what the input actually @italic{is}:
    241 
    242 @i[
    243 (define-syntax (show-me stx)
    244   (print stx)
    245   #'(void))
    246 (show-me '(+ 1 2))
    247 ]
    248 
    249 The @racket[(print stx)] shows what our transformer is given: a syntax
    250 object.
    251 
    252 A syntax object consists of several things. The first part is the
    253 S-expression representing the code, such as @racket['(+ 1 2)].
    254 
    255 Racket syntax is also decorated with some interesting information such
    256 as the source file, line number, and column. Finally, it has
    257 information about lexical scoping (which you don't need to worry about
    258 now, but will turn out to be important later.)
    259 
    260 There are a variety of functions available to access a syntax object.
    261 Let's define a piece of syntax:
    262 
    263 @i[
    264 (define stx #'(if x (list "true") #f))
    265 stx
    266 ]
    267 
    268 Now let's use functions that access the syntax object. The source
    269 information functions are:
    270 
    271 @margin-note{@racket[(syntax-source stx)] is returning @racket['eval],
    272 only because of how I'm generating this documentation, using an
    273 evaluator to run code snippets in Scribble. Normally this would be
    274 something like "my-file.rkt".}
    275 
    276 @i[
    277 (syntax-source stx)
    278 (syntax-line stx)
    279 (syntax-column stx)
    280 ]
    281 
    282 More interesting is the syntax "stuff" itself. @racket[syntax->datum]
    283 converts it completely into an S-expression:
    284 
    285 @i[
    286 (syntax->datum stx)
    287 ]
    288 
    289 Whereas @racket[syntax-e] only goes "one level down". It may return a
    290 list that has syntax objects:
    291 
    292 @i[
    293 (syntax-e stx)
    294 ]
    295 
    296 Each of those syntax objects could be converted by @racket[syntax-e],
    297 and so on recursively---which is what @racket[syntax->datum] does.
    298 
    299 In most cases, @racket[syntax->list] gives the same result as
    300 @racket[syntax-e]:
    301 
    302 @i[
    303 (syntax->list stx)
    304 ]
    305 
    306 (When would @racket[syntax-e] and @racket[syntax->list] differ? Let's
    307 not get side-tracked now.)
    308 
    309 When we want to transform syntax, we'll generally take the pieces we
    310 were given, maybe rearrange their order, perhaps change some of the
    311 pieces, and often introduce brand-new pieces.
    312 
    313 
    314 @; ----------------------------------------------------------------------------
    315 
    316 @subsection{Actually transforming the input}
    317 
    318 Let's write a transformer function that reverses the syntax it was
    319 given:
    320 
    321 @margin-note{The @racket[values] at the end of the example allows the
    322 result to evaluate nicely.  Try
    323 @racket[(reverse-me "backwards" "am" "i")] to see why it's handy.}
    324 @i[
    325 (define-syntax (reverse-me stx)
    326   (datum->syntax stx (reverse (cdr (syntax->datum stx)))))
    327 (reverse-me "backwards" "am" "i" values)
    328 ]
    329 
    330 Understand Yoda, can we. Great, but how does this work?
    331 
    332 First we take the input syntax, and give it to
    333 @racket[syntax->datum]. This converts the syntax into a plain old
    334 list:
    335 
    336 @i[
    337 (syntax->datum #'(reverse-me "backwards" "am" "i" values))
    338 ]
    339 
    340 Using @racket[cdr] slices off the first item of the list,
    341 @racket[reverse-me], leaving the remainder:
    342 @racket[("backwards" "am" "i" values)]. Passing that to
    343 @racket[reverse] changes it to @racket[(values "i" "am" "backwards")]:
    344 
    345 @i[
    346 (reverse (cdr '(reverse-me "backwards" "am" "i" values)))
    347 ]
    348 
    349 Finally we use @racket[datum->syntax] to convert this back to
    350 @racket[syntax]:
    351 
    352 @i[
    353 (datum->syntax #f '(values "i" "am" "backwards"))
    354 ]
    355 
    356 That's what our transformer function gives back to the Racket
    357 compiler, and @italic{that} syntax is evaluated:
    358 
    359 @i[
    360 (values "i" "am" "backwards")
    361 ]
    362 
    363 @; ----------------------------------------------------------------------------
    364 
    365 @subsection{Compile time vs. run time}
    366 
    367 @codeblock0{
    368 (define-syntax (foo stx)
    369   (make-pipe) ;Ce n'est pas le temps d'exécution
    370   #'(void))
    371 }
    372 
    373 Normal Racket code runs at ... run time. Duh.
    374 
    375 @margin-note{Instead of "compile time vs. run time", you may hear it
    376 described as "syntax phase vs. runtime phase". Same difference.}
    377 
    378 But a syntax transformer is called by Racket as part of the process of
    379 parsing, expanding, and compiling our program. In other words, our
    380 syntax transformer function is evaluated at compile time.
    381 
    382 This aspect of macros lets you do things that simply aren't possible
    383 in normal code. One of the classic examples is something like the
    384 Racket form, @racket[if]:
    385 
    386 @racket[(if <condition> <true-expression> <false-expression>)]
    387 
    388 If we implemented @racket[if] as a function, all of the arguments
    389 would be evaluated before being provided to the function.
    390 
    391 @i[
    392 (define (our-if condition true-expr false-expr)
    393   (cond [condition true-expr]
    394         [else false-expr]))
    395 (our-if #t
    396         "true"
    397         "false")
    398 ]
    399 
    400 That seems to work. However, how about this:
    401 
    402 @i[
    403 (define (display-and-return x)
    404   (displayln x)
    405   x)
    406 (our-if #t
    407         (display-and-return "true")
    408         (display-and-return "false"))
    409 ]
    410 
    411 @margin-note{One answer is that functional programming is good, and
    412 side-effects are bad. But avoiding side-effects isn't always
    413 practical.}
    414 
    415 Oops. Because the expressions have a side-effect, it's obvious that
    416 they are both evaluated. And that could be a problem---what if the
    417 side-effect includes deleting a file on disk?  You wouldn't want
    418 @racket[(if user-wants-file-deleted? (delete-file) (void))] to delete
    419 a file even when @racket[user-wants-file-deleted?] is @racket[#f].
    420 
    421 So this simply can't work as a plain function. However a syntax
    422 transformer can rearrange the syntax -- rewrite the code -- at compile
    423 time.  The pieces of syntax are moved around, but they aren't actually
    424 evaluated until run time.
    425 
    426 Here is one way to do this:
    427 
    428 @i[
    429 (define-syntax (our-if-v2 stx)
    430   (define xs (syntax->list stx))
    431   (datum->syntax stx `(cond [,(cadr xs) ,(caddr xs)]
    432                             [else ,(cadddr xs)])))
    433 (our-if-v2 #t
    434            (display-and-return "true")
    435            (display-and-return "false"))
    436 (our-if-v2 #f
    437            (display-and-return "true")
    438            (display-and-return "false"))
    439 ]
    440 
    441 That gave the right answer. But how?  Let's pull out the transformer
    442 function itself, and see what it did. We start with an example of some
    443 input syntax:
    444 
    445 @i[
    446 (define stx (syntax (our-if-v2 #t "true" "false")))
    447 (displayln stx)
    448 ]
    449 
    450 1. We take the original syntax, and use @racket[syntax->list] to
    451 change it into a @racket[list] of syntax objects:
    452 
    453 @i[
    454 (define xs (syntax->list stx))
    455 (displayln xs)
    456 ]
    457 
    458 2. To change this into a Racket @racket[cond] form, we need to take
    459 the three interesting pieces---the condition, true-expression, and
    460 false-expression---from the list using @racket[cadr], @racket[caddr],
    461 and @racket[cadddr] and arrange them into a @racket[cond] form:
    462 
    463 @racketblock[
    464 `(cond [,(cadr xs) ,(caddr xs)]
    465        [else ,(cadddr xs)])
    466 ]
    467 
    468 3. Finally, we change that into @racket[syntax] using
    469 @racket[datum->syntax]:
    470 
    471 @i[
    472 (datum->syntax stx `(cond [,(cadr xs) ,(caddr xs)]
    473                           [else ,(cadddr xs)]))
    474 ]
    475 
    476 So that works, but using @racket[cadddr] etc. to destructure a list is
    477 painful and error-prone. Maybe you know Racket's @racket[match]?
    478 Using that would let us do pattern-matching.
    479 
    480 @margin-note{Notice that we don't care about the first item in the
    481 syntax list. We didn't take @racket[(car xs)] in our-if-v2, and we
    482 didn't use @racket[name] when we used pattern-matching. In general, a
    483 syntax transformer won't care about that, because it is the name of
    484 the transformer binding. In other words, a macro usually doesn't care
    485 about its own name.}
    486 
    487 Instead of:
    488 
    489 @i[
    490 (define-syntax (our-if-v2 stx)
    491   (define xs (syntax->list stx))
    492   (datum->syntax stx `(cond [,(cadr xs) ,(caddr xs)]
    493                             [else ,(cadddr xs)])))
    494 ]
    495 
    496 We can write:
    497 
    498 @i[
    499 (define-syntax (our-if-using-match stx)
    500   (match (syntax->list stx)
    501     [(list name condition true-expr false-expr)
    502      (datum->syntax stx `(cond [,condition ,true-expr]
    503                                [else ,false-expr]))]))]
    504 
    505 Great. Now let's try using it:
    506 
    507 @i[
    508 (our-if-using-match #t "true" "false")
    509 ]
    510 
    511 Oops. It's complaining that @racket[match] isn't defined.
    512 
    513 Our transformer function is working at compile time, not run time. And
    514 at compile time, only @racket[racket/base] is required for you
    515 automatically---not the full @racket[racket].
    516 
    517 Anything beyond @racket[racket/base], we have to require
    518 ourselves---and require it for compile time using the
    519 @racket[for-syntax] form of @racket[require].
    520 
    521 In this case, instead of using plain @racket[(require racket/match)],
    522 we want @racket[(require (for-syntax racket/match))]---the
    523 @racket[for-syntax] part meaning, "for compile time".
    524 
    525 So let's try that:
    526 
    527 @i[
    528 (require (for-syntax racket/match))
    529 (define-syntax (our-if-using-match-v2 stx)
    530   (match (syntax->list stx)
    531     [(list _ condition true-expr false-expr)
    532      (datum->syntax stx `(cond [,condition ,true-expr]
    533                                [else ,false-expr]))]))
    534 (our-if-using-match-v2 #t "true" "false")
    535 ]
    536 
    537 Joy.
    538 
    539 @; ----------------------------------------------------------------------------
    540 
    541 @subsection{@racket[begin-for-syntax]}
    542 
    543 We used @racket[for-syntax] to @racket[require] the
    544 @racket[racket/match] module because we needed to use @racket[match]
    545 at compile time.
    546 
    547 What if we wanted to define our own helper function to be used by a
    548 macro?  One way to do that is put it in another module, and
    549 @racket[require] it using @racket[for-syntax], just like we did with
    550 the @racket[racket/match] module.
    551 
    552 If instead we want to put the helper in the same module, we can't
    553 simply @racket[define] it and use it---the definition would exist at
    554 run time, but we need it at compile time. The answer is to put the
    555 definition of the helper function(s) inside @racket[begin-for-syntax]:
    556 
    557 @racketblock[
    558 (begin-for-syntax
    559  (define (my-helper-function ....)
    560    ....))
    561 (define-syntax (macro-using-my-helper-function stx)
    562   (my-helper-function ....)
    563   ....)
    564 ]
    565 
    566 In the simple case, we can also use @racket[define-for-syntax], which
    567 composes @racket[begin-for-syntax] and @racket[define]:
    568 
    569 @racketblock[
    570 (define-for-syntax (my-helper-function ....)
    571   ....)
    572 (define-syntax (macro-using-my-helper-function stx)
    573   (my-helper-function ....)
    574   ....)
    575 ]
    576 
    577 To review:
    578 
    579 @itemize[
    580 
    581 @item{Syntax transformers work at compile time, not run time. The good
    582 news is this means we can do things like rearrange the pieces of
    583 syntax without evaluating them. We can implement forms like
    584 @racket[if] that simply couldn't work properly as run time functions.}
    585 
    586 @item{More good news is that there isn't some special, weird language
    587 for writing syntax transformers. We can write these transformer
    588 functions using the Racket language we already know and love.}
    589 
    590 @item{The semi-bad news is that the familiarity can make it easy to forget
    591 that we're not working at run time. Sometimes that's important to
    592 remember.
    593 
    594   @itemize[
    595 
    596   @item{For example only @racket[racket/base] is required for us
    597 automatically. If we need other modules, we have to require them, and
    598 do so @italic{for compile time} using @racket[for-syntax].}
    599 
    600   @item{Similarly, if we want to define helper functions in the same
    601 file/module as the macros that use them, we need to wrap the
    602 definitions inside a @racket[begin-for-syntax] form. Doing so makes
    603 them available at compile time.}
    604 
    605   ]
    606 }
    607 ]
    608 
    609 @; ----------------------------------------------------------------------------
    610 @; ----------------------------------------------------------------------------
    611 
    612 @section[#:tag "pattern-matching"]{Pattern matching: syntax-case and syntax-rules}
    613 
    614 Most useful syntax transformers work by taking some input syntax, and
    615 rearranging the pieces into something else.  As we saw, this is
    616 possible but tedious using list accessors such as
    617 @racket[cadddr]. It's more convenient and less error-prone to use
    618 @racket[match] to do pattern-matching.
    619 
    620 @margin-note{Historically, @racket[syntax-case] and
    621 @racket[syntax-rules] pattern matching came first. @racket[match] was
    622 added to Racket later.}
    623 
    624 It turns out that pattern-matching was one of the first improvements
    625 to be added to the Racket macro system. It's called
    626 @racket[syntax-case], and has a shorthand for simple situations called
    627 @racket[define-syntax-rule].
    628 
    629 Recall our previous example:
    630 
    631 @racketblock[
    632 (require (for-syntax racket/match))
    633 (define-syntax (our-if-using-match-v2 stx)
    634   (match (syntax->list stx)
    635     [(list _ condition true-expr false-expr)
    636      (datum->syntax stx `(cond [,condition ,true-expr]
    637                                [else ,false-expr]))]))
    638 ]
    639 
    640 Here's what it looks like using @racket[syntax-case]:
    641 
    642 @i[
    643 (define-syntax (our-if-using-syntax-case stx)
    644   (syntax-case stx ()
    645     [(_ condition true-expr false-expr)
    646      #'(cond [condition true-expr]
    647              [else false-expr])]))
    648 (our-if-using-syntax-case #t "true" "false")
    649 ]
    650 
    651 Pretty similar, huh? The pattern matching part looks almost exactly
    652 the same. The way we specify the new syntax is simpler. We don't need
    653 to do quasi-quoting and unquoting. We don't need to use
    654 @racket[datum->syntax]. Instead, we supply a "template", which uses
    655 variables from the pattern.
    656 
    657 There is a shorthand for simple pattern-matching cases, which expands
    658 into @racket[syntax-case]. It's called @racket[define-syntax-rule]:
    659 
    660 @i[
    661 (define-syntax-rule (our-if-using-syntax-rule condition true-expr false-expr)
    662   (cond [condition true-expr]
    663         [else false-expr]))
    664 (our-if-using-syntax-rule #t "true" "false")
    665 ]
    666 
    667 Here's the thing about @racket[define-syntax-rule]. Because it's so
    668 simple, @racket[define-syntax-rule] is often the first thing people are
    669 taught about macros. But it's almost deceptively simple. It looks so
    670 much like defining a normal run time function---yet it's not. It's
    671 working at compile time, not run time. Worse, the moment you want to
    672 do more than @racket[define-syntax-rule] can handle, you can fall off
    673 a cliff into what feels like complicated and confusing
    674 territory. Hopefully, because we started with a basic syntax
    675 transformer, and worked up from that, we won't have that problem. We
    676 can appreciate @racket[define-syntax-rule] as a convenient shorthand,
    677 but not be scared of, or confused about, that for which it's
    678 shorthand.
    679 
    680 Most of the materials I found for learning macros, including the
    681 Racket @italic{Guide}, do a very good job explaining
    682 @hyperlink["http://docs.racket-lang.org/guide/pattern-macros.html" "how
    683 patterns and templates work"]. So I won't regurgitate that here.
    684 
    685 Sometimes, we need to go a step beyond the pattern and template. Let's
    686 look at some examples, how we can get confused, and how to get it
    687 working.
    688 
    689 @; ----------------------------------------------------------------------------
    690 
    691 @subsection{Pattern variable vs. template---fight!}
    692 
    693 Let's say we want to define a function with a hyphenated name, a-b,
    694 but we supply the a and b parts separately. The Racket @racket[struct]
    695 macro does something like this: @racket[(struct foo (field1 field2))]
    696 automatically defines a number of functions whose names are variations
    697 on the name @racket[foo]---such as @racket[foo-field1],
    698 @racket[foo-field2], @racket[foo?], and so on.
    699 
    700 So let's pretend we're doing something like that. We want to transform
    701 the syntax @racket[(hyphen-define a b (args) body)] to the syntax
    702 @racket[(define (a-b args) body)].
    703 
    704 A wrong first attempt is:
    705 
    706 @i[
    707 (define-syntax (hyphen-define/wrong1 stx)
    708   (syntax-case stx ()
    709     [(_ a b (args ...) body0 body ...)
    710      (let ([name (string->symbol (format "~a-~a" a b))])
    711        #'(define (name args ...)
    712            body0 body ...))]))
    713 ]
    714 
    715 Huh. We have no idea what this error message means. Well, let's try to
    716 work it out.  The "template" the error message refers to is the
    717 @racket[#'(define (name args ...)  body0 body ...)] portion. The
    718 @racket[let] isn't part of that template. It sounds like we can't use
    719 @racket[a] (or @racket[b]) in the @racket[let] part.
    720 
    721 In fact, @racket[syntax-case] can have as many templates as you
    722 want. The obvious, required template is the final expression supplying
    723 the output syntax. But you can use @racket[syntax] (a.k.a. @tt{#'}) on a
    724 pattern variable. This makes another template, albeit a small, "fun
    725 size" template. Let's try that:
    726 
    727 @i[
    728 (define-syntax (hyphen-define/wrong1.1 stx)
    729   (syntax-case stx ()
    730     [(_ a b (args ...) body0 body ...)
    731      (let ([name (string->symbol (format "~a-~a" #'a #'b))])
    732        #'(define (name args ...)
    733            body0 body ...))]))
    734 ]
    735 
    736 No more error---good! Let's try to use it:
    737 
    738 @i[
    739 (hyphen-define/wrong1.1 foo bar () #t)
    740 (foo-bar)
    741 ]
    742 
    743 Apparently our macro is defining a function with some name other than
    744 @racket[foo-bar]. Huh.
    745 
    746 This is where the Macro Stepper in DrRacket is
    747 invaluable. @margin-note{Even if you prefer mostly to use Emacs, this
    748 is a situation where it's definitely worth temporarily using DrRacket
    749 for its Macro Stepper.}
    750 
    751 @image[#:scale 0.5 "macro-stepper.png"]
    752 
    753 The Macro Stepper says that the use of our macro:
    754 
    755 @racketblock[
    756 (hyphen-define/wrong1.1 foo bar () #t)
    757 ]
    758 
    759 expanded to:
    760 
    761 @racketblock[
    762 (define (name) #t)
    763 ]
    764 
    765 Well that explains it. Instead, we wanted to expand to:
    766 
    767 @racketblock[
    768 (define (foo-bar) #t)
    769 ]
    770 
    771 Our template is using the symbol @racket[name] but we wanted its
    772 value, such as @racket[foo-bar] in this use of our macro.
    773 
    774 Is there anything we already know that behaves like this---where using
    775 a variable in the template yields its value? Yes: Pattern
    776 variables. Our pattern doesn't include @racket[name] because we don't
    777 expect it in the original syntax---indeed the whole point of this
    778 macro is to create it. So @racket[name] can't be in the main
    779 pattern. Fine---let's make an @italic{additional} pattern. We can do
    780 that using an additional, nested @racket[syntax-case]:
    781 
    782 @i[
    783 (define-syntax (hyphen-define/wrong1.2 stx)
    784   (syntax-case stx ()
    785     [(_ a b (args ...) body0 body ...)
    786      (syntax-case (datum->syntax stx
    787                                  (string->symbol (format "~a-~a" #'a #'b)))
    788                   ()
    789        [name #'(define (name args ...)
    790                  body0 body ...)])]))
    791 ]
    792 
    793 Looks weird? Let's take a deep breath. Normally our transformer
    794 function is given syntax by Racket, and we pass that syntax to
    795 @racket[syntax-case].  But we can also create some syntax of our own,
    796 on the fly, and pass @italic{that} to @racket[syntax-case]. That's all
    797 we're doing here. The whole @racket[(datum->syntax ...)] expression is
    798 syntax that we're creating on the fly. We can give that to
    799 @racket[syntax-case], and match it using a pattern variable named
    800 @racket[name]. Voila, we have a new pattern variable. We can use it in
    801 a template, and its value will go in the template.
    802 
    803 We might have one more---just one, I promise!---small problem left.
    804 Let's try to use our new version:
    805 
    806 @i[
    807 (hyphen-define/wrong1.2 foo bar () #t)
    808 (foo-bar)
    809 ]
    810 
    811 Hmm. @racket[foo-bar] is @italic{still} not defined. Back to the Macro
    812 Stepper. It says now we're expanding to:
    813 
    814 @racketblock[(define (|#<syntax:11:24foo>-#<syntax:11:28 bar>|) #t)]
    815 
    816 Oh right: @racket[#'a] and @racket[#'b] are syntax objects. Therefore
    817 
    818 @racketblock[(string->symbol (format "~a-~a" #'a #'b))]
    819 
    820 is the printed form of both syntax objects, joined by a hyphen:
    821 
    822 @racketblock[|#<syntax:11:24foo>-#<syntax:11:28 bar>|]
    823 
    824 Instead we want the datum in the syntax objects, such as the symbols
    825 @racket[foo] and @racket[bar]. Which we get using
    826 @racket[syntax->datum]:
    827 
    828 @i[
    829 (define-syntax (hyphen-define/ok1 stx)
    830   (syntax-case stx ()
    831     [(_ a b (args ...) body0 body ...)
    832      (syntax-case (datum->syntax stx
    833                                  (string->symbol (format "~a-~a"
    834                                                          (syntax->datum #'a)
    835                                                          (syntax->datum #'b))))
    836                   ()
    837        [name #'(define (name args ...)
    838                  body0 body ...)])]))
    839 (hyphen-define/ok1 foo bar () #t)
    840 (foo-bar)
    841 ]
    842 
    843 And now it works!
    844 
    845 Next, some shortcuts.
    846 
    847 @subsubsection{@racket[with-syntax]}
    848 
    849 Instead of an additional, nested @racket[syntax-case], we could use
    850 @racket[with-syntax]@margin-note*{Another name for
    851 @racket[with-syntax] could be, "with new pattern variable".}. This
    852 rearranges the @racket[syntax-case] to look more like a @racket[let]
    853 statement---first the name, then the value. Also it's more convenient
    854 if we need to define more than one pattern variable.
    855 
    856 @i[
    857 (define-syntax (hyphen-define/ok2 stx)
    858   (syntax-case stx ()
    859     [(_ a b (args ...) body0 body ...)
    860      (with-syntax ([name (datum->syntax stx
    861                                         (string->symbol (format "~a-~a"
    862                                                                 (syntax->datum #'a)
    863                                                                 (syntax->datum #'b))))])
    864        #'(define (name args ...)
    865            body0 body ...))]))
    866 (hyphen-define/ok2 foo bar () #t)
    867 (foo-bar)
    868 ]
    869 
    870 Again, @racket[with-syntax] is simply @racket[syntax-case] rearranged:
    871 
    872 @racketblock[
    873 (syntax-case #,(italic "<syntax>") () [#,(bold "<pattern>") <body>])
    874 (with-syntax ([#,(bold "<pattern>") #,(italic "<syntax>")]) <body>)
    875 ]
    876 
    877 Whether you use an additional @racket[syntax-case] or use
    878 @racket[with-syntax], either way you are simply defining additional
    879 pattern variables. Don't let the terminology and structure make it
    880 seem mysterious.
    881 
    882 @subsubsection{@racket[with-syntax*]}
    883 
    884 We know that @racket[let] doesn't let us use a binding in a subsequent
    885 one:
    886 
    887 @i[
    888 (let ([a 0]
    889       [b a])
    890   b)
    891 ]
    892 
    893 Instead we can nest @racket[let]s:
    894 
    895 @i[
    896 (let ([a 0])
    897   (let ([b a])
    898     b))
    899 ]
    900 
    901 Or use a shorthand for nesting, @racket[let*]:
    902 
    903 @i[
    904 (let* ([a 0]
    905        [b a])
    906   b)
    907 ]
    908 
    909 Similarly, instead of writing nested @racket[with-syntax]s, we can use
    910 @racket[with-syntax*]:
    911 
    912 @i[
    913 (require (for-syntax racket/syntax))
    914 (define-syntax (foo stx)
    915   (syntax-case stx ()
    916     [(_ a)
    917       (with-syntax* ([b #'a]
    918                      [c #'b])
    919         #'c)]))
    920 ]
    921 
    922 One gotcha is that @racket[with-syntax*] isn't provided by
    923 @racket[racket/base]. We must @racket[(require (for-syntax
    924 racket/syntax))]. Otherwise we may get a rather bewildering error
    925 message:
    926 
    927 @italic{@tt{...: ellipses not allowed as an expression in: ...}}.
    928 
    929 
    930 @subsubsection{@racket[format-id]}
    931 
    932 There is a utility function in @racket[racket/syntax] called
    933 @racket[format-id] that lets us format identifier names more
    934 succinctly than what we did above:
    935 
    936 @i[
    937 (require (for-syntax racket/syntax))
    938 (define-syntax (hyphen-define/ok3 stx)
    939   (syntax-case stx ()
    940     [(_ a b (args ...) body0 body ...)
    941      (with-syntax ([name (format-id stx "~a-~a" #'a #'b)])
    942        #'(define (name args ...)
    943            body0 body ...))]))
    944 (hyphen-define/ok3 bar baz () #t)
    945 (bar-baz)
    946 ]
    947 
    948 Using @racket[format-id] is convenient as it handles the tedium of
    949 converting from syntax to symbol datum to string ... and all the way
    950 back.
    951 
    952 @subsubsection{Another example}
    953 
    954 Finally, here's a variation that accepts an arbitrary number of name
    955 parts to be joined with hyphens:
    956 
    957 @i[
    958 (require (for-syntax racket/string racket/syntax))
    959 (define-syntax (hyphen-define* stx)
    960   (syntax-case stx ()
    961     [(_ (names ...) (args ...) body0 body ...)
    962      (let* ([names/sym (map syntax-e (syntax->list #'(names ...)))]
    963             [names/str (map symbol->string names/sym)]
    964             [name/str (string-join names/str "-")]
    965             [name/sym (string->symbol name/str)])
    966        (with-syntax ([name (datum->syntax stx name/sym)])
    967          #`(define (name args ...)
    968              body0 body ...)))]))
    969 (hyphen-define* (foo bar baz) (v) (* 2 v))
    970 (foo-bar-baz 50)
    971 ]
    972 
    973 
    974 To review:
    975 
    976 @itemize[
    977 
    978   @item{You can't use a pattern variable outside of a template. But
    979 you can use @racket[syntax] or @tt{#'} on a pattern variable to make
    980 an ad hoc, "fun size" template.}
    981 
    982   @item{If you want to munge pattern variables for use in the
    983 template, @racket[with-syntax] is your friend, because it lets you
    984 create new pattern variables.}
    985 
    986   @item{Usually you'll need to use @racket[syntax->datum] to get the
    987 interesting value inside.}
    988 
    989   @item{@racket[format-id] is convenient for formatting identifier
    990 names.}
    991 
    992 ]
    993 
    994 @; ----------------------------------------------------------------------------
    995 
    996 @subsection{Making our own @racket[struct]}
    997 
    998 Let's apply what we just learned to a more-realistic example. We'll
    999 pretend that Racket doesn't already have a @racket[struct]
   1000 capability. Fortunately, we can write a macro to provide our own
   1001 system for defining and using structures.  To keep things simple, our
   1002 structure will be immutable (read-only) and it won't support
   1003 inheritance.
   1004 
   1005 Given a structure declaration like:
   1006 
   1007 @racketblock[
   1008 (our-struct name (field1 field2 ...))
   1009 ]
   1010 
   1011 We need to define some procedures:
   1012 
   1013 @itemize[
   1014 
   1015 @item{A constructor procedure whose name is the struct name. We'll
   1016 represent structures as a @racket[vector]. The structure name will be
   1017 element zero. The fields will be elements one onward.}
   1018 
   1019 @item{A predicate, whose name is the struct name with @tt{?}
   1020 appended.}
   1021 
   1022 @item{For each field, an accessor procedure to get its value. These
   1023 will be named struct-field (the name of the struct, a hyphen, and the
   1024 field name).}
   1025 
   1026 ]
   1027 
   1028 
   1029 @#reader scribble/comment-reader
   1030 (i
   1031 (require (for-syntax racket/syntax))
   1032 (define-syntax (our-struct stx)
   1033   (syntax-case stx ()
   1034     [(_ id (fields ...))
   1035      (with-syntax ([pred-id (format-id stx "~a?" #'id)])
   1036        #`(begin
   1037            ;; Define a constructor.
   1038            (define (id fields ...)
   1039              (apply vector (cons (quote id) (list fields ...))))
   1040            ;; Define a predicate.
   1041            (define (pred-id v)
   1042              (and (vector? v)
   1043                   (eq? (vector-ref v 0) 'id)))
   1044            ;; Define an accessor for each field.
   1045            #,@(for/list ([x (syntax->list #'(fields ...))]
   1046                          [n (in-naturals 1)])
   1047                 (with-syntax ([acc-id (format-id stx "~a-~a" #'id x)]
   1048                               [ix n])
   1049                   #`(define (acc-id v)
   1050                       (unless (pred-id v)
   1051                         (error 'acc-id "~a is not a ~a struct" v 'id))
   1052                       (vector-ref v ix))))))]))
   1053 
   1054 ;; Test it out
   1055 (require rackunit)
   1056 (our-struct foo (a b))
   1057 (define s (foo 1 2))
   1058 (check-true (foo? s))
   1059 (check-false (foo? 1))
   1060 (check-equal? (foo-a s) 1)
   1061 (check-equal? (foo-b s) 2)
   1062 (check-exn exn:fail?
   1063            (lambda () (foo-a "furble")))
   1064 
   1065 ;; The tests passed.
   1066 ;; Next, what if someone tries to declare:
   1067 (our-struct "blah" ("blah" "blah"))
   1068 )
   1069 
   1070 The error message is not very helpful. It's coming from
   1071 @racket[format-id], which is a private implementation detail of our macro.
   1072 
   1073 You may know that a @racket[syntax-case] clause can take an
   1074 optional "guard" or "fender" expression.  Instead of
   1075 
   1076 @racketblock[
   1077 [pattern template]
   1078 ]
   1079 
   1080 It can be:
   1081 
   1082 @racketblock[
   1083 [pattern guard template]
   1084 ]
   1085 
   1086 Let's add a guard expression to our clause:
   1087 
   1088 @#reader scribble/comment-reader
   1089 (i
   1090 (require (for-syntax racket/syntax))
   1091 (define-syntax (our-struct stx)
   1092   (syntax-case stx ()
   1093     [(_ id (fields ...))
   1094      ;; Guard or "fender" expression:
   1095      (for-each (lambda (x)
   1096                  (unless (identifier? x)
   1097                    (raise-syntax-error #f "not an identifier" stx x)))
   1098                (cons #'id (syntax->list #'(fields ...))))
   1099      (with-syntax ([pred-id (format-id stx "~a?" #'id)])
   1100        #`(begin
   1101            ;; Define a constructor.
   1102            (define (id fields ...)
   1103              (apply vector (cons (quote id) (list fields ...))))
   1104            ;; Define a predicate.
   1105            (define (pred-id v)
   1106              (and (vector? v)
   1107                   (eq? (vector-ref v 0) 'id)))
   1108            ;; Define an accessor for each field.
   1109            #,@(for/list ([x (syntax->list #'(fields ...))]
   1110                          [n (in-naturals 1)])
   1111                 (with-syntax ([acc-id (format-id stx "~a-~a" #'id x)]
   1112                               [ix n])
   1113                   #`(define (acc-id v)
   1114                       (unless (pred-id v)
   1115                         (error 'acc-id "~a is not a ~a struct" v 'id))
   1116                       (vector-ref v ix))))))]))
   1117 
   1118 ;; Now the same misuse gives a better error message:
   1119 (our-struct "blah" ("blah" "blah"))
   1120 )
   1121 
   1122 Later, we'll see how @racket[syntax-parse] makes it even easier to
   1123 check usage and provide helpful messages about mistakes.
   1124 
   1125 
   1126 @subsection[#:tag "hash.refs"]{Using dot notation for nested hash lookups}
   1127 
   1128 The previous two examples used a macro to define functions whose names
   1129 were made by joining identifiers provided to the macro. This example
   1130 does the opposite: The identifier given to the macro is split into
   1131 pieces.
   1132 
   1133 If you write programs for web services you deal with JSON, which is
   1134 represented in Racket by a @racket[jsexpr?]. JSON often has
   1135 dictionaries that contain other dictionaries. In a @racket[jsexpr?]
   1136 these are represented by nested @racket[hasheq] tables:
   1137 
   1138 @#reader scribble/comment-reader
   1139 (i
   1140 ; Nested `hasheq's typical of a jsexpr:
   1141 (define js (hasheq 'a (hasheq 'b (hasheq 'c "value"))))
   1142 )
   1143 
   1144 In JavaScript you can use dot notation:
   1145 
   1146 @codeblock{
   1147 foo = js.a.b.c;
   1148 }
   1149 
   1150 In Racket it's not so convenient:
   1151 
   1152 @racketblock[(hash-ref (hash-ref (hash-ref js 'a) 'b) 'c)]
   1153 
   1154 We can write a helper function to make this a bit cleaner:
   1155 
   1156 @#reader scribble/comment-reader
   1157 (i
   1158 ;; This helper function:
   1159 (define/contract (hash-refs h ks [def #f])
   1160   ((hash? (listof any/c)) (any/c) . ->* . any)
   1161   (with-handlers ([exn:fail? (const (cond [(procedure? def) (def)]
   1162                                           [else def]))])
   1163     (for/fold ([h h])
   1164       ([k (in-list ks)])
   1165       (hash-ref h k))))
   1166 
   1167 ;; Lets us say:
   1168 (hash-refs js '(a b c))
   1169 )
   1170 
   1171 That's better. Can we go even further and use a dot notation somewhat
   1172 like JavaScript?
   1173 
   1174 @#reader scribble/comment-reader
   1175 (i
   1176 ;; This macro:
   1177 (require (for-syntax racket/syntax))
   1178 (define-syntax (hash.refs stx)
   1179   (syntax-case stx ()
   1180     ;; If the optional `default' is missing, use #f.
   1181     [(_ chain)
   1182      #'(hash.refs chain #f)]
   1183     [(_ chain default)
   1184      (let* ([chain-str (symbol->string (syntax->datum #'chain))]
   1185             [ids (for/list ([str (in-list (regexp-split #rx"\\." chain-str))])
   1186                    (format-id #'chain "~a" str))])
   1187        (with-syntax ([hash-table (car ids)]
   1188                      [keys       (cdr ids)])
   1189          #'(hash-refs hash-table 'keys default)))]))
   1190 ;; Gives us "sugar" to say this:
   1191 (hash.refs js.a.b.c)
   1192 ;; Try finding a key that doesn't exist:
   1193 (hash.refs js.blah)
   1194 ;; Try finding a key that doesn't exist, specifying the default:
   1195 (hash.refs js.blah 'did-not-exist)
   1196 )
   1197 
   1198 It works!
   1199 
   1200 We've started to appreciate that our macros should give helpful
   1201 messages when used in error. Let's try to do that here.
   1202 
   1203 @#reader scribble/comment-reader
   1204 (i
   1205 (require (for-syntax racket/syntax))
   1206 (define-syntax (hash.refs stx)
   1207   (syntax-case stx ()
   1208     ;; Check for no args at all
   1209     [(_)
   1210      (raise-syntax-error #f "Expected hash.key0[.key1 ...] [default]" stx #'chain)]
   1211     ;; If the optional `default' is missing, use #f.
   1212     [(_ chain)
   1213      #'(hash.refs chain #f)]
   1214     [(_ chain default)
   1215      (unless (identifier? #'chain)
   1216        (raise-syntax-error #f "Expected hash.key0[.key1 ...] [default]" stx #'chain))
   1217      (let* ([chain-str (symbol->string (syntax->datum #'chain))]
   1218             [ids (for/list ([str (in-list (regexp-split #rx"\\." chain-str))])
   1219                    (format-id #'chain "~a" str))])
   1220        ;; Check that we have at least hash.key
   1221        (unless (and (>= (length ids) 2)
   1222                     (not (eq? (syntax-e (cadr ids)) '||)))
   1223          (raise-syntax-error #f "Expected hash.key" stx #'chain))
   1224        (with-syntax ([hash-table (car ids)]
   1225                      [keys       (cdr ids)])
   1226          #'(hash-refs hash-table 'keys default)))]))
   1227 
   1228 ;; See if we catch each of the misuses
   1229 (hash.refs)
   1230 (hash.refs 0)
   1231 (hash.refs js)
   1232 (hash.refs js.)
   1233 )
   1234 
   1235 Not too bad. Of course, the version with error-checking is quite a bit
   1236 longer. Error-checking code generally tends to obscure the logic, and
   1237 does here. Fortunately we'll soon see how @racket[syntax-parse] can
   1238 help mitigate that, in much the same way as contracts in normal
   1239 Racket or types in Typed Racket.
   1240 
   1241 Maybe we're not convinced that writing @racket[(hash.refs js.a.b.c)]
   1242 is really clearer than @racket[(hash-refs js '(a b c))]. Maybe we
   1243 won't actually use this approach. But the Racket macro system makes it
   1244 a possible choice.
   1245 
   1246 @; ----------------------------------------------------------------------------
   1247 @; ----------------------------------------------------------------------------
   1248 
   1249 @section{Syntax parameters}
   1250 
   1251 "Anaphoric if" or "aif" is a popular macro example. Instead of writing:
   1252 
   1253 @racketblock[
   1254 (let ([tmp (big-long-calculation)])
   1255   (if tmp
   1256       (foo tmp)
   1257       #f))
   1258 ]
   1259 
   1260 You could write:
   1261 
   1262 @racketblock[
   1263 (aif (big-long-calculation)
   1264      (foo it)
   1265      #f)
   1266 ]
   1267 
   1268 In other words, when the condition is true, an @racket[it] identifier
   1269 is automatically created and set to the value of the condition. This
   1270 should be easy:
   1271 
   1272 
   1273 @i[
   1274 (define-syntax-rule (aif condition true-expr false-expr)
   1275   (let ([it condition])
   1276     (if it
   1277         true-expr
   1278         false-expr)))
   1279 (aif #t (displayln it) (void))
   1280 ]
   1281 
   1282 Wait, what? @racket[it] is undefined?
   1283 
   1284 It turns out that all along we have been protected from making a
   1285 certain kind of mistake in our macros. The mistake is if our new
   1286 syntax introduces a variable that accidentally conflicts with one in
   1287 the code surrounding our macro.
   1288 
   1289 The Racket @italic{Reference} section,
   1290 @hyperlink["http://docs.racket-lang.org/reference/syntax-model.html#(part._transformer-model)" "Transformer
   1291 Bindings"], has a good explanation and example. Basically, syntax
   1292 has "marks" to preserve lexical scope. This makes your macro behave
   1293 like a normal function, for lexical scoping.
   1294 
   1295 If a normal function defines a variable named @racket[x], it won't
   1296 conflict with a variable named @racket[x] in an outer scope:
   1297 
   1298 @i[
   1299 (let ([x "outer"])
   1300   (let ([x "inner"])
   1301     (printf "The inner `x' is ~s\n" x))
   1302   (printf "The outer `x' is ~s\n" x))
   1303 ]
   1304 
   1305 When our macros also respect lexical scoping, it's easier to write
   1306 reliable macros that behave predictably.
   1307 
   1308 So that's wonderful default behavior. But sometimes we want to
   1309 introduce a magic variable on purpose---such as @racket[it] for
   1310 @racket[aif].
   1311 
   1312 There's a bad way to do this and a good way.
   1313 
   1314 The bad way is to use @racket[datum->syntax], which is tricky to use correctly. @margin-note*{See @hyperlink["http://www.schemeworkshop.org/2011/papers/Barzilay2011.pdf" "Keeping it Clean with Syntax Parameters (PDF)"].}
   1315 
   1316 The good way is with a syntax parameter, using
   1317 @racket[define-syntax-parameter] and
   1318 @racket[syntax-parameterize]. You're probably familiar with regular
   1319 parameters in Racket:
   1320 
   1321 @i[
   1322 (define current-foo (make-parameter "some default value"))
   1323 (current-foo)
   1324 (parameterize ([current-foo "I have a new value, for now"])
   1325   (current-foo))
   1326 (current-foo)
   1327 ]
   1328 
   1329 That's a normal parameter. The syntax variation works similarly. The
   1330 idea is that we'll define @racket[it] to mean an error by
   1331 default. Only inside of our @racket[aif] will it have a meaningful
   1332 value:
   1333 
   1334 @i[
   1335 (require racket/stxparam)
   1336 (define-syntax-parameter it
   1337   (lambda (stx)
   1338     (raise-syntax-error (syntax-e stx) "can only be used inside aif")))
   1339 (define-syntax-rule (aif condition true-expr false-expr)
   1340   (let ([tmp condition])
   1341     (if tmp
   1342         (syntax-parameterize ([it (make-rename-transformer #'tmp)])
   1343           true-expr)
   1344         false-expr)))
   1345 (aif 10 (displayln it) (void))
   1346 (aif #f (displayln it) (void))
   1347 ]
   1348 
   1349 Inside the @racket[syntax-parameterize], @racket[it] acts as an alias
   1350 for @racket[tmp]. The alias behavior is created by
   1351 @racket[make-rename-transformer].
   1352 
   1353 If we try to use @racket[it] outside of an @racket[aif] form, and
   1354 @racket[it] isn't otherwise defined, we get an error like we want:
   1355 
   1356 @i[
   1357 (displayln it)
   1358 ]
   1359 
   1360 But we can still define @racket[it] as a normal variable in local
   1361 definition contexts like:
   1362 
   1363 @i[
   1364 (let ([it 10])
   1365   it)
   1366 ]
   1367 
   1368 or:
   1369 
   1370 @i[
   1371 (define (foo)
   1372   (define it 10)
   1373   it)
   1374 (foo)
   1375 ]
   1376 
   1377 
   1378 For a deeper look, see @hyperlink["http://www.schemeworkshop.org/2011/papers/Barzilay2011.pdf" "Keeping it Clean with Syntax Parameters"].
   1379 
   1380 @; ----------------------------------------------------------------------------
   1381 @; ----------------------------------------------------------------------------
   1382 
   1383 @section{What's the point of @racket[splicing-let]?}
   1384 
   1385 I stared at @racket[racket/splicing] for the longest time. What does
   1386 it do? Why would I use it? Why is it in the Macros section of the
   1387 reference?
   1388 
   1389 Step one, @elem[#:style "strike"]{cut a hole in the box}
   1390 de-mythologize it. For example, using @racket[splicing-let] like this:
   1391 
   1392 @#reader scribble/comment-reader
   1393 (i
   1394 (require racket/splicing)
   1395 (splicing-let ([x 0])
   1396   (define (get-x)
   1397     x))
   1398 ;; get-x is visible out here:
   1399 (get-x)
   1400 ;; but x is not:
   1401 x
   1402 )
   1403 
   1404 is equivalent to:
   1405 
   1406 @#reader scribble/comment-reader
   1407 (i
   1408 (define get-y
   1409   (let ([y 0])
   1410     (lambda ()
   1411       y)))
   1412 ;; get-y is visible out here:
   1413 (get-y)
   1414 ;; but y is not:
   1415 y
   1416 )
   1417 
   1418 This is the classic Lisp/Scheme/Racket idiom sometimes called "let
   1419 over lambda". @margin-note*{A
   1420 @hyperlink["http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg03277.html" "koan"]
   1421 about closures and objects.} A closure hides @racket[y], which can
   1422 only be accessed via @racket[get-y].
   1423 
   1424 So why would we care about the splicing forms? They can be more
   1425 concise, especially when there are multiple body forms:
   1426 
   1427 @i[
   1428 (require racket/splicing)
   1429 (splicing-let ([x 0])
   1430   (define (inc)
   1431     (set! x (+ x 1)))
   1432   (define (dec)
   1433     (set! x (- x 1)))
   1434   (define (get)
   1435     x))
   1436 ]
   1437 
   1438 The splicing variation is more convenient than the usual way:
   1439 
   1440 @#reader scribble/comment-reader
   1441 (i
   1442 (define-values (inc dec get)
   1443   (let ([x 0])
   1444     (values (lambda ()  ;inc
   1445               (set! x (+ 1 x)))
   1446             (lambda ()  ;dec
   1447               (set! x (- 1 x)))
   1448             (lambda ()  ;get
   1449               x))))
   1450 )
   1451 
   1452 When there are many body forms---and we're generating them in a
   1453 macro---the splicing variations can be much easier.
   1454 
   1455 @; ----------------------------------------------------------------------------
   1456 @; ----------------------------------------------------------------------------
   1457 
   1458 @section{Robust macros: syntax-parse}
   1459 
   1460 Functions can be used in error. So can macros.
   1461 
   1462 @subsection{Error-handling strategies for functions}
   1463 
   1464 With plain old functions, we have several choices how to handle
   1465 misuse.
   1466 
   1467 1. Don't check at all.
   1468 
   1469 @#reader scribble/comment-reader
   1470 (i
   1471 (define (misuse s)
   1472   (string-append s " snazzy suffix"))
   1473 ;; User of the function:
   1474 (misuse 0)
   1475 ;; I guess I goofed, but -- what is this "string-append" of which you
   1476 ;; speak??
   1477 )
   1478 
   1479 The problem is that the resulting error message will be confusing. Our
   1480 user thinks they're calling @racket[misuse], but is getting an error
   1481 message from @racket[string-append].  In this simple example they
   1482 could probably guess what's happening, but in most cases they won't.
   1483 
   1484 2. Write some error handling code.
   1485 
   1486 @#reader scribble/comment-reader
   1487 (i
   1488 (define (misuse s)
   1489   (unless (string? s)
   1490     (error 'misuse "expected a string, but got ~a" s))
   1491   (string-append s " snazzy suffix"))
   1492 ;; User of the function:
   1493 (misuse 0)
   1494 ;; I goofed, and understand why! It's a shame the writer of the
   1495 ;; function had to work so hard to tell me.
   1496 )
   1497 
   1498 Unfortunately the error code tends to overwhelm and/or obscure our
   1499 function definition. Also, the error message is good but not
   1500 great. Improving it would require even more error code.
   1501 
   1502 3. Use a contract.
   1503 
   1504 @#reader scribble/comment-reader
   1505 (i
   1506 (define/contract (misuse s)
   1507   (string? . -> . string?)
   1508   (string-append s " snazzy suffix"))
   1509 ;; User of the function:
   1510 (misuse 0)
   1511 ;; I goofed, and understand why! I'm happier, and I hear the writer of
   1512 ;; the function is happier, too.
   1513 )
   1514 
   1515 This is the best of both worlds.
   1516 
   1517 The contract is a simple and concise. Even better, it's
   1518 declarative. We say what we want to happen, not how.
   1519 
   1520 On the other hand the user of our function gets a very detailed error
   1521 message. Plus, the message is in a standard, familiar format.
   1522 
   1523 4. Use Typed Racket.
   1524 
   1525 @codeblock{#lang typed/racket}
   1526 @interaction[#:eval typed/evaluator
   1527 (: misuse (String -> String))
   1528 (define (misuse s)
   1529   (string-append s " snazzy suffix"))
   1530 (misuse 0)
   1531 ]
   1532 
   1533 Even better, Typed Racket can catch usage mistakes up-front at compile
   1534 time.
   1535 
   1536 @subsection{Error-handling strategies for macros}
   1537 
   1538 For macros, we have similar choices.
   1539 
   1540 1. Ignore the possibility of misuse. This choice is even worse for
   1541 macros. The default error messages are even less likely to make sense,
   1542 much less help our user know what to do.
   1543 
   1544 2. Write error-handling code. We saw how much this complicated our
   1545 macros in our example of @secref["hash.refs"]. And while we're still
   1546 learning how to write macros, we especially don't want more cognitive
   1547 load and obfuscation.
   1548 
   1549 3. Use @racket[syntax-parse]. For macros, this is the equivalent of
   1550 using contracts or types for functions. We can declare that input
   1551 pattern elements must be certain kinds of things, such as an
   1552 identifier. Instead of "types", the kinds are referred to as "syntax
   1553 classes". There are predefined syntax classes, plus we can define our
   1554 own.
   1555 
   1556 @subsection{Using @racket[syntax-parse]}
   1557 
   1558 November 1, 2012: So here's the deal. After writing everything up to
   1559 this point, I sat down to re-read the documentation for
   1560 @racket[syntax-parse]. It was...very understandable. I didn't feel
   1561 confused.
   1562 
   1563 @codeblock{
   1564 <span style='accent: "Keanu-Reeves"'>
   1565 Whoa.
   1566 </span>
   1567 }
   1568 
   1569 Why? The documentation has a nice
   1570 @hyperlink["http://docs.racket-lang.org/syntax/stxparse-intro.html" "Introduction"]
   1571 with many simple examples, followed by an
   1572 @hyperlink["http://docs.racket-lang.org/syntax/stxparse-examples.html" "Examples"]
   1573 section illustrating many real-world scenarios.
   1574 
   1575 Furthermore, everything I'd learned up to this point prepared me to
   1576 appreciate what @racket[syntax-parse] does, and why. The details of
   1577 how to use it seem pretty straightforward, so far.
   1578 
   1579 This might well be a temporary state of me "not knowing what I don't
   1580 know". As I dig in and use it more, maybe I'll discover something
   1581 confusing or tricky. If/when I do, I'll come back here and update
   1582 this.
   1583 
   1584 But for now I'll focus on improving the previous parts.
   1585 
   1586 @; ----------------------------------------------------------------------------
   1587 @; ----------------------------------------------------------------------------
   1588 
   1589 @section{References and Acknowledgments}
   1590 
   1591 Eli Barzilay's blog post,
   1592 @hyperlink["http://blog.racket-lang.org/2011/04/writing-syntax-case-macros.html" "Writing
   1593 ‘syntax-case’ Macros"], helped me understand many key details and
   1594 concepts, and inspired me to use a "bottom-up" approach.
   1595 
   1596 Eli wrote another blog post,
   1597 @hyperlink["http://blog.racket-lang.org/2008/02/dirty-looking-hygiene.html" "Dirty
   1598 Looking Hygiene"], which explains @racket[syntax-parameterize]. I
   1599 relied heavily on that, mostly just updating it since his post was
   1600 written before PLT Scheme was renamed to Racket.
   1601 
   1602 Matthew Flatt's
   1603 @hyperlink["http://www.cs.utah.edu/plt/publications/macromod.pdf" "Composable
   1604 and Compilable Macros: You Want it When? (PDF)"] explains how Racket
   1605 handles compile time vs. run time.
   1606 
   1607 @hyperlink["http://www.scheme.com/tspl4/syntax.html#./syntax:h0" "Chapter
   1608 8"] of @italic{The Scheme Programming Language} by Kent Dybvig
   1609 explains @racket[syntax-rules] and @racket[syntax-case].
   1610 
   1611 @hyperlink["http://www.ccs.neu.edu/racket/pubs/icfp10-cf.pdf" "Fortifying
   1612 Macros (PDF)"] is the paper by Ryan Culpepper and Matthias Felleisen
   1613 introducing @racket[syntax-parse].
   1614 
   1615 Shriram Krishnamurthi looked at a very early draft and encouraged me
   1616 to keep going. Sam Tobin-Hochstadt and Robby Findler also encouraged
   1617 me. Matthew Flatt showed me how to make a Scribble
   1618 @racket[interaction] print @racket[syntax] as @racket["syntax"] rather
   1619 than as @racket["#'"]. Jay McCarthy helped me catch some mistakes and
   1620 confusions. Jon Rafkind provided suggestions. Kieron Hardy reported a
   1621 font issue and some typos.
   1622 
   1623 Finally, I noticed something strange. After writing much of this, when
   1624 I returned to some parts of the Racket documentation, I noticed it had
   1625 improved since I last read it. Of course, it was the same; I'd
   1626 changed. It's interesting how much of what we already know is
   1627 projected between the lines. My point is, the Racket documentation is
   1628 very good. The @italic{Guide} provides helpful examples and
   1629 tutorials. The @italic{Reference} is very clear and precise.
   1630 
   1631 @; ----------------------------------------------------------------------------
   1632 @; ----------------------------------------------------------------------------
   1633 
   1634 @section{Epilogue}
   1635 
   1636 @centered{
   1637 "Before I had studied Chan (Zen) for thirty years, I saw mountains as
   1638 mountains, and rivers as rivers. When I arrived at a more intimate
   1639 knowledge, I came to the point where I saw that mountains are not
   1640 mountains, and rivers are not rivers. But now that I have got its very
   1641 substance I am at rest. For it's just that I see mountains once again
   1642 as mountains, and rivers once again as rivers"
   1643 
   1644 @smaller{--Buddhist saying originally formulated by Qingyuan Weixin,
   1645 later translated by D.T. Suzuki in his @italic{Essays in Zen
   1646 Buddhism}.}
   1647 }
   1648 
   1649 Translated into Racket:
   1650 
   1651 @racketblock[
   1652 (dynamic-wind (lambda ()
   1653                 (and (eq? 'mountains 'mountains)
   1654                      (eq? 'rivers 'rivers)))
   1655               (lambda ()
   1656                 (not (and (eq? 'mountains 'mountains)
   1657                           (eq? 'rivers 'rivers))))
   1658               (lambda ()
   1659                 (and (eq? 'mountains 'mountains)
   1660                      (eq? 'rivers 'rivers))))
   1661 
   1662 ]