Bug again

Found a new bug, in the compiler, for the `Stat{ b, e } expression AST. This node's purpose is to include a block of statements b where an expression was expected. The node's value is the value of expression e, evaluated as if it were in the block's context, So that `Stat{ +{block: local x=3}, +{expr: x} } evaluates to 3. If the block declares local variables holding upvalues (i.e. values used by closures), they might be overridden before being saved out of the stack (so that they would survive the block's scope). Quick fix: replace function expr.Stat in compile.lua with the following code:

function expr.Stat (fs, ast, v)
   local save_nactvar = fs.nactvar
   local dest_reg = fs.freereg
   local save_actvar = { }
   local last_unreg_var = #fs.actvar
   if last_unreg_var > 0 or fs.actvar[0] then
      for i = fs.nactvar, last_unreg_var do
         save_actvar[i] = fs.actvar[i]
      end
   end
   fs.nactvar = fs.freereg
   enterblock (fs, { }, false)
   chunk (fs, ast[1])
   expr.expr (fs, ast[2], v)
   luaK:exp2nextreg (fs, v)
   leaveblock (fs)
   luaK:exp2reg (fs, v, dest_reg)
   fs.freereg = fs.freereg+1
   fs.nactvar = save_nactvar
   for i, j in pairs(save_actvar) do fs.actvar[i] = j end
end

A slightly better job could be done by checking whether there are upvalues in the block, and if not, save a register copy, but I don't have time to test this thoroughly yet so I stick to the safer version.

Metalua isn't Lisp

(Some of the subjects of this post are already addressed in the manual, grep for "Metalua design philosophy") It's quite hard to think of a macro-enabled language without thinking of Lisp. I thought it was worth explaining why I've started working on Metalua instead of sticking to Scheme. The most meaningful features of metalua are: a "normal" syntax Metalua isn't the only meta-language supporting algol-like syntax, but in the category of mature dynamic languages (maturity to be credited to Lua, not to metalua obviously), it's pretty much alone. There are very interesting works in which the language and the macro system are co-designed, my personal favorite being converge, but I'm afraid it takes a decade for a language+compiler+runtime to mature. That's why I felt so excited when I discovered how macro-friendly Lua syntax and semantics were: the decade of language maturing already mostly took place, quite efficiently due to the tastefulness of its designers and their fearlessness of breaking backward compatibility across versions. To the best of my knowledge, all other attempts to retrofit meta-programming into an existing language ended up with a clunky, mostly impractical chimera. There's something important to notice about Lua's syntax: since it's designed for fast and easy parsing, and with a strong focus on the Principle of Least Surprise, the parser included in metalua is really simple. In my opinion, it has a very gentle (and short) learning curve, and simply doesn't come in your way when you write a macro: no need to cope with an advanced grammar DSL. That's something that Lisps' sexps have obviously right, since they hardly distinguish abstract and concrete syntaxes. Moreover, this simple no-frills parser encourages to stick with simple, sound syntaxes, which tend to cohabit well together. And peaceful cohabitation of separately written syntax extensions is not a small issue. Finally, in the algol+macro category, the case of Dylan has to be mentioned. I have a bad feeling about this language, but I can't spot why exactly. That would be worth a separate post. Now what good is it to have an algol syntax? Anyone who gave a serious try to some Lisp dialect knows that it's not that hard to get used to sexps, provided that your editor handles indentation. I won't rant about people's prejudice against sexps when they never gave it a try, Lispers do that better than me. There's something more meaningful: AST are great when you want to think about your code as data, but 95% of the time you [ought to] think at a different abstraction level. Separating the representations for meta-operations and regular programming helps the separation of concerns. That's a form of encapsulation, if you will. It also makes it easier to use third party extensions without any meta-programming concern. Granted, metalua lets you choose and mix freely between syntax and AST representations: "function(x) return x+1 end" or "|x| x+1" are really the same as: -{`Function{ {x}, { `Return{`Op{`Add, `Id "x", `Number 1 } } } } } from the compiler's point of view; but these notations outline a very different intent from the developer. You should be able to choose the most relevant form for each task, and to translate easily from one form to the other (if you can't, you probably shouldn't write macros yet; but you might be interested by those written by third parties). clear staging That's the most important difference with Lisp IMO: shifting to the compile-time level is obvious through the use of -{...} markers, and feels wrong when done gratuitously. A Lisp program is often an intricate mix of macros and functions, where metalua design is more staged: it quickly becomes fastidious to mix macros and functions. The best way usually is to design language extensions separately, then include them with the -{ extension "foobar" } idiom. It's expected to favor the following results:

• writing of reusable extensions, rather than ad-hoc hacks. Macros are hard to design and debug, so macros addressing pervasive needs should be designed, maintained and shared by a community rather than reinvented again and again. For that, idioms which encourage modular designs are a good thing.
• slightly higher barrier to entry: one reason why Java and C++ programmers actually reuse standard structure libraries is that even writing a binary tree in these languages is a pain in the ass. in Lisp or ML dialects, it's so fun and easy people rewrite their own again and again. It causes problems of code maturity, inexistent or poor documentation, readability by other developers... all the classic symptoms of the Not-Invented-Here disease. So I hope it will help standard extension libs to emerge, and make people think twice before unrolling their own 42th variant of an OO framework.
• it's visually obvious when something interesting happens. Seeing a -{...} in the code should put code readers in warning mode, a bit like C++'s ugly cast syntax help people realize when they're writing something ugly with little good reasons. Moreover, in my experience, thinking of macros require a mental shift: it's a good think that this switch is outlined by a consistent visual clue. I love Pavlovian [meta]programming.

As written among others by Paul Graham, half of writing a Lisp program is about designing the perfect language to express it, through macros. Metalua is based on the hypothesis that this half of the work should be more clearly separated from the other one. The price to pay is a slightly lower flexibility, the expected benefit are better clarity and code sharing. solipsism considered harmful This quality is directly inherited from plain Lua. Lua is intended to integrate smoothly with C and C++. A typical Lua application is a mix of C core functions, higher-level code in Lua, and easy communication between them: extensive control of Lua through a dedicated C API, easy reification of C data and structures as first-class Lua entities, call of C from Lua as well as Lua from C... Lua doesn't pretend to do everything perfectly; instead, it focuses on doing right what C does poorly. Since C is the lingua franca with OSes and between other languages, it makes Lua a very good citizen: I've even integrated it into awfully proprietary embedded platforms (little RAM, no real OS, no threads/sync/blocking calls, not even a full stdlib) with surprising ease. There are also interesting experiences of integrating Lua with the JVM and the CLR; check luaforge for details. This focus on proper integration with the outside world is extremely important IMO, and something Lisps had notoriously wrong.

bug fix in pattern matching

The pattern matching extension, in metalua 0.3, contained a known variable scoping bug: all used local variables where declared once for the whole match statement, whereas they should have been local to a single case. It caused erroneous variable captures. That's fixed in this updated version.