Wednesday, February 27, 2008

One more catch

(This post might only be interesting to people with fairly good Lua proficiency). Some time ago, I wrote a post about designing a try...catch extension. The point was to show that even seemingly simple extensions require care to many corner cases, if one wants them to work reliably; and the implied consequence was that lower level approaches to meta-programming, akin to source preprocessing, are not suitable to write robust language features. I unwillingly demonstrated this by leaving an corner case unattended. The extension wraps portions of code that might fail into a "pcall(function()...end)"; this breaks return statements: they would return from the newly created function, not from the surrounding user's function as he would expect. Replacing return statements with something suitable was one of the extension's tasks (and one that would be hard to achieve with a code preprocessor). However, I forgot that "...", the extra arguments of a function call, can't be passed across function boundaries either. If they're used in a pcall+function wrapper, they have to be stored in a temporary table and unpacked when used. Here's the piece of code that has shown the issue:
-{ extension 'withdo' }

function run (fmt, ...)
   with h = io.popen(string.format(fmt, ...), 'r') do
      return h:read '*a'
   end
end
It uses "with...do...end", which limits the lifetime of a resource (here a file handle) to a lexical scope: the "h" declared in "with h = ... do ... end" will be closed once "end" is reached, even if returns and/or errors happen in the block; of course, it is implemented using the try/catch extension, and expands to:
function run (fmt, ...)
   do
      local h
      try 
         h = io.popen(string.format(fmt, ...), 'r') do
         return h:read '*a'
      finally
         h:close()
      end
   end
end
This in turns is equivalent to the following plain Lua code:
function run (fmt, ...)
   do
      local h
      local args = {...}
      local caught_return
      local user_success, user_error = pcall( function()
         h = io.popen(string.format(fmt, unpack(args)), 'r')
         caught_return =  { h:read '*a' }
      end)
      h:close()
      if user_success then
         if caught_return then return unpack(caught_return) end
      else
         error (user_error)
      end
   end
end
The (repeated) moral of the story is: language extensions are even harder than regular code to make reusable. If your language encourages writing quick and dirty macros, it will have a hard time growing extensive libraries. And if your meta-programming framework hasn't got a deep understanding of the code it manipulates, it's simply pointless.

Tuesday, February 26, 2008

shorter lambdas

In a stunning illustration of the bikeshed principle, Arc's feature that got the most attention and praises has been its short notation for anonymous functions: "[foo _]" is equivalent to "(fn (_) (foo _))". Since I'm in an experimental mood these days, let's give it a try, as it's merely a one-liner. Using "[...]" to mark the function's boundaries wouldn't be a great idea, as it would cause ambiguities in table constructors, and when used at the beginning of a statement, might be agregated to the previous statement as an `Index accessor. Instead, I'll use a prefix antislash; there's no need for a closing marker: in Lua, the end of an expression can be determined implicitly. Here's the meta-code, to put directly in the file where the notation is to be used: -{ mlp.expr.prefix:add{ '\\', prec=5, builder=|_,x| +{|_|-{x}} } } It can be used, for instance, as: table.foreach (mlp, \print('key:', _)) I chose to give it a very low precedence, this is what seemed the most sensible. When you're not happy with this, use parentheses. What good is that extension? It can be used instead of the already short "|_| foo(_)" notation; it saves two characters, and its usage is restricted to single parameter functions; so in terms of code shortening, it only makes sense for the tiniest functions. The most decent example I could think of is "\_[1]" to extract the first element of a list. Clearly this has no interest in terms of saving tokens nor AST nodes, yet I can't help but finding this notation rather likeable. Maybe it has some auto-documentary mojo in it? By not even bothering to give a name to the parameter of a tiny function, we give it some sort of point-free flavor: "\_[1]" is read as: "[1] as a function", rather than "the function that takes '_' and returns '_[1]'". It would also look nice in "\_+1"; in short, it seems to have a role equivalent to Haskell's sections, which allow to write "(+)" instead of "|x,y| x+y", "(+1)" instead of "|x| x+1", "(1+)" instead of "|x| 1+x" etc. In any case, the notation seems to only make sense in very functional code. It should be tried next time I have an occasion...

Thursday, February 21, 2008

Syntax experiments

Now that metalua 0.4 has been released, I took a bit of time to play with syntax. I'm usually reluctant to superficial syntax tweaking: 90% of the time, it breaks source compatibility for a marginal or even negative increase in readability. However, it's impossible to identify the remaining 10% interesting cases unless you give them a fair shot. Hence I introduced two experimental extensions:
  • xloop the extended loop syntax;
  • xmatch the extended pattern matching syntax

xloop

The original idea behind xloop stemmed from Common Lisp's infamous loop macro. This is an incredibly powerful and arcane macro. It's quite fun to hack it, but it's also a surefire way to get unmaintainable code in most team coding conditions. So I reduced its scope so dramatically that, well, it's got almost nothing left from the original loop :) But I've got what I believe is a sound base, on which I'll be able to gradually add new features to try. The central idea is that a Lua loop consists of a loop header, followed by a body "do ... end". What I'm going to provide is the ability to chain several headers, so that you can write:
for i=0,9 for j=0,90,10 do
  print (i+j)
end
instead of:
for i=0, 9 do
   for j=0, 90, 10 do
      print (i+j)
   end
end
The headers can be:
  • for, both in its numeric and enumerator versions. These are composed by nesting the loops, as shown above.
  • while, which gives a condition not to exit the loop.
  • until, which gives a condition to exit the loop.
  • if, which allows to zap an iteration of the loop without breaking it.
while and until header elements act as conditionial break statements, whereas if acts as a conditional continue. If a break appears in the loop's body, all the (implicit) nested loops are broken at once. Some things that might turn out to be useful idioms include "for i=1,100 if not prime[i] do ... end", or "for i=1, 1000 while not canceled do ... end", "for i=1,10 for j=1,10 if i~=j do ... end"... This might look not-so-interesting, but I plan to go on playing with it, in case something truly interesting emerges. I think that introducing some fold and map operators, as alternatives or complements to the loop's body, might be interesting (they would be roughly equivalent to directives such as collect or sum for Lisp's loop macro). There's a point where too many directives will hurt readability, and you can only tell after you've tried, but that's why I'm not merging these extensions with metalua's more mature stuff: they're virtually guaranteed to be at least partially rolled back.

xmatch

Structural pattern matching is already a metalua extension. However, it's the one I used and tuned the most extensively, and it's directly inspired by established functional programming languages, so it's fairly mature. The painfully repetitive idioms I've met are:
  • functions which solely consist of a match statement;
  • contrived uses of match statements when I meant it to return an expression.
Moreover, patterns are only usable in match statements, where several alternative patterns might apply. In some cases, we know that a given pattern will match, we only want to use it as a way to destructurate a piece of data, in order to easily create assignments.
match functions
instead of:
function f(a, b, c)
  match a, b, c with
  | x1, y1, z1 -> foo()
  | x2, y2, z2 -> bar()
  end
end
you can write:
match function f
| x1, y1, z1 -> foo()
| x2, y2, z2 -> bar()
end
This is very similar to Lua's "function f()... end" as a shortcut for "f=function()...end". Still in Lua's spirit, if you want to create a local match function, as in "local function f(...) match ... end end", you can use "local match function f ... end".
anonymous match functions
Of course, you can also declare anonymous functions, this is not Python after all :)
f = match function
| x if x%2==0 -> return 'even'
| _ -> return 'odd'
end
expression match
You are now allowed to put a match...with where an expression is expected. In that case, the conditional blocks are replaced by conditional expressions: print(match x with 1 -> 'one' | _ -> many). Notice that the end keyword in Lua terminates statement blocks, not expressions, and therefore isn't expected by the match expression.
Destructuring bind
This could be summed up as a single-case match statement which escapes its scope. Suppose that you know that x contains something of the form `If{ cond, { stat1, stat2, ... } } and you want to get the value of cond and the first statement of the body. You can write the full code:
assert(x.tag=='If')
cond = x[1]
stat = x[2][1]
Or you can use a bind: bind `If{ cond, { stat, ... } } = x. There is also a local bind, which declare the variables as new locals instead of merely assigning them: local bind `If{ cond, { stat, ... } } = x. Of course, if the pattern doesn't match, you get a runtime error.

Friday, February 15, 2008

Metalua 0.4.1 (rc1)

A new version of metalua is available, with the following improvements:
  • A properly working runtime error reporting: source line infos are now correctly included in the bytecode. thanks to V. Egorov.
  • Support for 64 bits processors, thanks to O. Gournet.
  • Interactive REPL in the front-end, with optional readline support. Interactions with scoped extensions is not fully functional yet.
  • Update of Pluto to version 2.2: dependencies to patched VM are dropped (the previous version required LUAI_FUNC symbols to be exported by the VM, which wasn't the case for non BSD-based platforms). Thanks to B. Sunshine-Hill for his very reactive user support.
  • Build for visual studio .NET, thanks to D. Manura
  • Update of the included VM from the Pluto-modified 5.1.2 to the regular 5.1.3.
  • a couple of minor bug fixes

Friday, February 8, 2008

Metalua 0.4 released

Metalua 0.4 has been released. Unfortunately it hasn't got a very convincing support for macro hygiene yet, but there's just too much work and experimentation left on this subject. If you're interested you're welcome to dig in the extension H and its runtime, and ask questions on the mailing list!
Metalua 0.4 rc2
Look at the README.TXT file for details on what's new. There probably are some issues left; I've already received reports that the Pluto integration doesn't work on some systems. If you have that issue, set the environment variable LUA_NOSPRINGS to true, and you should be fine. If you have a problem, either this one or another one, please take the time to report it.