Creating your own DSL - Parsing (with Ruby)

1. Why use a DSL?
2. Why create your own DSL?
3. What makes a good DSL?
4. Creating your own DSL - Parsing
5. Creating your own DSL - Parsing (with Ruby)
6. Creating the Packager DSL - Initial steps
7. Creating the Packager DSL - First feature

Ruby is a full-featured generic programming language with all the standard bells and whistles. So, it may seem odd that I'm suggesting it is also useful for parsing DSLs. But, Ruby has a few interesting features (both in syntax and semantics) which make it ideal for parsing DSLs:

• All functions are method calls on an implicit object
• Code blocks as the last parameter of a method call
• Greedy implicit binding rules

In short, Ruby makes it possible for a DSL author to create a class that, with a little help, will treat all key-value pairs as method invocations on an object. This includes nested blocks (though we need to create a new class for each level of descent). Because these nested blocks are values of some key, that key is the method call that receives the code block. And, because Ruby doesn't require a lot of symbols like parentheses and the like, the DSL ends up looking very clean and non-program-ish. (All the DSL examples in this series are parsable in Ruby.)

Most importantly, all of the features of full Ruby language (such as branching and looping) are available for free. The DSL author doesn't have to do anything special - it's just there. The DSL user simply has to be pointed at the standard Ruby documentation (and all the internet resources) to know how to solve any problem. If the DSL author desires, they can even elide over the use of Ruby and document Ruby's syntax as part of their language. There's no requirement to trumpet the use of Ruby in your DSL.

Ruby's only contribution to DSL parsing is a function called instance_exec(). It is the magic sauce that makes a code block act as if all the functions are method calls on the object of our choosing (vs whatever object is appropriately in scope). It's also extremely low-level, which is an obstacle to DSL authors.

The docile gem (aka, a Ruby package) provides a very nice way of mapping classes (and their objects) to different levels while enhancing the error-handling. It's definitely more usable than instance_exec() on its own. But, you still have to know far more about Ruby classes and objects that a DSL author should. It also doesn't provide any facilities for validation or production, leaving those as exercises for the reader.

The dsl_maker gem wraps docile with a more explicit way of declaring the DSL structure and validation. In essence, it provides a quasi-DSL for declaring DSL parsing and validation rules. It also allows the DSL author to work with concepts that map more closely to DSL creation, such as types and nested structures without having to maintain the mapping between DSL nesting and Ruby classes.

Over the next few posts, I'm going to walk through the creation of a non-trivial DSL designed to describe how a package should be created. I will discuss creating and maintaining the parser, validator, and executor. I will also discuss distribution, testing, and all the other aspects of a good software project and how those things are handled within a DSL. You can follow along at the packager GitHub repository.

prev

next