Creating the Packager DSL - Initial steps

1. Why use a DSL?
2. Why create your own DSL?
3. What makes a good DSL?
4. Creating your own DSL - Parsing
5. Creating your own DSL - Parsing (with Ruby)
6. Creating the Packager DSL - Initial steps
7. Creating the Packager DSL - First feature

The Packager DSL is, first and foremost, a software development project. In order for a software project to function properly, there's a lot of scaffolding that needs to be set up. This post is about that. (Please feel free to skip it if you are comfortable setting up a Ruby project.) When this post is completed, we will have a full walking skeleton. We can then add each new feature quickly. We'll stop when we're just about to add the first DSL-ish feature.

If you're creating your own DSL with your own project name, just put it everywhere you see packager and Packager accordingly.

Repository choice

Distributed version control systems (aka, DVCS) provide significant capabilities in terms of managing developer contributions over previous centralized systems (such as Subversion and CVS). Namely, Git (and Mercurial or Bazaar) have extremely lightweight branches and very good merging strategies. There's a ton of good work on the net about choosing one of these systems and how to use it.

I use Git. For my opensource work, I use GitHub. If I'm within an enterprise, Gitlab or Stash are excellent repository management solutions. The minimum scaffolding for any project in Git is:

• .gitignore file
• These files are never to be checked into the repository. Usually intermediate files for various developer purposes (coverage, build artifacts, etc). The contents of this file are usually language-specific.
• .gitattributes file
• Ensure line-endings are consistent between Windows, Linux, and OSX.

Repository scaffolding

The minimum scaffolding for a Ruby project is:

• lib/ directory
• Where our library code lives. Leave this empty (for now).
• spec/ directory
• Where our tests (or specifications) live. Leave this empty (for now).
• bin/ directory
• Where our executables live. Leave this empty (for now).
• .rspec file
• Default options for rspec (Ruby's specification runner).
• Rakefile file
• Ruby's version of Makefile
• packager.gemspec file
• How to package our project.
• Gemfile file
• Dependency management and repository location. I write mine so that it will delegate to the gemspec file. I hate repeating myself, especially when I don't have to.

You are welcome to copy any or all of these files from the ruby-packager project and edit them accordingly or create them yourself. There is ridiculous amounts of documentation on this process and each of the tools (Rake, Rspec, Gem, and Bundler) all over the web. Ruby, moreso than most software communities, has made a virtue of clear, clean, and comprehensive documentation.

You also need to have installed Bundler using gem install bundler. Bundler will then make sure your dependencies are always up-to-date with the simple bundle install command. I tend to install RVM first, just to make sure I have can upgrade (or downgrade!) my version of Ruby as needed.

At this point, go ahead and add all of these files to your checkout and commit. I like using the message "Create initial commit". (If you use git, the empty lib/ and spec/ directories won't be added. Don't worry - they will be added before we're finished with Day 1.)

Note: If your DSL is for use within an enterprise, please make sure that you know where to download and install your gems from. Depending on the enterprise, you probably already have an internal gems repository and a standard process for requesting new gems and new versions. You can set the location of that internal gems repository within the Gemfile instead of https://rubygems.org. If you have to set a proxy in order to reach rubygems.org, please talk your IT administrator first.

Documentation

It may seem odd to consider documentation before writing anything to document (and we will revisit this when we have an actual API to document), but there are two documentation files we should create right away:

• README.md
• This is the file GitHub (and most other repository systems) will display when someone goes to the repository. This should describe your project and give a basic overview of how to use it.
• Changes / Changelog
• Development is the act of changing software. Let's document what we've changed and when. I prefer the name Changes, but several other names will work.

I prefer to use Markdown (specifically, GitHub-flavored Markdown) in my README files. Hence, the .md suffix. Please use whatever markup language you prefer and which will work within your repository system.

Your changelog should be a reverse-sorted (newest-first) list of all the changes grouped by version. The goal is to provide a way for users to determine when something was changed and why. So, wherever possible, you should provide a link to the issue or task that describes what changed in detail. The best changelog is just a list of versions with issue descriptions linking to the issue.

Finally, git add . && git commit -am "Create documentation stubs"

SDLC

Before we write any software, let's talk quickly about how we're going to get the changes into the software and out the door. While the first set of changes are going to be driven by what we want, we definitely want a place for our users to open requests. GitHub provides a very simple issue tracker with every project, so packager will use that. If you're within an enterprise, you probably already have a bug tracker to use. If you don't, I recommend Jira (and the rest of the Atlassian suite) if you're a larger organization and Trac if you're not.

Either way, every change to the project should have an associated issue in your issue tracker. That way, there's a description of why the change was made associated with the blob that is the change.

Speaking of changes, use pull requests wherever possible. Pull requests do two very important things:

1. They promote code reviews, the single strongest guard against bugs.
2. They make it easy to have a blob that is the change for an issue.

I will confess, however, that when I'm working by myself on a project, I tend to commit to master without creating issues. But, I like knowing that the process can be easily changed to accomodate more than one developer.

Release process

Our release process will be pretty simple - we'll use the gem format. We can add a couple lines to our Rakefile and Ruby will handle all the work for us.

require 'rubygems/tasks'
Gem::Tasks.new

We will also need to add it to our gemspec

s.add_development_dependency 'rubygems-tasks', '~> 0'

and bundle install. That provides us with the release Rake task (among others). Please read the documentation for what else it provides and what it does.

Note: If your DSL is for use within an enterprise, please make sure that you know where and how you are going to release your gem. This will likely be the place you will download your dependencies from, but not always. Please talk your IT administrator first.

First passing spec

I am a strong proponent of TDD. Since we're starting from scratch, let's begin as we mean to continue. So, first, we add a spec/spec_helper.rb file that looks like:

require 'simplecov'
SimpleCov.configure do
    add_filter '/spec/'
    add_filter '/vendor/'
    minimum_coverage 100
    refuse_coverage_drop
end
SimpleCov.start

require 'packager'

Then, add a spec/first_spec.rb file with:

describe "DSL" do
    it "can compile"
        expect(true).to be(true)
    end
end

(This is a throwaway test, and that's okay. We will get rid of it when we have something better to test.)

rake spec fails right away because simplecov isn't installed. You'll need to add simplecov to the packager.gemspec and run bundle install to install simplecov.

We haven't created lib/packager.rb yet, so when we execute rake spec, it will fail (compile error). So, let's create lib/packager.rb with:

class Packager
end

rake spec now passes. git add . && git commit -am "Create initial test" to mark the success.

We also have 100% test coverage from the start. While this isn't a silver bullet that cures cancer, it does tell us when we're writing code that may not always work like we think it will. By forcing every line of code to be executed at least once by something our test suite (which is what 100% coverage guarantees), we are forced to write at least one test to reach every nook and cranny. Hopefully, we'll be adult enough to recognize when there's a use-case that isn't tested, even if the code under question is executed.

Continuous Integration

Finally, before we start adding actual features, let's end with continuous integration. If you're on GitHub, then use Travis. Just copy the one in ruby-packager and it will work properly (JRuby needs some special handling for code coverage). Travis will even run the test suite on pull requests and note the success (or lack thereof) right there. In order to enable Travis to run against your GitHub repository, you will need to register with Travis and point-and-click on its site to set things up. There is plenty of documentation (including StackOverflow Q&A) on the process.

Otherwise, talk to your devops or automation team in order to set up integration with Jenkins, Bamboo, or whatever you're using in your enterprise. Whatever you do, it should be set up to run the whole test suite on every single push on every single branch. More importantly, it should act as a veto for pull requests (if that's supported by your tooling).

Summary

It may not seem like we've actually done anything, but we've done quite a lot. A development project isn't about writing code (though that's a necessary part). It's about managing requests for change and delivering them in a sane and predictable fashion. Everything we've done here is necessary to support exactly that.

prev

next