Creating the Packager DSL - Integration

1. Why use a DSL?
2. Why create your own DSL?
3. What makes a good DSL?
4. Creating your own DSL - Parsing
5. Creating your own DSL - Parsing (with Ruby)
6. Creating the Packager DSL - Initial steps
7. Creating the Packager DSL - First feature
8. Creating the Packager DSL - The executor
9. Creating the Packager DSL - The CLI
10. Creating the Packager DSL - Integration

Our user story:

I want to run a script, passing in the name of my DSL file. This should create an empty package by specifying the name, version, and package format. If any of them are missing, print an error message and stop. Otherwise, an empty package of the requested format should be created in the directory I am in.

Our progress:

• We can parse the DSL into a Struct. We can handle name, version, and package format. If any of them are missing, we raise an appropriate error message.
• We can create a package from the parsed DSL
• We have a script the executes everything

So, we're done, right? Not quite. We have no idea if it actually works. Sure, you can run it manually and see the script works. But, that's useful only when someone remembers to do it and remembers how to interpret it. Much better is a test that runs every single time in our CI server (so no-one has to remember to do it) and knows how to interpret itself. In other words, a spec.

We have tests for each of the pieces in isolation. That was deliberate - we want to make sure that each piece works without involving more complexity than is necessary. But, our user story doesn't care about that. It says the user wants to execute a script and get their package. The user doesn't care about the Parser vs. the Executor (or anything else we've written). Those distinctions are for us, the developers, to accommodate the inevitable changes that will happen. A developer's job (and raison d'etre, frankly) is to create change. Without change, a developer has nothing to do. So, we organize our project to make it as easy as possible to make change.

But, at the end of the day, it's the integration of these pieces that matters. So, we need to create end-to-end integration tests that show how all the pieces will work together. Where the unit tests we've written so far test the inner workings of each unit, the integration tests will test the coordination of all the units together. We are interested in checking that the output of one piece fits the expected input of the next piece.

Said another way, our unit tests should provide 100% code coverage (which you can see with rspec spec/{cli,dsl,executor}). The integration tests will provide 100% user-expectation coverage.

As always, first thing we do is write a test. We have a subdirectory in spec/ for the unit tests for each component. Let's add another one called spec/integration with a file called empty_spec.rb and contents of

require 'tempfile'

describe "Packager integration" do
    let(:dsl_file) { Tempfile.new('foo').path }
    it "creates an empty package" do
        append_to_file(dsl_file, "
            package {
                name 'foo'
                version '0.0.1'
                package_format 'dir'
            }
        ")

        Packager::CLI.start('execute', dsl_file)

        expect(File).to exist('foo.dir')
        expect(Dir['foo.dir/*'].empty?).to be(true)
    end
end

Take a file with an empty package definition, create a package in the directory we're in, then verify. Seems pretty simple. We immediately run into a problem - no package is created. If you remember back when when we were creating the executor, we never actually call out to FPM. It's relatively simple to add in an #execute method to the Executor which does a system() call. That should make this test pass.

But, that's not enough. After you run it, do a git status. You'll immediately see another problem - the package was created in the directory you ran rspec in. Which sucks. But, it's fixable.

In the same way we have tempfiles, we have temp directories. Sysadmins used to bash are familiar with mktemp and mktemp -d. Ruby has Tempfile and Dir.mktmpdir, respectively. So, let's run the test within a temporary directory - that should solve the problem.

require 'tempfile'
require 'tmpdir'

describe "Packager integration" do
    let(:dsl_file) { Tempfile.new('foo').path }

    it "creates an empty package" do
        Dir.mktmpdir do |tempdir|
            Dir.chdir(tempdir) do
                # Rest of test
            end
        end
    end
end

That keeps the mess out of the main directory. Commit and push.

Though, when I look at what's been written, the tempdir handling is both manually-done (the author has to remember to do it every time) and creates two more levels of indentation. The manual part means it's possible for someone to screw up. The indentations part means that it's harder to read what's happening - there's boilerplate in every test. Which is somewhat ironic given that the whole point of this process is to create a DSL - removing the boilerplate. We can do better. Red-Green-Refactor doesn't just apply to the code. (Or, put another way, tests are also code.)

RSpec allows us to do things before-or-after all-or-each of the specs in a context. Let's take advantage of this to ensure that every integration test will always happen within a tempdir.

require 'fileutils'
require 'tempfile'
require 'tmpdir'

describe "Packager integration" do
    let(:dsl_file) { Tempfile.new('foo').path }

    let(:workdir) { Dir.mktmpdir }
    before(:all)  { @orig_dir = Dir.pwd }
    before(:each) { Dir.chdir workdir }
    after(:each) {
        Dir.chdir @orig_dir
        FileUtils.remove_entry_secure workdir
    }

    it "creates an empty package" do
        # Rest of test
    end
end

A few notes here.

1. When we used the block versions of Dir.mktmpdir and Dir.chdir, the block cleaned up whatever we did (e.g., changed back to the original directory). When we use the direct invocation, we have to do our own cleanup.
2. before(:all) will always run before before(:each) (guaranteed by RSpec).
3. We don't want to use let() for the original directory. let() is lazy, meaning it only gets set the first time it's invoked. Instead, we set an attribute of the test (as provided helpfully to us by RSpec).
1. We could have used let!() instead (which is eager), but it's too easy to overlook the !, so I don't like to use it. Sometimes, subtlety is overly so.
4. Tests should be runnable in any order. And this includes all the other tests in all the other spec files. You should never assume that any two tests will ever run in a specific order or even that any test will run in a specific test run. So, we always make sure to change directory back to the original directory (whatever that was). There's nothing here that assumes anything about the setup.
5. FileUtils has many ways to remove something. #remove_entry_secure is the most conservative, so the best for something that needs to be accurate more than it needs to be fast.
6. We need to leave the tempdir that we're in before trying to remove it. On some OSes, the OS will refuse to remove a directory if a process has it as its working directory.

prev