The Packager DSL - The second user story

1. DSLs - An Overview
2. The Packager DSL - The first user story
3. The Packager DSL - The second user story

Our main user is an odd person. So far, all we've made is a DSL that can create empty packages with a name and a version. No files, no dependencies, no before/after scripts - nothing. But, instead of asking for anything useful, the first response we get is "Huh - I didn't think 'abcd' was a legal version number." Some people just think in weird ways.

Our second user story:

Throw an error whenever the version isn't an acceptable version string by whatever criteria are normally employed.

Before we dig into the user story itself, let's talk about why this isn't a "bug" or a "defect" or any of the other terms normally bandied about whenever deployed software doesn't meet user expectations. Every time the user asks us to change something, it doesn't matter whether we call it a "bug", "defect", "enhancement", or any other word. It's still a change to the system as deployed. Underneath all the fancy words, we need to treat every single change with the same processes. Bugfixes don't get a special pass to production. Hotfixes don't get a special pass to production. Everything is "just a change", nothing less and nothing more.

In addition, the "defect" wasn't in our implementation. It was in the first user story if it was anywhere. The first user story didn't provide any restrictions on the version string, so we didn't place any upon it. And that was correct - you should never do more than the user story requires. If you think that a user story is incomplete in its description, you should go back to the user and negotiate the story. Otherwise, the user doesn't know what they're going to receive. Even worse, you might add something to the story that the user does not want.

Really, this shouldn't be considered a defect in specification, either. That concept assumes an all-knowing specifier that is able to lay out fully-formed and fully-correct specifications that never need updating. Which is ridiculous on its face. No-one can possibly be that person and no-one should ever be forced to try. This much tighter feedback loop between specification to production to next specification is one of the key concepts behind Agile development. (The original name for devops was agile systems administration or agile operations.)

All of this makes sense. When you first conceive of a thing, you have a vague idea of how it should work. So, you make the smallest possible thing that could work and use it. While you start with a few ideas of where to go next, the moment you start using it, you realize other things that you never thought of. All of them (older ideas and newer realizations) become user stories and we (the developers and the users together) agree on what each story means, then prioritizes them in collaboration. Maybe the user wants X, but the developers point out Y would be really quick to do, so everyone agrees to do Y first to get it out of the way. It's an ongoing conversation, not a series of dictates.

All of which leads us back to our user story about bad version strings. The first question I have is "what makes a good or bad version string?" The answer is "whatever criteria are normally employed". That means it's up to us to come up with a first pass. And, given that we're building this in a Ruby world, the easiest thing would be to see what Ruby does.

Ruby's interface with version strings would be in its packages - gems. Since everything in Ruby is an object, then we would look at Gem and its version-handling class Gem::Version. Reading through that documentation, it looks like the Ruby community has given a lot of thought to the issue. More thought than I would have realized was necessary, but it's good stuff. More importantly, if we use Gem::Version to do our version string validation, then we have a ready-documented worldview of how we expect version strings to be used.

Granted, we will have to conform to whatever the package formats our users will want to generate require. FreeBSD may require something different from RedHat and maybe neither is exactly what Gem::Version allows. At that point, we'll have failing tests we can write from user stories saying things like "For Debian, allow this and disallow that." For now, let's start with throwing an error on values like "bad" (and "good"). Anything more than that will be another user story.

As always, the first thing is to write a test. Because we can easily imagine needing to add more tests for this as we get more user stories, let's make a new file at spec/dsl/version_spec.rb. That way, we have a place to add more variations.

describe Packager::DSL do
    context "has version strings that" do
        it "rejects just letters" do
            expect {
                Packager::DSL.execute_dsl {
                    package {
                        name 'foo'
                        version 'abcd'
                        type 'whatever'
                    }
                }
            }.to raise("'abcd' is not a legal version string")
        end
    end
end

Once we have our failing test, let's think about how to fix this. We have three possible places to put this validation. We may even choose to put pieces of it in multiple places.

1. The first is one we've already done - adding a validation to the :package entrypoint. That solution is good for doing validations that require knowing everything about the package, such as the version string and the type together.
2. The second is to add a Packager::Validator class, similar to the DSL and Executor classes we already have. This is most useful for doing validation of the entire DSL file, for example if multiple packages need to be coordinated.
3. The third is to create a new type, similar to the String coercion type we're currently using for the version.

Because it's the simplest and is sufficient for the user story, let's go with option #3. I'm pretty sure that, over time, we'll need to exercise option #1 as well, and possibly even #2. But, YAGNI. If we have to change it, we will. That's the beauty of well-built software - it's cheap and easy to change as needed.

class Packager::DSL < DSL::Maker
    add_type(VersionString = {}) do |attr_name, *args|
        unless args.empty?
            begin
                ___set(attr_name, Gem::Version.new(args[0]).to_s)
            rescue ArgumentError
                raise "'#{args[0]}' is not a legal version string" 
            end
        end
        ___get(attr_name)
    end

    add_entrypoint(:package, {
        ...,
        :version => VersionString,
        ...,
    }) ...
end

Note how we use Gem::Version to do the validation, but we don't save it as a Gem::Version object. We could keep the value as such, but there's no real reason (yet) to do so. ___get() and ___set() (with three underscores each) are provided by DSL::Maker to do the sets and gets. The attr_name is provided for us. So, if we wanted, we could reuse this type for many different attributes.

We could add more tests, documenting exactly what we've done. But, I'm happy with this. Ship it!

There's something else we haven't discussed about this option in particular. Type validations happen immediately when the item is set. Since values can be reused within a definition, they are safely reusable. For example (once we have defined how to include files in our packages), we could have something like:

package {
    name 'some-product'
    version '1.2.44'
    files {
        source "/downloads/vendor/#{name}/#{version}/*"
        dest "/place/for/files"
    }
}

If we defer validation until the end, we aren't sure we've caught all the places an invalid value may have been used in another value. This is why we attempt a "defense in depth", a concept normally seen in infosec circles. We want to make sure the version value is as correct as we can make it with just the version string. Then, later, we want to make sure it's even more correct once we can correlate it with the package format (assuming we ever get a user story asking us to do so).