Using Ed

15 May 2019

Quick! You have about 100 Jenkinsfiles to edit. You started editing some manually but you have found a pattern. How do you automate this but also share it with everyone else? Do you write it in python? Which version? What about some macros for a text editor like vim? What about some VS Code script thing?

This is an old problem with old solutions. A classic that we all have installed is ed. Believe it or not, this is not some minimal editor that old diehard real programmers use. Like a lot of classic UNIX utilities it is really a small program that implements a ‘little language’ (or a DSL, if you like). In the case of ed it implements a text editing language, and one of the first ever to use regular expressions. The syntax of the language is similar to the more popular sed, which isn’t surprising when you think that sed is “streaming ed”.

Let’s get started - we have a lot of files to edit. We have a Jenkinsfile which starts like this:

pipeline {
  agent {
    label 'sbt-1-2-8'
  }
}

You want it to look like this:

pipeline {
  agent {
    kubernetes {
      label 'sbt'
      defaultContainer 'sbt'
    }
  }
}

Don’t read the code. Just look at it from a distance. What we want to do is replace all text in the agent block. If we were using our mouse we would click and drag from the opening { to the closing } and start typing. But of course we’re not going to automate like that. ed has a command, c which replaces text between two addresses. The syntax is:

addr1,addr2 c

The simplest thing we can do is use line numbers. In the above example we would run:

3,3 c

Changes from line 3 to line 3 (i.e. changes line 3). ed reads input until it reads a literal .. Our editing session would be:

3,3 c
    kubernetes {
      label 'sbt'
      defaultContainer 'sbt'
    }
.

We can save the above snippet to a file, say k8sedit, and we have our script that we can share with others. ed reads commands from standard input. Now anyone can edit a Jenkinsfile like so:

ed Jenkinsfile < k8sedit

Or in a basic shell script across many files:

ed */Jenkinsfile < k8sedit

Refactor done? Almost.

Imagine that we have a comment at the top of Jenkinsfile. Now our script is wrong; the agent block is not between lines 2 and 4 any more. Instead of using line numbers for the addresses, we can use regular expressions. We will write the commands first then explain it afterwards piece by piece. The command is:

/agent/,/}/c

From the line containing ‘agent’ to the line containing ‘}’, change the contents. Our new script:

/agent/,/}/c
  agent {
    kubernetes {
      label 'sbt'
      defaultContainer 'sbt'
    }
  }
.

This isn’t precisely what we want. We want to change the contents of the agent block, not rewrite the whole thing. We can use the + and - characters to do what we want.

/agent/+,/^  }/-c
    kubernetes {
      label 'sbt'
      defaultContainer 'sbt'
    }
.

From one line after (+) line containing ‘agent’, to one line before (-) the line beginning with ‘ }’ (including two spaces of indentation), change the contents. We have squeezed our selection to be between the beginning and end of the agent block.

As you refactor Jenkinsfile you find that an entire block called ‘environment’ can be deleted. It looks like this:

environment {
  VERSION = "1.2.8"
  UPSTREAM = "example.com"
}

We can use ed’s ‘d’ command to get rid of the whole thing. Before looking at the command, we can think ahead of what we want to do. We want to delete the entire environment block; i.e.

/^  environment {/
/environment/,/^  }/d

We have two commands. First we move the cursor down to the line which begins with environment {. Then we delete everything from the line containing ‘environment’, to the line beginning with ‘ }’.

One more example. We need to add a new stage called ‘publish’ to Jenkinsfile. It needs to be the last stage of the pipeline.

pipeline {
  agent {
    ...
  }
  stages {
    stage('build') {
      ...
    }
    stage('test') {
      ...
    }
  }
}

We can express this in ed like so:

?stage
/stage/,/^    }/a
    stage('publish') {
      ...
    }
.

Search from the end of the file (?) a line matching ‘stage’ (i.e. the last stage in the file). From the beginning of that stage block to its closing brace, append (a) text.

Our entire script is not too long or difficult to understand now that we understand what we want to do to our file.

/agent/+,/^  }/-c
    kubernetes {
      label 'sbt'
      defaultContainer 'sbt'
    }
.
/^  environment {/
/environment/,/^  }/d
?stage
/stage/,/^    }/a
    stage('publish') {
      ...
    }
.
w

One final command: w writes the file to disk.

We can share this script easily; it’s just a few bytes. We could keep it in version control and update it if we need it again. It is very portable; it can run on decades old original UNIX machines, a raspberry pi without any packages installed, or in an alpine linux container in a kubernetes cluster using the ed from busybox.

There are some caveats to using this little text editing language. The first is that this solution requires consistent indentation of code (but this is already done by a linter, right?). Secondly, like with any use of regular expressions, it is important to not go overboard. Keep it simple and be clear about the problem to be solved.

Using ed might seem a bit obscure, but programatic editing of text is something that we do every day at work. When was the last time you ran git diff or git pull? It relies on the diff utility. That utility writes ed scripts.