Charlie Stross Predicts 2017

Charlie Stross has written a series of blog posts which extrapolates current events into the next 12 months. It’s a glorious imagining of the future, but I’m afraid it doesn’t end well for humanity…

The Hosting of exubero.com has Changed

This is just an administrative announcement: I have moved exubero.com to be hosted by wordpress.com. It was previously hosted on a EC2 server and database. At the time I moved exubero.com to AWS, I felt it was an interesting project to learn the AWS tools and infrastructure. In that sense, it was a good way to learn. However, years later I’m running months behind on keeping everything updated with security patches and paying too much for AWS services. Given that 90% of my website was already running under WordPress, I decided to bite the bullet and let WordPress.com manage my site for me, allowing me to kill my EC2 server and save some money.

The transfer wasn’t quite painless: some of the URLs have changed. All the content is still available, but if you have any bookmarks, they may need to be updated, and the RSS feed location has changed too, so you may need to update your subscription.

Software Development is an Engineering Discipline

There is a view that software development is a craft; that “Software Craftsmanship” is a better metaphor for software development than is engineering or science. I’m going to argue against this view: software development is an engineering discipline, and relies strongly on scientific principles. Firstly, what is an engineering? Here’s one definition of the engineering method:

The Engineering Method is the use of heuristics to cause the best change in a poorly understood or uncertain situation within the available resources. – Billy Vaughn Koen

What heuristics? In general, they look something like this:

Engineering method

(from the article “Comparing the Engineering Design Process and the Scientific Method“)

The first thing to notice is the feedback loops here. The steps brainstorm ⇒ develop ⇒ test corresponds to the TDD cycle in software development:

TDD cycle(from “Growing Object-Oriented Software, Guided by Tests“)

The difference between the engineering design process with physical products and TDD in software is mainly the speed of the feedback loop. A skilled developer can often go through the red-green-refactor loop in less than 5 minutes, as there is no physical materials to manipulate. When you then add in the steps specify requirements ⇒ … ⇒ meets requirements around the TDD cycle, you get the ATDD cycle: ATDD Cycle

Depending upon the size of feature that the the acceptance test describes, this could take anywhere from an hour to a week, though high-performing teams learn that quicker feedback is critical to quality.

Zooming out, there is a larger feedback loop surrounding the ATDD loop: the lead time between new software capabilities being prioritised and the capability being used by an end user. The DevOps movement is built on the focus on the software and organisational structures required to get software into the hands of end users as quickly as possible.

The size of the feedback loop is probably the most critical part of successful engineering. Consider, for example, the first human powered flight across the English Channel. The Kremer Prize was established in 1959 to give some motivation to show that human powered flight was both possible, as able to go long distances.

The challenge was picked up by many peoples and teams, but there was very little progress until the problem was examined by Paul MacCready, who noted that the engineers were trying to solve the wrong problem.

MacCready’s insight was that everyone working on solving human-powered flight would spend upwards of a year building an airplane on conjecture and theory without the grounding of empirical tests. Triumphantly, they’d complete their plane and wheel it out for a test flight. Minutes later, a years worth of work would smash into the ground.

The problem was the process itself… He came up with a new problem that he set out to solve: how can you build a plane that could be rebuilt in hours not months. And he did. He built a plane with Mylar, aluminium tubing, and wire.

After two and a half years, MacCready team successfully flew the Gossamer Albatross II across the English Channel in just under 3 hours.

Gossamer Albatross

(Gossamer Albatross II in flight, from Wikimedia Commons, public domain)

Compare this feedback loop to a more traditional engineering discipline based upon something less malleable than software: traffic engineering. This is a branch of civil engineering that uses engineering techniques to achieve the safe and efficient movement of people and goods on roadways. In this case the engineering medium is road signs, lane markings, asphalt, road size and shape. In some cases, the traffic engineer will look to move roads through entire buildings. The behaviour of traffic can be very complex, with surprising effects from interventions (think of adding a lane to a busy road, causing induced demand, causing worse traffic delays). In all these examples, the feedback loop for understanding if the intervention was successful can range anywhere from days to years. In order to reduce the risk of untested changes, traffic engineers must use models of the road, and run the simulation in software.

In summary: software development that makes use of feedback loops to help understand and direct the problem solution is most definitely engineering.

One question is left to ask: where does Software Craftsmanship fit in the context of software engineering? My take is that Craftsmanship is the focus on the skills that are known to be effective, a body of knowledge on the techniques that are effective. Engineering without skills won’t get you very far. Skills without engineering means that you can build the wrong thing very well. Both are required.

(This post was prompted by a discussion between Dave Farley and myself when we were both at the book launch of Infrastructure as Code by Kief Morris. A subsequent discussion with Andy Parker helped refine the topic)

Devops 101

I recently saw a reference to Devops 101 on Twitter. It looked interesting, so I bookmarked it, and forgot about it for a few weeks.

Devops 101 - Part 4

Last night I finally got around to trying the exercises – it was a bit of an eye-opener. There’s a huge amount of functionality on AWS that I’m not really up-to-date with (and I say this as a person who has been running an AWS EC2 instance for many years now).

The “101” part of the name implies that it’s a starting place for graduate developers. I would clarify to say that it’s very useful for anyone who isn’t working on infrastructure-as-code day to day. If you’re developer who is not yet embedded in a team working devopsly, give it a go.

I had never had a chance to place with the AWS command line, and leveraging that with cloudformation and ansible was an real pleasure to use.

I previously worked at a company where we had built our own stack builder tool. I never got to grips with it at the time, but I now want to go over it again.

Keybase

Well, I finally received an invitation to join keybase.io. My Keybase profile is:

https://keybase.io/tumbarumba.

I first heard about Keybase from a post by Tim Bray in March. I’ve been quite interested in crypto stuff since forever, but I’ve never been able to use it consistently in all my communications. Given the increasing surveillance issues on-line, it seems prudent to increase use of encryption, but it’s still very hard when pretty much everyone I would ever chat with can’t decrypt. Wider use of tools like GPG and usability wrappers like Keybase might improve the situation. As a first step, this is my announcement of my public key and availability to receive encrypted messages, should you so desire.

Keybase is currently still in a closed alpha. You can request to join, but it will probably take a while (it was 6 months for me). Even without Keybase, you can still use GPG directly, just retrieve my public key from https://keybase.io/tumbarumba/key.asc.

Management Training for the Twentieth Century

So, I recently took management training course. There was some useful stuff taught in the course (such as delegation, coaching, and performance management, among other things). I learnt a lot, most of it useful. I’m not here to talk about that, though. I’m here to rant…

First problem: the feedback method they were teaching was the “shit sandwich“. I thought it was known for years that the shit sandwich was shit. However, in this course this shit sandwich was still king of the dung-heap. They had videos of actors giving each other the praise-shit-praise rigmarole, and then turning around giving the thumbs up and a shit eating grin. I kid you not.

Second problem: Myers Briggs. At the time, I didn’t know better. It’s an interesting topic. I felt that I was learning a really useful tool that would help me to understand myself and others. (I’m an INTJ, thanks for asking!). However, as I recently found out, Myers Briggs is completely meaningless. That’s a really interesting article, and explains why Myers Briggs is so popular: the Forer effect.

Now, if you will excuse me, I have to send some feedback to the course administrators…

 

It’s the People, Stupid!

I’m a career technologist. Building software systems that people find useful (as possibly love) gives me a great thrill. I’ve spent more than 25 years learning many arcane bits of knowledge that makes it possible for me to contribute to the construction and maintenance of these software constructs.

Historically, I have mainly focused on technical skills and software development techniques. I tended to avoid practising “softer” skills such as communication, leadership and mentoring. In the last couple of years, I had an epiphany, and changed emphasis to include those things.

The first thing that made me reconsider was a private conversation with Jeff Fredrick (who should really tweet and blog more – he knows too much good stuff). I’ve forgotten the exact details of the conversation (it was a couple of years ago), but Jeff talked about his career development, and his deliberate choice to move into technical leadership (he’s now CTO of TIM Group), and the people skills he needed to make the transition. We contrasted that with my own career choices (mainly “keep my hands in the code”). I recall his main point was that learning the people skills made him much more effective as a technologist (in the land of the blind, the one-eyed man is king!). Essentially: it’s the people, stupid!

It didn’t change my behaviour straight away, but it made me think harder about the skills I had been neglecting, and seek out better ways of working with people. With a hat-tip to people like Bob Marshall, Ben Mitchell and Jeff Fredrick who pointed me towards the reference material, here’s the main things that I feel have contributed most to my development over the last couple of years:

I’m still working in the code, but as the above list shows, I’m working towards using empathy and humanity in my work. As always, it’s harder than it looks.