enjoys technical writing, and a cheeky drink 🥃

How I added CircleCI Runner support for Terraform provider

I have released support for the CircleCI Runner resource-class and token in my unofficial Terraform provider for CircleCI, as per v0.10.3.

Developers can now manage the provisioning (and teardown) of CircleCI self-hosted Runners within Terraform.
You can explore an example here.

This was a fun challenge, and I wanted to document my journey on this work.


Unlike other resources, self-hosted runners are not manageable under the official V2 API.
Developers had to use the CircleCI CLI to manage resource-classes and tokens instead.

To port this into my Terraform provider, I was hoping there was a HTTP API available. This way, I can continue using my approach of abstracting the HTTP API away to a Go SDK.

I assumed (wrongly) the CLI was using GraphQL under the hood for Runner operations, as with many others (e.g., for Orbs).

Digging into the source-code, I then realized Runner resource-classes and tokens can be managed via a HTTP API;
It was simply not publicly documented, yet.


After the legwork mentioned above, I tested the HTTP APIs with an OpenAPI (Swagger 2.0) document.
This enabled me to generate a Go SDK for Runner APIs.

Why are the Go SDKs separated? Wouldn't it be easier to keep it all in one?

That was something I mulled over for some time indeed.
I have documented my reasons for keeping them separate.


With the Go SDK published, I “simply” have to then expose the Runner resource-classes and tokens in the Terraform provider codebase.

The main work was done within a pull request here.
This also included acceptance tests, and examples.

Ultimately, I noted self-hosted (machine) runners are also available for CircleCI's Server customers (i.e., self-hosted CircleCI).
We would want to extend and ensure this addition can be used by platform teams using CircleCI Server.

It turned out that the Runner API:

Thankfully, this was a quick patch.
I was also able to verify this fix against my own CircleCI Server instance.

Things I learnt

This feature was satisfying for me to build, and I had many learning points along the journey.

  1. Read the code: This feature would not have been completed if I did not dig deeper into the publicly-available source code 📖

  2. Keep trying: I am still learning (and failing) at Go. However, I think it is important to keep trying and learning. Keeping this source code open-sourced forces me to keep myself honest too about my lack of knowledge. For fellow engineers out there, let's keep at it! 🤓

#terraform #circleci #runner #go

buy Kelvin a cup of coffee ☕