Speed Up Your Workflow with Prebuilds

22 Aug 2023 - @bamurtaugh @craiglpeters

Getting dev containers up and running for your projects is exciting - you’ve unlocked environments that include all the dependencies your projects need to run, and you can spend so much more time on coding rather than configuration.

Once your dev container has everything it needs, you might start thinking more about ways to optimize it. For instance, it might take a while to build. Maybe it takes 5 minutes. Maybe it takes an hour!

You can get back to working fast and productively after that initial container build, but what if you need to work on another machine and build the container again? Or what if some of your teammates want to use the container on their machines and will need to build it too? It’d be great to make the build time faster for everyone, every time.

After configuring your dev container, a great next step is to prebuild your image.

In this guide, we’ll explore what it means to prebuild an image and the benefits of doing so, such as speeding up your workflow, simplifying your environment, and pinning to specific versions of tools.

We have a variety of tools designed to help you with prebuilds. In this guide, we’ll explore two different repos as examples of how our team uses different combinations of these tools:

The prebuilt image for the Kubernetes repo developed by one of our spec maintainers Craig
The prebuilt images we host in the devcontainers/images repo as part of the dev container spec

What is prebuilding?

We should first define: What is prebuilding?

If you’re already using dev containers, you’re likely already familiar with the idea of building a container, where you package everything your app needs to run into a single unit.

You need to build your container once it has all the dependencies it needs, and rebuild anytime you add new dependencies. Since you may not need to rebuild often, it might be alright if it takes a while for that initial build. But if you or your teammates need to use that container on another machine, you’ll need to wait for it to build again in those new environments.

Note: The dev container CLI doc is another great resource on prebuilding.

Prebuilt Codespaces

You may have heard (or will hear about) GitHub Codespaces prebuilds. Codespaces prebuilds are similar to prebuilt container images, with some additional focus on the other code in your repo.

GitHub Codespaces prebuilds help to speed up the creation of new codespaces for large or complex repositories. A prebuild assembles the main components of a codespace for a particular combination of repository, branch, and devcontainer.json file.

By default, whenever you push changes to your repository, GitHub Codespaces uses GitHub Actions to automatically update your prebuilds.

You can learn more about codespaces prebuilds and how to manage them in the codespaces docs.

How do I prebuild my image?

We try to make prebuilding an image and using a prebuilt image as easy as possible. Let’s walk through the couple of steps to get started.

Prebuilding an image:

Install the Dev Container CLI:
```
   npm install -g @devcontainers/cli
```
Build your image and push it to a container registry (like the Azure Container Registry, GitHub Container Registry, or Docker Hub):
```
   devcontainer build --workspace-folder . --push true --image-name <my_image_name>:<optional_image_version>
```
You can automate pre-building your image by scheduling the build using a DevOps or continuous integration (CI) service like GitHub Actions. We’ve created a GitHub Action and Azure DevOps task to help with this.

Using a prebuilt image:

Determine the published URL of the prebuilt image you want to use
Reference it in your devcontainer.json, Dockerfile, or Docker Compose file
- In our previous guide on “Using Images, Dockerfiles, and Docker Compose,” we also showed how you can use prebuilt images, Dockerfiles, or Docker Compose files for your configurations

Prebuild Examples

As mentioned above, let’s walk through a couple examples of these steps, one using Craig’s Kubernetes repo, and the other using our devcontainers/images repo.

Kubernetes

It’s a fork of the main Kubernetes repo and contributes a prebuilt dev container for use in the main Kubernetes repo or any other forks
The dev container it’s prebuilding is defined in the .github/.devcontainer folder
Any time a change is made to the dev container, the repo currently uses the dev container GitHub Action to build the image and push it to GHCR
- You can check out its latest prebuilt image in the Packages tab of its GitHub Repo. In this tab, you can see its GHCR URL is ghcr.io/craiglpeters/kubernetes-devcontainer:latest
The main Kubernetes repo and any fork of it can now define a .devcontainer folder and reference this prebuilt image through: "image": "ghcr.io/craiglpeters/kubernetes-devcontainer:latest"

Dev container spec images

This repo prebuilds a variety of dev containers, each of which is defined in their individual folders in the src folder
- As an example, the Python image is defined in the src/python/.devcontainer folder
Any time a change is made to the dev container, the repo uses a GitHub Action to build the image and push it to MCR
- Using the Python image as an example again, its MCR URL is mcr.microsoft.com/devcontainers/python
Any projects can now reference this prebuilt image through: "image": "mcr.microsoft.com/devcontainers/python"

Where do the dependencies come from?

If your devcontainer.json is as simple as just an image property referencing a prebuilt image, you may wonder: How can I tell what dependencies will be installed for my project? And how can I modify them?

Let’s walk through the Kubernetes prebuild as an example of how you can determine which dependencies are installed and where:

Start at your end user dev container
- We start at the .devcontainer/devcontainer.json designed for end use in the Kubernetes repo and other forks of it
  - It sets a few properties, such as hostRequirements, onCreateCommand, and otherPortsAttributes
  - We see it references a prebuilt image, which will include dependencies that don’t need to be explicitly mentioned in this end user dev container. Let’s next go explore the dev container defining this prebuilt image
Explore the dev container defining your prebuilt image
- We next open the config that defines the prebuilt image. This is contained in the .github/.devcontainer folder
- We see there’s a devcontainer.json. It’s much more detailed than the end user dev container we explored above and includes a variety of Features
Explore content in the prebuilt dev container’s config
- Each Feature defines additional functionality
- We can explore what each of them installs in their associated repo. Most appear to be defined in the devcontainers/features repo as part of the dev container spec
Modify and rebuild as desired
- If I’d like to add more content to my dev container, I can either modify my end user dev container (i.e. the one designed for the main Kubernetes repo), or modify the config defining the prebuilt image (i.e. the content in Craig’s dev container)
  - For universal changes that anyone using the prebuilt image should get, update the prebuilt image
  - For more project or user specific changes (i.e. a language I need in my project but other forks won’t necessarily need, or user settings I prefer for my editor environment), update the end user dev container
- Features are a great way to add dependencies in a clear, easily packaged way

Benefits

There are a variety of benefits (some of which we’ve already explored) to creating and using prebuilt images:

Faster container startup
- Pull an already built dev container config rather than having to build it freshly on any new machine
Simpler configuration
- Your devcontainer.json can be as simple as just an image property
Pin to a specific version of tools
- This can improve supply-chain security and avoid breaks

Tips and Tricks

We explored the prebuilt images we host as part of the spec in devcontainers/images. These can form a great base for other dev containers you’d like to create for more complex scenarios
The spec has a concept of Development container “Templates” which are source files packaged together that encode configuration for a complete development environment
- A Template may be as simple as a devcontainer.json referencing a prebuilt image, and a devcontainer-template.json
- You can learn more about Templates in our Templates documentation
- You can adopt and iterate on existing Templates from the spec and community, or you can create and share your own
You can include Dev Container configuration and Feature metadata in prebuilt images via image labels. This makes the image self-contained since these settings are automatically picked up when the image is referenced - whether directly, in a FROM in a referenced Dockerfile, or in a Docker Compose file. You can learn more in our reference docs
You can use multi-stage Dockerfiles to create a prod container from your dev container
- You’d typically start with your prod image, then add to it
- Features provide a quick way to add development and CI specific layers that you wouldn’t use in production
- For more information and an example, check out our discussion on multi-stage builds

Feedback and Closing

We hope this guide will help you optimize your dev container workflows! We can’t wait to hear your tips, tricks, and feedback. How are you prebuilding your images? Would anything in the spec or tooling make the process easier for you?

If you haven’t already, we recommend joining our dev container community Slack channel where you can connect with the dev container spec maintainers and community at large. If you have any feature requests or experience any issues as you use the above tools, please feel free to also open an issue in the corresponding repo in the dev containers org on GitHub.