In this post we will look at GitHub’s current CI/CD capabilities.  We will see what they do well, what they don’t, and what you need to know before moving your pipelines to GitHub.

The Good

Actions

Pipelines

GitHub Actions is GitHub’s pipeline/workflow automation feature. This feature is roughly equivalent to Pipelines in Azure DevOps or Bitbucket Cloud.  It’s YAML based and is primarily used for CI/CD (continuous integration/continuous delivery or deployment) but can also be used to automated process workflows such as deleting stale branches, closing stale issues, automatically adding labels to new issues, and/or other tasks. Actions workflows are very flexible. You would be hard pressed to find a CI/CD or workflow process you couldn’t implement with Actions.

Configurability

Actions pipelines are highly configurable.  They can be configured to include and/or exclude certain branched.  They can be triggered by many different system events including PR (pull request), CI , CD, scheduled, and manual events to name a few. Pipelines can be broken up into separated jobs.  Jobs can be setup with complex rules which allow them to be run in parallel and/or following other jobs. This just scratches the surface of how versatile GitHub Actions pipelines can be.

Enterprise Quality Agents

Actions agents are really solid.  There are multiple Mac, Ubuntu and Windows cloud agent versions available.  These agents come preloaded with some of the most common build and deployment tools.  The full list of what comes preloaded on the agents can be found in the Actions Virtual Environments public GitHub project (https://github.com/actions/virtual-environments).  In some cases, you may need to add tooling such as a specific JDK or Dotnet core version.  Adding such tooling to a cloud agent at runtime can be accomplished by simply add a setup step or steps to your Actions YAML. Most tooling can be setup in under a minute.

If the GitHub hosted agents don’t meet your needs, you can also self-host your own dedicated agents.  Setting up these agents is fairly straight forward. Once you know what you are doing, setting up a new agent can be done in just a couple of minutes. There are however a couple of things you need to know before moving to self-hosted agents.  The agents installed on your self-hosted machines will automatically get updated by GitHub, but you are 100% responsible for updating the OS and any other installed tooling.  And finally, it must be noted that for security reasons; GitHub recommends only using self-hosted agents with private GitHub projects (https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners#self-hosted-runner-security-with-public-repositories).

As far as I can tell, Azure DevOps and GitHub Actions use the exact same agents.

Killer Quality of Life Features

Dependabot

Dependabot is one of GitHub’s best features. It automatically scans your code dependencies and creates PRs when newer versions of your dependencies are released. This makes keeping your dependencies current really simple.  It also allows you to provide your own YAML based configuration.  With this configuration you can set how often it runs, what branches it looks at, how to label its PRs, and more. On top of all that, it can also generate alerts when any of your dependencies have known vulnerabilities.

Security Code Scanning

In addition to Dependabot, GitHub also provides security code scanning.  By default, it uses CodeQL but there are several other options available.  These scans can easily be added to your PR process, helping to prevent vulnerable code from ever getting merged in.

Extensions

GitHub has an extensive extension marketplace.  Some of the extension help you integrate GitHub with other service such as Azure DevOps, Slack, Jira, etc. Other extensions extend GitHub Actions.  The Actions extension can help simplify common pipeline activities such installing SDKs, uploading build artifacts, building Docker images and so much more.

One of my favorite extensions is Release Drafter (https://github.com/marketplace/actions/release-drafter). It almost entirely automates the creation and updating of release notes.

Package Registry

GitHub has a built-in package registry feature called Packages. Packages hit generally available in July 2021.  It supports the major package types (NPM, Maven, Gradle, Gems, and NuGet) as well as containers. This is a great option if you want your own package registry but don’t want to stand up your own Nexus server or that type of thing.

Generous Free Trier, Especially for Opensource Projects

Outside of some hard-core corporate features, such as SSO, public repositories have access to pretty much the same set of features as those on the paid tiers.  The differences are largely in how many build minutes you get and how much space you can use.   These limits are typically more than enough for most projects.  Those on the free tier with private projects do miss out on some of the more advanced feature, such as automated core review assignment, but nothing I would personally consider a deal breaker.

Pull Requests

PRs are fairly straight forward. Reviews see exactly what changes were made and can comment directly on those changes.  If PR builds are setup the reviewer(s) and requester will be all be able see the build results in the PR.  Reviewers and requesters (and other contributors) can also discuss a PR in the conversation tab.  This unified kind of view makes doing PRs in GitHub very user friendly.

REST API

GitHub has a comprehensive and well documented set of REST APIs (https://docs.github.com/en/rest).  These APIs are leveraged heavily by many GitHub extensions. You can also use them (usually in Actions pipelines) to accomplish your more complicated tasks.

Branch Protection

Branches can be protected using branch protection rules.  This allows you to do things like require code reviews, require specific build checks to pass, limit PRs from non-members, and more.

Dogfooding

Most of the GitHub tool and extension projects are developed in public GitHub projects.

Maybe Not So Good

YAML Or Nothing

Actions can have a steep learning curve for those new to YAML based pipelines.

Actions Templates Are Limited

If you find you're repeating the same steps again and again in your Actions pipeline you can take advantage of templates.  Templates a basically reusable YAML snippets.  The problem is that these snippets can only execute basic command actions.

Upshot - You cannot use anything from the Actions market in a template.

Limited Manual Approval Gates

Outside environment-based approvals there are really no manual approval gates available.  This is not a huge issue if you are doing continuous deployments but can be a bit of a challenge when doing continues delivery.

Limited Reporting

If you want pretty reports and graphs, Actions will be disappointing. For the most part GitHub basically just gives you yellow, red or green checkmarks.  This is fine for things like CI builds and unit tests where you expect everything to pass.  This is a little less than ideal when you're dealing with things like automated regression suites.  There are some extensions that can make this better but nothing I would put on par with Azure DevOps or Jira.

Primitive Secret Management

There is basic secret management, but you only get string key values pairs.

The Interesting

Branching

GitHub Actions can work with pretty much any branching strategies your throw at it.  That being said, it works best with simpler strategies. One of the best branching strategies for GitHub is called GitHub flow (https://guides.github.com/introduction/flow/), which is basically a trunk base branching strategy.

Access

Access management within GitHub can be a little confounding.  Member access within an organization and/or project can be managed by specific user or by team or both. Then you have workflow access of third-party and/or automated processes. Depending on what triggered those workflows you may find the accesses is different.  For example, Dependabot triggered runs will not have access to secrets, but if you have a CI, PR or manually trigger of the same workflow it will typically have secrets access.  Then there is API access where you may get different results based on the type of user and/or if you have been authenticated.  For the most part these access differences make sense, but they can be little unintuitive.

Conclusion

GitHub provides a very strong CI/CD capabilities with GitHub Actions. I would put it almost on par with Azure DevOps YAML pipelines. Actions however goes beyond simple CI/CD, it also provides workflow automation which can greatly reduce manual project maintenance. There are also aspects such as Dependabot which one-up several other services.  That is not to say Actions is for everyone. Those new to YAML based pipelines will likely find Actions have a steep learning curve. Additionally, associated reporting features leave a lot to be desired, especially from a quality engineering perspective. Overall, GitHub is a solid choice provided you can live with the slight lack of polish.