Start tracking DORA metrics for your team in just 15 minutes with Apache Dev Lake

DORA (DevOps Research and Assessment) metrics are an excellent way for engineering organisations to measure and improve their performance.

Up until now, monitoring the DORA metrics across Github, Jira, Azure Devops etc required custom tooling or a tedious manual process.

With Apache Dev Lake you can get beautiful reporting for DORA metrics on your local machine in as little as 15 minutes (honestly!).

DORA Metrics

DORA metrics are 4 dimensions for improving your engineering team. The metrics came out of a report by Google in 2020. They're designed so they balance velocity and stability across an engineering teams output.

• Deployment Frequency
• Lead time for changes
• Median Time to Restore Service
• Change Failure Rate

These metrics are explained in detail in books (here) and blogs (here) so I won't go in to any detail the metrics themselves here. If you haven't heard of them before, I recommend reading up on them.

I do have a note at the end of this article describing some of the updates to DORA since 2020 and I discuss the newer Microsoft SPACE framework. Let's get started with the tutorial though.

The old way of tracking DORA metrics

Most teams will have various support systems like Github, Azure DevOps, Jira etc.

Extracting the metrics required for DORA across all of our developer tools is difficult to automate. You have to write custom software or pay for tooling. The time cost for custom tooling is too much, especially for smaller teams.

When I first started tracking DORA I used a spreadsheet and did everything manually. Copy and paste data from Github, Jira and Azure devops.

It wasn't pretty and it was a lot of work but I was able to see a noticeable increase in deployed work when deployment frequency increased.

Tracking DORA metrics in 2022: Apache Dev Lake

Apache Dev Lake is a project from the Apache Foundation specifically for tracking engineering performance metrics.

The project is new and it is in the incubation stage at the Apache Foundation.

Dev Lake is a set of services in containers - a database, a data pipeline, a grafana instance with pre-built reports and a configuration web application.

The dev lake team provide helm charts and docker compose files for easy setup of the infrastructure.

The benefits of tracking your DORA metrics in Dev Lake are:

• It connects to most major platforms out of the box (Jira, Github, DevOps)
• Comes with pre built Grafana charts for DORA and many other metrics
• Free and you can easily run it locally to get started
• Your data or your client's data never leaves your organisation

Having the data processed and stored locally is a huge benefit for many organisations. It means you don't have to add an additional supplier to get started with DORA metrics.

Visit the dev lake site for more details.

Setting up a local instance of Apache Dev Lake

You should have docker desktop installed before continuing.

1. Get the docker compose and .env files from the releases page - https://github.com/apache/incubator-devlake/releases/latest and place them in a folder
2. Open a terminal in that folder and mv .env.example .env to rename the env var file
3. create the infrastructure: docker compose up
4. That's it! - Wait for the 4 services to start
5. You should be able to access the configuration site at https://localhost:4000

There are some sites already configured on the instance in the image above. Yours will be empty for now.

Now you can add any connections to your engineering tools like Jira or Github.

Adding connections to common dev tools

Github is very popular so let's use that to demonstrate setting up a tool connection.

1. Click on Data Connections > Github
2. Give the connection a name you'll remember
3. The endpoint is https://api.github.com/ unless you have a special enterprise account - then you'll need to use your own endpoint
4. Get a personal access token from Github and paste it in the token field (https://github.com/settings/tokens)

The PAT (personal access token) permissions you need are:

• repo:status
• repo_deployment
• public_repo
• repo:invite
• read:user
• read:org

I set the github rate limit to 5000 requests per hour but you can leave that as default if you like.

Configuring projects in Dev Lake

Now that you have a connection to your dev tools you can configure projects. In Dev Lake a project is called a "Blueprint".

I'll add one of my open source projects (https://github.com/darraghoriordan/eslint-plugin-nestjs-typed) to show how it works.

1. Click on Blueprints > Add Blueprint in the main left-hand menu
2. Give it a name and select the Github connection you created earlier

3. Click next and enter the name of the repo(s) associated with your project

4. Click next and on the summary screen click on the "add transformation" link. You have to tell Dev Lake how to identify deploys and incidents for your setup.