Pull Request Environments

At Phlo, we operate on three strategic pillars – great engineering, great design and great patient care.

Our Engineering team is constantly testing new and improved ways of working which allows the team to deliver new product features, quickly. In this blog, Phlo’s Lead Mobile Engineer Jamie MacDonald discusses the importance of Pull Request (PR) environments and our experience building our own PR environments.

Last year at Phlo our engineers wanted to surface ongoing work to the rest of the team as soon as possible for feedback.

We had noticed we were running into some problems with getting reviews of our ongoing blocks of work, which we refer to as Pull Requests, these problems included:

Product Managers, Designers, and other members of staff wishing to see new functionality had to have engineers run through the changes on their local machine and demoed via screen sharing. This reduced the ability for those testing to see and interact with the new features themselves, as well as having to plan in these demos which can prove difficult with varying schedules.

We wanted to avoid the old developer adage “but it works on my machine!”. If an engineer reviewing the PR wanted to verify the changes they had to pull down the branch and run locally to test things out. With this bringing the additional complexity of having to deal with changing database migrations and rebuilding locally we found this to be a barrier for reviewers wanting to run through a visual change.

We previously ensured all PRs were correctly built and automated tests had passed but in some cases, we had discovered that the changes would fail to deploy. This left our master branch in a state where it couldn’t be deployed. We then had to put up another PR to fix the previous issue causing delays in our development process.

We had grown accustomed to using workarounds such as demoing functionality via a screen share or waiting for the new functionality to be completed and released to our testing environment, but these were merely avoiding the problem rather than tackling it head-on. After identifying these problems, we sought out a solution that would allow us to rapidly spin up an individual environment for each of our ongoing PRs, these PR environments we hoped would allow us to move quickly while reducing manual processes.

You may be asking yourself, what exactly is a PR Environment? Sometimes named “ephemeral environments”, a PR environment is a short-lived environment containing changes that require review before being merged in and released to production. These environments can be quickly created and torn down to allow rapid development and testing, essential when working on a fast-moving application.

‍

Creating Pull Request Environments

So how did we go about creating these PR environments? Luckily, our infrastructure choices of Azure DevOps and Kubernetes lend themselves well to the task (not without some trial and error!). We knew that we’d need to accomplish four key tasks for our PR environments to be a success, these were:

‍

Trigger a build when a PR was opened

Use that build to be able to deploy to our infrastructure

Host some data storage for our PR environment’s Database

Finally, tear down the resources once the PR has been closed

‍

We started by first trying to figure out how to use Azure DevOps to run a pipeline based on a Pull Request as previously we had relied on commits to master as our trigger. We discovered that we could amend our pipeline in Azure DevOps to ensure that we would create a build artifact on a pull request trigger, this meant that each PR we opened (and subsequent commit to that PR) would build our application.

Now we had the build artifacts after a PR was opened we needed to find a way to have a stage in our pipeline that was triggered only for PR artifacts. To do this we created a new PR stage that had a pre-deployment condition set to “pull request deployment”; in this stage, we were able to reuse most of our existing deployment process, however, we did need to extend our typical deployment tasks to include a way to create some data storage to be used by our PR environment for its database.

To ensure we could quickly create some data storage we decided to provision a Persistent Volume Claim (PVC) which could then be used for setting up a database. We chose a self-hosted database for our PR environments as we didn’t need the full weight of our production setup using Google’s Cloud SQL.

Once the database was set up we could finish our deployment process and finally post a comment on the PR with convenient links to each section of the app as well as details required for connecting to the database for any specific checks that may need to be made there.

Now we had the environments set up with easy access links to them we found getting a comprehensive PR review was much easier, it allowed us to quickly show the new functionality to other members of the team for feedback before they were merged in.

The final step we had to consider was when a PR is merged, how do we ensure these environments are adequately torn down, so we don’t have unused PVCs or deployments hanging about?

After attempting to do this with Azure DevOps and finding some shortcomings we decided the best way to approach this would be using GitHub Actions; we first created a script that would be able to call the Azure DevOps API to trigger a “PR Teardown” stage, this stage deleted the PVC and all deployments associated with the PR. Once we had this script written and included in our repo all we had to do in the GitHub Action was set it up to call our tear down script whenever a PR was merged or closed, the action automatically fires and tidies up all resources without any manual intervention required.

Results and Next Steps

Since introducing PR Environments we’ve iterated on them (as we do with everything at Phlo) and now have a robust and reliable way to quickly spin up new environments. The improvements to our process have been invaluable, not only to engineers but all involved with the product; we no longer must schedule demos, find that code doesn’t run correctly on our infrastructure or discover that our builds are unemployable.

What's next for our PR Environments?

We're expanding our PR environments to run an end-to-end test suite on the changes that have been made, allowing us to prevent regressions from slipping into our application. This means that instead of spending our time manually testing changes we can focus on creating new features instead

The environments will be seeded with representative real-world data; this will bring us even closer to a production-like environment, improve our ability to conduct performance testing, and find regressions when data is in a particular state. We previously noticed some issues once the changes had reached production, this allows us to be pre-emptive and discover the issues before they are released.

‍