Circle CI with Nx
In this tutorial we're going to learn how to leverage Nx to setup a scalable CI pipeline on Circle CI. As repositories get bigger, making sure that the CI is fast, reliable and maintainable can get very challenging. Nx provides a solution.
- Nx reduces wasted time in CI with the
affectedcommand. - Nx Replay's remote caching will reuse task artifacts from different CI executions making sure you will never run the same computation twice.
- Nx Agents efficiently distribute tasks across machines ensuring constant CI time regardless of the repository size. The right number of machines is allocated for each PR to ensure good performance without wasting compute.
- Nx Atomizer automatically splits large e2e tests to distribute them across machines. Nx can also automatically identify and rerun flaky e2e tests.
Example Repository
To follow along with this tutorial, we recommend using the nx-shops sample repository.
Example repository/nrwl/nx-shops
The nx-shops repo is useful to demonstrate the value of the CI pipeline because it has the following characteristics:
- Multiple Nx projects with interdependencies
- Defined lint, test, build and e2e tasks
- Running all the tasks takes more than a minute to finish
To get started:
Fork the nx-shop repo and then clone it to your local machine
❯
git clone https://github.com/<your-username>/nx-shops.git
Install dependencies (this repo uses PNPM but you should be able to also use any other package manager)
❯
pnpm i
Make sure all tasks are working on your machine, by running lint, test, build and e2e on all projects of the workspace
❯
pnpm exec nx run-many -t lint test build e2e
Connect to Nx Cloud
Nx Cloud is a companion app for your CI system that provides remote caching, task distribution, e2e test deflaking, better DX and more.
Let's connect your repository to Nx Cloud with the following command:
❯
pnpm exec nx connect
Once the script is finished, it will print a link in the terminal to register your repository in your Nx Cloud account. Click the link and follow the steps provided to make sure Nx Cloud is enabled on the main branch of your repository.
The set up process will create a PR on your repository that you'll need to merge in order connect to Nx Cloud.
When you're finished, you'll need to pull down the changes that were merged into the hosted repository with this command:
❯
git pull
You should now have an nxCloudAccessToken property specified in the nx.json file.
Connect to Circle CI
In order to use Circle CI, you need to sign up and create an organization. Follow the steps in the Circle CI documentation to connect to your GitHub repository. When you are asked to configure a pipeline, choose any option, since we'll overwrite it in the next step.
Make sure to merge the PR that Circle CI creates and then pull the changes to your local machine with the following command:
❯
git pull
Create a CI Workflow
First, we'll create a new branch to start adding a CI workflow.
❯
git checkout -b setup-ci
Now we can use an Nx generator to create a default CI workflow file.
❯
pnpm exec nx generate ci-workflow --ci=circleci
This generator will overwrite Circle CI's default .circleci/config.yml file to create a CI pipeline that will run the lint, test, build and e2e tasks for projects that are affected by any given PR.
The key lines in the CI pipeline are highlighted in this excerpt:
1- run: pnpm install --frozen-lockfile
2- nx/set-shas:
3 main-branch-name: 'main'
4
5# Prepend any command with "nx-cloud record --" to record its logs to Nx Cloud
6# - run: pnpm exec nx-cloud record -- echo Hello World
7- run: pnpm exec nx affected --base=$NX_BASE --head=$NX_HEAD -t lint test build
8- run: pnpm exec nx affected --base=$NX_BASE --head=$NX_HEAD --parallel 1 -t e2e-ci
9The nx affected command will run the specified tasks only for projects that have been affected by a particular PR, which can save a lot of time as repositories grow larger. In this particular branch, however, the only file that has changed is the CI.yml file which is not associated with any project. This will cause nx affected to not run any tasks. Let's fix this by adding the CI workflow file to the sharedGlobals defined in the nx.json - this will tell Nx that changes to the CI pipeline should affect every project in the repository.
1{
2 // ...
3 "namedInputs": {
4 "default": ["{projectRoot}/**/*", "sharedGlobals"],
5 "sharedGlobals": [
6 "{workspaceRoot}/babel.config.json",
7 "{workspaceRoot}/.circleci/config.yml"
8 ]
9 // ...
10 }
11}
12Commit your changes and push your branch:
❯
git add .
❯
git commit -am "basic ci workflow"
❯
git push -u origin HEAD
Now create a pull request with the new branch.
Create Your PR on Your Own RepositoryMake sure that the PR you create is against your own repository's main branch - not the nrwl/nx-shops repository.
Nx Cloud will create a comment on your PR that gives you a summary of the CI run and a link to dig into logs and understand everything that happened during the CI run.
Understand Remote Caching
Nx Cloud provides Nx Replay, which is a powerful, scalable and, very importantly, secure way to share task artifacts across machines. It lets you configure permissions and guarantees the cached artifacts cannot be tempered with.
Nx Replay is enabled by default. We can see it in action by running a few commands locally. First, let's build every project in the repository:
❯
pnpm exec nx run-many -t build
Nx will store the output of those tasks locally in the .nx/cache folder and remotely in Nx Cloud. If someone else in the organization were to run the same build command on the same source code, they would receive the remotely cached outputs instead of re-running the build task themselves. We can simulate this by deleting the .nx/cache folder and re-running the build command.
❯
rm -rf .nx/cache
❯
pnpm exec nx run-many -t build
The build tasks complete almost instantly, and you can see in the logs that Nx has pulled the outputs from the remote cache:
1❯ nx run-many -t build
2
3 ✔ nx run shared-product-types:build [remote cache]
4 ✔ nx run shared-product-ui:build [remote cache]
5 ✔ nx run shared-header:build [remote cache]
6 ✔ nx run landing-page:build:production [remote cache]
7 ✔ nx run admin:build:production [remote cache]
8 ✔ nx run cart:build:production [remote cache]
9This remote cache is useful to speed up tasks when developing on a local machine, but it is incredibly useful for CI to be able share task results across different CI pipeline executions. When a small commit is added to a large PR, the CI is able to download the results for most of the tasks instead of recomputing everything from scratch.
You might also want to learn more about how to fine-tune caching to get even better results.
Parallelize Tasks Across Multiple Machines Using Nx Agents
The affected command and Nx Replay help speed up the average CI time, but there will be some PRs that affect everything in the repository. The only way to speed up that worst case scenario is through efficient parallelization. The best way to parallelize CI with Nx is to use Nx Agents.
The Nx Agents feature
- takes a command (e.g.
run-many -t build lint test e2e-ci) and splits it into individual tasks which it then distributes across multiple agents - distributes tasks by considering the dependencies between them; e.g. if
e2e-cidepends onbuild, Nx Cloud will make sure thatbuildis executed beforee2e-ci; it does this across machines - distributes tasks to optimize for CPU processing time and reduce idle time by taking into account historical data about how long each task takes to run
- collects the results and logs of all the tasks and presents them in a single view
- automatically shuts down agents when they are no longer needed
To enable Nx Agents, make sure the following line is uncommented in the .circleci/config.yml file.
1# Connect your workspace on nx.app and uncomment this to enable task distribution.
2# The "--stop-agents-after" is optional, but allows idle agents to shut down once the "e2e-ci" targets have been requested
3- run: pnpm dlx nx-cloud start-ci-run --distribute-on="5 linux-medium-js" --stop-agents-after="e2e-ci"
4We recommend you add this line right after you check out the repo, before installing node modules.
nx-cloud start-ci-run --distribute-on="5 linux-medium-jslets Nx know that all the tasks after this line should using Nx Agents and that Nx Cloud should use 5 instances of thelinux-medium-jslaunch template. See the separate reference on how to configure a custom launch template.--stop-agents-after="e2e-ci"lets Nx Cloud know which line is the last command in this pipeline. Once there are no more e2e tasks for an agent to run, Nx Cloud will automatically shut them down. This way you're not wasting money on idle agents while a particularly long e2e task is running on a single agent.
Try it out by creating a new PR with the above changes.
Once Circle CI starts, you can click on the Nx Cloud report to see what tasks agents are executing in real time.
With this pipeline configuration in place, no matter how large the repository scales, Nx Cloud will adjust and distribute tasks across agents in the optimal way. If CI pipelines start to slow down, just add some agents. One of the main advantages is that this pipeline definition is declarative. We tell Nx what commands to run, but not how to distribute them. That way even if our monorepo structure changes and evolves over time, the distribution will be taken care of by Nx Cloud.
Next Steps
You now have a highly optimized CI configuration that will scale as your repository scales. See what else you can do with Nx Cloud.
- Configure dynamic agent allocation
- Learn about automatically splitting e2e tasks
- Identify and re-run flaky tasks