Automated build and deploy on VIP Go

VIP Go platform specific

This document is for sites running on VIP Go.

Learn more

Overview #

VIP Go has built-in support for CI/CD integration. Our system allows you to use a Continuous Integration service like Circle CI to run your build processes (which may include tasks like optimizing static resources, bundling CSS and JS, using composer to fetch and install dependencies, etc.) and then deploy a built copy to your environment.

How does it work? #

The following example describes how a production site is developed and deployed using a build and deploy flow:

  1. Branch from master for a new feature
  2. Make the necessary modifications to your source files and committed them to the branch.
  3. Commit any changes to your dependencies, e.g. package.json, package.lock, but .gitignore the directories and files that npm modifies, e.g. /node_modules/ and anything else that gets built
  4. Create a pull request, get it reviewed and approved, then merge to master
  5. Build: Your build steps run on the CI service.
  6. Deploy: Our deploy script commits and pushes the build code to the master-built branch (and from there it is immediately deployed to your production site)

We have specific instructions below for Travis CI or Circle CI. If you wish to use your own script or your own CI service, you are welcome to do so; the instructions below and scripts referenced are provided as a convenience only.

↑ Top ↑

How do I build my code? #

It is up to you and your team to develop a method to build, e.g. run npm install, transpile, optimize, concatenate, minify, etc, your code. You should develop the method so that it can be automatically run by a script on a Continuous Integration (CI) service, without intervention from a human.

You should also ensure any versioning updates (needed for cache busting, etc) is part of the build process or part of the commit that triggers the build.

When running the build process for production, i.e. master-built, you should ensure that your build process does not include development dependencies. You might also want to ensure that your CI script tests the build, and flags any issues to your team.

↑ Top ↑

What branches should I deploy built code to? #

Your deployed branch should end in -built, e.g. if your working branch is master (for your production environment) then your built branch should be master-built, for your Develop environment your working branch should be develop and your built and deployed branch should be develop-built.

↑ Top ↑

Should I commit directly to my built branches? #

No. You should never push code to your build branches, e.g. master-built, you should only ever push to your working branch, e.g. master, which should then build the code and push a commit to your build branch.

↑ Top ↑

My built files are in .gitignore! How do I deploy them? #

By default, files and folders referenced in your repo’s .gitignore file will not be pushed to master-branch. This usually includes files generated by your build process, which is actually the opposite result we want here!

To allow the built files to be pushed to your built branch, you can create and use a .deployignore file. This file acts as a replacement for all .gitignore files in your repo. When preparing the deploy, our script removes all .gitignore files and uses .deployignore as the canonical, global .gitignore instead.

If you’d like to use a .deployignore file, you should doing the following:

  1. Make a copy your global gitignore file and name it .deployignore.
    • (If you don’t have one, feel free to use this one as a starting point.)
  2. Review any other .gitignore files in your repo and make sure any relevant rules are copied over to the .deployignore file. (You may need to update paths for these rules so that they start from the root of the repo).
  3. Remove any rules that reference built or auto-generated files that do need to be deployed.
  4. (Optional) Add any rules that reference source files that do not need to be deployed.

As an example, here’s an trimmed-down .gitignore file:

# node_modules should almost never be committed.
node_modules

# /plugins/my-ui/src is omitted here since we do want that committed for development purposes.

# This is where our built files are generated. Don't need to commit it.
/plugins/my-ui/dist

And the .deployignore file derived from that:

# node_modules should almost never be committed.
node_modules

# Our source files don't need to be deployed.
/plugins/my-ui/src

# /plugins/my-ui/dist is omitted here since we do want it deployed.

↑ Top ↑

Configuring builds on Circle CI #

It’s a good idea to read the Circle CI getting started documentation (but don’t add the suggested Circle CI config at this point).

The following instructions reference the master and master-built branch, but can be adapted for other branches, e.g. develop and develop-built.

Before you start: We’ll need to enable Circle CI for your repository. Open a ticket and we’ll take care of that for you. Once done, you can follow the remaining steps below.

  1. Generate a deploy key. This key can be generated locally, as it will be used only by Circle CI to communicate with your GitHub repository; it does not come from or communicate with our servers.
  2. On GitHub, add the key to your repository under “Settings > Deploy Keys”. Note that the key needs “write” access.
  3. On Circle CI, you’ll need to:
    1. navigate to https://circleci.com/gh/wpcomvip/your-github-repo (use your GitHub account to access).
    2. add the key to your project (Settings | SSH Permissions). Note: It’s important that you set the hostname to github.com.
  4. Create a new Pull Request to add or adapt a config for Circle CI:
    • If you have no Circle CI config in your repository, copy this config to .circleci/config.yml in your repo; you will need to add the build command(s) you’re using in the section under “@TODO: Configure build steps”
    • If you already have a Circle CI config, you’ll need to:
      1. Add the build command(s), referencing the section in our example config commented with “@TODO: Configure build steps”
      2. Add the two sets of two lines referenced by the “REQUIRED:” comments
    • Important: Add the deploy key’s fingerprint to the repo’s /.circleci/config.yml file in the master branch.
  5. If necessary, add and update a .deployignore file.
  6. You can now trigger a build by merging a PR to master (this can be a non-significant change like a code comment). If the setup worked, Circle CI will have pushed a built copy of your application to the master-built branch on GitHub. You should verify the branch exists and contains the changes you made.
  7. Now contact VIP again to have your environment updated to deploy from master-built
  8. And that’s it! Happy building!

↑ Top ↑

Testing your CircleCI Config #

If your build script is failing on CircleCI, it might be a good idea to test your config locally since new builds will only run when a commit is made.  CircleCI has a Local CLI you can use with Docker to execute jobs.  Take a look at the Using the CircleCI Local CLI documentation.  You can validate your config.yml file using the Local CLI, but it only checks for syntax errors and not build errors.

Installing CircleCI Config on macOS or Linux:

  • Public CLI Github Repository: https://github.com/CircleCI-Public/circleci-cli
  • Install using Homebrew, cURL, or Snapcraft
  • Make sure you have Docker installed and you’re logged in with docker login (Tip: Many people report logging in with your email might cause issues so if you encounter that problem, try logging in with your Docker username)
  • Connect with your CircleCI account: circleci setup
  • Now you can validate a config file circleci config validate or run a job locally with circleci local execute --job JOBNAME (Note: this command only runs a single job and not a workflow)

↑ Top ↑

Configuring builds on Travis CI #

It’s a good idea to read the Travis CI getting started documentation, but don’t add the Travis CI config at this point.

  1. Visit https://travis-ci.com, authenticate with your GitHub account, and add your repository to Travis CI
  2. Create or adapt a config for Travis CI:
    • If you have no Travis CI config in your repository, copy this config to .travis.yml in your repo; you will need to add the build command(s) you’re using in the section under “@TODO: Configure build steps”
    • If you already have a config, you’ll need to:
      1. Add the build command(s), referencing the before_script section in our example config commented with “@TODO: Configure build steps”
      2. Add the two sets of two lines referenced by the “REQUIRED:” comments
  3. Ensure you have a machine user, this is a regular GitHub user account which is used purely for scripted functions, e.g. used by Travis CI to commit and push the built code (GitHub call this user a “machine user”):
    • If you have no dedicated “machine user” account, create a new GitHub account, named appropriately.
    • If you already have a machine user for your team, then use that account for the following steps.
  4. Setup a key pair for your machine user:
    1. Use the commandline on your local machine to create a public private key pair (documentation)
    2. Set the public portion of the key pair as a deploy key with write permissions on the GitHub repository (documentation)
    3. Add the private key as a setting on your Travis repository (see “Adding a deploy key in repository settings on Travis CI” below)
  5. If necessary, add and update a .deployignore file.
  6. Merge a PR to your master branch… it should be built by Travis CI, committed to your master-built branch, and pushed up to GitHub (verify the branch now exists on GitHub and contains the changes you made)
  7. Contact VIP to have your environment changed to deploy from master-built
  8. …that’s it!

↑ Top ↑

Adding a deploy key as a repository variable on Travis CI #

Please read these instructions through before executing the steps.

Add the public portion of the key as a deploy key on your GitHub repository; GitHub documentation on deploy keys.

Set the private portion of the key as a repository variable in the Travis settings. You will need to replace newlines with \n and surround it with double quotes, e.g. “THIS\nIS\A\KEY\nABC\n123\n”; Travis documentation on repository variables in settings.

You must set the “Display value in build log” toggle for the repository variable to “off”.

You must name the Travis setting that contains the key BUILT_BRANCH_DEPLOY_KEY.

↑ Top ↑

Other Notes #

↑ Top ↑

How is code reviewed when using automated build and deploy? #

If you are on a plan where we review your code, we will review the source code that your team writes and commits to your development branch.

However, third-party dependencies brought in by the build process will not be reviewed (e.g. Javascript dependencies added via npm or yarn or PHP SDKs/plugins/libraries added via Composer).

↑ Top ↑

Where is the source code for the deploy script? #

You can see the source code on the Automattic/vip-go-build Github repo.

↑ Top ↑

My CI service is down or broken, how do I deploy files? #

The step that you use to build files in your CI service should be scripted, by the nature of CI tasks. You should be able to follow this process:

  1. Clone your repo.
  2. Checkout the working branch with the code you need to build (e.g. master).
  3. Run your build process (e.g. npm install).
  4. Create another clone of your repo, and checkout the deployment/built branch (e.g. git checkout master-built).
  5. Copy changes from the development copy to the deployment copy using a tool like rsync.
  6. Verify your changes: they should only include built code (and not development modules and dependencies).
  7. Commit and push the changes the remote repository (e.g. git push origin master-built).
    • Note: you do not need to open a PR, unless you would like a quick validation review from the VIP team.
  8. At this point, our infrastructure detects the changes and deploys them to your application.

Ready to get started?

Drop us a note.

No matter where you are in the planning process, we’re happy to help, and we’re actual humans here on the other side of the form. 👋 We’re here to discuss your challenges and plans, evaluate your existing resources or a potential partner, or even make some initial recommendations. And, of course, we’re here to help any time you’re in the market for some robust WordPress awesomeness.