What’s next?

Now that you have an overview of how to achieve a smooth launch on the VIP platform, you’re ready to get started!

Don’t forget that you can review our documentation at any time, and reach out to our team via Zendesk if you need help. We can’t wait to help you launch!

Further reading

Disaster recovery for VIP

Site outage protocol

Though we have 24×7 automated monitoring and would be aware of the outage immediately, you should let us know if there is an issue with your application not loading. To notify our entire team immediately, you can send in a ticket with the word “urgent” in the subject line. Someone will respond to you within minutes.

Urgent tickets should only be used for outages, security issues, or workflow-blocking concerns. And please remember to include as many details as you can to help us solve the issue faster for you.

Platform-wide outages

VIP have documented and agreed procedures for outage scenarios and VIP staff are trained in and required to understand these procedures. Before we touch on how we would respond, let’s go over what is done to prevent outages from happening.

VIP offers different code review levels based on the contract type, all intended to make sure that your site will be secure, performant, and adhering to best practices. This is an extra layer of stability that helps to identify and prevent code-related issues. We also run high-performance checks ahead of site launches to ensure the application is optimized before going into production.

VIP also maintains a blend of documented and automated procedures for dealing with various equipment or other failures. All VIP production environments are backed up every hour, and 30 days of backups are held in the origin data center. A recent backup for each production environment is stored in a separate data center.

VIP maintain several origin data centers, each with additional capacity that can be used if sites require to move data center in the event of a failure. Our primary data centers are located in or around Dallas Fort Worth, Los Angeles, and Washington D.C.

The VIP hosting infrastructure is designed to mitigate against equipment failures causing a given application environment, e.g. the production environment, to be unavailable. All VIP application environments are spread across networking (e.g. switch) and power (e.g. rack power) infrastructure, to mitigate against equipment issues with networking or power affecting all the resources assigned with running a given VIP production environment.

VIP maintain procedures on multiple communications channels and knowledge sharing systems, both on Automattic and third-party infrastructure, in order to mitigate against issues with only a single system in the event of some wide-ranging issue with our systems. We provide our clients with access to New Relic, which includes the availability and monitoring service Synthetics Lite. We have specific thresholds to set off warnings for performance issues, often flagging concerning items that could contribute to or indicate that an application environment is unavailable or has service issues.

In outage scenarios, VIP uses an established Outage Mode protocol. There are different leads assigned to different roles, allowing our engineering team to focus on resolving the issue. This also ensures someone is updating the VIP Status Twitter feed and VIP Lobby. Once the outage has been resolved, we follow up with an After Action Report with details about what led to the issue and preventative measures that may have been put in place to keep a similar scenario from occurring again.

Enterprise touch

In addition to the communication outlined above, Enterprise clients should expect to hear from their designated Technical Account Manager (TAM). The TAM will alert their primary contact about the situation with the details gathered so far, and then provide status updates throughout. Typically an email, Slack Direct Message, or text message is best while the situation is being tended to so we can contribute to the resolution. Once the issue has been resolved, the TAM will follow up so you know when the problem has been resolved. This could be an email, a Zendesk ticket, or a quick phone call if desired.

Your VIP team

A Technical Account Manager, or TAM, will serve as your main point of contact from onboarding up to your time of launch. The first time you’ll meet a TAM will likely be on the project kickoff call, where you will establish your launch timeline and get access to VIP Support.

In addition to the TAM, at different points of your onboarding process, you’ll also engage with VIP Engineers and Relationship Managers:

 

Further reading

The VIP platform

The VIP platform (also referred to in our documentation as VIP Go) has container-based infrastructure that allows clients to run core WordPress with custom themes and plugins on Automattic’s world-class hardware and network infrastructure.

On our platform, the codebase consists of core WordPress, a handful of platform-specific mu-plugins, and the client’s custom code. Media is served from the VIP Files Service, a globally distributed object store, and sites are cached on the edge with Varnish. VIP looks after hourly platform backups and 24/7 monitoring.

VIP infrastructure diagram
An overview of the VIP infrastructure (click to enlarge)

While your site is in development, it will use a convenience URL ending in .go-vip.net. The production environment tracks the master branch of the connected GitHub repository, which auto-deploys. VIP can also set up child environments for development purposes, which will track a specific branch in the same repo. We also encourage developing locally.

The VIP platform supports both single-site and multisite installations of WordPress.

Tools

VIP clients are offered a variety of tools:

  • Jetpack, VaultPress, and Akismet are connected by default to each VIP Go application.
  • Sites are connected to New Relic for application monitoring.
  • VIP clients can take advantage of supercharged search via Jetpack Search, powered by Elasticsearch.
  • Scanning for coding issues on each PR.

Further Reading

Steps to launch

Once your site is ready, your TAM will guide you through the launch process. This will include scheduling a date and time for the launch when VIP can be on-hand to assist.

A sample launch timeline

Typically, the launch steps will be something like:

12 hours before launch:

  • Code freeze
  • Editorial freeze
  • Final SQL import
  • Delta media import

At the time of launch:

  • Everybody joins a StormChat, which is a unique chat room for your launch
  • VIP changes the site to use the production domain
  • Before the DNS switch, the client is welcome to QA their site by changing their local hosts file
  • DNS is switched to point to VIP
  • VIP installs an SSL certificate via Let’s Encrypt
  • Congratulations! Your site is launched! 🎉

Keep in mind that the steps to launch each site may differ. Your TAM will be able to provide you with the exact steps, tailored to the needs of your site.

Further reading

Welcome to your VIP launch

Hello there! We’re excited to launch your site on VIP.

The VIP team

This series is an overview of what you can expect when launching with VIP. We’ll guide you through the launch process, and link to important resources along the way.

In this guide, you’ll learn:

  • The VIP team members you’ll have contact with
  • How to contact VIP Support
  • An overview of the VIP platform
  • How we review code
  • What you can expect if you’re migrating your site to VIP
  • The typical steps to launch your site
  • How to find further resources and help

Ready to get started? Let’s go!

Domains and SSL

DNS

If you plan on hosting your DNS with us, contact us via support ticket and we will set up domain management for you, where you can self-service create DNS entries to mirror your current setup. Once setup is complete, you can verify and switch to our nameservers. This should be done at least a week before launch.

If you plan on hosting your own DNS, we’ll ask you to point the domain to us.

When your site is set up, it will be using a “convenience” domain like example-com.go-vip.net. At launch, we’ll change the site to using the production domain.

Once the DNS for a domain points to VIP, and the domain is mapped to an application, visiting the mapped domain will direct visitors to the example-com.go-vip.net domain. If there is content on the site that should not be discoverable before launch, you may want to enable Maintenance Mode until the official launch.

TTL

When migrating a site from another platform, you may want to time the DNS switch with launch, to minimise downtime. In this case, the TTL (Time To Live) on the domain should be lowered as far as possible before the launch. Your TAM will work with you on the launch steps.

SSL

VIP provides an SSL certificate via Let’s Encrypt. There is no additional charge for this, and we’ll handle the renewals. If you prefer to provide your own cert, you can contact us for a CSR.

Further reading

Content migration

When you are ready to import content into your VIP site, we’ll need two things:

  1. A SQL dump of your site’s WordPress database
  2. A compressed archive of the uploads directory.

Most clients will want VIP to import content into their environment early in the development process, so they can ensure that everything looks great on the platform. Then, at the time of launch, we can update the content on your site with the most up-to-date content you’ve been creating.

If your content files are too large to attach to a Zendesk ticket, you can provide a link to Dropbox or a similar cloud storage service. VIP can also import content from an S3 bucket.

Further reading

Code review

VIP’s priority is to ensure that your site is there when you need it, which means we care about its performance and security. Code review is a key component of ensuring your site is secure and performance. We offer both automated checks and manual reviews to clients.

VIP’s code review focuses on the performance and security considerations in PHP, custom JavaScript, and SVG files. We do not review HTML, CSS, SASS, many popular third-party JavaScript libraries, or built JavaScript files.

We’ll schedule an initial code review of the entire code base. You will continue receiving this feedback automatically on all pull requests to your GitHub repository.

We offer two levels of manual code review:

  • Full: A developer will read every line of your code, including themes and custom plugins.
  • Enhanced: Your theme and custom plugins are reviewed line-by-line. Third-party plugins will go through an automated scan.

Automated scan: Even if you don’t receive manual review, your entire code base will be automatically scanned with VIP’s PHP CodeSniffer (PHPCS) standard with an initial report sent to your developers. VIP will answer any questions about specific errors or warnings if the client wishes to refactor the code.

Initial review

For clients on the Full or Enhanced levels of review, please allow for 10-15 business days in your project timeline to complete the first and subsequent review cycles. Please note, exact timeframes can vary depending on various factors – ask your TAM for more details. Before you submit your code for review, ensure it’s been thoroughly tested, scanned using PHPCS with the WordPress-VIP-Go ruleset, and as many errors and warnings as possible have been addressed.

Ongoing review

After the initial review, clients with Full and Enhanced reviews will have a GitHub pull request workflow enabled. This protects the master branch from merges without our review. For clients without manual review, we recommend following a similar workflow to enable the VIP code analysis bot to provide automated feedback.

We take code review seriously and understand that there may be many questions along the way. If you need assistance, please open a ticket and we’d be happy to provide guidance.

Further reading

Accessing VIP support

Your support team is here to help, every step of the way.

Zendesk

VIP uses the Zendesk ticketing system for helping support clients with technical issues and launch planning.

Support tickets can be opened in one of these ways:

  • Log in to the Zendesk portal (access provided after kickoff call)
  • Submit tickets from “VIP” portal in the wp-admin of your site
  • Use the ? Support button in the lower left-hand corner of the VIP Dashboard
  • Email our support address

Tips for using Zendesk:

  • Keep each issue in its own ticket. This allows our team to effectively route your questions and can help avoid confusion where multiple issues and resolutions are being discussed in one ticket.
  • When you create a ticket, you’ll have the option of four different priority levels: low, normal, high, and urgent. Urgent tickets page the entire support team, so we appreciate that you use them sparingly for true emergencies like outages, time-sensitive security concerns, and workflow-blocking situations where the site isn’t functioning at all.
  • Access your existing tickets in the Zendesk portal.
  • Add additional stakeholders using cc field in the original email, or any subsequent responses.
  • Please follow our guidelines for opening a ticket, to help us reach a timely resolution.

The VIP Lobby

  • The VIP Lobby is the first place we share information with VIP clients from outages to event announcements.
  • Invitations to the VIP Lobby are sent to WordPress.com usernames or email addresses.
  • To ensure you receive the latest updates, you’ll want to subscribe to new posts. When logged in, you can do this by clicking “Subscribe to Email Updates” or the “Follow” button in the bottom right-hand corner.

 

Follow posts from the VIP Lobby
Follow posts from the VIP Lobby

Twitter

Updates about VIP service issues and maintenance are posted on @WPVIPStatus on Twitter.

Further reading

Automated build and deploy on VIP Go

VIP Go has built-in support for CI/CD integration. Our system allows you to use a Continuous Integration service like Circle CI to run your build processes (which may include tasks like optimizing static resources, bundling CSS and JS, using composer to fetch and install dependencies, etc.) and then deploy a built copy to your environment.

How does it work?

The following example describes how a production site is developed and deployed using a build and deploy flow:

  1. Branch from master for a new feature
  2. Make the necessary modifications to your source files and committed them to the branch.
  3. Commit any changes to your dependencies, e.g. package.json, package.lock, but .gitignore the directories and files that npm modifies, e.g. /node_modules/ and anything else that gets built
  4. Create a pull request, get it reviewed and approved, then merge to master
  5. Build: Your build steps run on the CI service.
  6. Deploy: Our deploy script commits and pushes the build code to the master-built branch (and from there it is immediately deployed to your production site)

We have specific instructions below for Travis CI or Circle CI. If you wish to use your own script or your own CI service, you are welcome to do so; the instructions below and scripts referenced are provided as a convenience only.

How do I build my code?

It is up to you and your team to develop a method to build, e.g. run npm install, transpile, optimize, concatenate, minify, etc, your code. You should develop the method so that it can be automatically run by a script on a Continuous Integration (CI) service, without intervention from a human.

You should also ensure any versioning updates (needed for cache busting, etc) is part of the build process or part of the commit that triggers the build.

When running the build process for production, i.e. master-built, you should ensure that your build process does not include development dependencies. You might also want to ensure that your CI script tests the build, and flags any issues to your team.

What branches should I deploy built code to?

Your deployed branch should end in -built, e.g. if your working branch is master (for your production environment) then your built branch should be master-built, for your Develop environment your working branch should be develop and your built and deployed branch should be develop-built.

Should I commit directly to my built branches?

No. You should never push code to your build branches, e.g. master-built, you should only ever push to your working branch, e.g. master, which should then build the code and push a commit to your build branch.

My built files are in .gitignore! How do I deploy them?

By default, files and folders referenced in your repo’s .gitignore file will not be pushed to master-branch. This usually includes files generated by your build process, which is actually the opposite result we want here!

To allow the built files to be pushed to your built branch, you can create and use a .deployignore file. This file acts as a replacement for all .gitignore files in your repo. When preparing the deploy, our script removes all .gitignore files and uses .deployignore as the canonical, global .gitignore instead.

If you’d like to use a .deployignore file, you should doing the following:

  1. Make a copy of your global gitignore file and name it .deployignore.
    • (If you don’t have one, feel free to use this one as a starting point.)
  2. Review any other .gitignore files in your repo and make sure any relevant rules are copied over to the .deployignore file. (You may need to update paths for these rules so that they start from the root of the repo).
  3. Remove any rules that reference built or auto-generated files that do need to be deployed.
  4. (Optional) Add any rules that reference source files that do not need to be deployed.

As an example, here’s an trimmed-down .gitignore file:

# node_modules should almost never be committed.
node_modules

# /plugins/my-ui/src is omitted here since we do want that committed for development purposes.

# This is where our built files are generated. Don't need to commit it.
/plugins/my-ui/dist

And the .deployignore file derived from that:

# node_modules should almost never be committed.
node_modules

# Our source files don't need to be deployed.
/plugins/my-ui/src

# /plugins/my-ui/dist is omitted here since we do want it deployed.

Configuring builds on Circle CI

It’s a good idea to read the Circle CI getting started documentation (but don’t add the suggested Circle CI config at this point).

The following instructions reference the master and master-built branch, but can be adapted for other branches, e.g. develop and develop-built.

Before you start: We’ll need to enable Circle CI for your repository. Open a ticket and we’ll take care of that for you. Once done, you can follow the remaining steps below.

  1. Generate a deploy key. This key can be generated locally, as it will be used only by Circle CI to communicate with your GitHub repository; it does not come from or communicate with our servers.
  2. On GitHub, add the key to your repository under “Settings > Deploy Keys”. Note that the key needs “write” access.
  3. On Circle CI, you’ll need to:
    1. navigate to https://circleci.com/gh/wpcomvip/your-github-repo (use your GitHub account to access).
    2. add the key to your project (Settings | SSH Permissions). Note: It’s important that you set the hostname to github.com.
  4. Create a new Pull Request to add or adapt a config for Circle CI:
    • If you have no Circle CI config in your repository, copy this config to .circleci/config.yml in your repo; you will need to add the build command(s) you’re using in the section under “@TODO: Configure build steps”
    • If you already have a Circle CI config, you’ll need to:
      1. Add the build command(s), referencing the section in our example config commented with “@TODO: Configure build steps”
      2. Add the two sets of two lines referenced by the “REQUIRED:” comments
    • Important: Add the deploy key’s fingerprint to the repo’s /.circleci/config.yml file in the master branch.
  5. If necessary, add and update a .deployignore file.
  6. You can now trigger a build by merging a PR to master (this can be a non-significant change like a code comment). If the setup worked, Circle CI will have pushed a built copy of your application to the master-built branch on GitHub. You should verify the branch exists and contains the changes you made.
  7. Now contact VIP again to have your environment updated to deploy from master-built
  8. And that’s it! Happy building!

Testing your CircleCI Config

If your build script is failing on CircleCI, it might be a good idea to test your config locally since new builds will only run when a commit is made.  CircleCI has a Local CLI you can use with Docker to execute jobs.  Take a look at the Using the CircleCI Local CLI documentation.  You can validate your config.yml file using the Local CLI, but it only checks for syntax errors and not build errors.

Installing CircleCI Config on macOS or Linux:

  • Public CLI Github Repository: https://github.com/CircleCI-Public/circleci-cli
  • Install using Homebrew, cURL, or Snapcraft
  • Make sure you have Docker installed and you’re logged in with docker login (Tip: Many people report logging in with your email might cause issues so if you encounter that problem, try logging in with your Docker username)
  • Connect with your CircleCI account: circleci setup
  • Now you can validate a config file circleci config validate or run a job locally with circleci local execute --job JOBNAME (Note: this command only runs a single job and not a workflow)

Configuring builds on Travis CI

It’s a good idea to read the Travis CI getting started documentation, but don’t add the Travis CI config at this point.

  1. Visit https://travis-ci.com, authenticate with your GitHub account, and add your repository to Travis CI
  2. Create or adapt a config for Travis CI:
    • If you have no Travis CI config in your repository, copy this config to .travis.yml in your repo; you will need to add the build command(s) you’re using in the section under “@TODO: Configure build steps”
    • If you already have a config, you’ll need to:
      1. Add the build command(s), referencing the before_script section in our example config commented with “@TODO: Configure build steps”
      2. Add the two sets of two lines referenced by the “REQUIRED:” comments
  3. Ensure you have a machine user, this is a regular GitHub user account which is used purely for scripted functions, e.g. used by Travis CI to commit and push the built code (GitHub call this user a “machine user”):
    • If you have no dedicated “machine user” account, create a new GitHub account, named appropriately.
    • If you already have a machine user for your team, then use that account for the following steps.
  4. Setup a key pair for your machine user:
    1. Use the commandline on your local machine to create a public private key pair (documentation)
    2. Set the public portion of the key pair as a deploy key with write permissions on the GitHub repository (documentation)
    3. Add the private key as a setting on your Travis repository (see “Adding a deploy key in repository settings on Travis CI” below)
  5. If necessary, add and update a .deployignore file.
  6. Merge a PR to your master branch… it should be built by Travis CI, committed to your master-built branch, and pushed up to GitHub (verify the branch now exists on GitHub and contains the changes you made)
  7. Contact VIP to have your environment changed to deploy from master-built
  8. …that’s it!

Adding a deploy key as a repository variable on Travis CI

Please read these instructions through before executing the steps.

Add the public portion of the key as a deploy key on your GitHub repository; GitHub documentation on deploy keys.

Set the private portion of the key as a repository variable in the Travis settings. You will need to replace newlines with \n and surround it with double quotes, e.g. “THIS\nIS\A\KEY\nABC\n123\n”; Travis documentation on repository variables in settings.

You must set the “Display value in build log” toggle for the repository variable to “off”.

You must name the Travis setting that contains the key BUILT_BRANCH_DEPLOY_KEY.

Other Notes

How is code reviewed when using automated build and deploy?

If you are on a plan where we review your code, we will review the source code that your team writes and commits to your development branch.

However, third-party dependencies brought in by the build process will not be reviewed (e.g. Javascript dependencies added via npm or yarn or PHP SDKs/plugins/libraries added via Composer).

Where is the source code for the deploy script?

You can see the source code on the Automattic/vip-go-build Github repo.

My CI service is down or broken, how do I deploy files?

The step that you use to build files in your CI service should be scripted, by the nature of CI tasks. You should be able to follow this process:

  1. Clone your repo.
  2. Checkout the working branch with the code you need to build (e.g. master).
  3. Run your build process (e.g. npm install).
  4. Create another clone of your repo, and checkout the deployment/built branch (e.g. git checkout master-built).
  5. Copy changes from the development copy to the deployment copy using a tool like rsync.
  6. Verify your changes: they should only include built code (and not development modules and dependencies).
  7. Commit and push the changes the remote repository (e.g. git push origin master-built).
    • Note: you do not need to open a PR, unless you would like a quick validation review from the VIP team.
  8. At this point, our infrastructure detects the changes and deploys them to your application.

What’s next?

Now that you have an overview of the VIP development workflow, you’re ready to get started!

Don’t forget that you can review our documentation at any time, and reach out to our team via Zendesk if you need help. Speak to you soon!

Further reading

VIP CLI

VIP CLI is the command line interface for VIP Go. You can use VIP CLI to interact with your VIP applications, query information about your applications, and perform actions like syncing data from production to development environments.

VIP CLI is aimed at developers, so familiarity with installing CLI applications and using the Terminal is required. VIP also offers the web-based VIP Dashboard, which will provide access to similar functionality in a graphical user interface.

What commands does the CLI offer?

You can view the available commands with the vip --help command:

$ vip --help

Usage: vip [options] [command]

Commands:

app List and modify your VIP Go apps
help Display help
logout Logout from your current session
sync Sync production to a development environment

Options:

-h, --help Output usage information
-v, --version Output the version number

Installing VIP CLI locally

VIP CLI is a Node.js Package and can be installed through a package manager like npm.

VIP CLI requires Node.js v8.9.0+ and npm v5.7.1+. If either or both are below the required versions, see the upgrade steps in our troubleshooting section below.

To begin installation, visit our VIP CLI page installation page and follow the instructions.

Note: It is extremely important not to install VIP CLI using the sudo command; if this has been done you can follow the directions below.

Specifying App and Environment

On VIP, each application has one or more environments, such as production, staging, or develop. For VIP CLI commands that run on a specific environment, VIP CLI will prompt you to choose an environment.

Environments can also be specified with an alias in the command, in the form of @<app-name>.<env>. Here’s an example command using an environment alias:

vip @wpvip.production -- wp option get home

Each environment can be specified by the unique app name, a dot separator, and the environment name. An environment’s alias can be found in the VIP Dashboard.

Troubleshooting

Status code 401

You may receive an error including the line Received status code 401, for example:

Failed to fetch apps: Error: Network error: Response not successful: Received status code 401

Please try the following:

  1. Run vip logout
  2. Run vip and follow the prompts to log in again

I can’t see all my applications

For you to access an application via VIP CLI, you must have access to the GitHubrepository for that application. If you do not see all the applications you expect when running vip app list, please follow this procedure:

  1. Check you have access to the GitHub repository for the application(s) you expect to see, if you do not have access to these repositories on GitHub then you should contact someone in your team who has admin rights or alternatively contact VIP support
  2. If you do have access to the application repository on GitHub, but you do not see the application listed via vip app list then…
  3. Run vip logout
  4. Run vip and follow the prompts to log in
  5. Run vip app list, If you are still not seeing all the applications you expect then please get in touch with VIP support

Installing Node

Please follow the Node.js project’s installation instructions.

Installing npm

Installing Node.js (instructions above) will include npm in the installation.

Upgrading Node.js

Follow the Node.js installation instructions to acquire and install the latest version of Node.js, or use the upgrade command if you installed via a package manager like Homebrew for macOS.

Upgrading npm

If you have npm installed already, you can run the following command to update to the latest version:

npm install -g npm

Fixing NPM/Node permissions

Read this npm guide: How to Prevent Permissions Errors.

If you have installed various commands with sudo you will need to reverse the process, fix the permissions errors, then install the command without the use of sudo.

Help, I used sudo to install VIP CLI!

You should not use sudo to fix access permissions; instead, please see “Fixing permissions” above.

WARN install EACCES: permission denied

See “Fixing permissions” above.

Loading Gutenberg on VIP

On the WordPress VIP Go platform, Gutenberg editor is opt-in. If you do nothing, legacy editing experience will be preserved. If you wish to enable Gutenberg, load it in your theme code using the helper function wpcom_vip_load_gutenberg().

You can load Gutenberg everywhere, or selectively by post ID (load only for specific posts) and post type (load only for specific post types). It is agnostic about whether Gutenberg is loading from core or via the plugin, allowing for a seamless transition.

Loading Gutenberg

Loading behavior is controlled by the wpcom_vip_load_gutenberg() function, this needs to be added in your theme/functions.php. Calling this function without its single optional parameter causes Gutenberg to load on all post-edit screens.

An optional associative array of criteria can be passed. The possible keys and values are:

  • load (Int): 0|1: never or always load Gutenberg
  • posts (Array of post_ids): loads Gutenberg for the specified post_ids
  • post_types` (Array of post_types): loads Gutenberg for the specified post types.

Post Types
Make sure that the post type is registered with 'show_in_rest' => true otherwise Gutenberg can’t be enabled.

Examples

In theme/functions.php:

Load Gutenberg for all posts:

if ( function_exists( 'wpcom_vip_load_gutenberg' ) ) {
    wpcom_vip_load_gutenberg( true );
}

Do not load Gutenberg for any posts:

wpcom_vip_load_gutenberg( false );

Load Gutenberg for posts with ids 12, 13 and 122:

wpcom_vip_load_gutenberg( [ 'post_ids' => [ 12, 13, 122 ] ] );

Load Gutenberg for post_id 12 and all posts of type test and scratch:

wpcom_vip_load_gutenberg( [ 'post_types' => [ 'test', 'scratch' ], 'post_ids' => [ 12 ] ] );

Advanced

The typical use case is as shown above, the parameters do not change except when theme code is updated.

If making more dynamic changes, note that the parameter supplied is persisted in a site option; when the parameters are changed in code, one page load is necessary to update the site option before the editor can use the new setting.

In rare cases you may prefer a different Gutenberg transition solution (eg Classic Editor). Thus, Ramp can be disabled:

// in vip-config.php
define('VIP_GO_DISABLE_RAMP', true);

Local Development

wpcom_vip_load_gutenberg() and the above functionality is available by default on your VIP Go Development Environment.

It is also separately available in the Gutenberg Ramp plugin, available here. If you are experimenting with this plugin in a non-Go development environment please be aware that the name of the function differs slightly on the VIP vs public version (although it accepts the same parameters). Also, the UI is intentionally switched off on the VIP version so that developers maintain control, rather than wp-admin users. Contributions are welcome, via the plugin repository.

Using Gutenberg Plugin, rather than Core

When running WordPress Core 5.0+, Gutenberg is loaded from core. However, you can choose to keep using the plugin if you’re interested in being on the forefront for Phase 2 development.

Phase 2 is about thinking outside the box, namely the post and page box, to allow Gutenberg to handle entire-site layouts. We will replace widgets with blocks, so any block will be able to be used in any registered “sidebar” for legacy themes, and we will upgrade “menus” to a navigation block.

In order to keep using the Gutenberg plugin instead of loading Gutenberg from Core, you need to be using at least Gutenberg plugin version 4.5.1. Then, add the following line of code to /vip-config/vip-config.php:

define( 'GUTENBERG_USE_PLUGIN', true );

While 4.5.1 is the minimum version needed to keep using the plugin, we strongly urge you to use at least 4.6.1, which introduces some parsing improvements.

How to disable the privacy tools in WordPress

WordPress 4.9.6 and above will include five new GDPR tools: a privacy policy page, a privacy policy editing helper, a personal data export tool, a personal data erasure tool, and a permissions checkbox that is used with comments.

You can learn more about the release schedule and current feature set for WordPress 4.9.6 on the WordPress.org site.

On the WordPress.com VIP platform these tools will be disabled. On the VIP Go platform these tools will be available, but clients can choose to disable them. By default, the tools are disabled in Multisite for single-site administrators, but are still available for Super Admins.

You can use the map_meta_cap() filter to hide the following tools for all users:

  • privacy policy page
  • privacy policy editing helper
  • personal data export tool
  • personal data erasure tool

If you just want to restrict access to a small group of users (admins for example) the user_has_cap() filter would work as well, with some modifications.

/**
* Disable the privacy tools added in WordPress 4.9.6.
*
* @param array $required_capabilities The primitive capabilities that are required to perform the requested meta capability.
* @param string $requested_capability The requested meta capability
* @param int $user_id The user ID.
* @param array $args Adds the context to the cap. Typically the object ID.
*
* @return array The primitive capabilities that are required to perform the requested meta capability.
*/
function disable_496_privacy_tools( $required_capabilities, $requested_capability, $user_id, $args ) {
$privacy_capabilities = array( 'manage_privacy_options', 'erase_others_personal_data', 'export_others_personal_data' );

if ( in_array( $requested_capability, $privacy_capabilities ) ) {
$required_capabilities[] = 'do_not_allow';
}

return $required_capabilities;
}
add_filter( 'map_meta_cap', 'disable_496_privacy_tools', 10, 4 );

The permissions checkbox that is used with comments, can be disabled using the comment_form_default_fields filter.

Setting up redirects on VIP Go

When you need to implement redirects involving your VIP site, you’ll want to choose one of the methods listed below based mostly upon how many redirects you have and expect to have going forward. This guide will help you figure out your best approach.

Because WordPress.com VIP uses nginx and not apache, there aren’t any .htaccess files to modify. Instead, we have one of three methods available. Which one you use (or combination of them) depends upon how many redirects you need to implement:

  1. For a small number of redirects and redirects which will change frequently, we recommend Safe Redirect Manager plugin;
  2. For larger numbers of redirects (> 300), you will want to use the WPCOM Legacy Redirector plugin
  3. Alternately, you can write redirects directly into your theme code.
  4. On VIP Go you can use vip-config.php for “general” redirects, e.g. to redirect all traffic from one domain to another

If you’re using our VIP Go platform, you may be interested in the cache control of redirects; see “Controlling VIP Go Page Cache”.

Safe Redirect Manager plugin

If you have fewer than 300 redirects, the Safe Redirect Manager plugin is the way to go. This plugin has an admin UI component, so your editors can edit redirects easily. This can be especially important for redirects that are going to change frequently. Safe Redirect Manager also supports regex and wildcards, so if you require that, this is a good option.

Safe Redirect Manager

WPCOM Legacy Redirector plugin

With this plugin, site managers can take advantage of adding Redirects directly from their Dashboards.

Three options come with the plugin:

  1. Add a new redirect rule by going to Redirect Manager -> Add Redirect in the WP Admin of the site
  2. Build redirects from CSV. We just need a CSV file with the following mapping structure: `redirect_from_path,(redirect_to_post_id|redirect_to_path|redirect_to_url)`
  3. Build redirect from a meta field. Just provide the meta_key for posts where the legacy URLs are stored.

Be sure to review the documentation http://wpvip.com/plugins/wpcom-legacy-redirector/. Please contact us before using this plugin to verify your redirect strategy.

Writing redirects directly into your theme code

VIP has a couple of helper functions for broader redirects.

  1. vip_regex_redirects  – advanced 301 redirects using regex to match and redirect URLs. Warning: Since regex is expensive and this will be run on every uncached pageload, you’ll want to keep this small, lean, and mean.
  2. vip_substr_redirects – wildcard redirects based on the beginning of the request path. This is basically an alternative to vip_regex_redirects() for when you only need to redirect /foo/bar/* to somewhere else. Using regex to do this simple check would add lots of overhead.

VIP Go: Domain redirects in vip-config.php

This method will only work on our VIP Go platform. The main advantage of this method is to respond to a request before WordPress is loaded, thus improving the speed and performance of the response. The caveat is that because WordPress has not loaded, you cannot rely on any WordPress APIs or functions.

When redirecting in the vip-config.php file, please omit the /cache-healthcheck? URI from your redirects. This URI is used by the VIP Go platform to check the health of your site, and a redirect response will cause issues with your site being served correctly.

Here’s an example snippet of code which redirects requests for example.net or example.org to the same path on the main site, example.com; e.g. example.net/some/path would be redirected to example.com/some/path:

if ( isset( $_SERVER['HTTP_HOST'] ) ) {
	$http_host   = $_SERVER['HTTP_HOST'];
	$request_uri = $_SERVER['REQUEST_URI'];

	$redirect_to_domain = 'www.example.com';
	$redirect_domains   = [
		'example.net',
		'www.example.net',
		'example.org',
		'www.example.org',
	];

	// Safety checks for redirection:
	// 1. Don't redirect for '/cache-healthcheck?' or monitoring will break
	// 2. Don't redirect in WP CLI context
	if (
			'/cache-healthcheck?' !== $request_uri && // safety
			! ( defined( 'WP_CLI' ) && WP_CLI ) && // safety
			$redirect_to_domain !== $http_host && in_array( $http_host, $redirect_domains, true )
		) {
		header( 'Location: https://' . $redirect_to_domain . $request_uri, true, 301 );
		exit;
	}
}

In a multisite network, code such as the above is necessary to handle redirecting http://www.example.com to example.com (or vice versa) for each subsite. (In single-site installations, this redirect is handled automatically, and no additional configuration is necessary.)

Here’s another example for multisites where there may be redirects for multiple sites, from different domains, resulting in using non-www as the canonical version. Note that for A => [B, C], B and C redirect to A.

if ( isset( $_SERVER['HTTP_HOST'] ) ) {
	$http_host   = $_SERVER['HTTP_HOST'];
	$request_uri = $_SERVER['REQUEST_URI'];

	$redirect_domains = [
		/*
		'to-this-domain.com' => [
			'from-this-domain.com',
			'or-from-this-domain.com',
		],
		*/
		'example-one.com'   => [
			'www.example-one.com',
			'www.exampleone.com',
		],
		'example-two.com'   => [
			'www.example-two.com',
			'www.exampletwo.com',
		],
		'example-three.com' => [
			'www.example-three.com',
			'www.examplethree.com',
		],
	];

	// Safety checks for redirection:
	// 1. Don't redirect for '/cache-healthcheck?' or monitoring will break
	// 2. Don't redirect in WP CLI context 
	foreach ( $redirect_domains as $redirect_to => $redirect_from_domains ) {
		if (
				'/cache-healthcheck?' !== $request_uri && // safety
				! ( defined( 'WP_CLI' ) && WP_CLI ) && // safety
				$redirect_to !== $http_host && in_array( $http_host, $redirect_from_domains, true )
			) {
			header( 'Location: https://' . $redirect_to . $request_uri, true, 301 );
			exit;
		}
	}
}

 

Additional Resources

VIP Dashboard

The VIP Dashboard is the home for managing your VIP Go applications. Upon signing in, a list of all your apps is visible:

For updates & news on VIP, click on the info icon in the upper right-hand corner. You will also find shortcuts to documentation.

Click on your profile icon to set up the VIP CLI.

Tools

Currently, each application has Dashboard, Sync, and Domains tools:


Dashboard
– Displays an activity log for each of the app’s environments.
Click on the shortcuts for each environment to see the corresponding WordPress login and GitHub repositories.

Sync – Features a real-time data syncing tool to copy content from production to a child environment. Find out more about data sync here.

Domains – View a list of your domains available to the app. Switch environments and see which domains are associated with each environment, map domains, view DNS Instructions, and provision LE Certificates.

As we continue to design and develop new platform functionality, we are looking for testers to provide feedback on early ideas and prototypes. If you would like to get involved, please get in touch via Zendesk.

Authentication

GitHub is used to authenticate, which allows access to VIP Go applications to be provided by checking which VIP Go repositories the user has access to.

To work with VIP Go applications on the command line, install VIP CLI.

Troubleshooting

e002 User Not Found!

This error states we were unable to find any VIP Go repositories the user has access to. If the GitHub user account does not have access to any VIP Go repositories, authentication will not be successful. 

To gain access to a VIP Go repository, please contact our Support team.

Copying production data to your non-production environments on VIP Go

VIP Go has built-in support for syncing/copying data from your production to non-production environments.

This feature facilitates testing and QA of new features and allows teams to accurately reproduce and examine errors in a non-production environment. Our sync process seamlessly and speedily handles the data copying, modifies URLs to match the new environment, and adds a hook for further client-specific customization.

If you have any questions about the data sync process, please contact our Support team.

Running a data sync via the VIP Dashboard

The VIP Dashboard offers a tool to sync data from production to non-production environments.

Running a data sync via CLI

You will need the VIP CLI installed on your local machine: VIP CLI installation instructions.

Step 1: Find the application name or ID you wish to sync

You can do this with the vip app list command, which will give you back a list of apps. You want the name or the id.

Step 2: Initiate the sync

Run the sync command with the app name, e.g. vip sync --app=my-app, or the app id, e.g. vip sync --app=000. You will be prompted for the environment you wish to sync to, i.e. the target environment. You can get more information about sync command options by using the help parameter, i.e. vip sync --help.

What happens during a Data Sync?

The operation is designed to not impact the production site in any way; all processing happens on the target environment:

  1. The target environment is protected via “maintenance mode”, this prevents access to the site while the data is being restored and manipulated, and ensures no changes can be made (and lost) during this time
  2. The most recent backup from production is loaded into the target environment
  3. We update all references to production domains to match the target environment
    • The primary production domain is replaced with the primary domain for the target environment; e.g. if your production site is at example.com and your target non-production site is example-com-develop.go-vip.net, then we’ll replace all instances of example.com with example-com-develop.go-vip.net
    • For any additional domains, we use config file with a URL map stored in your site’s code repository. See “domain mapping” below for more details
  4. Caches are flushed
  5. Jetpack is re-connected
  6. Maintenance mode is deactivated
  7. A cron event (vip_go_migration_cleanup) is fired for further cleanup
    • See below for more detail on this hook, and associated considerations

Domain Mapping Config

If your application has multiple domains in use (this is common in a multisite install, for example), Data Sync needs a bit more information to correctly update URL references in your non-production environments. This can be done using config files in your code repository.

If you don’t have multiple domains, you can skip this section.

Structure

Data Sync relies on YAML-based config files to correctly handle URL replacements. Here’s what the basic structure looks like:

data_sync:
  domain_map:
    prod-url-1: nonprod-url-1
    prod-url-2: nonprod-url-2

Note that YAML requires spaces for indents, and indents are critical to YAML syntax. Two spaces are usually good.

To illustrate, here’s an example. Let’s say your production environment uses the following domains:

  1. en.example.com
  2. fr.example.com

And your non-production environment has the following matching domains:

  1. en-preprod.example.com
  2. fr-preprod.example.com

The config file would look something like this:

data_sync:
  domain_map:
    en.example.com: en-preprod.example.com
    fr.example.com: fr-preprod.example.com

When Data Sync runs, all instances of en.example.com will be replaced with en-preprod.example.com and all instances of fr.example.com will be replaced with fr-preprod.example.com.

If your environments use a mixture of domain-, subdomain- and/or subdirectory-based sites, you can still specify those via config.

Let’s say we had another site in our example above (example.com/translate) which had a matching non-production site (preprod.example.com/translate) our config would look something like this:

data_sync:
  domain_map:
    en.example.com: en-preprod.example.com
    fr.example.com: fr-preprod.example.com
    example.com/translate: preprod.example.com/translate

Naming

If you have multiple applications or environments using the same repo, chances are that each of them will need a unique config. We use a specific naming convention for config files to correctly identify the application and environment:

.vip.[app].[environment].yml

The placeholders are:

  • [app] is the name/slug used to identify your (production) application.
  • [environment] is the environment that this config file applies to.

Here’s an example: .vip.example.develop.yml (this config file would be used for the example application’s develop environment).

In some (very rare) circumstances, you may have multiple environments using the same environment name (e.g. multiple develop environments). In this case, you can add the unique name of your non-production environment at the end (i.e. .vip.[app].[environment].[child].yml; an example would be .vip.example.develop.instance-01.yml).

The “Sync” section of the VIP Dashboard contains a link that will generate this file in the appropriate place in the repo, with the correct naming, for any child environment that is missing one.

If extra assistance is needed, please let us know and we’ll be happy to help.

Location

Config files for Data Sync should live in a config folder in the root of the branch that tracks your production environment (usually master).

Here’s what the root of your master branch might look like:

|– client-mu-plugins/
|– config/
|—– .vip.example.develop.yml
|—– .vip.example.preprod.yml
|– images/
|– languages/
|– plugins/
|– themes/
|– vip-config/

Maintenance

Please note that you will need to update the config files if any domains are added, removed, or modified across any of your environments. We’ll be adding tools in the future to help simplify this process.

Custom cleanup operations

The vip_go_migration_cleanup hook can be used for custom cleanup of your application data. Some examples of cleanup operation include removing production API keys or performing brief data manipulation.

Note: The cleanup hook runs after maintenance mode is lifted.

Here’s some example code that shows you how to use this cleanup event:

/**
 * Run some custom cleanup after a migration.
 *
 * @uses vip_go_migration_cleanup
 */
function my_action_vip_go_migration_cleanup() {
	// Safety first: Don't do anything in 
	// the production environment
	if ( 'production' === VIP_GO_APP_ENVIRONMENT ) {
		return;
	}

	delete_option( 'my_social_api_token' );
}
add_action( 'vip_go_migration_cleanup', 'my_action_vip_go_migration_cleanup' );

On a multisite, the cleanup hook is run individually on each subsite within that multisite.

Media library files and UnionFS

Instead of copying all the media library files from the production environment, for large media libraries of millions of files and hundreds of gigabytes of data could take many hours, VIP Go provides a service called UnionFS.

UnionFS works by making the files for the production site available to all non-production sites in read-only mode. Files shared by UnionFS in this way are served from the same infrastructure and have the same caching rules applied.

The non-production environment can upload a file of the same name to the same URL, and this will override the file shown on the non-production site (the production file remains untouched).

Caveats:

  • The non-production environment cannot delete a file on the production site. The operation will not error, but the production file will still be shared and available from the non-production environment.
  • If the file is deleted from production, it will no longer be available to be shared with the non-production environments.

Installing and activating plugins on VIP Go

Before reading this, it will be useful to read our documentation on your VIP Go codebase.

This is the preferred approach for installing and activating plugins for VIP Go:

  • Add the plugins to your plugins directory
  • Activate the plugins via a plugin loader file in client-mu-plugins

Plugins can also be activated via the “Plugins” screen on your WordPress admin area.

If you are familiar with developing for WordPress.com hosted VIP sites, you no longer need to bundle your custom plugins with your theme and we would encourage installing to the plugins directory in your site’s repository.

Installing plugins

On VIP Go, all plugins must be installed by adding them to your Git version control repository, hosted on GitHub. Having the plugin code in git version control allows multiple developers in your own team, and of course in the VIP team, to work on your code together, maintain and improve the site, and debug issues. On VIP Go, we deploy the code from git, which also allows us to roll back to a previous “known good” state, if the site encounters issues. (Please note, adding plugins via the WordPress admin area is not supported.)

As the VIP Go workflow is Git-based, you can also reference submodules in your repository which will make maintaining third-party code easier. VIP Go currently only supports public submodules, e.g. you cannot submodule a private GitHub repository or a Git repository hosted elsewhere which requires authentication.

Note that some plugin repositories, such as that for AMP for WordPress, contain unbuilt versions whereby the command, composer install, needs to be run for the plugin to work properly. Repositories like this one cannot be referenced as submodules as we do not run these build commands on the server. In such cases, developers should use one of the other methods outlined on this page.

Installing to the plugins directory

On VIP Go, there is a dedicated plugins directory in the root of the site git repository; this directory works similarly to WP_CONTENT_DIR . '/plugins/' in a self-hosted WordPress installation. Installing plugins to this location is a matter of adding the plugin to a unique directory in plugins, e.g. plugins/my-plugin, commit, then push to GitHub (see our documentation on how code is deployed on VIP Go).

We recommend that third-party plugins always go in the plugins directory, to allow for better compatibility with systems (like security-related plugin updates) that expect plugins to be in a specific location. Custom plugins can go in either the plugins or client-mu-plugins directory, depending on the preferred loading behaviour.

Loading or activating plugins in the plugins directory

The plugins in the plugins directory can be activated and deactivated from the standard Plugins menu in the WordPress admin area, however, we recommend loading your plugins from code. Loading plugins in code results in more control and a greater consistency across your production, non-production environments, and local development environments.

If loading your plugins via code, you should use a plugin loader file in client-mu-plugins (example) using the wpcom_vip_load_plugin() function:

wpcom_vip_load_plugin( 'plugin-name' );
// Note this requires a specific naming structure: /plugin-name/plugin-name.php
// You can also specify a specific root file: wpcom_vip_load_plugin( 'plugin-name/plugin.php' );

Installing to the client-mu-plugins directory

In addition to the directory of `mu-plugins` that loads by default into all VIP environments, there is a dedicated client-mu-plugins directory in the root of the site git repository; this directory works similarly to WP_CONTENT_DIR . '/mu-plugins/' in a self-hosted WordPress installation.

If you want your plugins to be installed as MU (“must use”) plugins then you can place them in the client-mu-plugins directory. We strongly recommend that only custom (and not third-party) plugins go in the client-mu-plugins directory, depending on the preferred loading behaviour. Specifically, client-mu-plugins should be for any code that should be auto-loaded or run earlier in the WordPress load process.

MU plugins behave differently than plugins installed in the plugins directory, so be sure to read up on the pros and cons and make a measured choice. If you have any questions, we’re happy to help.

Note that plugins contained within a directory in the client-mu-plugins directory will need a “loader” file in the root of this directory to require the main plugin file. The contents of the loader file can be as simple as:

<?php
require WPCOM_VIP_CLIENT_MU_PLUGIN_DIR . '/my-plugin/my-plugin.php';

When do plugins run?

Depending on how the plugin is loaded, it will hook into the WordPress load order at different points (from latest to earliest):

  • Manually activated from the wp-admin Plugins screen (supported, but code activation is preferred): before the plugins_loaded hook
  • wpcom_vip_load_plugin( 'plugin-name' ) in your theme functions.php (supported, but code activation from client-mu-plugins is preferred): before the after_setup_theme hook
  • wpcom_vip_load_plugin( 'plugin-name' ) from client-mu-plugins (recommended): on the muplugins_loaded hook

Sometimes plugins may have dependencies on running before specific hooks, in order to hook on very early actions or filters you will need to load from client-mu-plugins.

Version updates and maintenance

We recommend that you always run the latest version of each of the plugins on your site, however, this is not a requirement as we recognise that different development teams run different maintenance and patch cycles.

Automatically generated Pull Requests from our bot

The VIP team maintains a bot, wpcomvip-bot, to generate pull requests updating some plugins and themes. We do not currently have full coverage of all plugins used by all VIPs and the bot generated PRs are offered as a convenience; it remains the responsibility of individual VIP client developer teams to maintain the plugins used for their sites.

Remaining consistent with standard VIP practice, PRs will not be merged on your behalf unless they contain critical security fixes or resolve an existing or potential site outage. We recommend you merge the code into a development and/or non-production environments and test there before deploying to production.

Security fixes

Where the VIP team become aware of a security update, we will issue and merge PRs for affected production sites as soon as we can.

Featured partner plugins

On VIP Go, our Featured Partner plugins are installed and maintained the same as other plugins, read the documentation above and follow the directions. If you have any questions, please get in touch.

ACF 5 and VIP Go

Advanced Custom Fields (ACF) is a popular plugin that many VIP clients choose to use on their sites. However, ACF is not available for clients on the WordPress.com VIP platform, and has not undergone a full, line-by-line review for use on VIP Go. Clients wishing to use ACF on VIP Go accept the security and performance risks of using it. This page outlines some additional steps needed to make ACF more secure and performant, but should not be interpreted as the equivalent of a line-by-line review, or VIP’s approval of the plugin.

Please note that while ACF 5 can be used on most VIP Go sites, ACF 4 is unavailable. If in doubt about whether ACF is allowed for use on your VIP site, please contact us.

When using ACF 5 and ACF 5 Pro, several additional steps are needed in order to make ACF secure, and avoid performance issues:

  • Hide the Admin UI
  • Define fields in PHP
  • Use taxonomies for searchable fields
  • Avoid the_field and escape
  • Secure fields that allow arbitrary output

Hide the ACF Admin UI

The fields UI can be used to add arbitrary fields, including unsafe fields. For example it can allow an admin to display the passwords of users. You can disable the UI using this filter:


add_filter('acf/settings/show_admin', '__return_false');

Define Fields in PHP

In order to make sure that all ACF usage is secure, define the fields in PHP or local json, rather than at runtime. This way they remained versioned and safe. This can be done via the import export menu of a local developer environment to setup the fields available and export them to PHP.

Documentation on how to do this can be found here on the ACF website.

Alternatively, fields can be defined via the local JSON feature as described here, but keep in mind that saving local JSON will not work in production as the filesystem is read only, nor is it desirable as it would bypass the security benefits.

Being Mindful of Taxonomy Term Storage

If an ACF field is going to be queried, filtered, or searched for in a post query, use the taxonomy data checkbox so that the field is stored as a term, not a post meta value. This ensures performance is not impacted by expensive meta queries on the frontend

the_field and Escaping

the_field has no context as to when or where it is called. So how is it to know if it should be using esc_url, esc_attr or wp_kses_post? It doesn’t, which makes it dangerous from a security point of view. Instead, use get_field in combination with an escaping function, e.g.

$url = get_field( 'custom_link' );
echo esc_url( $url );

Flexible content is the exception to this, and should be clearly marked on usage via comments

Fields That Use Arbitrary Output

If the field types that allow arbitrary output are to be used, they must be accounted for in the acf/format_value and equivalent filters such as acf/format_value/type=textarea.

For example:

function vip_make_acf_text_areas_safe( $value, $post_id, $field ) {
	return wp_kses_post( $value );
}

add_filter('acf/format_value/type=textarea', 'vip_make_acf_text_areas_safe', 10, 3);

This way different escaping can be applied via different `$field` values. Alternatively, if all fields of that type use the same escaping, this can be done instead:

add_filter('acf/format_value/type=textarea', 'wp_kses_post', 10, 1);

For more information, see: https://www.advancedcustomfields.com/resources/acf-format_value/

New Relic on VIP Go

New Relic is available for clients running on our VIP Go platform. New Relic is an application monitoring platform and on VIP Go, you can use it to monitor the PHP (WordPress) code and browser performance of your site or application.

If your team would like to use New Relic, please contact us and we’ll be happy to arrange access for you. We do not charge for access to New Relic.

What New Relic plan will I be running?

The New Relic plan we run is as follows

  • Web Pro Annual – this is the PHP application monitoring which covers your WordPress application code
  • Mobile Lite
  • Insights Free
  • Browser Lite
  • Synthetics Lite

The differentiation mainly affects the retention of monitoring data, with Pro elements of the plan retaining data for longer than Lite elements.

Which parts of the VIP infrastructure is New Relic installed on?

New Relic is installed and activated on a maximum of 50% of web containers per WordPress environment. Typically, a non-production environment will run just two web containers (and will therefore have one New Relic instance), and the majority of our production environments run less than 15 web containers. The maximum number of New Relic instances per environment is 15.

By default, the New Relic agent is not activated on CLI containers for WordPress environments; these are the containers which run WP-CLI commands and these containers also run WordPress Cron events. If it is important to monitor the performance of your Cron events and WP-CLI commands, please get in touch and we can activate the New Relic activation on the CLI containers for any of your environments.

The New Relic agent is not installed on any other parts of our infrastructure (for example, the VIP CDN or database containers).

How many users can I have accessing New Relic?

As many as you require, please contact us with the email addresses of any team members who need access to your New Relic monitoring.

It is your responsibility to request leaving members of your teams have their New Relic access removed.

Can I turn off New Relic Browser Monitoring?

Yes! New Relic have some documentation on disabling Browser monitoring and real user monitoring.

Can I use the New Relic PHP API and SDK to enhance the data captured by New Relic?

New Relic maintains a suite of PHP functions which can be used to  add data to transactions, name transactions, etc, you can read more in their documentation. WordPress.com VIP uses this API to enhance the data provided by all VIP Go WordPress applications (see the code).

We don’t offer custom PHP INI configuration for individual WordPress applications (sites), but you may find that some of the configuration can be set in PHP at runtime using ini_set().

Can I use my own New Relic account?

The VIP team uses New Relic to monitor all the sites and applications on the VIP Go platform; therefore we require that your application uses the WordPress.com VIP New Relic account in order for us to continue doing so.

How do I use New Relic

New Relic themselves maintain documentation:

If you have any further questions, please feel free to contact us and we’ll be happy to help where we can.

wpcomvip-bot

The wpcomvip-bot is a GitHub bot for VIP Go that keeps repositories up-to-date and helps with code reviews.

What is this Pull Request?

If you see a Pull Request from wpcomvip-bot, it means there is an update available for a plugin in your repository. Remaining consistent with standard VIP practice, these updates will never be merged on your behalf unless they contain critical security or performance updates required to maintain stability on your site. It is, however, recommended that you test and deploy these changes in a timely manner.

Note: The changes have not necessarily been tested by VIP. Please test all changes before merging them to production.

Testing

You can git merge the PR branch into a development branch. PRs are opened with branch names prefixed with vip/. You can find the name of the branch under the Pull Request title in the GitHub UI.

If that doesn’t work,  you can also git cherry-pick the commit into your testing branch. The SHA-1 hash of the commit can also be found in the GitHub UI, in the Commits tab.

Code Review

The bot also helps with code reviews. You may see comments in Pull Requests with review feedback. If there are any questions about the feedback, feel free to leave them in the Pull Request or open a ticket with VIP Support.

Increasing Cron event concurrency on VIP Go

By default, Cron Control will only run one event of a given type (i.e. with the same action) at any one time. This is designed to avoid issues where multiple events simultaneously write to the same values in the database, particularly in alloptions, causing issues with data loss.

In some circumstances, multiple events with the same action can safely run in parallel. This document explains how to whitelist these events and specify the concurrency for them.

When to increase concurrency for a Cron event

If you are expecting only low numbers of your Cron event at any one time, e.g. less than 5, then it is not worth increasing concurrency.

If your event writes data to an option which is autoloaded, it is not safe for increased concurrency.

How to increase Cron event concurrency

To specify the concurrency level for an event, filter the a8c_cron_control_concurrent_event_whitelist hook, adding the event to the array using the action name as the key and an integer representing the number of simultaneous events of that action which you want to run. Here is an example:

/*
 * Increase Cron Control's concurrency limit for my_async_send_ping action to 10
 */
add_filter( 'a8c_cron_control_concurrent_event_whitelist', function( $wl ) {
	$wl['my_async_send_ping'] = 10;
	return $wl;
} );

If you have any doubts or questions, please send us a support ticket and we’ll be happy to help.

Asynchronous publishing actions on VIP Go

Often when a post is published, there’s a number of actions associated with that publication. Some of those actions can take a while to perform, for example syndicating content out to sibling sites, pushing to social media, etc.

On VIP Go you can easily offload actions to be processed asynchronously on the Cron infrastructure for your site when a post is published or transitions status. Offloading actions reduces the processing time required for your publish action, which makes for a much faster and nicer publishing experience for your editors.

How to offload actions

Offloading an action on publish for asynchronous processing is as simple as changing the publish action hook to one of our async_ equivalents. The hooks we have available for asynchronous processing are:

  • async_transition_post_status, the asynchronous equivalent of transition_post_status (core hook docs)
  • async_{$old_status}_to_{$new_status}, the asynchronous equivalent of {$old_status}_to_{$new_status} (core hook docs)
  • async_{$new_status}_{$post->post_type}, the asynchronous equivalent of {$new_status}_{$post->post_type} (core hook docs)

This hypothetical code example splits off some functionality to be processed as the post is published, and some longer running functionality to be processed asynchronously:

/**
 * Runs when a post of type `post` is published.
 */
function my_post_publish_actions() {
	// This function performs quick actions,
	// and things which must happen immediately
}
add_action( 'publish_post', 'my_post_publish_actions' );

/**
 * Runs asynchronously, when a post of type `post` is published.
 */
function my_slower_post_publish_actions() {
	// This function performs slower actions,
	// like syndication and other longer 
	// running tasks
}
add_action( 'async_publish_post', 'my_slower_post_publish_actions' );

The code which powers our asynchronous offloading functionality works by scheduling an immediate Cron event to be processed on our dedicated Cron infrastructure, during which these async_* action hooks are called.

Skipping Async Actions

There may be cases where you do not need to queue actions for specific post types (e.g. private post types with high volumes of activity). You can easily skip queueing async actions for them using the wpcom_async_transition_post_status_schedule_async filter.

The following snippet skips async transition actions for the all posts with the `x-transactions` post type:

add_filter( 'wpcom_async_transition_post_status_schedule_async', function( $value, $args ) {
    if ( 'x-transactions' === get_post_type( $args['post_id'] ) ) {
        $value = false;
    }
    return $value;
}, 10, 2 );

Caveats

Separate Contexts: The asynchronous actions will be processed on separate infrastructure to the web requests; this means that the asynchronous action will not have access to anything which has been placed in the global context, or access files placed in a System tmp directory during a previous request, etc.

Non-Atomic: The events are not guaranteed to fire in the order that they occur since we process them concurrently. This means that if you trash a post and then untrash it, the async events for those actions may fire out or of order or at the same time. This means that if you need to take action based on the status of a post, you should verify it using get_post( $post->ID )->post_status before doing so.

The File System on VIP Go

The VIP Platform has been designed with for high-scale performance and security. One of the ways we do that is to run all web servers in read-only mode. This helps protect applications on the platform against many common form of attack (e.g. vulnerabilities that allow installation of backdoor shells and other malicious files); allows for automated, high-capacity horizontal scaling; and dynamic image resizing and manipulation.

Media Uploads

The VIP Platform stores all uploaded media in a globally distributed object store called the VIP Files Service. This is seamlessly integrated with your WordPress application and all common operation including uploads, cropping, editing, and deletes work as expected.

All images uploaded to the VIP Files Service can make use of the Photon API, which allows for dynamic resizing and other manipulation (cropping, letterboxing, filters, etc.). Because image sizes all are created on-the-fly, uploads are significantly faster and changes to image sizes is much easier (no need to run scripts to regenerate files).

Programmatic Access to Media Uploads

For programmatic access to media stored in the VIP Files Service, such as uploading and modifying files, you can use several different APIs.

For simple file uploads, the media_handle_sideload, media_sideload_image, or wp_upload_bits functions are very easy to work with.

For more complex interactions, we’ve integrated a custom PHP Stream Wrapper, which allows most filesystem functions (e.g. fopen, fwrite, file_put_contents, etc.) to work with media uploads. This means that most plugins that interact with media uploads will work on the VIP Platform without any modifications. There are some caveats

A few caveats to be aware of:

  • This only works for the uploads directory (i.e. paths that begin with /wp-content/uploads/; other paths will not work).
  • We default to a year/month directory structure for media uploads. This can be overridden by adding the following filter to your theme’s functions.php file:
    add_filter( 'pre_option_uploads_use_yearmonth_folders', function() {return '0';}, 9999 );
    
  • You must use the WordPress function wp_get_upload_dir() or wp_upload_dir() to generate the correct, writeable path (hard-coding a path like /wp-content/uploads/... will not work).
  • Currently, not all filesystem functions and operations are supported (e.g. directory traversal).
  • Not all use cases are ideal or supported (e.g. files with high volumes of updates, such as logging).
  • Because communication with the VIP Files Service happens over HTTP, high number of function calls (e.g. lots of file_exists checks) can result in a performance hit and should be used with caution.
  • file_exists will always return true for extension-free values (e.g. directory names).

Examples

Here’s a simple example of how to upload a file using PHP functions:


$csv_content = '1,hello,admin';

$upload_dir = wp_get_upload_dir()['basedir'];

$file_path = $upload_dir . '/csv/updated.csv';

file_put_contents( $file_path, $csv_content );

// The file will now be accessible at <a href="https://example-com.go-vip.net/wp-content/uploads/csv/updated.csv">https://example-com.go-vip.net/wp-content/uploads/csv/updated.csv</a>

Here’s an example that parses an uploaded CSV and stores the contents in a variable:


$csv_attachment = get_attached_file( 1234 );

$csv_file = file( $csv_attachment );

$csv_content = array_map( 'str_getcsv', $csv_file );

Local File Operations

There may be cases where you need to perform local manipulations to temporary files (e.g. download a zip, extract its contents, and then upload them to WordPress). For these cases, you can use the system temp directory, which is the only local, writeable path on web servers on the VIP Platform.

A few things to keep in mind:

  • Use get_temp_dir() to find the right path. Or if you need to generate a temporary file, use wp_tempnam().
  • You should clean up and remove all files after they’re no longer needed. Use unlink() to remove those files.
  • Files and directories can only be relied on for the duration of the current request.
    • Each request may be served by a different container and the sequence of requests from a given user are not guaranteed to be served by the same container.
    • Containers are transient and may be created and destroyed due to autoscaling.

If you have any questions about the VIP Files Service or working with the filesystem on the VIP Platform, please don’t hesitate to get in touch with the VIP team so we can work with you on the implementation.

sunrise.php on VIP Go

When developing for a WordPress multisite, it is often necessary to load some custom code very early in the WordPress application’s loading sequence, e.g. to implement some custom domain mapping functionality. For multisite instances on VIP Go, we make it easy to load custom code as part of WordPress Core’s sunrise.php.

Before discussing how, it’s important to note that sunrise.php loads very early in WordPress, and most of its functions won’t be available. Further, sunrise.php loads before mu-plugins, any active plugins, and the active theme. As such, what can be done in sunrise.php is limited, and largely confined to using pure PHP to set constants that override WordPress Core behaviour.

That said, to load additional code in sunrise.php, please follow these steps:

  1. Create a vip-config/client-sunrise.php file in your site’s repository. On VIP Go, if the file is present, it will be loaded automatically (ref).To achieve the same behaviour on a local development environment:
  2. Copy wp-content/mu-plugins/lib/sunrise/sunrise.php to wp-content/sunrise.php.
  3. Add define(‘SUNRISE’, true) to wp-config.php.

Guidelines

As already mentioned, the purpose of vip-config/client-sunrise.php is to execute PHP early in the WordPress cycle, before most of WordPress, including any theme or plugin code, loads. Typical uses for this file therefore share a few characteristics:

  • The problem being solved requires early execution (for instance, for performance reasons or to override aspects of the request very early on.) Code that can run later in execution should typically be written in the plugin or theme context and leverage WordPress’ Plugin API.
  • The problem can be solved with (mostly) pure PHP.
  • The problem being solved does not require database access or access to any persistent data, but often makes decisions or performs actions based on intrinsic characteristics of the request, such as the domain or URI.  Because it executes on every single request handled by WordPress, make sure your use of sunrise.php does not introduce the kinds of performance penalties associated with queries or other expensive operations.
  • The scope of the problem being solved is well understood and minimized. Code executed within sunrise.php is kind of like a sledgehammer, and can have far-reaching effects. Do as much as you can to limit the conditions under which the code executes (such as limiting it to only certain URLs or certain domains) to reduce the probability of unintended side-effects.

Example: multiple domains in multisite on Go

By default, VIP Go enforces the use of a single domain per site in a multisite (notably, this is not exactly the same as core multisite, which will cause a domain not associated with any site in the network to resolve to site_id 1.) Some VIPs have found it useful to allow multiple domains to resolve to a single site in the network either temporarily or permanently. In the example below, this multi-domain configuration is limited to REST API requests, but it could be more general, or differently targeted.

// Allow REST API requests to be served on one of several domains
$clientslug_custom_sunrise_domains = ['example.com', 'example.blog', 'example.go-vip.net'];
// Cause each of these domains to load site_id 1
$clientslug_custom_sunrise_site_id = 1;
// @codingStandardsIgnoreLine -- use of $_SERVER okay here
if ( in_array( $_SERVER['HTTP_HOST'], $clientslug_custom_sunrise_domains, true ) && 0 === strpos( $_SERVER[ 'REQUEST_URI' ], '/wp-json' ) ) {
	// These domains are each associated with site_id 1 of network 1
	$current_blog = get_site( $clientslug_custom_sunrise_site_id );
	$current_site = get_network( 1 ); // This should always be 1, unless you are running multiple WordPress networks.
}

Other uses of client-sunrise.php might include logic around redirects (that don’t require database lookups) or other manipulation of fundamental WordPress constants or globals based on the request.

If you have questions about whether a solution involving sunrise is appropriate for your situation, please reach out to VIP support — we’d be happy to discuss your issue further.

Elasticsearch on VIP Go

Jetpack Search is a powerful replacement for WordPress’ built-in search, available to all sites hosted on the VIP Go platform. It provides higher quality results, better scalability, and runs in the WordPress.com cloud. It’s powered by Elasticsearch (ES), the industry-leading open-source search engine.

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries.

Why use Jetpack Search?

With Jetpack Search, you can easily replace the WordPress’ built-in search functionality with a supercharged search that is more accurate and significantly faster (in some cases, we’ve seen queries that are 10-300x faster). You can also utilize powerful features such as Aggregations (to allow search filters/drill-downs), Geolocation-based queries, and other complex queries that often don’t scale well within WordPress (e.g. NOT IN, taxonomy, postmeta, etc).

You get a fully-managed search service powered by Elasticsearch, where we handle all the heavy lifting around mapping, indexing, and making your site’s data available in an easily searchable format.

How Does It Work?

There are two prerequisites for using Jetpack Search on a VIP Go site:

  1. An active Jetpack connection on your site.
  2. An active Elasticsearch Index on WordPress.com.

The content from your site is synced to the WordPress.com cloud via the Jetpack Sync module and then indexed in Elasticsearch. Most content updates are synced and indexed within a minute.

Note that the index needs to be created and set up by the VIP team; please open a support ticket and we’ll be happy to do that for you.

Integrating Jetpack Search

The easiest way to integrate Jetpack Search is to enable the module.

How to activate the Search module

Activating the search module requires two steps:

  1. Add the JETPACK_SEARCH_VIP_INDEX constant and set it to true.
  2. Enable the Search module via code.
define( 'JETPACK_SEARCH_VIP_INDEX', true );

add_filter( 'jetpack_active_modules', 'x_enable_jetpack_search_module', 9999 );

function x_enable_jetpack_search_module( $modules ) {
    if ( ! in_array( 'search', $modules, true ) ) {
        $modules[] = 'search';
    }
    return $modules;
}

Note that this needs to be loaded before the plugins_loaded hook is fired and we recommend doing so via client-mu-plugins.

Filtering Search

Jetpack Search provides powerful filtering capabilities to allow users to refine the results across a number of criteria, like post type, category, and date. These are also sometimes referred to as Aggregations (and in earlier versions as Facets).

To enable filters on Jetpack Search, you need to define which filters you want to enable and then integrate the filtering into your site.

To define the filters, use Jetpack_Search::set_filters():

add_action( 'init', 'x_init_jetpack_search_filters' );
function x_init_jetpack_search_filters() {
    Jetpack_Search::instance()->set_filters( [
        'Content Type' => [
            'type'     => 'post_type',
            'count'    => 10,
        ],
        'Categories' => [
            'type'     => 'taxonomy',
            'taxonomy' => 'category',
            'count'    => 10,
        ],
        'Tags' => [
            'type'     => 'taxonomy',
            'taxonomy' => 'post_tag',
            'count'    => 10,
        ],
        'Year' => [
            'type'     => 'date_histogram',
            'field'    => 'post_date',
            'interval' => 'year',
            'count'    => 10,
        ],
        'Month' => [
            'type'     => 'date_histogram',
            'field'    => 'post_date',
            'interval' => 'month',
            'count'    => 10,
        ],
    ] );
}

To integrate with your site, you can use the “Search Filters” widget included in Jetpack. Alternatively, for more control, you can integrate the filters yourself. See the code for the widget as a guideline on how to do so.

Modifying Queries

You can modify the Elasticsearch queries generated by the plugin by hooking into the jetpack_search_es_query_args filter and adjusting the args as needed.

For example, here’s an easy way to limit search results to a specific date range:

add_filter( 'jetpack_search_es_query_args', function( $args, $query ) {
    $args['filter']['range']['date'] = array(
        'gte' => '2016-01-01',
        'lte' => '2016-12-31',
    );

    return $args;
}, 10, 2 );

Searching Through And Indexing Post Meta

By default, only a small subset of post metadata is currently synced and indexed. To index and be able to query across additional metadata, you can whitelist them:

add_filter( 'jetpack_sync_post_meta_whitelist', function( $keys ) {
    // Any keys
    $keys[] = 'vip_post_subtitle';
    return $keys;
} );

After the whitelist addition, the site’s content will need to be re-synced and re-indexed. This can be done from the WordPress.com Dashboard under Manage > Settings > Site Tools > Manage your connection > initiate a sync manually.

You can then do a term query against meta.[NAME].value where [NAME] is the meta key you want to search through.

Note: if you add the jetpack_search_es_wp_query_args filter to your code you must include all fields you wish to index as this filter overrides the default filters. For example, to continue to index  title, content, and author they must be included in the array or they won’t be indexed.

add_filter( 'jetpack_search_es_wp_query_args', function( $args, $query ) {
 $args['query_fields'] = [
 'title',
 'content',
 'author',
 'tag',
 'category',
 'meta.vip_post_subtitle.value',

 // You can also search across taxonomy fields.
 // e.g. searching through terms in a "location" taxonomy
 // 'taxonomy.location.name',
 ];

 return $args;
}, 10, 2 );

You can also filter based on metadata. Here’s an example that excludes sponsored posts from search results:

add_filter( 'jetpack_search_es_query_args', function ( $es_args, $query ) {
	$filter = [
		'not' => [
			'term' => [
				'meta.is_sponsored_post.value' => 1,
			],
		],
	];

	if ( ! isset( $es_args['filter'] ) ) {
		$es_args['filter'] = $filter;
	} elseif ( ! isset( $es_args['filter'] ) ) {
		$es_args['filter']['and'] = [ $filter ];
	} else {
		$es_args['filter']['and'][] = $filter;
	}

	return $es_args;
}, 10, 2 );

Complex Data Types

While it is possible to whitelist additional postmeta, there are two types of data that are not indexed:

  1. “private” postmeta (i.e. keys prefixed with _)
  2. non-primitive data types like arrays and objects

All content in the default index is meant to be public, so private metadata is ignored by default. Serialized versions of objects and arrays tend to include additional metadata which doesn’t make sense to index as-is.

To work around these restrictions, you can extract out searchable pieces of content into separate post meta entries on add/update (and remove on delete). The only use for these entries is for indexing and querying.

For example, let’s say we have a post meta entry (x_content_blocks) which stores an object with various content strings and other metadata used to build out our page. As-is, the content within this object will not be searchable. However, we can extract out all the searchable content any time this post meta entry is added or updated. The extracted content can be combined together into a single string and inserted into a new post meta entry called es_x_content_blocks. Similarly, we can remove the meta entry when all the content blocks are deleted.

We can then whitelist the meta key, proceed with a full re-sync and index, and then be able to search through this field.

Here’s some sample code that you can use as a headstart:

add_action( 'updated_post_meta', 'x_maybe_require_search_extraction', 10, 4 );
add_action( 'added_post_meta', 'x_maybe_require_search_extraction', 10, 4 );
add_action( 'deleted_post_meta', 'x_maybe_require_search_extraction', 10, 4 );

function x_maybe_require_search_extraction( $meta_id, $post_id, $meta_key, $meta_value ) {
    if ( ! in_array( $meta_key, [ /* list of content keys go here */ ], true ) ) {
        return;
    }

    global $x_require_search_extraction;
    if ( ! isset( $x_require_search_extraction ) ) {
        $x_require_search_extraction = [];
    }

    if ( ! in_array( $post_id, $x_require_search_extraction, true ) ) {
        $x_require_search_extraction[] = $post_id;
    }
}

add_action( 'shutdown', 'x_extract_searchable_content' );
function x_extract_searchable_content() {
    global $x_require_search_extraction;

    if ( empty( $x_require_search_extraction ) ) {
        return;
    }

    foreach ( $x_require_search_extraction as $post_id ) {
        // grab all postmeta containing our content
        // extract into a string
        // update_postmeta( ... )
    }
}

Changing Sort Order

By default, search results will be returned ordered by relevancy with some weight given to recency. You can change this sort order by hooking into the jetpack_search_es_query_args filter and modifying the ES query.

This snippet changes the results to be ordered by most recent posts:

add_filter( 'jetpack_search_es_query_args', function( $args, $query ) {
    // Re-sort the results based on their modified date
    $args['sort'] = [
        [
            'modified' => [
                'order' => 'desc'
            ],
        ],
    ];

    return $args;
}, 10, 2 );

You can modify the sort order in any you choose including weighting various fields depending on your requirements. For more details on sorting, check out the Elasticsearch documentation.

Testing in Local / Preprod Environments

You can use ngrok to connect your local development environments to Jetpack: https://gist.github.com/mjangda/7fcafffda4c6c5c1de18c4af647bd111

Some things worth noting:

  • ngrok is a 3rd party service, which does come with a small cost for custom/reserved domains (which you’ll need to ensure your Jetpack connection stays up).
  • Once connected, we will need to set up an Elasticsearch index for your development site. Please open a ticket with your reserved ngrok domain and we’d be happy to set things up.

Migrating from WPCOM_Elasticsearch to Jetpack Search

Jetpack Search replaces the legacy WPCOM Elasticsearch plugin, which does not support Elasticsearch > 2.0. For stability and customization, we recommend migrating from WPCOM Elasticsearch to Jetpack Search, which works with infinite scroll and has better support for core WP query parameters.

The key changes are:

  • Jetpack Search is directly integrated into Jetpack; no additional plugin required, however, the Search module needs to be activated.
  • The WPCOM_Elasticsearch class has been replaced with Jetpack_Search.
  • Jetpack Search runs off a newer version of Elasticsearch, which has replaced Facets with Aggregations.
  • If you are using the built-in widget, it may need to be re-added as the slug has been changed.
  • The filters from WPCOM Elasticsearch have been renamed:
    • wpcom_elasticsearch_wp_query_args => jetpack_search_es_wp_query_args
    • wpcom_elasticsearch_query_args => jetpack_search_es_query_args
    • wpcom_elasticsearch_found_posts => no replacement available
    • wpcom_elasticsearch_taxonomy_query_var => jetpack_search_taxonomy_query_var

Advanced Topics

Debugging Elasticsearch

The Debug Bar on VIP sites includes an Elasticsearch panel which provides details on all queries performed on the page, including the full JSON-based query sent to the index, timing data (how fast was the query, the network request, etc.), and so on. This can be incredibly useful when things aren’t working as expected.

Replacing WP_Query

We also support other advanced use cases with Elasticsearch outside of search like offloading expensive queries.

You can use the es-wp-query plugin from Alley Interactive to offload any WP_Query lookup to Elasticsearch for the increased performance of a variety of workloads. Use of the plugin is as simple as adding ’es’ => true to your query params when building the WP_Query object.

While Elasticsearch does generally speed up most queries, there may be cases where Elasticsearch may actually be slower. This depends on various factors like the speed of the original MySQL query (using a primary key like ID or an index), the complexity of the Elasticsearch query, cost of the network round-trip, and so on. Query times and complexity should be carefully measured before deciding to offload.

Search API

You can send fully qualified Elasticsearch queries directly to the API as well:

curl https://public-api.wordpress.com/rest/v1/sites/${SITEID_OR_URL}/search --data '{"query": { "term": { "title": "elasticsearch" } } }'

Where ${SITEID_OR_URL} should be replaced with your site’s WordPress.com ID or the site URL.

Querying multiple sites

Searching across sites is not currently supported via Jetpack Search but it’s something we’re interested in exploring. If this is functionality that you are interested in, please let us know.

Non-Public Content

To search for non-public content, you can make authenticated requests to the API. This can be done by filtering the Elasticsearch arguments.

add_filter( 'jetpack_search_es_query_args', function( $es_query_args ) {
 $es_query_args['authenticated_request'] = true;

 return $es_query_args;
} );

This will make an authenticated (as your JP site) request to the Elasticsearch API, which provides access to non-public content.

Admin search

We’re exploring ways to add Jetpack Search capabilities to the admin to allow for faster and more relevant searches for editorial teams. Stay tuned for more.

Mapping

To learn more about how content from your WordPress site is mapped to the Elasticsearch index, please see https://developer.wordpress.com/elasticsearch-post-doc-schema-v2/

The code that handles the mapping/translation can be found here: https://github.com/Automattic/wpes-lib

FAQs & Common Problems

Help, ES didn’t return the data I expected!

There are several cases where indexing operations can manifest themselves in “missing content” when queried:

  1. A delay in sync, e.g. large volumes of cron jobs may be backlogged on the site which may take some time to work through or you just initiated a full sync, which can take some time on larger sites.
  2. A delay in indexing, e.g.: a backup in the processing of indexing jobs. The system design here is “eventual consistency”, so some delays are to be expected, but during normal operation, content queued for indexing should be processed in < 60 sec at the outside and ~15 sec on average.
  3. A failure in indexing. There is retry logic for failed indexing jobs, however during normal operation, the failure rate is very low. When the failure rate spikes, there are usually other problems in dependent systems eg: the databases fail to return the content to index.
  4. The site or the specific post_type is not set to public.

If you are missing data that you definitely expect to be in the index, please reach out and we’d be happy to help investigate.

Where does Elasticsearch live?

WordPress.com VIP site indexes all reside on one ES cluster. However, that ES cluster is distributed across our datacenters to provide redundancy and optimize response times.

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries.

Restricting access to a site hosted on VIP Go

Many VIP Go clients have the requirement to control access to their site so only a specific group of users are able to access it. For launched production sites, your solution must be architected around anonymous users and authenticated users:

Anonymous users can be served with a login screen, a page with information on how to authenticate, or a simple denial notice, etc. This response should be served with an HTTP Status of 200, and you may choose to include information about how a user can authenticate to access the site fully. This response will be cached by the VIP Go page cache.

Authorized users must log in using a WordPress user account, and you can tailor the output to these users, vary the content for different users, etc. Logged in users bypass VIP Go page caching. If your organization has a Single Sign On system, then integrating this with your VIP Go site will smooth the log in experience for your users (see “Single Sign On (SSO)” below).

Requirements and Notes

Restricting access by IP Allow List

Using an IP Allow List you can limit all access to a specified list of IP addresses or ranges of IP addresses (aka subnets). Once you have applied the IP Allow List to an environment, any and all requests from an IP address list outside of the allowed list or range will be denied.

Single Sign On (SSO)

Single Sign On systems provide a central log in system for all of your company’s services, they simplify life for your users by removing the need to configure separate usernames and passwords for each system each user needs to use. VIP Go allows integration with SSO systems to simplify authentication. See our page about Single Sign On on VIP Go for more information about how to set up SSO for your application.

The WordPress REST API

Your site content can be accessed via the WordPress REST API, so if you need to constrain access to site content then your solution must take this into account.

As many VIP Go features utilise the WordPress REST API, we do not allow sites to disable this API completely. If you need to restrict access to your site via the WordPress REST API, you should force authentication for API endpoints. An example of how to do this can be found in the WordPress REST API FAQ.

Restricting access to a list of allowed IP address in WordPress

If you need to restrict the display of certain content to authorized users, then those users must be logged in, this is to ensure that responses to these users are not cached. During the log in process (and on every subsequent request) you can hook into WordPress to reject the log in if the user is not on an authorized IP address.

In order for the VIP team to support your site our users must still be able to log in to your site, despite not being on your list of allowed IP addresses. Requests from inside our authenticated network can be identified by checking that the A8C_PROXIED_REQUEST constant is true (see “Checking for requests from inside Automattic’s network” below).

You must not add IP address restrictions for anonymous users in PHP code, and it is not possible for us to configure your site to restrict access to a limited range of IP addresses in our web server or caching layers. Implementing IP restrictions in PHP code will cause issues within the cached responses to your site; consider the following scenario:

  1. User A visits page 1, this user is from an “authorized IP address”
  2. The VIP Go site serves a tailored response to User A, showing them content only available to certain specific users
  3. The VIP Go page cache caches the response
  4. User B visits page 2, this user is not from an “authorized IP address”
  5. The VIP Go page cache responds to User B with the cached information, i.e. the content which was served to User A, content which was only intended to be available to certain specific users

Note: This method will not restrict access to user uploaded files.

Restricting access to user uploaded files

If you need to restrict access to user uploaded files by IP address, you can use the IP Allow List feature in the VIP Dashboard; note that this limits all requests to the environment with the IP Allow List, not just files requests.

You must not block xmlrpc.php

Various VIP Go platform services use the /xmlrpc.php endpoint, so this must not be blocked by any access restrictions you put into place. Access to the /xmlrpc.php endpoint is restricted to authorised requests only, so additional constraints are not necessary.

Checking for requests from inside Automattic’s network

The following code example shows how to check that the request is from a user inside Automattic’s network.

if ( defined( 'A8C_PROXIED_REQUEST' ) and true === A8C_PROXIED_REQUEST ) {
    // The request originates from WordPress.com VIP (Automattic)
}

Can I mark some entries as “no cache” to avoid issues with caching?

Anonymous user requests should always be cached, and you must not send “no cache” headers to anonymous users. If you want to show the user restricted content and not have this content cached the user should be logged in, logged in users bypass the VIP Go .

Is there a scaling issue with so many uncached users browsing my site?

Your site can scale to handle many thousands of logged in users with the help of our standard VIP Go guidelines for scaling websites. We are happy to talk through your particular use cases, please get in touch.

Basic Authentication for un-launched sites

Basic Authentication is the “pop up” prompt for a username and password which is displayed by your browser when you visit a site which is protected in this way.

While a site is under development, and for non-production sites, we can set up Basic Authentication for you. If you’d like us to do this, please contact us and we’ll be happy to do so.

It is not possible for launched production sites to use Basic Authentication, as this form of access control breaks various VIP Go platform services.

Maintenance Mode

While a site is under development, you can also use the Maintenance Mode plugin to restrict access to your site. Add this plugin to your site by following the instructions for enabling plugins on VIP Go and configuring Maintenance Mode.

Ready to get started?

Drop us a note.

No matter where you are in the planning process, we’re happy to help, and we’re actual humans here on the other side of the form. 👋 We’re here to discuss your challenges and plans, evaluate your existing resources or a potential partner, or even make some initial recommendations. And, of course, we’re here to help any time you’re in the market for some robust WordPress awesomeness.