Scale High-Volume WooCommerce Sites with Enterprise Search

We’ve heard the whispers. “WordPress and commerce solutions like WooCommerce simply weren’t built for high-volume mega stores with deep product catalogs.” 

But is that reality or just outdated perception not worth paying attention to? 

We believe it’s the latter. The key for commerce brands who want to rapidly innovate, save customers time and money, and unlock easy-to-use commerce solutions like WooCommerce at scale—say, 1-million-products-in-stock scale—is via Enterprise Search, developed by WordPress VIP.

To prove this, we took up the challenge in the WordPress VIP lab, designing and stress testing a high-volume WooCommerce shop with stock across multiple languages, categories, and dozens of attributes. Check out what we learned. 

High-volume WooCommerce store challenges

From our experience, the more data that goes into a MySQL database, the harder it becomes to search for it. Certainly, displaying a catalog of more than 1 million products—and making it easy for customers to navigate—is a serious challenge.

To begin, we decided to make use of WooCommerce internal methods to generate products with enough taxonomies and terms relationships to resemble a real ecommerce store. 

One package that helped was WooCommerce Smooth Generator. It let us make use of scalable WordPress VIP Cron architecture, which helped us accelerate WooCommerce products generation tenfold. Good news: in only 48 hours, we had a store with high-volume data—and those 1 million products—to test against. 

Tech tip: To keep the initial import process speedy, we disabled the terms recount process and only ran that again at the final stage:

add_filter('woocommerce_product_recount_terms', '__return_false');

For this test, we used the latest WooCommerce (version 6.1.1) and the default Storefront theme.

Assessing the frontend, we began noticing large delays in rendering pages—MySQL slow queries were the root cause. The main areas affected were related to:

• Product search product search
• Category product listing pages
• Filtered search 

Enter Enterprise Search—fast, easy, secure

It was clear that one of our newest products, Enterprise Search, would be essential in addressing these issues in a scalable way. We found that it helped our high-volume WooCommerce store function without major code or database schema changes.

First, some background: Enterprise Search was designed from the ground up for the needs of enterprise customers. It’s lightning-fast, even as database and site volume scale. Built on Elasticsearch, it gives accurate, nearly instant results even across millions of pieces of content.

Because it’s directly integrated with VIP Cloud with no external services required, Enterprise Search immediately indexes new pieces of content, rapidly surfacing the newest, and most relevant, results.  

We enabled Enterprise Search and ran an initial index on the product data we had, allowing us to horizontally store the data from our database.

Tech tip: Enabling Enterprise Search is as simple as:

define( 'VIP_ENABLE_VIP_SEARCH', true ); // Enables Enterprise Search.
define( 'VIP_ENABLE_VIP_SEARCH_QUERY_INTEGRATION', true ); // Integrates search queries with Enterprise Search.

Lightening the load by offloading queries to Enterprise Search

Knowing that any WP_Query can be offloaded to Enterprise Search, we started checking for appropriate filters that allowed us to inject the basic es => true to the slow MySQL queries found in the site frontend. Instead of querying the database, the result of each query is fetched from the Enterprise Search engine using a fast API call.

Using this method, the response time for products search, category products listing pages, and filtered search improved tenfold. Yes, 10x!

Sample code for how the product frontend queries related to WP_Query were offloaded to Enterprise Search is below:

add_action( 'pre_get_posts', ‘offload_query_to_ES’ );
function offload_query_to_ES( $query ) {
   // Avoid unintentional query offloading for backend queries.
    if ( is_admin() ) {
        return;
    }

    if ( ! $query->is_main_query() ) {
        return;
    }
 
    if( 'product' !== $query->get( 'post_type' ) ) {
        return;
    }

     // Offloading query to  Enterprise Search.
    $query->set( 'es', true );
}

The result, achieved in short order, was a fast and scalable high volume WooCommerce site, which, in combination with existing VIP Cloud architecture, allowed it to handle large amounts of traffic.

In addition, the WooCommerce backend speed was also improved by offloading expensive MySQL queries to Enterprise Search, allowing a more optimized experience for shop owners.

Tech tip: All queries sent to Enterprise Search are fully customizable as they are translated to JSON ElasticSearch Query DSL using the following filter:

 ep_pre_request_args

Going big on order processing

Our next step in testing was to generate a high volume of orders and related customer data.

Using similar steps as noted above, we managed to create ~1 million orders and related customer data.

During our testing, we found that most of the optimizations were related to slow MySQL backend queries. This meant, again, offloading more expensive queries to Enterprise Search and indexing related data.

Note: WooCommerce orders are stored as a custom post type in WooCommerce, so they need to be registered in Enterprise Search’s allowed post types along with their custom post statuses. Sample code is below:

// The `shop_order` post type is not public, so we need to add this to get them to index.
add_filter( 'ep_indexable_post_types', function( $post_types ) {
    $post_types['shop_order'] = 'shop_order';
    return $post_types;
} );

// Adding the custom order statuses to the index.
add_filter( 'ep_indexable_post_status', function( $statuses ) {
    return array_merge( $statuses, array_keys(wc_get_order_statuses()) );
});

As an aside, the WooCommerce team is working to introduce a new custom orders table to help query related data faster and in a more scalable way. This will help Enterprise Search continue to increase the scalability and performance of an ecommerce application.

Speaking of multilingual options…

One approach that scales well during setting up a high-volume, multilingual ecommerce site is creating separate sites within a WordPress network install and allowing each to be connected separately to Enterprise Search. 

Inpsyde, a WPVIP technology partner, for example, offers a MultilingualPress plugin that facilitates the creation of a similar network setup that supports an unlimited number of languages on high-volume sites.

However, this does create challenges related to syncing stock between sites. Multiple solutions exist to address this, including a Central Stock for WooCommerce plugin, also from Inpsyde.

A final word at checkout

Besides offloading more expensive MySQL queries, Enterprise Search also offers new search functionality—including search terms fuzziness, synonyms, and weighting—that can help make a high-volume WooCommerce store search even more efficient.

Can you tell we love Enterprise Search? Frankly, it’s a real solution to increasing the scalability of your business, cutting costs related to development and implementation time, and opening up the road to new ecommerce implementations.

We’re already seeing existing WordPress VIP and WooCommerce customers successfully and profitably pushing the boundaries of high-volume ecommerce. Here’s to a more scalable future!

Author

Ovidiu Liuta, Premier Engineer, WordPress VIP