TechCrunch and Non-Blocking WordPress – Now With Full Transcript

Alex Khadiwala and Nico Vincent from TechCrunch presented on “Non-Blocking WordPress” at the recent Big Media & Enterprise Meetup in San Francisco, California. We’ve shared this presentation before, but now we’re publishing it again now with full transcript below.Download the slides here.

Nico: Today, we’re going to talk about non-blocking WordPress. By non-blocking, we’re meaning asynchronous tasks. First of all, who is a developer in the audience here? Okay, so more than half, awesome. So we’re going to start by talking about non-technical stuff and then we’re going to go a little deeper, under the hood.

Basically the team right now is me and David Lake, as a software engineer and a project manager. Alex here was former development lead for more than a year. I want to give props also to 10up which is here tonight and John Bloch and Eric Mann who were really helpful in the redesign we did in the past year.

So the whole point of this presentation is to talk about having a fast website. Basically when you have a fast website, the user engagement is much better. I’ve heard that under 3 second websites, 40 percent of the people drop and only 40 percent of them will come back on your site.

The more they stay, the more pages they see, which is good for ad revenue and on a business point. Also, like Google and other search engines, like Google or Hummingbird penalizes when the website takes longer to load.

Also it’s good for the bounce rate and I know we’re hosting on WordPress VIP but it’s better for your hardware, when you have a really performant website.

I’m going to let Alex talk about our old TechCrunch to our new TechCrunch.

Alex: Yes, so our old site was really slow, it was about 17.4 seconds for a single page load. We’re part of the AOL network and from all the sites from AOL, we were the second slowest, pretty embarrassing against the peers.

Also, like Google and other search engines, like Google or Hummingbird penalizes when the website takes longer to load.

If you guys recall the old site, for the new site, after we finished the redesign, this is hearsay, but what we were told is that we were one of the fastest sites on VIP. I think Matt Mullenweg was kind of excited about that too which makes us excited.

Blocking is: a process that is blocked is waiting for some event such as a resource becoming available or the completion of an operation.

This little graph at the bottom, you can see, first page load of DOM complete, when you can start interacting with the page is about 2.9 seconds and after the first load, and second load about 1.9 seconds and fully loaded it’s 4.2 on the first load and about 2.3 seconds on the repeat, on the second load.

Basically when everything gets cached and such, definitely very exciting, pretty fast. So it’s good that there’s all these non-technical presentations so we can balance it out with a little more technical presentation.

We’re going to talk about blocking, basically, this is kind of a general definition of what blocking is: a process that is blocked is waiting for some event such as a resource becoming available or the completion of an operation.

The general idea is that you have a page request and it has to do a bunch of things before it can render the HTML and send it back to the user’s browser. So by architecting the page – one of the big things was to make our site extremely performant for all the reasons that Nico had listed off a moment ago. One of the big things that we did was try to identify anything that was blocking and push it off to an asynchronous task.

Some of the ways that you generally do it with other non-WordPress stacks are message queues, using something like Rabbit MQ or Amazon SQS to kind of send a list of tasks and then using workers, or in past I’ve used Gearman as a worker for processing things in a queue. Sidekick and Resque are big popular ones with rails apps.

Unfortunately these options are unavailable on WordPress but with a lot of help from 10up, we built out something that is referred to as a wp-Async-Task. What this is is an abstract class that we extend for different tasks that we wanted to make asynchronous.

So the nice thing is essentially it’s kind of starting a whole new thread to kind of do this work that you need done asynchronously.

So basically, it performs. When you make a request for something, if we kick off a second request back to the same site, which is a non-blocking HP request and it basically sends some information to the server and says do this task.

So the nice thing is essentially it’s kind of starting a whole new thread to kind of do this work that you need done asynchronously. A couple of examples of what we did on our site that you can see are CrunchBase.

We have, on most of our articles we have CrunchBase cards and you can kind of see – here Facebook and Snapchat as an example.This information all comes from the CrunchBase API. So imagine that one circumstance you’re loading the page, you have to get this information from CrunchBase.

Imagine that we have to go to CrunchBase, wait like a second or even half a second to get that information, for each of those different companies and then pull the information back and basically have it available then to render the page. It would be painful for the end user.

So what we do actually is we kick off an, if the information is not available, we kick off an async task, it goes and gets the information, caches it in our site, we basically store it in a taxonomy – a custom taxonomy for CrunchBase.

One of the big things that we did was try to identify anything that was blocking and push it off to an asynchronous task.

Then the next time that particular CrunchBase information is requested, it will be available to the page.So we then load nothing for the first request if it’s not available, then the next time the request comes through, it’ll be available for that page.

Another example is for recirculation. On the left rail of our site, we have some for example, we have tags and categories basically for that particular post and we pull in articles that are related just for recirculation purposes. Pretty much the same exact thing as a CrunchBase.

One nice thing that we do here as well is that when the page finishes, if the information is not available it does not render on the page. Instead it sets up, it just kind of writes a Javascript callback so that when that page finishes loading, that it’ll call back and try to get the information.

Basically, as far as images, we were just putting a 1×1 image on a CEN in the SRC attribute and then once the DOM was done loading, we were just using another attribute that we call data-srce and we were replacing the image source so that was super fast, seamless.

Again, it waits ’til the page is completely done loading, before it makes that call back, just for the best user experience. That’s it for my part, I’m going to pass it back to Nico to talk about some other things.

Nico: Alright, I’m going to talk about something we call the Lazy Loader. Basically on our front page and many of our different pages. We’re loading more than 100 assets, and it was a big part of us having a 17 second loading time.

We have different blocking API’s and the social buttons. Let’s say for a blog when you’re displaying 30 articles, you have to load a like button, a linkedIn button, a Twitter Button 30 times and these API’s are really slow and sometimes it’s a real hassle to finish the DOM to load with that.

Basically, as far as images, we were just putting a 1×1 image on a CEN in the SRC attribute and then once the DOM was done loading, we were just using another attribute that we call data-srce and we were replacing the image source so that was super fast, seamless.

And then like Alex was talking about, the tag accordion – basically all the accordion was closed so we were just loading the content as the page was finishing loading.

So once again, no one could see it and it was much better in terms of page load.

Also, our social buttons, they’re just like little images and as soon as you hover it, that would just trigger the loading, so these are examples that really really cut down our page load down from like more than 10 or 15 seconds.

Another example is caching, so here, basically what we’re doing, we’re having a refresh key that’s usually like 10 minutes and then when the refresh key is still okay and the data is still good we just returning the data and serving it. If it’s not the case, we only have one person that can trigger the refresh and then that will set the transient.

Another example, it’s a plugin actually, so you guys can check it out if you’re interested. It’s from another 10up person that used to work with us.It’s called the Live Cache.

So where you see the two arrows, it’s a little module on the homepage, when the stream is on and we wanted to be able to change the information there so the person wouldn’t have to refresh the page.

We tricked a little bit with a URL and a timestamp, so basically the browser will trigger a call to our backend every ten seconds based on a different timestamp and we will just get the information back if someone in the backend had refreshed it before.

I will let Alex here conclude on this.

Alex: So yeah basically, the point of our presentation was just to – it’s something you kind of have to do from the beginning and especially if you’re doing a redesign.

Just general idea, is you want to architect your site to asynchronously handle any heavy task to maximize your task performance.

Yeah,using a pattern like wp Async Task or things like the Live Cache plugin or just Javascript Lazy Loading or the 1×1 pixel source when you’re loading the page will definitely help improve the page performance.

One other note we’re told to tell you if we’re hiring, so we are hiring and if you are looking or know somebody that is looking in the Bay area, would love if you guys could email them now at dev@techcrunch as I’m no longer there.

Thank you guys so much and thank you so much to Automattic for having us present here.


Q: How do you factor speed with features, it’s a balance, so how do you deal with those decisions? Is it just the feature set wins and you just figure out how to do it?

A: Pretty much. So a lot of our features are driven by editorial, and editorial typically is very opinionated and had a lot of pull. I’m sure everyone has dealt with that for any media site.

So I think most of the time, if it’s not them coming to us, it’s them going to the higher ups and talking them into basically adding a certain feature and then us kind of figuring out how to architect it.

It gives some interesting challenges, but it’s been great, it’s been fun. I mean, I think Live Cache was a good example with that.

Like one of the big things was working against batcache, because we’re on VIP and having batcache, we have to kind of come up with a way that we can make page requests back and get up to date information.

But without taxing VIP too heavily and getting passcode review on VIP.

Q: Just a follow up, it seems like you’ve built tools to help you quickly integrate speed into everything you build, rather than having to tackle it individually for each piece, is that correct?

A: For sure, just kind of abstracting as much as we can, to make it reusable and I will give props to 10up again for a lot of that.

Nico and I, like I started about a year and a half ago, Nico’s been almost 2 years now and we both came from non-WordPress backgrounds.

We started probably a few months into our tenure at TechCrunch doing the redesign, so it was a lot of stuff we had to learn. It’s kind of a different world doing WordPress development.

Participant:Can you go back on a slide? Two slides, one more.
Alex: The one on Live Cache?
Participant: Yeah, thanks
Alex: Cool.

Q: Yeah, just curious what your best practices are around caching, and content like that, what’s your TTLs and whether you’ve found any optimizations around caching that you’d like to share?

A: That’s a good question. So that’s a good question because we had issues with that plugin. It didn’t work for like two live events because first of all.

We were logged in and VIP has a different behaviour as far as refresh and cache for logged in users. When you’re just someone who’s chilling on the website, we we’re getting the headers from that cache.

Sometimes, so I don’t remember if it’s like too much or not enough, but sometimes we were having the right data from the back end, and then ten seconds later, the old data because of issues with cache timestamps.

Alex: Do you mind if I?
Nico: You want to chime in?
Alex: Yeah I’ll chime in for this one.

Alex: We were having an issue with Live Cache, you might be speaking of other assets…

Q: Yeah, I’d just like to hear general, just caching in general, what best practices you have for that like new content versus page speed.

A: Yeah, everything is basically, we said no headers from our side. Everything is kind of managed from VIP, all of our information, all of our content is through their CDN and through their hosting.

They kind of take care of that for all our static assets and such as far as the issue that Nico was just talking about.

One of the big issues with Live Cache is that basically you have to make the request back to the server with the right timestamp to make sure you’re getting the right text.

So for example basically we have these presentations that disrupt and they’re like 10 minutes long, which we’ve gone over a little bit but we’ll be quick.

When they switch presentations, somebody goes into the backend, in the widget and switches out the text. But, you have to synchronize the clock with the end user to make sure that they’re using the right, the same clock as you.

So our solution was to make that first request live_cache_check/1/. Fortunately the header on the batcache, the response is coming from batcache. It does have the proper timestamp so we use that to synchronize the clock and then accordingly all subsequent requests, which are every ten seconds will be based on that, based on the synchronization.

Q: So assuming no one messes with the clock between the first request and after that?

A: I guess, it’s valid, I mean just to speculate, I guess it’s OS specific. So it’s kind of relying on the timer within the browser and within the OS to kind of, as to the interval that’s been set.

Participant: makes sense, cool.

Q: Is the wp Async plugin, is that available right now?

A: Unfortunately not, we have it, we were looking, we were talking about like open sourcing that and I guess we just have to get it cleared by a few different people but I think it would be useful. We think it would be something useful to open source.

Participant: Sounds like it.

Alex: Yeah yeah, for sure. Cool, thank you guys so much.

Nico: Thanks.

See the presentations from previous Big Media & Enterprise WordPress Meetups. For Big Media & Enterprise WordPress Meetup groups in other cities, see the full list on VIP Events and join your local group. 

Ready to get started?

Drop us a note.

No matter where you are in the planning process, we’re happy to help, and we’re actual humans here on the other side of the form. 👋 We’re here to discuss your challenges and plans, evaluate your existing resources or a potential partner, or even make some initial recommendations. And, of course, we’re here to help any time you’re in the market for some robust WordPress awesomeness.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.