Melkweg.nl - Serving over 30,000 web pages blazingly fast.
At GRRR, we think website speed is very important. It not only influences search engine rankings and page visits, but it also has a major impact on user experience. You can have a website that is very well thought through, and beautifully designed, but if it doesn’t perform, that all falls apart.
So, if we deliver a website, we want it to be fast and responsive. By this, we mean that you experience the website as if it would be running on your local device and experience no lag at all. So we set the bar high.
With that in mind, we’ve created a new website for our client the Melkweg, a famous venue in Amsterdam. But, there were some major challenges along the way... Melkweg has a lot of events, which results in serving more than 30,000 static web pages. So, the main question was, how to get this performant?
Thanks to Martijn for giving valuable input and reviewing.
Static site generation
For many years now, we’ve gained experience with static site generation — in our opinion, an elegant way of getting a website fast and responsive. Like we did for The Ocean Cleanup and World Benchmarking Alliance. But, the Melkweg is something different, with its large amount of web pages. We thought of adding a new solution to our current stack, but we’ve decided that this would introduce new issues, and maybe our current stack would just not be sufficient enough anymore.
So, how did we tackle this beast?
Renew our stack
With static site generation, static pages will be (re-)generated when updates must be published. This requires build time. So, imagine, if I update something on the Melkweg website, then initiate a static build to publish this update, the build will take a long time with this amount of pages… which is kind of an understatement. So, we need to decrease this build time drastically.
There are many ways to approach this, but before re-inventing the wheel, we’ve looked around at what is already available — with the experience we’ve already got in the team in mind. Because we’ve used this technology for many years now, we’ve tackled many of its drawbacks. We’ve created several fixes and solutions for this, with which we eventually concluded that the core technology works for small sites but not for this one.
Time for something new…
Decoupling and headless CMS
Renewing our stack, now and then, is the default mode for us. But how to see if a new technology is a hype or actually something that will last for longer? You never know for sure, but to be less dependent on a certain technology decision, we decouple as much as possible. Decoupling in this case means dividing the stack into loosely coupled components, like front-end, back-end, and hosting, but also the protocol that ties it all together.
This comes with a lot of advantages, like scalability and replaceability, which makes decisions like which technology to use for a certain component easier. In our team, we’ve already had experience with Laravel Nova as a headless CMS and Next.js as a front-end. So, we’ve combined these two technologies. Also, we love standards, and we’ve used existing standards for connecting these dots: the JSON API. So we’ve tied this all together, and tada… a new stack arises.
Mocks
But, we took this decoupling philosophy a little bit further. Using Next.js as our front-end, we’ve made standards on how to use it using this stack, so that every developer easily understands how to develop with it, and write understandable code for other developers. This makes it transparent and easy to onboard new developers if needed.
With this standard, we’ve introduced so-called mocks which completely simulate the back-end API. With mocks, front-end developers don’t need anything from the back-end and can develop without a locally running CMS. So, no need to set up a dedicated local environment, server, and database with the hassle that comes with it. Because mocks are very fast as well, it also meets our requirement to keep everything snappy during development. And because it prevents (production) data from spreading around within the team, it also meets security standards.
The API mocks automatically document how the CMS API should look and help with quality assurance when anything changes in the API. And last, but not least, it improves decoupling, and it isn’t dependent on which tech the CMS runs on, so we can change this in the future if we want to.
Ok, but how does this tackle our static site generation build time? Sorry, got a bit distracted there, so to get back on track, we needed a new thing for this.
Therefore, we’ve introduced Netlify to the game.
Netlify
Instead of hosting static builds ourselves, we’ve stumbled across Netlify, which comes with a lot of nice features out of the box. Also, it connects with Next.js pretty well, so we’ve decided that this would be a good addition to this stack.
Because Melkweg is based in Amsterdam a server running within the EU was important to keep up speed, which Netlify supports also. Finally, in addition to decoupling as much as we can, this also takes away the dependency on our current hosting. Therefore it takes away a lot of worries so we can focus more on development instead of designing infrastructure.
So. The building and static site generation can be done with Next.js and Netlify, which also offloads the web server. But still, how to deal with this large amount of pages? For this, Netlify comes with so-called on-demand builds.
On-demand builds
With Next.js, we can control what we want to build, and therefore control its build time. We saw that there is no need to build everything at once, so we made rules for what we want to build immediately or what we want to build later. Pages that are visited often, for instance, events that occur in the future we want to serve super fast. But we saw that events that have already occurred in the past are less popular, so we can build this later.
For this, we use Netlify’s on-demand builds. This basically means that the page will be generated statically (built) during a first visit. The first user that visits a web page that hasn’t been statically generated yet, will automatically initiate an on-demand build. This means that the page will be generated in real-time. Because of this, a user can experience a little bit of delay, but when it has been built, it is very fast, also for other users visiting this page.
This does not mean that it takes very long. Netlify’s on-demand builds are building very fast, and it could be that the user doesn’t even notice an on-demand build has happened. With Netlify, we know that this will be optimized as much as possible.
So, what’s next?
We are continuously improving, and during the renewal of our stack, we’ve already got a lot of new insights. So, also for the Melkweg, we have some new ideas already, like on-demand revalidation, which is a new feature from Netlify. Also, we would like to learn more from how on-demand builds are triggered with our current configuration and optimize this where possible.
Basically, to keep a website blazingly fast, we need to improve and monitor continuously. This demands good communication and setting priorities with our clients. And this makes it so much fun to work on projects like this so that we can keep learning by doing.
Finally, see it for yourselves at Melkweg.nl!