Migrating Large-Scale Websites From Gatsby to Next.js
DATE
CATEGORY
Development
READ TIME
7 minutes
VIEWS
...Migrating Large-Scale Websites From Gatsby to Next.js
Why migrate a perfectly functioning website from Gatsby to Next.js? Will the end user benefit from all this, or is it just to satisfy the development team?” Those were some of the questions we had before starting this project.
With more than 30 static pages, around 600 dynamic pages, and 15 language translations for all of those, solana.com was filled with plugins and escape hatches, and unbearable build times. The more we talked with Solana Devs, the more we agreed on this: that better DX, faster build times, and versatile APIs would put developers in a “pit of success” that would lead to faster iteration cycles and a better website for end users.
And so we took on the ambitious project of migrating solana.com from Gatsby to Next.js, in the shortest amount of time possible—as the Solana Dev Team also needed to ship new features.
Thinking in Next.js
When starting a migration from Gatsby to Next.js, there are a couple of things to consider. We must get into “thinking in Next.js” (ode to the famous Thinking in React).
Next.js is a full-stack framework.
At the surface, it provides developers with APIs that unlock several rendering strategies—SSR, SSG, ISR; wraps basic web primitives such as images, links, scripts, and fonts with well-thought-out abstractions, and makes working with React and every styling solution you’d want a bliss.
At a deeper level, it provides ways of handling different edge cases that may arise seamlessly. And that leads us to one of the critical differences between Next.js and Gatsby: how they approach extensibility.
Extensibility
Next.js provides developers with primitives that can be adapted to any use case, whereas Gatsby—while it still offers many built-in primitives—tends to suggest a plugin for almost everything.
This means that if you want to add a new feature to a Gatsby site, you’ll likely need to find and install a plugin that provides that feature. This can make it easier to start, but it can also make it harder to customize your site to meet specific needs. Let's explore an example:
Image Optimization
Optimizing images is a critical step to take to build a lean and performant website.
How does Gatsby solve image optimization? You will need the
gatsby-plugin-image
plugin. To make that work, you will also needgatsby-plugin-sharp
. Additionally, installgatsby-source-filesystem
if you are using static images andgatsby-transformer-sharp
in case you use dynamic ones.How does Next.js solve image optimization? Just
import Image from 'next/image'
.
This is how you’d do image optimization for each framework:
Gatsby
1yarn add gatsby-plugin-image gatsby-plugin-sharp gatsby-source-filesystem gatsby-transformer-sharp
1// gatsby.config.js2module.exports = {3 plugins: [4 `gatsby-plugin-image`,5 `gatsby-plugin-sharp`,6 `gatsby-transformer-sharp`, // Needed for dynamic images7 ],8}
1// component.jsx2import { graphql, useStaticQuery } from "gatsby";3import { GatsbyImage, getImage } from 'gatsby-plugin-image'45() => {6const { file } = useStaticQuery(7 graphql`8 {9 file(relativePath: { eq: "demo.png" }) {10 childImageSharp {11 gatsbyImageData(12 quality: 9013 layout: FULL_WIDTH14 placeholder: BLURRED15 formats: [AUTO, WEBP, AVIF]16 )17 }18 }19 }20 `21 );22 return <GatsbyImage image={getImage(file)} />23}
Next.js
1// component.jsx2import Image from 'next/image'3import demoSrc from '~/public/demo.png'45() => {6 return <Image src={demoSrc} quality={90} placeholder="blur" />7}
That’s a big difference. Thirty-one lines of code, plus three npm packages, versus just seven lines of code.
Additionally, Next.js’ Image component allows the developer to tweak how the image will load and look by just passing props into it. This declarative and colocated approach scales really well and makes the codebase super easy to understand.
When migrating solana.com, images were not as much work as you’d think. It was mainly deleting code and checking styles that didn’t break.
Fetching Remote Data
Every big website, at some point, needs to get remote data from somewhere. This was certainly the case for solana.com, which read data from:
Ghost to get blog posts
YouTube to get the latest videos from the Solana YT channel
GitHub to get information about the Solana repo
In Gatsby, for each data source, we’d use a different plugin. For Ghost, we’d use gatsby-source-ghost
and gatsby-plugin-ghost-images
. For YouTube, we’d use gatsby-source-youtube-v3
. For GitHub, we’d use gatsby-source-custom-api
. These were four different dependencies, which needed to be understood by the developer and consumed in different ways across the application.
In Next.js, we simply fetch
ed data from wherever and passed the data down via getStaticProps
. Solana developers now don’t need to go through different plugin documentations, explore various files, and do lots of cmd + shift + F
(global searches), just to understand how the data gets to the UI: now they just see the data being fetched on the spot and then passed down via props (a common pattern for React developers).
Again, let’s look at a practical code example taken from the Solana repository:
Gatsby
1yarn add gatsby-source-custom-api
1// gatsby-config.js2module.exports = {3 plugins: [4 {5 resolve: "gatsby-source-custom-api",6 options: {7 rootKey: "github",8 url: "<https://api.github.com/repos/solana-labs/solana>",9 schemas: {10 github: `stargazers_count: Int`,11 },12 },13 },14 ],15}
1// gatsby-node.js2exports.createPages = async ({ actions, graphql }) => {3 const { createPage } = actions;45 // Create Community pages67 // 1. Fetch related data8 const githubData = await graphql(`9 {10 github {11 stargazers_count12 }13 }14 `);1516 // 2. Create Community page17 const communityPage = require.resolve(`./src/templates/community.js`);1819 createPage({20 path: "community",21 component: communityPage,22 context: {23 githubData: githubData.data.github,24 },25 });26};
1// templates/community.jsx2export default function ({ pageContext }) {3 return (4 <p>{pageContext.githubData.stargazers_count}</p>5 );6};
Next.js
1// pages/community.jsx2export default function ({ stargazers }) {3 return <p>{stargazers}</p>4}56export const getStaticProps = async () => {7 const res = await fetch("<https://api.github.com/repos/solana-labs/solana>", {8 method: "GET",9 headers: {10 Accept: "application/json",11 "Content-Type": "application/json",12 },13 });14 const jsonData = await res.json();15 const stargazers = jsonData?.stargazers_count || 0;1617 return {18 props: {19 stargazers20 },21 revalidate: 122 }23}
This is a real example, taken from solana.com on Gatsby (left), and solana.com on Next.js (right). Again, more than the diff in terms of the number of lines of code, the most important difference here is that in Gatsby, there are several layers you need to understand just to know how that stargazers_count
reaches the UI (an npm package, gatsby-config.js
→ gatsby-node.js
→ render function). In the Next.js example, it’s all condensed into one file and two functions. Colocation is important for us, and Solana developers really enjoyed not having to deal with gatsby-config
and gatsby-node
stuff.
Additionally, Gatsby forces all the data to pass through GraphQL, which gets really hard when dealing with remote APIs that do not expose a GraphQL endpoint. In Next.js, you can use GraphQL, fetch
, or node’s fs
... basically whatever you want.
Build Times
One big issue we had with Solana in Gatsby was build times. We were circling 35-minute build times. That means, on every push, the Developer would need to wait half an hour (sometimes even more) just to share progress with coworkers. This made iteration cycles really slow, and therefore shipping new features was a drag.
Let’s be fair: the site is big. At each build, we were creating close to 10k static pages ((static + dynamic pages) * locales
). How could we go faster than this if we were not server rendering?
Luckily, Next.js is very flexible. By taking a hybrid approach, generating the most important pages at build time, while not generating less frequently visited pages and localized ones, we could make builds much leaner. But how would we ensure that every page of the site remains fast? We’d use Incremental Static Regeneration: the first time a user requests a page that’s not cached, the Vercel server would:
generate the page and return the HTML + CSS + JS
push the newly generated page to its CDN
This means, yes, the first user would take a loading time hit by needing to go to the origin server, but subsequent users would get a fast response from the CDN! That was a tradeoff we were willing to make.
Let’s look at how generating static pages works for both of these frameworks:
Gatsby
1yarn add gatsby-source-filesystem
1// gatsby-node.js2exports.createPages = async ({ actions }) => {3 const ecosystemData = await fetchEcosystemDataViaGraphQL();45 const ecosystemProjectTemplate = require.resolve(6 `./src/templates/ecosystemProject.js`7 );89 // Create ecosystem partner pages.10 ecosystemData.pages.forEach(({ node }) => {11 actions.createPage({12 path: `ecosystem/${node.frontmatter.slug}`,13 component: ecosystemProjectTemplate,14 context: {15 slug: node.frontmatter.slug,16 logline: node.frontmatter.logline,17 },18 });19 });20};
1// templates/ecosystemProject.jsx2export default function EcosystemProjectTemplate(props) {3 return <div>{...}</div>4}
Next.js
1// pages/ecosystem/[slug].jsx2export default function EcosystemPage(props) {3 return <div>{...}</div>4}56// ————— Here ⬇️7**// only pre-render `importantEcosystemPages`**8export const getStaticPaths = async () => {9 const importantEcosystemPages = await fetchMostImportantEcosystemPages();1011 return {12 paths: importantEcosystemPages.map((p) => ({13 params: { slug: node.frontmatter.slug },14 })),15 fallback: "blocking",16 };17};1819export const getStaticProps = async (ctx) => {20 const data = await fetchEcosystemPageBySlug(ctx.params.slug);2122 return {23 props: {24 slug: data.slug,25 logline: data.logline,26 },27 revalidate: 1,28 };29};
With this strategy, our build times went down from 35 minutes to just 5 minutes. Learn more about ISR in the Next.js docs.
Results
Overall, migrating from Gatsby to Next.js was a great decision. The project took 3 weeks from start to finish. We moved fast to prevent blocking Solana Devs from shipping new features. Vercel Previews were key here, as we needed to test every single page we migrated to make sure we didn’t break anything. Those 5 minute build times Vercel was giving us made us really happy.
Vercel Analytics showed us that the Real Experience Score improved from 79 to 90 points since we migrated. This gave us even more confidence to know that the project was successful.
The Solana Team now knows that this architecture will scale as more and more users land on solana.com, and new features need to be built and shipped.
This article was originally published on Vercel.