An Inside Look into Our SEO Process

Introduction

Taro is a community of 1000s of software engineers focused on career growth. With so many engineers creating masterclasses, contributing discussions, and running live events, search engine optimization (SEO) is a natural growth strategy.

We started to invest more time into improving our SEO at the end of 2022 because we were getting poor results from our organic search traffic.

We decided to follow Google's official SEO guide closely when optimizing our website since they are the source of truth when it comes to ranking well in search results: Google SEO Starter Guide

In this post, I will cover the most significant changes positively affected our site's performance on the Google Search Console dashboard. These changes helped our page indexing and user performance, and we hope you are able to take these findings and make use of them for your own website.

Optimizing Core Web Vitals

Optimize your LCP and CLS to go from "poor" URLs and "needs improvement" URLs to "good" URLs

Instead of getting too caught up in a debate about whether Core Web Vitals was a good ranking signal, we decided to proactively optimize them. Since these metrics are shown on the Google Search Console, optimizing them would improve how Google perceives our site, and it would also improve the overall user experience.

By investing time into fixing our issues with CLS (Cumulative Layout Shift) and LCP (Largest Contentful Paint), we were able to improve our Core Web Vitals. Google Search Console initially flagged most of our URLS as being "poor" or "needs improvement" because of CLS and LCP issues. I will cover the changes that we made to improve our CLS and LCP scores.

CLS (Cumulative Layout Shift)

CLS measures the distance that elements shift on a page once they have already been rendered. A poor CLS score can provide a jarring user experience because each forces the user to visually recalculate how elements are laid out on a page. Here is an excellent illustration from web.dev of how CLS can cause a user to misclick an element, which can result in a frustrating user experience:

0:00

The following changes made the most difference in affecting our CLS score.

Use Static Site Generation

One of the most powerful features of Next.js is its static site generation (SSG) feature. With SSG, you now have the ability to generate fully fleshed out HTML pages containing dynamically fetched data serve these pages to your users and continuously serve these built pages from a CDN. This approach offers two significant benefits that enhance website performance and user experience: 1. you can eliminate unneeded API calls that fetch the same data that all of your users will see and 2. you can send a rendered page to the user eliminating content layout shifts and ensuring a seamless browsing experience

SSG comes enabled out of the box. You only need to export a getStaticProps function from any of your page files, and Next.js will automatically know to build these pages in advance:

export const getStaticProps: GetStaticProps<{
  data: Data;
}> = async () => {
  const res = await fetch('https://domain.com/api/data');
  const data = await res.json();
  return { props: { data } };
};

We use SSG for virtually all of the pages that you see on Taro to improve our CLS scores.

Use Skeleton Loaders

Skeleton loaders serve as placeholder elements on a webpage, indicating to the user that content being loaded. These visual elements replicate the layout hierarchy of the fully loaded page, providing a familiar structure. There will be fewer layout shifts once the new data is loaded onto the page. This is possible because skeleton loaders reserve the same amount of space in the overall page layout.

We use skeleton loaders on the Taro homepage while the user is being authenticated. By implementing skeleton loaders during this stage, we communicate to the user that content is being loaded. This approach helps maintain a seamless and intuitive user experience throughout the authentication process.

They are fairly simple to implement since the skeleton loaders are elements with rounded corners, fixed sizes, and a grey background color. Since we use Tailwind, we make use of the animate-pulse class to give the skeleton loaders movement to indicate to the user that we are doing something in the background.

<div>
  <div className="flex flex-col space-y-12">
    <div className="flex flex-col space-y-2">
      <div className="h-8 w-80 animate-pulse rounded-sm bg-light-intermediate" />
      <div className="h-8 w-80 animate-pulse rounded-sm bg-light-intermediate" />
    </div>
    <div className="h-16 w-80 animate-pulse rounded-sm bg-light-intermediate"></div>
  </div>
</div>

LCP (Largest Contentful Paint)

LCP measures the time it takes for the largest element to render in the initial page's viewport. An example of the largest element could be an image, a block of text, or any other visually significant component. Optimizing the LCP is crucial for having a more performant user experience because the largest element is usually the first element that a user will engage with when accessing a page

Optimize Loading Third Party Dependencies

You can utilize the Next.js webpack bundle analyzer to investigate which webpack bundles have the largest dependencies. The tool generates a visual mapping of each bundle, where each bundle appears larger in the visualization the larger the bundle size is. Once you determine which dependencies are the largest, there are three actions you can take to improve performance:

1. You can remove unused dependencies

Review your imports and eliminate any unused dependencies. By removing unnecessary imports, you can significantly reduce the size of your bundles, resulting in faster load times.

2. You can replace the dependency with a smaller dependency

There are some libraries that require you to import the entire package, even if you only need to import a single function. There is a common pattern called tree shaking where webpack will remove any unused code at build time. This will reduce the overall bundle size. Make sure any libraries you use can support tree shaking.

3. You can defer loading the dependency until the user interacts with the dependency

You can lazy load large dependencies if users don't need to immediately interact with the library. If you are using Next.js, you can make use of next/dynamic to load larger libraries conditionally. In our case, we use a larger Markdown WYSIWYG text editor library called tiptap. Since only a small subset of our users will post a question, we conditionally load this library only when the user opens up the modal to create a new question.

Disable Link Prefetching

Next.js comes with a Link component that will automatically prefetch the contents of the destination URL. I only discovered this when I used webpack bundle analyzer to examine the bundles that were initially loaded on the homepage. I saw a bundle contain a library that was never used on the homepage. I started to comment out all of the code, and I saw when I added back in the Link component, I saw that the bundle was being loaded

By default, the Link component has a prefetch prop that is initially set to be true. The prefetch will only happen for Link components that are rendered in the initial viewport. You can set prefetch={false} on the Link component if there is a Link component that is referencing a page with a large bundle size.

Prioritize Image Loading

Next.js provides an Image component with a priority prop which you can use to load the highest priority images first. We use this to load the largest images above the fold. The largest images that render above the fold are a good candidate for using the priority prop since they will affect LCP. Make sure to use the priority prop with other image optimization techniques to reduce LCP even more.

Using Sitemaps

Submit your sitemaps to Google Search Console for faster indexing

Sitemaps can be a cheat code for indexing because they provide a comprehensive list of pages to search engine crawlers. In our experience, submitting sitemaps proved to be more effective than waiting and hoping that the crawler would discover all of our pages since the feedback loop can be weeks or months. They turn the page indexing process from a passive process into a more active process.

We have three sitemaps, which are all implemented differently:

Next.js sitemap for our questions and video pages
Custom sitemap for our videos (this is for our actual video files, which is different from our video pages)
Blog sitemap

Next.js Pages Sitemap

We use a library called next-sitemap to automatically generate a sitemap for all of our statically generated Next.js routes. The setup is very minimal. The only real work we had to do here was to exclude protected routes.

Custom Videos Sitemap

Our approach for generating a video sitemap is more manual since we build it from scratch. We create a custom Next.js file at pages/video-sitemap.xml/index.tsx which generates a sitemap with video data including information like the thumbnail url, video url, video title, video description, and more. It looks like the following:

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
<url>
<loc>https://www.jointaro.com/lesson/CsRpEPk394g52mQz04FY/at-big-tech-youre-just-a-cog-in-the-machine/</loc>
<video:video>
<video:thumbnail_loc>https://firebasestorage.googleapis.com/v0/b/tech-career-growth.appspot.com/o/TCGHighlights%2FFAANGLife%2F2023_04_01_5_faang_machine_cog_thumbnail.jpeg?alt=media&token=71eeeeba-42c2-4029-b4a7-2df7f6e8ee39</video:thumbnail_loc>
<video:title>At Big Tech, You're Just A Cog In The Machine</video:title>
<video:description>Many people idolize FAANG companies, because they own these amazing products that everyone uses</video:description>
<video:content_loc>https://firebasestorage.googleapis.com/v0/b/tech-career-growth.appspot.com/o/TCGHighlights%2FFAANGLife%2F2023_04_01_5_faang_machine_cog.mp4?alt=media&token=77c42eda-69b0-47c5-aa83-537f2fbb2b7a</video:content_loc>
<video:duration>298</video:duration>
<video:rating>5.0</video:rating>
<video:view_count>290</video:view_count>
<video:publication_date>2023-05-21</video:publication_date>
</video:video>
</url>

Blog Sitemap

The blog sitemap contains all of blog post information. This is automatically included as part of our Ghost blog. Our only work on this end was to submit the sitemap to the Google Search Console.

Using Structured Data

Finally, we generate structured data on our video pages, Q&A pages, and event pages to help Google as much as possible determine the content of our pages. This also helps our organic search traffic because Google displays these results in a way that makes them visually stand out more in search results. All of our structured data is generated manually in our Next.js pages.

Video Structured Data

If you check the source on one of our video pages, you'll see the following video structured data in the <head> section:

    <script type="application/ld+json">
      {
        "@context": "https://schema.org/",
        "@type": "VideoObject",
        "name": "At Big Tech, You're Just A Cog In The Machine",
        "description": "Many people idolize FAANG companies, because they own these amazing products that everyone uses",
        "uploadDate": "2023-05-21",
        "thumbnailUrl": [
          "https://firebasestorage.googleapis.com/v0/b/tech-career-growth.appspot.com/o/TCGHighlights%2FFAANGLife%2F2023_04_01_5_faang_machine_cog_thumbnail.jpeg?alt=media&token=71eeeeba-42c2-4029-b4a7-2df7f6e8ee39"
        ],
        "contentUrl": "https://firebasestorage.googleapis.com/v0/b/tech-career-growth.appspot.com/o/TCGHighlights%2FFAANGLife%2F2023_04_01_5_faang_machine_cog.mp4?alt=media&token=77c42eda-69b0-47c5-aa83-537f2fbb2b7a",
        "duration": "PT0H4M58S"
      }
    </script>

Our video page file, pages/lesson/[lessonId]/[slug].tsx, adds the structured data between the Next.js <head> tags for each of our video pages:

<Head>
  <script
    type="application/ld+json"
    dangerouslySetInnerHTML={addVideoJsonLd(lesson, lessonDescription)}
    key="video-jsonld"
  />
</Head>

function addVideoJsonLd(lesson: Lesson, description: string) {
  const uploadDate = lesson ? getLessonUploadDate(lesson) : '';
  const duration = lesson ? getLessonDuration(lesson) : '';

  return {
    __html: `{
      "@context": "https://schema.org/",
      "@type": "VideoObject",
      "name": "${lesson.name}",
      "description": "${description}",
      "uploadDate": "${uploadDate}",
      "thumbnailUrl": ["${lesson.thumbnailUrl}"],
      "contentUrl": "${lesson.videoUrl}",
      "duration": "${duration}"
    }
  `,
  };
}

Q&A Structured Data

Here is our Q&A structured data when you visit one of our Q&A pages:

    <script type="application/ld+json">
      {
        "@context": "https://schema.org/",
        "@type": "QAPage",
        "mainEntity": {
          "@type": "Question",
          "name": "How to develop and express opinions as a Senior Engineer?",
          "text": "I am been working as a Senior software engineer for quite some time now. To move to the Staff level, one feedback/pointer I got was, that start developing opinions on how to do certain things. When people start respecting your opinions, they start to see you as a leader.   The problem I have is, in many cases, I don't have enough relevant experience for the problem at hand. Also, I find, that in discussions I am listening more than I am speaking. Having no opinions, and finding it hard to express opinions for fear of being judged and losing respect in case my opinion is wrong, are the problems I am facing.   Is there any advice on how to develop these skills?",
          "answerCount": 3,
          "upvoteCount": 2,
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "The framework I usually operate on is \"strong opinions, weakly held\". As a senior engineer, you need to break discussion stalemates to move the project forward. Often times, people don't exactly disagree in discussions (so they're fine with most/any reasonable outcome, but don't really want to say it) or they don't even know where the discussion should begin. So that's where \"strong opinions\" come in: you need to be ready to put your opinion down to serve as the starting point. Although you need to put something strongly down, you should be explicit about what specific details that helped you formulate your decision as you vocalize your opinion. This is so the broader group can understand how you arrived at your line of thought and see where they should/shouldn't be building on. Once you've done that, you should end with asking people what their thoughts are? This is where \"weakly held\" comes on: you need to pull input from other participants and keep the discussion going until the group has arrived at a consensus. It doesn't matter if the final outcome is completely different from your opinion, someone else's opinion, or something completely new that was thought of mid-meeting: you are simply a conductor to drive people into agreeing on something reasonable.",
            "upvoteCount": 5
          }
        }
      }
    </script>

Our Q&A page file, pages/questions/[questionId]/[slug].tsx, adds the structured data between the Next.js <head> tags for each of our question pages:

      <Head>
        {threadItems.length > 1 && (
          <script
            type="application/ld+json"
            dangerouslySetInnerHTML={addQAJsonLd(lesson, threadItems)}
            key="qa-jsonld"
          />
        )}
      </Head>

function addQAJsonLd(lesson: Lesson, threadItems: ThreadItem[]) {
  return {
    __html: `{
      "@context": "https://schema.org/",
      "@type": "QAPage",
      "mainEntity": {
        "@type": "Question",
        "name": "${lesson.questionText}",
        "text": "${bodyText}",
        "answerCount": ${validThreadItems.length},
        "upvoteCount": ${(lesson.likingUserIds || []).length},
        "acceptedAnswer": ${JSON.stringify(acceptedItemJson)}
      }
    }
  `,
  };
}

Event Structured Data

Here is our event structured data when you visit one of our event pages:

    <script type="application/ld+json">
      {
        "@context": "https://schema.org/",
        "@type": "Event",
        "eventAttendanceMode": "https://schema.org/OnlineEventAttendanceMode",
        "eventStatus": "https://schema.org/EventScheduled",
        "name": "How To Become A Debugging Master And Fix Issues Faster",
        "startDate": "2023-05-27T17:00:00+00:00",
        "endDate": "2023-05-27T17:00:00+00:00",
        "image": [
          "https://firebasestorage.googleapis.com/v0/b/tech-career-growth.appspot.com/o/events%2Fdebugging_event_banner.png?alt=media&token=d10cff15-96b7-4ab4-8de2-58efc671b896"
        ],
        "description": "Bugs: There will always be there to annoy us",
        "performer": {
          "@type": "Person",
          "name": "Alex Chiou"
        }
      }
    </script>

Our event page file, pages/event/[slug].tsx, adds the structured data between the Next.js <head> tags for each of our event pages:

      <Head>
        <script
          type="application/ld+json"
          dangerouslySetInnerHTML={addEventJsonLd(event)}
          key="event-jsonld"
        />
      </Head>

function addEventJsonLd(event: Event) {
  return {
    __html: `{
      "@context": "https://schema.org/",
      "@type": "Event",
      "eventAttendanceMode": "https://schema.org/OnlineEventAttendanceMode",
      "eventStatus": "https://schema.org/EventScheduled",
      "name": "${event.title}",
      "startDate": "${formattedStartTime}",
      "endDate": "${formattedEndTime}",
      "image": [
        "${event.bannerImageUrl}"
      ],
      "description": "${description}",
      "performer": {
        "@type": "Person",
        "name": "${event.speakers[0].title}"
      }
    }
  `,
  };
}

Conclusion

Our approach to technical SEO was centered around ensuring that our site would not incur any penalties from Google. To achieve this, we heavily relied on the Google Search Console, using it as a primary tool to identify and address any performance issues within our web application. Whenever we encountered problems related to Core Web Vitals, we promptly addressed the flagged issues. This proactive strategy allowed us to optimize all the metrics monitored by the Google Search Console.

If you're working on optimizing SEO for your Next.js app, we recommend following a similar strategy to streamline the indexing process and improve Core Web Vitals. By leveraging the insights provided by the Google Search Console, you can effectively identify areas that need improvement and take the necessary steps to enhance your website's performance and user experience.

If you have any questions or need further assistance, feel free to reach out to me on our Taro Slack channel.