Integrate CATF Time with Lighthouse Audit

What is CATF?

Content Above the Fold (CATF) refers to the content a viewer sees before they scroll down.
Here we measure the time it takes to render the content above the fold.

Image source: Walmart.com The first screen a user visits on a website is called “Content Above The Fold”. The content that a user sees after scrolling down the page is referred to as “Content Below the Fold”.

image source: https://www.walmart.com/

Why is the CATF time important?

As per studies, if the user sees the loader on a webpage for a long time, the probability of leaving that site is higher as it reduces the interest of the user. To improve the user experience, it is important to reduce CATF time, so that the content is visible as soon as possible.

Problem Statement

If you really think improving CATF is a one-time activity, then you are wrong. As the code grows, the CATF tends to degrade over time. And it happens if regular checkpoints are not kept in place to measure, track, and fix the performance degradation.

Proposal

Calculate CATF time on Pull Request(PR) merge flow and upload it to the monitoring dashboard from where we can refer to it in the future.

How to measure CATF time?

We can measure it using the Lighthouse. It is an open-source, automated tool for improving the quality of web pages. It gives us some metrics related to performance checks for the website. But CATF time is not present by default. Lighthouse gives us the flexibility to create custom audits, i.e. we can add or edit various metrics according to our requirements. We can extend its power by adding custom audits to it.We can read in more detail about Lighthouse architecture from its documentation.

Now, as a developer, the question comes to mind, what should be the point which marks above the fold? As the screen height is different for each device, what should be the point that bifurcates between above the fold and below the fold content?

For this, let’s take an example from the home page of the Walmart website. On mobile, content up to the top banner is included in the content above the fold.

In this tutorial we are going to discuss how we have integrated the CATF metric into Lighthouse audits.

Flow Diagram

First Lighthouse requests the URL after CATF adds it to the Lighthouse Audit. Write the output of Lighthouse Audit to a JSON file. Send results to the monitoring dashboard.

Marking CATF Time

We created a React component, ATFMarker. Here we will set the time for CATF. We used the performance.mark function, which creates a timestamp under the name atf-marker and performance.clearMarks to clear the previous value.

useEffect(() => {
performance.clearMarks("atf-marker");
performance.mark("atf-marker");
},[]);

We will include an ATFMarker component at the end of content above the fold.

<div>
<TopBanner/>
</div>
<ATFMarker/>

As soon as the ATFMarker component renders on the client, it marks the timestamp. It represents the moment when ATFMarker finishes client side rendering, which we consider the time taken to render above the fold content. Since we have marked the atf-marker timestamp, it will come under user-timings in the Lighthouse report.

Configure NX to calculate CATF time?

At Walmart, we use nx to manage our monorepo architecture. We created an executor to run our code.
We have created a wm-workspace-library generator which is used to generate an empty workspace based library. It extends the @nrwl/workspace:library generator and adds some Walmart specific changes.
We can create a library using the NX extension. We will generate it inside the workspace library as it is used to manage our code base.

Click on NX extension -> Generate -> wm-workspace-library

A Snapshot of Visual Studio Code, provide the name and click on the “Run” button.

Provide a scope and name, then click on Run. Here we provide the name as performance and scope as workspace. It creates a library named performance under workspace. Inside the performance library, we will write code.

Or we can generate it using shell commands as well.

pnpm g wm-workspace-library -- --name=<scope>/<name> --forceGenerate

We configure our workspaces in workspace.json. This file contains the workspace projects with their architect targets. Suppose we have a demo-app for which we will define a browser-perf target. Every target uses an executor which runs this target.

"demo-app": {
"root": "apps/demo/app",
"sourceRoot": "apps/demo/app",
"projectType": "application",
"targets": {
"browser-perf": {
"executor": "./libs/workspace/performance/src/executors:run",
"options": {
"urls": ["/url1","/url2"],
"serveTarget": "demo-app:serve"
}
},....

When we run the below command, the code under the performance library gets executed.

nx run demo-app:browser-perf

Inside options, we have provided a URL array on which we will run the Lighthouse.
Create a schema.json file inside the executors folder to define a schema to validate inputs.

Integrating with Lighthouse CI

Using Lighthouse CI, we can automate it and further enhance this feature.
Lighthouse CI is typically used through one of the following approaches:

  • Running Lighthouse CI as part of continuous integration.
  • Using a Lighthouse CI GitHub Action that runs and comments on every pull request.
  • Tracking performance over time via the dashboard provided by Lighthouse Server.

How to run Lighthouse CI on a local machine

Before proceeding further, let’s discuss a few commands:

lhci collect-It is used to run the Lighthouse and collect its results in a local .lighthouseci/ folder.

Options: --url  A URL to run Lighthouse on. Use this flag multiple times to evaluate multiple URLs.
To run on multiple URLs
lhci collect --url=https://example-1.com --url=https://example-2.com

lhci assert — In this we can define multiple strategies for aggregating the results and at which result we need to show warning, error etc.

lhci upload — Saves the runs in .lighthouseci/ folder to the desired target
Options: --outputDir [filesystem only] The directory in which to dump Lighthouse results.

lhci autorunAutomatically run, with sensible defaults, lhci collect, lhci assert and lhci upload, depending on the options in your config

Lighthouse CI configuration can be managed through a config file, environment variables, and CLI flag overrides. If you want to provide some specific configuration to the Lighthouse, create a .lighthouserc.js file in the root directory which contains the configuration related to the Lighthouse.

Now to run lhci on local, it requires three steps:
1. Add lhci/cli package.
2. Start Server
3. Run lhci command

Let’s discuss these steps:

Add lhci/cli package locally in your code base.

npm install -g @lhci/cli
Let’s write code to start the server and run lhci on different URLs.
From the workspace.json , we have provided URLs.

We can start the server using the runExecutor utility which invokes the executor. It will find the target in the configuration, find the executor, construct the options (as if you invoked it in the terminal) and invoke the executor. It returns an iterable from where we can evaluate the base URL.

Now we run the lhci command autorun
We use execa to execute the commands with arguments

const args = [“autorun”,`--collect.url=${url1}` `--collect.url=${url2}`]
const lhResult = await execa(lhciPath, args, {stdio: “inherit”});

Inside the .lighthouseci folder, output HTML and JSON files are recorded. We can read data from those JSON files. You can see all predefined audits in the default Lighthouse config.

Result of the lighthouse audit in JSON file. In “user-timings”, object “atf-marker” is present with a startTime of 2537.63 msatf-marker in JSON fileResult of the lighthouse audit in HTML file. In “user-timings”, object “atf-marker” is present with a startTime of 2537.63 ms.atf-marker in html file

In JSON output files, we have atf-marker data as well. We can use it according to our requirements. We have extracted the required data and written it in a file, let’s say demo.json.
On the CI merge flow, we read the demo.json file and send data to the monitoring dashboard.

Future Work

We will add it to the PR flow of CI as well so that developers know the CATF time before their PR merge.

Potential Issues

Since Lighthouse loads the page to find the metrics, as URLs increase, it becomes a time-consuming process. To resolve this issue, we should run URLs in batches.

Conclusion

To check the performance related data, the Lighthouse provides us with some of the default metrics. We can extend them by adding custom metrics and updating the audit.

Integrate CATF Time with Lighthouse Audit was originally published in Walmart Global Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Article Link: https://medium.com/walmartglobaltech/integrate-catf-time-with-lighthouse-audit-2cc4852d0698?source=rss----905ea2b3d4d1---4