The speed of your website is important to users. But in this age of pay-per-compute, reducing processing all along the chain is important to keep owner costs down too.

Fortunately, in today’s modern web, both of these seemingly competing requirements actually complement each other; reducing processing costs can be achieved by improving site speed.

To explore how to improve site speed and reduce overall processing, lets start with the end in mind and work backwards.

Starting with the customer, what consititutes a ‘good’ user experience, in terms of speed?

Ultimate User Experience?

Imagine you are a first time visitor to your site. The ideal situation would be if an immediate view of your site was instantly available to the device you’re using. No javascript processing, no querying for data, no image optimizations, just the browser showing HTML and [inline] CSS (or even one big snapshot image of your content).

Bear with me, I know some sites need to show dynamic data and some do not, but remember, at this point, you’re just an ordinary user. You dont care about what goes on in the browser and on the server. Neither do I care what static vs dynamic is, or what pain it takes to achieve results. You just want a great experience.

As a user, I want to see the content immediately. By the time I’ve made sense of the initial page (0-3.8secs), I want to interact with it.

If the data a customer is viewing is updated server-side while the page is open, these updates should be pushed to me automagically. Getting new data should not rely on me to fetch the data e.g. by hitting some kind of refresh button to call back to the server.

If the customer leaves the site and comes back to it, by the time I have indicated that I wish to view the page (preemptive rendering?), it should already be fully loaded and no stale data exists on the screen. If any updates have been made to the page since I last saw it, the changes, and only the changes, should have been pushed to my device, using as little bandwidth as possible.

All sounds great? But are these demands even possible using the latest tools/technologies for web and server?

Server Side Rendering

Arguably one of the most important things is to show the content of your site in the quickest way possible.

Generally, the steps that a browser takes to display a web page are as follows:

1. A request to the server is made for a webpage.
2. The server decodes this request and the page, and its resources (files), are downloaded.
3. The web browser uses the page resources to build the page.
4. The page then is rendered (displayed) to the user.

The bottle-neck in this process are steps 2 + 3 i.e. downloading the resources and ‘building’ the page as the resources become available. Rendering a ‘built’ page, stage 4, is what a browser is really good at.

Can we improve, or even skip, steps 2 + 3 to give users a better experience?

If the customer’s browser isnt going to do the hard work to pull all the resources together to build the page, who’s going to do it then?

Combining and building the page can be perfomed on the server instead. The subsequent ‘built’ page can be the resource that is served to the customer.

Server side rendering is an ‘old’ method of serving web pages. Before client-side Javascript was as powerful as it is today, SRR was a common way to serve dynamic web pages.

What’s changed is that now you can use Javascript on both the client and server. You can now replace PHP, Java, ASP etc on the server with NodeJs. This certainly helps with code reuse between client and server, but are we actually any further forward?

The principles are still the same. Client browser makes call to server, server dynamically creates a webpage containing an initial view of the page, server then delivers the page to client.

Server side rendering certainly solves the processing problem for the customer but we’ve just pushed the problem server-side. Not only have we increased the amount of processing that we, the site owner, has to pay for. We are also not much further in improving the overall speed of the site. Certainly, the customer may see a more complete view of your site sooner. Also, the server may have better access to some of the resources. But the overall amount of processing to build the page stays relatively the same.

If we, as a site owner, are now paying for the processing to ‘build’ an inital view of the site how can we make this process as efficient as possible?

Static First

The vast majority of content on the web changes infrequently. Even sites with ‘dynamic’ content, usually only a small amount of the total content of a page is truly dynamic.

If the same content is going to be viewed by more than one visitor, why generate it more than once? It might not be a big issue if you only have 2 visitors. But even with 11 visitors, you still might be doing 10x more processing than was needed.

If as much of your content can be precompiled i.e. the data has already been fetched, the styles applied and html generated preemptively, it can be delivered to the user quicker. If fact, if the content is already compiled, we can take the server out of the chain completely for this interaction, and allow the browser to access the content directly from a location close to the customer.

The ‘static first’ strategy is to compile a static view of the page first, serve it from a CDN, delay enabing javascript until the page has loaded, then hydrate any stale dynamic data.

By adopting static first you can potentially reduce your hosting costs to cents per month, as opposed to dollars, AND provide a blisteringly fast experience for your customers

But what about pages that are never viewed? To statically generate an entire site, you need to generate and manage updates for all potential routes in your website. You might be generating hundreds or thousands of web pages that real users may never visit. However, although ‘real’ users may not visit these pages, it is likely, and welcome, that at least 1x crawler bot will want to access these pages (unless it is content not for the eyes of search engines).

Caching Vs Static Site

So if having assets ready and available on the network, close to the user, is preferable. Is this not just server caching?

Yes and no. There are a number of different caching options available for your site. You can cache the individual items that are referenced by your page e.g. images, css files, database queries etc, or you can cache the page itself, or both. A static first strategy will try to cut the cord with the server. The strategy does not require database query caching and processes as much into a cachable unit i.e. page caching. Caching is generally performed dynamically i.e. the caching happens when one or more users access a particular page. Static site generation is performed pre-emptively i.e. before any users access a particular page. Caching and static site generation both have the aim of making reused assets as available and as close to a user’s device as possible; the difference is if this is done entirely pre-emptively or dynamically.

Static First, Only Update Dynamic Data

However, frequent updates may be unavoidable depending on the type of site. For dynamic sites, it is not feasible to continually pre-compile all views for all users, especially when the data is changing frequently.

But remember again, your user does not care. Mostly, they dont understand the difference between static and dynamic sites; they want to see the important content, fast.

You can aim to statically compile as much of the page as possible beforehand, but the ‘dynamic’ parts will involve some sort of processing. As a first time user, I may accept having the page load, seeing a placeholder where the dynamic data should be, and then ‘hydrating’ the data on page load. On the other hand, the user may, in this instance, prefer a slightly slower page load, if the page loads with the data already fully ‘hydrated’. The choice probably depends on what makes your customer happiest.

Subsequent Visits & Data caching

Up until now we’ve generally been concentrating on the scenario when customers first visit your site. When a customer visits your site again, the situation is slightly different. A returning customer will already have, on their device, many of the resources for the page. If nothing has changed, they may even already have everything to show the page again.

As a returning user, it makes little sense for me to have to contact the server again, have a new view generated by the server and download a new page. If only a small subsection of the page I already have has changed, this is unnecessary processing.

The ideal situation is if the server actually pushes my browser updates. When this happens my browser doesnt have to continually ask if new data is available. An even better scenario is if the server has already pushed me the data before I open the page again.

Even if you dont consider websockets and/or service workers you still have the opportunity to cache api data on the server. If a piece of data has not changed since the last time your browser (or any other browser) asked the server for it, it introduces unnessecary processing. Not for the feint hearted, but api caching can be achieved using the ETag header of a HTTP call.

Final Note. Software Development is hard.

There are only two hard things in Computer Science: cache invalidation and naming things.

— Phil Karlton

There are lots of difficult things about software development, but cache invalidation is the devil.

To reduce processing, all the methods above require caching in some shape or form. Static website generation is just page caching by another name. Putting things in cache is easy, knowing when to update them is incredibly difficult.

If your website page changes url, should you rebuild your entire site in case other pages reference the changed page? Does this re-introduce more processing than its worth?

If you statically compile your stylesheets inline into the page, what if a stylesheet changes? Does every page need compiled again, even if it doesnt make use of the changed style?

If a property in a row of data in your database changes, how do you invalidate your api cache? What if the data is referenced by another item in the cache?

If you are a masochist and like solving these type of problems, have at it. For the rest of us mere mortals, look to use a tool or service that helps you manage these problems.

Here is a non-exhaustive list of some newer tools and services that help in this space:

Hugo – Static website builder

Shifter – Compile your WordPress site to static

Vercel – JAM Stack + React Server Side Rendering + Hosting

Netlify – JAM Stack + Hosting

Webiny – Almost serverless Static website builder + API Generation

Svelte – Javascript framework that precompiles to plain JS so you dont need to deploy and load a framework with your webapp.

FOSDEM is a free and opensource conference held in Brussels every year. There is no admission charge to the 2 day event and the topics under discussion are incredibly diverse. If you are interested in anything and everything opensource, FOSDEM has you covered.

This year was my 2nd year at the conference and, yet again, it did not disappoint. Most of my time was spent in the Web performance & Javascript developer rooms which were packed out. I also managed to get to some ethics talks and to the Quantum Computing room (but quickly scarpered out of there when things got complicated).

I learned quite a bit there, but one quick win that I want to share with you was on web font performance. During the Sia Karamalegos talk on web fonts she shared some quick tips on getting your fonts to load quicker. The full talk is embedded below, but the tips are as follows:

• Preconnect External Fonts. If you are loading external fonts e.g. Google Fonts, a quick performance tip was to use the “preconnect” tag. Using this tag will establish a handshake with the external server before the resource is loaded. With the connection warmed up, it will take less time to download the resource when the time comes. *A small caveat to this, only preload things definately used on the page, otherwise it is doing extra work that might not even be needed.*

/* the usual way of loading google fonts */
<link href="https://fonts.googleapis.com/css?family=Muli:400"
      rel="stylesheet">

/* A better way of loading */
<link rel="preconnect" href="https://fonts.gstatic.com/" crossorigin>
<link href="https://fonts.googleapis.com/css?family=Muli:400"
      rel="stylesheet">

• Preload Self-hosted Fonts. If you are loading self-hosted fonts, the tip was to use the “preload” tag to load the fonts quicker. The “preload” tag will fetch a resource (without actually executing it). This means that the files will be downloaded right away and will not have to wait on other resources to be loaded.

/* loading self-hosted fonts */
<link as="font" type="font/woff2"
  href="./fonts/muli-v12-latin-regular.woff2" crossorigin>

<link as="font" type="font/woff2"
  href="./fonts/muli-v12-latin-700.woff2" crossorigin>

/* a better way */
<link rel="preload" as="font" type="font/woff2"
  href="./fonts/muli-v12-latin-regular.woff2" crossorigin>

<link rel="preload" as="font" type="font/woff2"
  href="./fonts/muli-v12-latin-700.woff2" crossorigin>

Resource Hints

To explain the difference in the “rel” tags, here is a quick cheet sheet from @addyosmani.

Full Video

Please check out the video below for the full talk which also includes tips on FOIT , variable fonts, and tooling.

NoSQL, huh, whats is it good for?

I am not a huge fan of NoSQL. If you can keep your data structured, you should keep your data structured. Also, I have never, to date, worked on a project where a relational database was not scale-able enough to cater for the amount of data I needed to throw at it. NoSQL databases, like Firestore, have their uses, but for most production projects I’ve worked on so far, a relational database was either the ‘best’ choice or ‘good enough’.

However, sometimes, the structure of the data is unknown or likely to change. One such occasion is when you are starting out on a project. For prototype web applications I have found that NoSQL (Mongo, Firebase etc) can really speed up the initial stages of the project. Firebase’s new Cloud Firestore offering is a really useful NoSQL database for prototyping projects.

Real-time Database Vs Cloud Firestore

Firebase used to call its database offering ‘Realtime Database’ it now has a new offering called ‘Cloud Firestore’. Both are currently supported and available through the console, but it does appear that Firebase are trying to coax users to use the newer, but still in beta, Firestore.

The main difference between the two seems to be the data model. One of the frustrating things about the Realtime database was that you needed to de-normalize your data in order to use the queries effectively. If your data was hierarchical in nature you found yourself having to jump through hoops in order to use the realtime database effectively e.g. because you are always returned the entire sub-tree for a query, you needed to create separate trees for your data to avoid bloat in your result sets.

Cloud Firestore FTW

Firestore is not just any old NoSQL database, it has the following compelling features:

• Easy to setup. The Firebase docs do a great job of helping you get started in whatever language you use (web, IOs, Android, Node, Go, Python, Java).
• Easy to use. I have tried to think of a way that the Google team could have made it even easier to use the SDK… but failed.
• Realtime. Real-time means real-time, try it out, create a simple project, hook your UI up to some data, watch the data change as you change the data in your console.
• Scalable. So they say, sadly I’ve not had to use the scalability super powers yet.

Recommendation

Even though its still in beta, I would thoroughly recommend choosing Firestore over the realtime database for new projects. For existing projects, which currently use the realtime database, the choice is not as simple. Migrating to the Firestore is not a simple task due to the vastly different datamodels. Take a look at the pros and cons to determine if it is indeed worth the migration effort.

 

In the next blog post I’ll show how you can get started with Firestore and build upon the Authentication post to tie authentication events to saved user data.

 

Tools For Your Tool-Belt

As a freelance developer its good to have a number of different tools in your tool-belt for when the situation demands. Firebase, and in particular, Firebase Authentication, has proven really useful over the past couple of years.

JWT

Most projects that I work on nowadays require some sort of user authentication. In the past I have used JWT or basic auth as my go-to solutions for authentication. However, even those these methods are fairly straightforward, I still had the nagging feeling that “This is a common problem, there must be a way to make this even easier.”.

JWT is great and easy to get started with but it is not always plain sailing; basic auth is well….. basic. JWT is very similar to Firebase Authentication in that you can use a 3rd party to authenticate users (using the Auth0 service) and not have to set up your own authentication server and process. However, manually persisting tokens on the client side and handling the clean up on expiry, logout etc, still felt verbose to a lazy dev like myself.

Firebase Auth

Firebase Authentication is easy to set up, the wide range of SDKs are easy to use, is free to start using, and “supports authentication using passwords, phone numbers, popular federated identity providers like Google, Facebook and Twitter, and more”.

Most of my experience using Firebase authentication has been in web apps or hybrid mobile app development; below are some really quick steps on how to get started for web:

1. Sign up for a Firebase account and create a new project.
2. Choose “Add firebase to your web app” and follow the instructions i.e. include the JS library and add the configuration details to your app.
3. Add a screen which asks the user for an email address and password. Then pass these details to the “createUserWithEmailAndPassword” SDK function to create a new user.
4. Add another screen which asks for an email address and password, then pass these details to the “signInWithEmailAndPassword” function. When the function returns you should now have an authenticated user! I could go into more detail on the other methods you can use e.g. “signOut”, but most of them really are self-explanatory and the docs do a great job of showing you how.

Facebook, Google, Twitter, GitHub

To add Facebook, Google, Twitter or Github authentication just:

1. Enable the required type of authentication through the “Authentication” tab on your Firebase console (following any specific instructions for each provider).
2. Add a button to your login page for each provider you wish to use.
3. Hook up the button click to use the appropriate SDK method on the client e.g. “signInWithPopup(new firebase.auth.GoogleAuthProvider())”
4. After the user has authenticated themselves through the popup window, your call should return and you should now have a [Google] authenticated user.
5. *A word of caution for hybrid mobile developers* Popups will not work on your hybrid mobile app, you will need to use native Facebook/Google libraries to achieve the same thing e.g. using facebook sign-in for Ionic.

Ionic 3 + Angular 4

This post follows on from posts about hybrid mobile app development using the new WordPress REST API and Ionic: Part 1, Part 2, Part 3

Part 3 contained a walk through of the Ionic 1 code. What follows is a walk through of the code for an Ionic 3 app.

From following the steps in part 2 By now you should have a [very basic] Ionic app running in your browser. The app will allow you to:

• Authenticate with your remote, WordPress REST API enabled, website.
• Make a post from the mobile app to your WordPress site.

Pretty simple stuff.

The code that performs the magic is pretty simple too.

Ionic Project Structure (Ionic 3)

As you can see from the folder structure below there are quite a few folders in our Ionic App.

However, the important files and folders are as follows:

• src/app: The 1st component to be shown on screen for the app. Things like navigation routes are configured here.
• src/pages: The pages for the app are in here i.e. the Login and Report pages.
• src/providers: This folder contains reusable services to perform business logic e.g. authentication and connection to WordPress
• ionic.config.json: This file contains configuration options for the app. In Part 2, we changed a setting in this file to point to our WordPress site.

The folder structure is much like any other Angular 4 application, so we will head straight to the code to see what the key lines are:

/app/app.component.ts

As this component is the entry component for this app we configure if the user should be navigated to the Login or Report screens on startup.

In this file we:

• Start listening to our UserProvider (see user.provider.ts) to see if the logged in/out event is fired.
• If the logged in event fires navigate to the Report page, otherwise go to the Login page.

constructor(
    private events: Events,
    private userData: UserProvider
  ) {
    // start listening to login and log out events
    this.listenToLoginEvents();
    // check to see if user has logged in before
    // if they have there will be a logged in event fired
    userData.checkedLoggedInStatus();
  }

  listenToLoginEvents() {
    // if the user is logged in then navigate to the Report page
    this.events.subscribe('user:login', () => this.rootPage = Report);
    this.events.subscribe('user:logout', () => this.rootPage = Login);
  }

/src/providers/user.provider.ts

In the previous step we saw that a UserProvider was referenced; this provider is declared in user.provider.ts.

The main job of the UserProvider is to check the username and password in order to receive an authentication token from the WordPress server. This job is outlined in the ‘login’ function.

The login function will:

• Contact the WordPress server with the username and password to ask for an authentication token.
• If successful the token will be used in every subsequent http call to the WordPress server. The way we do this is by creating our own http client and injecting the token in every header (see http-client.ts).
• The authentication token will be stored in case the user comes back to the app at a later date.
• An event is fired to tell subscribers that the user is now logged in.

// this is a unique token for storing auth tokens in your local storage
  // for later use
  AUTHTOKEN: string = "myauthtokenkey";

  // determine if the user/password can be authenticated and fire an event when finished
  login(username, password) {
    let data = { username: username, password: password };
    // remove any existing auth tokens from local storage
    this.stor.remove(this.AUTHTOKEN);
    // the important bit, contact the WP end point and ask for a token
    this.http.post('/server/wp-json/jwt-auth/v1/token', data).map(res => res.json())
      .subscribe(response => {
        // great we are authenticated, save the token in localstorage for future use
        this.stor.set(this.AUTHTOKEN, response.token);
        // and start using the token in every subsequent hhtp request to the WP server
        this.http.addHeader('Authorization', 'Bearer ' + response.token);
        // fire an event to say we are authenticated
        this.events.publish('user:login');
      },
      err => console.log(err)
      );
  }

/src/providers/http-client.ts

The HttpClient class has a very simple purpose; inject our authentication header into every Get and Post operation we make to the WordPress server.

// inject the header to every get or post operation
  get(url) {
    return this.http.get(url, {
      headers: this.headers
    });
  }
  post(url, data) {
    return this.http.post(url, data, {
      headers: this.headers
    });
  }

/src/providers/word-press-provider.ts

The WordPress provider has a single function: try to post the score and report data to WordPress.

The ‘createReport’ function:

• Sends a message to the app to tell it that a save operation has started (for the purpose of showing spinners etc).
• Sets the JSON data required by the WordPress REST API posts call (the full range of options can be seen here).
• Posts the information to our WordPress website.
• Gets back a success/failure message.
• Lets the App know we have finished the save.

createReport(score: string, report: string) {
    // let the app know we have started a save operation
    // to show spinners etc
    this.events.publish('wordpress:savestatus', { state: 'saving' });
    // set the JSON data for the call
    // see https://developer.wordpress.org/rest-api/reference/posts/#create-a-post for options
    let data = {
      title: score,
      excerpt: report,
      content: report,
      status: 'publish'
    };
    // the important bit, make a request to the server to create a new post
    // The Authentication header will be added to the request automatically by our Interceptor service
    this.http.post('/server/wp-json/wp/v2/posts', data).subscribe(data => {
      // tell the app that the operation was a success
      this.events.publish('wordpress:savestatus', { state: 'finished' });
      this.events.publish('wordpress:createdreport');
    }, error => {
      this.events.publish('wordpress:savestatus', { state: 'error', message: error });
    });
  }

/pages/login/login.ts

The login page is very simple. When the login button is pressed, the username and password input in the textboxes are passed to our UserProvider service.

username: string;
  password: string;
  constructor(public UserProvider: UserProvider) {
  }
  login() {
    this.UserProvider.login(this.username, this.password);
  }

/pages/report/report.ts

Again the report page is very simple. When the report button is pressed, the score and report input in the textboxes are passed to our WordPress service.

constructor(private events: Events, private wordpress: WordPressProvider, private user: UserProvider, private loadingCtrl: LoadingController, private toastCtrl: ToastController) {
    this.createLoader();
    this.listenToWordPressEvents();
  }
  createReport() {
    this.wordpress.createReport(this.score, this.report);
  }

 

Any Questions?

That’s basically all there is to it.

If you have any questions, or any amendments that I can make to the Github repo, then please comment below….