Tag Archives: performance

Thumbnail

Keeping Node.js Fast: Tools, Techniques, And Tips For Making High-Performance Node.js Servers




Keeping Node.js Fast: Tools, Techniques, And Tips For Making High-Performance Node.js Servers

David Mark Clements



If you’ve been building anything with Node.js for long enough, then you’ve no doubt experienced the pain of unexpected speed issues. JavaScript is an evented, asynchronous language. That can make reasoning about performance tricky, as will become apparent. The surging popularity of Node.js has exposed the need for tooling, techniques and thinking suited to the constraints of server-side JavaScript.

When it comes to performance, what works in the browser doesn’t necessarily suit Node.js. So, how do we make sure a Node.js implementation is fast and fit for purpose? Let’s walk through a hands-on example.

Tools

Node is a very versatile platform, but one of the predominant applications is creating networked processes. We’re going to focus on profiling the most common of these: HTTP web servers.

We’ll need a tool that can blast a server with lots of requests while measuring the performance. For example, we can use AutoCannon:

npm install -g autocannon

Other good HTTP benchmarking tools include Apache Bench (ab) and wrk2, but AutoCannon is written in Node, provides similar (or sometimes greater) load pressure, and is very easy to install on Windows, Linux, and Mac OS X.

After we’ve established a baseline performance measurement, if we decide our process could be faster we’ll need some way to diagnose problems with the process. A great tool for diagnosing various performance issues is Node Clinic, which can also be installed with npm:

npm --install -g clinic

This actually installs a suite of tools. We’ll be using Clinic Doctor and Clinic Flame (a wrapper around 0x) as we go.

Note: For this hands-on example we’ll need Node 8.11.2 or higher.

The Code

Our example case is a simple REST server with a single resource: a large JSON payload exposed as a GET route at /seed/v1. The server is an app folder which consists of a package.json file (depending on restify 7.1.0), an index.js file and a util.js file.

The index.js file for our server looks like so:

'use strict'

const restify = require('restify')
const  etagger, timestamp, fetchContent  = require('./util')()
const server = restify.createServer()

server.use(etagger().bind(server))

server.get('/seed/v1', function (req, res, next) 
  fetchContent(req.url, (err, content) => 
    if (err) return next(err)
    res.send(data: content, url: req.url, ts: timestamp())
    next()
  })
})

server.listen(3000)

This server is representative of the common case of serving client-cached dynamic content. This is achieved with the etagger middleware, which calculates an ETag header for the latest state of the content.

The util.js file provides implementation pieces that would commonly be used in such a scenario, a function to fetch the relevant content from a backend, the etag middleware and a timestamp function that supplies timestamps on a minute-by-minute basis:

'use strict'

require('events').defaultMaxListeners = Infinity
const crypto = require('crypto')

module.exports = () => 
  const content = crypto.rng(5000).toString('hex')
  const ONE_MINUTE = 60000
  var last = Date.now()

  function timestamp () 
    var now = Date.now()
    if (now — last >= ONE_MINUTE) last = now
    return last
  
  
  function etagger () 
    var cache = 
    var afterEventAttached = false
    function attachAfterEvent (server) 
      if (attachAfterEvent === true) return
      afterEventAttached = true
      server.on('after', (req, res) => 
        if (res.statusCode !== 200) return
        if (!res._body) return
        const key = crypto.createHash('sha512')
          .update(req.url)
          .digest()
          .toString('hex')
        const etag = crypto.createHash('sha512')
          .update(JSON.stringify(res._body))
          .digest()
          .toString('hex')
        if (cache[key] !== etag) cache[key] = etag
      )
    }
    return function (req, res, next) 
      attachAfterEvent(this)
      const key = crypto.createHash('sha512')
        .update(req.url)
        .digest()
        .toString('hex')
      if (key in cache) res.set('Etag', cache[key])
      res.set('Cache-Control', 'public, max-age=120')
      next()
    
  }

  function fetchContent (url, cb) 
    setImmediate(() => 
      if (url !== '/seed/v1') cb(Object.assign(Error('Not Found'), statusCode: 404))
      else cb(null, content)
    })
  }

  return  timestamp, etagger, fetchContent 
  
}

By no means take this code as an example of best practices! There are multiple code smells in this file, but we’ll locate them as we measure and profile the application.

To get the full source for our starting point, the slow server can be found over here.

Profiling

In order to profile, we need two terminals, one for starting the application, and the other for load testing it.

In one terminal, within the app, folder we can run:

node index.js

In another terminal we can profile it like so:

autocannon -c100 localhost:3000/seed/v1

This will open 100 concurrent connections and bombard the server with requests for ten seconds.

The results should be something similar to the following (Running 10s test @ http://localhost:3000/seed/v1 — 100 connections):

Stat Avg Stdev Max
Latency (ms) 3086.81 1725.2 5554
Req/Sec 23.1 19.18 65
Bytes/Sec 237.98 kB 197.7 kB 688.13 kB

231 requests in 10s, 2.4 MB read

Results will vary depending on the machine. However, considering that a “Hello World” Node.js server is easily capable of thirty thousand requests per second on that machine that produced these results, 23 requests per second with an average latency exceeding 3 seconds is dismal.

Diagnosing

Discovering The Problem Area

We can diagnose the application with a single command, thanks to Clinic Doctor’s –on-port command. Within the app folder we run:

clinic doctor --on-port=’autocannon -c100 localhost:$PORT/seed/v1’ -- node index.js

This will create an HTML file that will automatically open in our browser when profiling is complete.

The results should look something like the following:


Clinic Doctor has detected an Event Loop issue


Clinic Doctor results

The Doctor is telling us that we have probably had an Event Loop issue.

Along with the message near the top of the UI, we can also see that the Event Loop chart is red, and shows a constantly increasing delay. Before we dig deeper into what this means, let’s first understand the effect the diagnosed issue is having on the other metrics.

We can see the CPU is consistently at or above 100% as the process works hard to process queued requests. Node’s JavaScript engine (V8) actually uses two CPU cores. One for the Event Loop and the other for Garbage Collection. When we see the CPU spiking up to 120% in some cases, the process is collecting objects related to handled requests.

We see this correlated in the Memory graph. The solid line in the Memory chart is the Heap Used metric. Any time there’s a spike in CPU we see a fall in the Heap Used line, showing that memory is being deallocated.

Active Handles are unaffected by the Event Loop delay. An active handle is an object that represents either I/O (such as a socket or file handle) or a timer (such as a setInterval). We instructed AutoCannon to open 100 connections (-c100). Active handles stay a consistent count of 103. The other three are handles for STDOUT, STDERR, and the handle for the server itself.

If we click the Recommendations panel at the bottom of the screen, we should see something like the following:


Clinic Doctor recommendations panel opened


Viewing issue specific recommendations

Short-Term Mitigation

Root cause analysis of serious performance issues can take time. In the case of a live deployed project, it’s worth adding overload protection to servers or services. The idea of overload protection is to monitor event loop delay (among other things), and respond with “503 Service Unavailable” if a threshold is passed. This allows a load balancer to fail over to other instances, or in the worst case means users will have to refresh. The overload-protection module can provide this with minimum overhead for Express, Koa, and Restify. The Hapi framework has a load configuration setting which provides the same protection.

Understanding The Problem Area

As the short explanation in Clinic Doctor explains, if the Event Loop is delayed to the level that we’re observing it’s very likely that one or more functions are “blocking” the Event Loop.

It’s especially important with Node.js to recognize this primary JavaScript characteristic: asynchronous events cannot occur until currently executing code has completed.

This is why a setTimeout cannot be precise.

For instance, try running the following in a browser’s DevTools or the Node REPL:

console.time('timeout')
setTimeout(console.timeEnd, 100, 'timeout')
let n = 1e7
while (n--) Math.random()

The resulting time measurement will never be 100ms. It will likely be in the range of 150ms to 250ms. The setTimeout scheduled an asynchronous operation (console.timeEnd), but the currently executing code has not yet complete; there are two more lines. The currently executing code is known as the current “tick.” For the tick to complete, Math.random has to be called ten million times. If this takes 100ms, then the total time before the timeout resolves will be 200ms (plus however long it takes the setTimeout function to actually queue the timeout beforehand, usually a couple of milliseconds).

In a server-side context, if an operation in the current tick is taking a long time to complete requests cannot be handled, and data fetching cannot occur because asynchronous code will not be executed until the current tick has completed. This means that computationally expensive code will slow down all interactions with the server. So it’s recommended to split out resource intense work into separate processes and call them from the main server, this will avoid cases where on rarely used but expensive route slows down the performance of other frequently used but inexpensive routes.

The example server has some code that is blocking the Event Loop, so the next step is to locate that code.

Analyzing

One way to quickly identify poorly performing code is to create and analyze a flame graph. A flame graph represents function calls as blocks sitting on top of each other — not over time but in aggregate. The reason it’s called a ‘flame graph’ is because it typically uses an orange to red color scheme, where the redder a block is the “hotter” a function is, meaning, the more it’s likely to be blocking the event loop. Capturing data for a flame graph is conducted through sampling the CPU — meaning that a snapshot of the function that is currently being executed and it’s stack is taken. The heat is determined by the percentage of time during profiling that a given function is at the top of the stack (e.g. the function currently being executed) for each sample. If it’s not the last function to ever be called within that stack, then it’s likely to be blocking the event loop.

Let’s use clinic flame to generate a flame graph of the example application:

clinic flame --on-port=’autocannon -c100 localhost:$PORT/seed/v1’ -- node index.js

The result should open in our browser with something like the following:


Clinic’s flame graph shows that server.on is the bottleneck


Clinic’s flame graph visualization

The width of a block represents how much time it spent on CPU overall. Three main stacks can be observed taking up the most time, all of them highlighting server.on as the hottest function. In truth, all three stacks are the same. They diverge because during profiling optimized and unoptimized functions are treated as separate call frames. Functions prefixed with a * are optimized by the JavaScript engine, and those prefixed with a ~ are unoptimized. If the optimized state isn’t important to us, we can simplify the graph further by pressing the Merge button. This should lead to view similar to the following:


Merged flame graph


Merging the flame graph

From the outset, we can infer that the offending code is in the util.js file of the application code.

The slow function is also an event handler: the functions leading up to the function are part of the core events module, and server.on is a fallback name for an anonymous function provided as an event handling function. We can also see that this code isn’t in the same tick as code that actually handles the request. If there were functions in the core, http, net, and stream would be in the stack.

Such core functions can be found by expanding other, much smaller, parts of the flame graph. For instance, try using the search input on the top right of the UI to search for send (the name of both restify and http internal methods). It should be on the right of the graph (functions are alphabetically sorted):


Flame graph has two small blocks highlighted which represent HTTP processing function


Searching the flame graph for HTTP processing functions

Notice how comparatively small all the actual HTTP handling blocks are.

We can click one of the blocks highlighted in cyan which will expand to show functions like writeHead and write in the http_outgoing.js file (part of Node core http library):


Flame graph has zoomed into a different view showing HTTP related stacks


Expanding the flame graph into HTTP relevant stacks

We can click all stacks to return to the main view.

The key point here is that even though the server.on function isn’t in the same tick as the actual request handling code, it’s still affecting the overall server performance by delaying the execution of otherwise performant code.

Debugging

We know from the flame graph that the problematic function is the event handler passed to server.on in the util.js file.

Let’s take a look:

server.on('after', (req, res) => 
  if (res.statusCode !== 200) return
  if (!res._body) return
  const key = crypto.createHash('sha512')
    .update(req.url)
    .digest()
    .toString('hex')
  const etag = crypto.createHash('sha512')
    .update(JSON.stringify(res._body))
    .digest()
    .toString('hex')
  if (cache[key] !== etag) cache[key] = etag
)

It’s well known that cryptography tends to be expensive, as does serialization (JSON.stringify) but why don’t they appear in the flame graph? These operations are in the captured samples, but they’re hidden behind the cpp filter. If we press the cpp button we should see something like the following:


Additional blocks related to C++ have been revealed in the flame graph (main view)


Revealing serialization and cryptography C++ frames

The internal V8 instructions relating to both serialization and cryptography are now shown as the hottest stacks and as taking up most of the time. The JSON.stringify method directly calls C++ code; this is why we don’t see a JavaScript function. In the cryptography case, functions like createHash and update are in the data, but they are either inlined (which means they disappear in the merged view) or too small to render.

Once we start to reason about the code in the etagger function it can quickly become apparent that it’s poorly designed. Why are we taking the server instance from the function context? There’s a lot of hashing going on, is all of that necessary? There’s also no If-None-Match header support in the implementation which would mitigate some of the load in some real-world scenarios because clients would only make a head request to determine freshness.

Let’s ignore all of these points for the moment and validate the finding that the actual work being performed in server.on is indeed the bottleneck. This can be achieved by setting the server.on code to an empty function and generating a new flamegraph.

Alter the etagger function to the following:

function etagger () 
  var cache = 
  var afterEventAttached = false
  function attachAfterEvent (server) 
    if (attachAfterEvent === true) return
    afterEventAttached = true
    server.on('after', (req, res) => )
  }
  return function (req, res, next) 
    attachAfterEvent(this)
    const key = crypto.createHash('sha512')
      .update(req.url)
      .digest()
      .toString('hex')
    if (key in cache) res.set('Etag', cache[key])
    res.set('Cache-Control', 'public, max-age=120')
    next()
  
}

The event listener function passed to server.on is now a no-op.

Let’s run clinic flame again:

clinic flame --on-port='autocannon -c100 localhost:$PORT/seed/v1' -- node index.js

This should produce a flame graph similar to the following:


Flame graph shows that Node.js event system stacks are still the bottleneck


Flame graph of the server when server.on is an empty function

This looks better, and we should have noticed an increase in request per second. But why is the event emitting code so hot? We would expect at this point for the HTTP processing code to take up the majority of CPU time, there’s nothing executing at all in the server.on event.

This type of bottleneck is caused by a function being executed more than it should be.

The following suspicious code at the top of util.js may be a clue:

require('events').defaultMaxListeners = Infinity

Let’s remove this line and start our process with the --trace-warnings flag:

node --trace-warnings index.js

If we profile with AutoCannon in another terminal, like so:

autocannon -c100 localhost:3000/seed/v1

Our process will output something similar to:

(node:96371) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 after listeners added. Use emitter.setMaxListeners() to increase limit
  at _addListener (events.js:280:19)
  at Server.addListener (events.js:297:10)
  at attachAfterEvent 
    (/Users/davidclements/z/nearForm/keeping-node-fast/slow/util.js:22:14)
  at Server.
    (/Users/davidclements/z/nearForm/keeping-node-fast/slow/util.js:25:7)
  at call
    (/Users/davidclements/z/nearForm/keeping-node-fast/slow/node_modules/restify/lib/chain.js:164:9)
  at next
    (/Users/davidclements/z/nearForm/keeping-node-fast/slow/node_modules/restify/lib/chain.js:120:9)
  at Chain.run
    (/Users/davidclements/z/nearForm/keeping-node-fast/slow/node_modules/restify/lib/chain.js:123:5)
  at Server._runUse
    (/Users/davidclements/z/nearForm/keeping-node-fast/slow/node_modules/restify/lib/server.js:976:19)
  at Server._runRoute
    (/Users/davidclements/z/nearForm/keeping-node-fast/slow/node_modules/restify/lib/server.js:918:10)
  at Server._afterPre
    (/Users/davidclements/z/nearForm/keeping-node-fast/slow/node_modules/restify/lib/server.js:888:10)

Node is telling us that lots of events are being attached to the server object. This is strange because there’s a boolean that checks if the event has been attached and then returns early essentially making attachAfterEvent a no-op after the first event is attached.

Let’s take a look at the attachAfterEvent function:

var afterEventAttached = false
function attachAfterEvent (server) 
  if (attachAfterEvent === true) return
  afterEventAttached = true
  server.on('after', (req, res) => )
}

The conditional check is wrong! It checks whether attachAfterEvent is true instead of afterEventAttached. This means a new event is being attached to the server instance on every request, and then all prior attached events are being fired after each request. Whoops!

Optimizing

Now that we’ve discovered the problem areas, let’s see if we can make the server faster.

Low-Hanging Fruit

Let’s put the server.on listener code back (instead of an empty function) and use the correct boolean name in the conditional check. Our etagger function looks as follows:

function etagger () 
  var cache = 
  var afterEventAttached = false
  function attachAfterEvent (server) 
    if (afterEventAttached === true) return
    afterEventAttached = true
    server.on('after', (req, res) => 
      if (res.statusCode !== 200) return
      if (!res._body) return
      const key = crypto.createHash('sha512')
        .update(req.url)
        .digest()
        .toString('hex')
      const etag = crypto.createHash('sha512')
        .update(JSON.stringify(res._body))
        .digest()
        .toString('hex')
      if (cache[key] !== etag) cache[key] = etag
    )
  }
  return function (req, res, next) 
    attachAfterEvent(this)
    const key = crypto.createHash('sha512')
      .update(req.url)
      .digest()
      .toString('hex')
    if (key in cache) res.set('Etag', cache[key])
    res.set('Cache-Control', 'public, max-age=120')
    next()
  
}

Now we check our fix by profiling again. Start the server in one terminal:

node index.js

Then profile with AutoCannon:

autocannon -c100 localhost:3000/seed/v1

We should see results somewhere in the range of a 200 times improvement (Running 10s test @ http://localhost:3000/seed/v1 — 100 connections):

Stat Avg Stdev Max
Latency (ms) 19.47 4.29 103
Req/Sec 5011.11 506.2 5487
Bytes/Sec 51.8 MB 5.45 MB 58.72 MB

50k requests in 10s, 519.64 MB read

It’s important to balance potential server cost reductions with development costs. We need to define, in our own situational contexts, how far we need to go in optimizing a project. Otherwise, it can be all too easy to put 80% of the effort into 20% of the speed enhancements. Do the constraints of the project justify this?

In some scenarios, it could be appropriate to achieve a 200 times improvement with a low hanging fruit and call it a day. In others, we may want to make our implementation as fast as it can possibly be. It really depends on project priorities.

One way to control resource spend is to set a goal. For instance, 10 times improvement, or 4000 requests per second. Basing this on business needs makes the most sense. For instance, if server costs are 100% over budget, we can set a goal of 2x improvement.

Taking It Further

If we produce a new flame graph of our server, we should see something similar to the following:


Flame graph still shows server.on as the bottleneck, but a smaller bottleneck


Flame graph after the performance bug fix has been made

The event listener is still the bottleneck, it’s still taking up one-third of CPU time during profiling (the width is about one third the whole graph).

What additional gains can be made, and are the changes (along with their associated disruption) worth making?

With an optimized implementation, which is nonetheless slightly more constrained, the following performance characteristics can be achieved (Running 10s test @ http://localhost:3000/seed/v1 — 10 connections):

Stat Avg Stdev Max
Latency (ms) 0.64 0.86 17
Req/Sec 8330.91 757.63 8991
Bytes/Sec 84.17 MB 7.64 MB 92.27 MB

92k requests in 11s, 937.22 MB read

While a 1.6x improvement is significant, it arguable depends on the situation whether the effort, changes, and code disruption necessary to create this improvement are justified. Especially when compared to the 200x improvement on the original implementation with a single bug fix.

To achieve this improvement, the same iterative technique of profile, generate flamegraph, analyze, debug, and optimize was used to arrive at the final optimized server, the code for which can be found here.

The final changes to reach 8000 req/s were:

These changes are slightly more involved, a little more disruptive to the code base, and leave the etagger middleware a little less flexible because it puts the burden on the route to provide the Etag value. But it achieves an extra 3000 requests per second on the profiling machine.

Let’s take a look at a flame graph for these final improvements:


Flame graph shows that internal code related to the net module is now the bottleneck


Healthy flame graph after all performance improvements

The hottest part of the flame graph is part of Node core, in the net module. This is ideal.

Preventing Performance Problems

To round off, here are some suggestions on ways to prevent performance issues in before they are deployed.

Using performance tools as informal checkpoints during development can filter out performance bugs before they make it into production. Making AutoCannon and Clinic (or equivalents) part of everyday development tooling is recommended.

When buying into a framework, find out what it’s policy on performance is. If the framework does not prioritize performance, then it’s important to check whether that aligns with infrastructural practices and business goals. For instance, Restify has clearly (since the release of version 7) invested in enhancing the library’s performance. However, if low cost and high speed is an absolute priority, consider Fastify which has been measured as 17% faster by a Restify contributor.

Watch out for other widely impacting library choices — especially consider logging. As developers fix issues, they may decide to add additional log output to help debug related problems in the future. If an unperformant logger is used, this can strangle performance over time after the fashion of the boiling frog fable. The pino logger is the fastest newline delimited JSON logger available for Node.js.

Finally, always remember that the Event Loop is a shared resource. A Node.js server is ultimately constrained by the slowest logic in the hottest path.

Smashing Editorial
(rb, ra, il)


Continue at source:  

Keeping Node.js Fast: Tools, Techniques, And Tips For Making High-Performance Node.js Servers

Thumbnail

Monthly Web Development Update 5/2018: Browser Performance, Iteration Zero, And Web Authentication




Monthly Web Development Update 5/2018: Browser Performance, Iteration Zero, And Web Authentication

Anselm Hannemann



As developers, we often talk about performance and request browsers to render things faster. But when they finally do, we demand even more performance.

Alex Russel from the Chrome team now shared some thoughts on developers abusing browser performance and explains why websites are still slow even though browsers reinvented themselves with incredibly fast rendering engines. This is in line with an article by Oliver Williams in which he states that we’re focusing on the wrong things, and instead of delivering the fastest solutions for slower machines and browsers, we’re serving even bigger bundles with polyfills and transpiled code to every browser.

It certainly isn’t easy to break out of this pattern and keep bundle size to a minimum in the interest of the user, but we have the technologies to achieve that. So let’s explore non-traditional ways and think about the actual user experience more often — before defining a project workflow instead of afterward.

Front-End Performance Checklist 2018

To help you cater for fast and smooth experiences, Vitaly Friedman summarized everything you need to know to optimize your site’s performance in one handy checklist. Read more →

News

General

  • Oliver Williams wrote about how important it is that we rethink how we’re building websites and implement “progressive enhancement” to make the web work great for everyone. After all, it’s us who make the experience worse for our users when blindly transpiling all our ECMAScript code or serving tons of JavaScript polyfills to those who already use slow machines and old software.
  • Ian Feather reveals that around 1% of all requests for JavaScript on BuzzFeed time out. That’s about 13 million requests per month. A good reminder of how important it is to provide a solid fallback, progressive enhancement, and workarounds.
  • The new GDPR (General Data Protection Regulation) directive is coming very soon, and while our inboxes are full of privacy policy updates, one thing that’s still very unclear is which services can already provide so-called DPAs (Data Processing Agreements). Joschi Kuphal collects services that offer a DPA, so that we can easily look them up and see how we can obtain a copy in order to continue using their services. You can help by contributing to this resource via Pull Requests.

UI/UX

Product design principles
How to create a consistent, harmonious user experience when designing product cards? Mei Zhang shares some valuable tips. (Image credit)

Security

Privacy

  • The GDPR Checklist is another helpful resource for people to check whether a website is compliant with the upcoming EU directive.
  • Bloomberg published a story about the open-source privacy-protection project pi-hole, why it exists and what it wants to achieve. I use the software daily to keep my entire home and work network tracking-free.
GDPR Compliance Checklist
Achieving GDPR Compliance shouldn’t be a struggle. The GDPR Compliance Checklist helps you see clearer. (Image credit)

Web Performance

  • Postgres 10 has been here for quite a while already, but I personally struggled to find good information on how to use all these amazing features it brings along. Gabriel Enslein now shares Postgres 10 performance updates in a slide deck, shedding light on how to use the built-in JSON support, native partitioning for large datasets, hash index resiliency, and more.
  • Andrew Betts found out that a lot of websites are using outdated headers. He now shares why we should drop old headers and which ones to serve instead.

Accessibility

Page previews
Page previews open possibilities in multiple areas, as Nirzar Pangarkar explains. (Image credit: Nirzar Pangarkar)

CSS

  • Rarely talked about for years, CSS tables are still used on most websites to show (and that’s totally the correct way to do so) data in tables. But as they’re not responsive by default, we always struggled when making them responsive and most of us used JavaScript to make them work on mobile screens. Lea Verou now found two new ways to achieve responsive tables by using CSS: One is to use text-shadow to copy text to other rows, the other one uses element() to copy the entire <thead> to other rows — I still try to understand how Lea found these solutions, but this is amazing!
  • Rachel Andrew wrote an article about building and providing print stylesheets in 2018 and why they matter a lot for users even if they don’t own a printer anymore.
  • Osvaldas Valutis shares how to implement the so-called “Priority Plus” navigation pattern mostly with CSS, at least in modern browsers. If you need to support older browsers, you will need to extend this solution further, but it’s a great start to implement such a pattern without too much JavaScript.
  • Rachel Andrew shares what’s coming up in the CSS Grid Level 2 and Subgrid specifications and explains what it is, what it can solve, and how to use it once it is available in browsers.

JavaScript

  • Chris Ashton “used the web for a day with JavaScript turned off.” This piece highlights the importance of thinking about possible JavaScript failures on websites and why it matters if you provide fallbacks or not.
  • Sam Thorogood shares how we can build a “native undo & redo for the web”, as used in many text editors, games, planning or graphical software and other occasions such as a drag and drop reordering. And while it’s not easy to build, the article explains the concepts and technical aspects to help us understand this complicated matter.
  • There’s a new way to implement element/container queries into your application: eqio is a tiny library using IntersectionObserver.

Work & Life

  • Johannes Seitz shares his thoughts about project management at the start of projects. He calls the method “Iteration Zero”. An interesting concept to understand the scope and risks of a project better at a time when you still don’t have enough experience with the project itself but need to build a roadmap to get things started.
  • Arestia Rosenberg shares why her number one advice for freelancers is to ‘lean into the moment’. It’s about doing work when you can and using your chance to do something else when you don’t feel you can work productively. In the end, the summary results in a happy life and more productivity. I’d personally extend this to all people who can do that, but, of course, it’s best applicable to freelancers indeed.
  • Sam Altman shares a couple of handy productivity tips that are not just a ‘ten things to do’ list but actually really helpful thoughts about how to think about being productive.

Going Beyond…

  • Ethan Marcotte elaborates on the ethical issues with Google Duplex that is designed to imitate human voice so well that people don’t notice if it’s a machine or a human being. While this sounds quite interesting from a technical point of view, it will push the debate about fake news much further and cause more struggle to differentiate between something a human said or a machine imitated.
  • Our world is actually built on promises, and here’s why it’s so important to stick to your promises even if it’s hard sometimes.
  • I bet that most of you haven’t heard of Palantir yet. The company is funded by Peter Thiel and is a data-mining company that has the intention to collect as much data as possible about everybody in the world. It’s known to collaborate with various law enforcement authorities and even has connections to military services. What they do with data and which data they have from us isn’t known. My only hope right now is that this company will suffer a lot from the EU GDPR directive and that the European Union will try to stop their uncontrolled data collection. Facebook’s data practices are nothing compared to Palantir it seems.
  • Researchers sound the alarm after an analysis showed that buying a new smartphone consumes as much energy as using an existing phone for an entire decade. I guess I’ll not replace my iPhone 7 anytime soon — it’s still an absolutely great device and just enough for what I do with it.
  • Anton Sten shares his thoughts on Vanity Metrics, a common way to share numbers and statistics out of context. And since he realized what relevancy they have, he thinks differently about most of the commonly readable data such as investments or usage data of services now. Reading one number without having a context to compare it to doesn’t matter at all. We should keep that in mind.

We hope you enjoyed this Web Development Update. The next one is scheduled for Friday, June 15th. Stay tuned.

Smashing Editorial
(cm)


See the original article here: 

Monthly Web Development Update 5/2018: Browser Performance, Iteration Zero, And Web Authentication

Thumbnail

Measuring Websites With Mobile-First Optimization Tools




Measuring Websites With Mobile-First Optimization Tools

Jon Raasch



Performance on mobile can be particularly challenging: underpowered devices, slow networks, and poor connections are some of those challenges. With more and more users migrating to mobile, the rewards for mobile optimization are great. Most workflows have already adopted mobile-first design and development strategies, and it’s time to apply a similar mindset to performance.

In this article, we’ll take a look at studies linking page speed to real-world metrics, and discuss the specific ways mobile performance impacts your site. Then we’ll explore benchmarking tools you can use to measure your website’s mobile performance. Finally, we’ll work with tools to help identify and remove the code debt that bloats and weighs down your site.

Responsive Configurators

How would you design a responsive car configurator? How would you deal with accessibility, navigation, real-time previews, interaction and performance? Let’s figure it out. Read article →

Why Performance Matters

The benefits of performance optimization are well-documented. In short, performance matters because users prefer faster websites. But it’s more than a qualitative assumption about user experience. There are a variety of studies that directly link reduced load times to increased conversion and revenue, such as the now decade-old Amazon study that showed each 100ms of latency led to a 1% drop in sales.

Page Speed, Bounce Rate & Conversion

In the data world, poor performance leads to an increased bounce rate. And in the mobile world that bounce rate may occur sooner than you think. A recent study shows that 53% of mobile users abandon a site that takes more than 3 seconds to load.

That means if your site loads in 3.5 seconds, over half of your potential users are leaving (and most likely visiting a competitor). That may be tough to swallow, but it is as much a problem as it is an opportunity. If you can get your site to load more quickly, you are potentially doubling your conversion. And if your conversion is even indirectly linked to profits, you’re doubling your revenue.

SEO And Social Media

Beyond reduced conversion, slow load times create secondary effects that diminish your inbound traffic. Search engines already use page speed in their ranking algorithms, bubbling faster sites to the top. Additionally, Google is specifically factoring mobile speed for mobile searches as of July 2018.

Social media outlets have begun factoring page speed in their algorithms as well. In August 2017, Facebook announced that it would roll out specific changes to the newsfeed algorithm for mobile devices. These changes include page speed as a factor, which means that slow websites will see a decline in Facebook impressions, and in turn a decline in visitors from that source.

Search engines and social media companies aren’t punishing slow websites on a whim, they’ve made a calculated decision to improve the experience for their users. If two websites have effectively the same content, wouldn’t you rather visit one that loads faster?

Many websites depend on search engines and social media for a large portion of their traffic. The slowest of these will have an exacerbated problem, with a reduced number of visitors coming to their site, and over half of those visitors subsequently abandoning.

If the prognosis sounds alarming, that’s because it is! But the good news is that there are a few concrete steps you can take to improve your page speeds. Even the slowest sites can get “sub three seconds” with a good strategy and some work.

Profiling And Benchmarking Tools

Before you begin optimizing, it’s a good idea to take a snapshot of your website’s performance. With profiling, you can determine how much progress you will need to make. Later, you can compare against this benchmark to quantify the speed improvements you make.

There are a number of tools that assess a website’s performance. But before you get started, it’s important to understand that no tool provides a perfect measurement of client-side performance. Devices, connection speeds, and web browsers all impact performance, and it is impossible to analyze all combinations. Additionally, any tool that runs on your personal device can only approximate the experience on a different device or connection.

In one sense, whichever tool you use can provide meaningful insights. As long as you use the same tool before and after, the comparison of each should provide a decent snapshot of performance changes. But certain tools are better than others.

In this section, we’ll walk through two tools that provide a profile of how well your website performs in a mobile environment.

Note: If can be difficult to benchmark an entire site, so I recommend that you choose one or two of your most important pages for benchmarking.

Lighthouse

Lighthouse audit tab


Lighthouse in the Google’s Web Developer Tools. (Large preview)

One of the more useful tools for profiling mobile performance is Google’s Lighthouse. It’s a nice starting point for optimization since it not only analyzes page performance but also provides insights into specific performance issues. Additionally, Lighthouse provides high-level suggestions for speed improvements.

Lighthouse is available in the Audits tab of the Chrome Developer Tools. To get started, open the page you want to optimize in Chrome Dev Tools and perform an audit. I typically perform all the audits, but for our purposes, you only need to check the ‘Performance’ checkbox:

Lighthouse audit selection


All the audits are useful, but we’ll only need the Performance audit. (Large preview)

Lighthouse focuses on mobile, so when you run the audit, Lighthouse will pop your page into the inspector’s responsive viewer and throttle the connection to simulate a mobile experience.

Lighthouse Reports

When the audit finishes, you’ll see an overall performance score, a timeline view of how the page rendered over time, as well as a variety of metrics:

Lighthouse performance audit results


In the performance audit, pay attention to the first meaningful paint. (Large preview)

It’s a lot of information, but one report to emphasize is the first meaningful paint, since that directly influences user bounce rates. You may notice that the tool doesn’t even list the total load time, and that’s because it rarely matters for user experience.

Mobile users expect a first view of the page very quickly, and it may be some time before they scroll to the lower content. In the timeline above, the first paint occurs quickly at 1.3s, then a full above-the-fold content paint occurs at 3.9s. The user can now engage with the above-the-fold content, and anything below-the-fold can take a few seconds longer to load.

Lighthouse’s first meaningful paint is a great metric for benchmarking, but also take a look at the opportunities section. This list helps to identify the key problem areas of your site. Keep these recommendations on your radar, since they may provide your biggest improvements.

Lighthouse Caveats

While Lighthouse provides great insights, it is important to bear in mind that it only simulates a mobile experience. The device is simulated in Chrome, and a mobile connection is simulated with throttling. Actual experiences will vary.

Additionally, you may notice that if you run the audit multiple times, you will get different reports. That’s again because it is simulating the experience, and variances in your device, connection, and the server will impact the results. That said, you can still use Lighthouse for benchmarking, but it is important that you run it several times. It is more relevant as a range of values than a single report.

WebPageTest

In order to get an idea of how quickly your page loads in an actual mobile device, use WebPageTest. One of the nice things about WebPageTest is that it tests on a variety of real devices. Additionally, it will perform the test a number of times and take the average to provide a more accurate benchmark.

To get started, navigate to WebPageTest.org, enter the URL for the page you want to test and then select the mobile device you’d like to use for testing. Also, open up the advanced settings and change the connection speed. I like testing at Fast 3G, because even when users are connected to LTE the connection speed is rarely LTE (#sad):

WebPageTest advanced settings form


WebPageTest provides actual mobile devices for profiling. (Large preview)

After submitting the test (and waiting for any queue), you’ll get a report on the speed of the page:

WebPageTest profiling results


In WebPageTest’s results, pay attention to the start render and first byte. (Large preview)

The summary view consists of a short list of metrics and links to timelines. Again, the value of the start render is more important than the load time. The first byte is useful for analyzing the server response speed. You can also dig into the more in-depth reports for additional insights.

Benchmarking

Now that you’ve profiled your page in Lighthouse and WebPageTest, it’s time to record the values. These benchmarks will provide a useful comparison as you optimize your page. If the metrics improve, your changes are worthwhile. If they stay static (or worse decline), you’ll need to rethink your strategy.

Lighthouse results are simulated which makes it less useful for benchmarking and more useful for in-depth reports and optimization suggestions. However, Lighthouse’s performance score and first meaningful paint are nice benchmarks so run it a few times and take the median for each.

WebPageTest’s values are more reliable for benchmarking since it tests on real devices, so these will be your primary benchmarks. Record the value for the first byte, start to render, and overall load time.

Bloat Reduction

Now that you’ve assessed the performance of your site, let’s take a look at a tool that can help reduce the size of your files. This tool can identify extra, unnecessary pieces of code that bloat your files and cause resources to be larger than they should.

In a perfect world, users would only download the code that they actually need. But the production and maintenance process can lead to unused artifacts in the codebase. Even the most diligent developers can forget to remove pieces of old CSS and JavaScript while making changes. Over time these bits of dead code accumulate and become unnecessary bloat.

Additionally, certain resources are intended to be cached and then used throughout multiple pages, such as a site-wide stylesheet. Site-wide resources often make sense, but how can you tell when a stylesheet is mostly underused?

The Coverage Tab

Fortunately, Chrome Developer Tools has a tool that helps assess the bloat in files: The Coverage tab. The Coverage tab analyzes code coverage as you navigate your site. It provides an interface that shows how much code in a given CSS or JS file is actually being used.

To access the Coverage tab, open up Chrome Developer Tools, and click on the three dots in the top right. Navigate to More Tools > Coverage.

Access the Coverage tab by clicking on More tools and then Coverage


The Coverage tab is a bit hidden in the web developer tools console. (Large preview)

Next, start instrumenting coverage by clicking the reload button on the right. That will reload the page and begin the code coverage analysis. It brings up a report similar to this:

The Coverage report identifies unused code


An example of a Coverage report. (Large preview)

Here, pay attention to the unused bytes:

The coverage report UI shows a breakdown of used and unused bytes


The unused bytes are represented by red lines. (Large preview)

This UI shows the amount of code that is currently unused, colored red. In this particular page, the first file shown is 73% bloat. You may see significant bloat at first, but it only represents the current render. Change your screen size and you should see the CSS coverage go up as media queries get applied. Open any interactive elements like modals and toggles, and it should go up further.

Once you’ve activated every view, you will have an idea of how much code you are actually using. Next, you can dig into the report further to find out just which pieces of code are unused, simply click on one of the resources and look in the main window:

Detail view of a file in the Coverage report, showing which pieces of code aren’t being used


Click on a file in the Coverage report to see the specific portions of unused code. (Large preview)

In this CSS file, look at the highlights to the left of each ruleset; green indicates used code and red indicates bloat. If you are building a single page app or using specialized resources for this particular page, you may be inclined to go in and remove this garbage. But don’t be too hasty. You should definitely remove dead code, but be careful to make sure that you haven’t missed a breakpoint or interactive element.

Next Steps

In this article, we’ve shown the quantitative benefits of optimizing page speed. I hope you’re convinced, and that you have the tools you need to convince others. We’ve also set a minimum goal for mobile page speed: sub three seconds.

To hit this goal, it’s important that you prioritize the highest impact optimizations first. There are a lot of resources online that can help define this roadmap, such as this checklist. Lighthouse can also be a great tool for identifying specific issues in your codebase, so I encourage you to tackle those bottlenecks first. Sometimes the smallest optimizations can have the biggest impact.

Smashing Editorial
(da, lf, ra, yk, il)


Visit link: 

Measuring Websites With Mobile-First Optimization Tools

Infographic: The Surprising Words That Get Content Shared on Social Media

surprising words social media

It always comes back to copywriting. You can blindly post to social media, or you can take a step back and carefully wordsmith the language of your updates. The right concoction of words can multiply the effect of your social media efforts. Today’s infographic will help you get started in the right direction. Just be sure to track what you’re doing and measure the results. And even though most social media networks track your performance for you (using their native analytics systems), I find it’s always a good exercise to record your performance in a Google spreadsheet. It makes you…

The post Infographic: The Surprising Words That Get Content Shared on Social Media appeared first on The Daily Egg.

From: 

Infographic: The Surprising Words That Get Content Shared on Social Media

SmashingConf 2018: Fetch Those Early-Bird Tickets!

Great conferences are all about learning new skills and making new connections. That’s why we’ve set up a couple of new adventures for SmashingConf 2018 — just practical sessions, new formats, new lightning talks, evening sessions and genuine, interesting conversations — with a dash of friendly networking! Taking place in London, San Francisco, Toronto. Tickets? Glad you asked!
SmashingConf London / #perfmatters / Feb 7–8 Performance matters. Next year, we’re thrilled to venture to London for our brand new conference fully dedicated to everything front-end performance.

Read more: 

SmashingConf 2018: Fetch Those Early-Bird Tickets!

Quick Wins For Improving Performance And Security Of Your Website

When it comes to building and maintaining a website, one has to take a ton of things into consideration. However, in an era when people want to see results fast, while at the same time knowing that their information online is secure, all webmasters should strive for a) improving the performance of their website, and b) increasing their website’s security.

Quick Wins For Improving Performance And Security Of Your Website

Both of these goals are vital in order to run a successful website. So, we’ve put together a list of five technologies you should consider implementing to improve both the performance and security of your website.

The post Quick Wins For Improving Performance And Security Of Your Website appeared first on Smashing Magazine.

Read the article – 

Quick Wins For Improving Performance And Security Of Your Website

Front End Performance Checklist 2017 [PDF, Apple Pages]

Are you using progressive booting already? What about tree-shaking and code-splitting in React and Angular? Have you set up Brotli or Zopfli compression, OCSP stapling and HPACK compression? Also, how about resource hints, client hints and CSS containment — not to mention IPv6, HTTP/2 and service workers?
Back in the day, performance was often a mere afterthought. Often deferred till the very end of the project, it would boil down to minification, concatenation, asset optimization and potentially a few fine adjustments on the server’s config file.

Read more:

Front End Performance Checklist 2017 [PDF, Apple Pages]

Responsive Images In WordPress With Art Direction

Support for responsive images was added to WordPress core in version 4.4 to address the use case for viewport-based image selection, where the browser requests the image size that best fits the layout for its particular viewport.

Responsive Images In WordPress With Art Direction

Images that are inserted within the text of a post automatically get the responsive treatment, while images that are handled by the theme or plugins — like featured images and image galleries — can be coded by developers using the new responsive image functions and filters. With a few additions, WordPress websites can accommodate another responsive image use case known as art direction. Art direction gives us the ability to design with images whose crop or composition changes at certain breakpoints.

The post Responsive Images In WordPress With Art Direction appeared first on Smashing Magazine.

Follow this link:

Responsive Images In WordPress With Art Direction