Tag Archives: service

Thumbnail

Building A Pub/Sub Service In-House Using Node.js And Redis




Building A Pub/Sub Service In-House Using Node.js And Redis

Dhimil Gosalia



Today’s world operates in real time. Whether it’s trading stock or ordering food, consumers today expect immediate results. Likewise, we all expect to know things immediately — whether it’s in news or sports. Zero, in other words, is the new hero.

This applies to software developers as well — arguably some of the most impatient people! Before diving into BrowserStack’s story, it would be remiss of me not to provide some background about Pub/Sub. For those of you who are familiar with the basics, feel free to skip the next two paragraphs.

Many applications today rely on real-time data transfer. Let’s look closer at an example: social networks. The likes of Facebook and Twitter generate relevant feeds, and you (via their app) consume it and spy on your friends. They accomplish this with a messaging feature, wherein if a user generates data, it will be posted for others to consume in nothing short of a blink. Any significant delays and users will complain, usage will drop, and if it persists, churn out. The stakes are high, and so are user expectations. So how do services like WhatsApp, Facebook, TD Ameritrade, Wall Street Journal and GrubHub support high volumes of real-time data transfers?

All of them use a similar software architecture at a high level called a “Publish-Subscribe” model, commonly referred to as Pub/Sub.

“In software architecture, publish–subscribe is a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers, but instead categorize published messages into classes without knowledge of which subscribers, if any, there may be. Similarly, subscribers express interest in one or more classes and only receive messages that are of interest, without knowledge of which publishers, if any, there are.“

Wikipedia

Bored by the definition? Back to our story.

At BrowserStack, all of our products support (in one way or another) software with a substantial real-time dependency component — whether its automate tests logs, freshly baked browser screenshots, or 15fps mobile streaming.

In such cases, if a single message drops, a customer can lose information vital for preventing a bug. Therefore, we needed to scale for varied data size requirements. For example, with device logger services at a given point of time, there may be 50MB of data generated under a single message. Sizes like this could crash the browser. Not to mention that BrowserStack’s system would need to scale for additional products in the future.

As the size of data for each message differs from a few bytes to up to 100MB, we needed a scalable solution that could support a multitude of scenarios. In other words, we sought a sword that could cut all cakes. In this article, I will discuss the why, how, and results of building our Pub/Sub service in-house.

Through the lens of BrowserStack’s real-world problem, you will get a deeper understanding of the requirements and process of building your very own Pub/Sub.

Our Need For A Pub/Sub Service

BrowserStack has around 100M+ messages, each of which is somewhere between approximately 2 bytes and 100+ MB. These are passed around the world at any moment, all at different Internet speeds.

The largest generators of these messages, by message size, are our BrowserStack Automate products. Both have real-time dashboards displaying all requests and responses for each command of a user test. So, if someone runs a test with 100 requests where the average request-response size is 10 bytes, this transmits 1×100×10 = 1000 bytes.

Now let’s consider the larger picture as — of course — we don’t run just one test a day. More than approximately 850,000 BrowserStack Automate and App Automate tests are run with BrowserStack each and every day. And yes, we average around 235 request-response per test. Since users can take screenshots or ask for page sources in Selenium, our average request-response size is approximately 220 bytes.

So, going back to our calculator:

850,000×235×220 = 43,945,000,000 bytes (approx.) or only 43.945GB per day

Now let’s talk about BrowserStack Live and App Live. Surely we have Automate as our winner in form of size of data. However, Live products take the lead when it comes to the number of messages passed. For every live test, about 20 messages are passed each minute it turns. We run around 100,000 live tests, which each test averaging around 12 mins meaning:

100,000×12×20 = 24,000,000 messages per day

Now for the awesome and remarkable bit: We build, run, and maintain the application for this called pusher with 6 t1.micro instances of ec2. The cost of running the service? About $70 per month.

Choosing To Build vs. Buying

First things first: As a startup, like most others, we were always excited to build things in-house. But we still evaluated a few services out there. The primary requirements we had were:

  1. Reliability and stability,
  2. High performance, and
  3. Cost-effectiveness.

Let’s leave the cost-effectiveness criteria out, as I can’t think of any external services that cost under $70 a month (tweet me if know you one that does!). So our answer there is obvious.

In terms of reliability and stability, we found companies that provided Pub/Sub as a service with 99.9+ percent uptime SLA, but there were many T&C’s attached. The problem is not as simple as you think, especially when you consider the vast lands of the open Internet that lie between the system and client. Anyone familiar with Internet infrastructure knows stable connectivity is the biggest challenge. Additionally, the amount of data sent depends on traffic. For example, a data pipe that’s at zero for one minute may burst during the next. Services providing adequate reliability during such burst moments are rare (Google and Amazon).

Performance for our project means obtaining and sending data to all listening nodes at near zero latency. At BrowserStack, we utilize cloud services (AWS) along with co-location hosting. However, our publishers and/or subscribers could be placed anywhere. For example, it may involve an AWS application server generating much-needed log data, or terminals (machines where users can securely connect for testing). Coming back to the open Internet issue again, if we were to reduce our risk we would have to ensure our Pub/Sub leveraged the best host services and AWS.

Another essential requirement was the ability to transmit all types of data (Bytes, text, weird media data, etc.). With all considered, it did not make sense to rely on a third-party solution to support our products. In turn, we decided to revive our startup spirit, rolling up our sleeves to code our own solution.

Building Our Solution

Pub/Sub by design means there will be a publisher, generating and sending data, and a Subscriber accepting and processing it. This is similar to a radio: A radio channel broadcasts (publishes) content everywhere within a range. As a subscriber, you can decide whether to tune into that channel and listen (or turn off your radio altogether). 

Unlike the radio analogy where data is free for all and anyone can decide to tune in, in our digital scenario we need authentication which means data generated by the publisher could only be for a single particular client or subscriber.


Basic working of Pub/Sub


Basic working of Pub/Sub (Large preview)

Above is a diagram providing an example of a good Pub/Sub with:

  • Publishers
    Here we have two publishers generating messages based on pre-defined logic. In our radio analogy, these are our radio jockeys creating the content.
  • Topics
    There are two here, meaning there are two types of data. We can say these are our radio channels 1 and 2.
  • Subscribers
    We have three that each read data on a particular topic. One thing to notice is that Subscriber 2 is reading from multiple topics. In our radio analogy, these are the people who are tuned into a radio channel. 

Let’s start understanding the necessary requirements for the service.

  1. An evented component
    This kicks in only when there is something to kick in.
  2. Transient storage
    This keeps data persisted for a short duration so if the subscriber is slow, it still has a window to consume it.
  3. Reducing the latency
    Connecting two entities over a network with minimum hops and distance.

We picked a technology stack that fulfilled the above requirements:

  1. Node.js
    Because why not? Evented, we wouldn’t need heavy data processing, plus it’s easy to onboard.
  2. Redis
    Supports perfectly short-lived data. It has all the capabilities to initiate, update and auto-expire. It also puts less load on the application.

Node.js For Business Logic Connectivity

Node.js is a nearly perfect language when it comes to writing code incorporating IO and events. Our particular given problem had both, making this option the most practical for our needs.

Surely other languages such as Java could be more optimized, or a language like Python offers scalability. However, the cost of starting with these languages is so high that a developer could finish writing code in Node in the same duration. 

To be honest, if the service had a chance of adding more complicated features, we could have looked at other languages or a completed stack. But here it is a marriage made in heaven. Here is our package.json:


  "name": "Pusher",
  "version": "1.0.0",
  "dependencies": 
    "bstack-analytics": "*****", // Hidden for BrowserStack reasons. :)
    "ioredis": "^2.5.0",
    "socket.io": "^1.4.4"
  ,
  "devDependencies": {},
  "scripts": 
    "start": "node server.js"
  
}

Very simply put, we believe in minimalism especially when it comes to writing code. On the other hand, we could have used libraries like Express to write extensible code for this project. However, our startup instincts decided to pass on this and to save it for the next project. Additional tools we used:

  • ioredis
    This is one of the most supported libraries for Redis connectivity with Node.js used by companies including Alibaba.
  • socket.io
    The best library for graceful connectivity and fallback with WebSocket and HTTP.

Redis For Transient Storage

Redis as a service scales is heavily reliable and configurable. Plus there are many reliable managed service providers for Redis, including AWS. Even if you don’t want to use a provider, Redis is easy to get started with.

Let’s break down the configurable part. We started off with the usual master-slave configuration, but Redis also comes with cluster or sentinel modes. Every mode has its own advantages.

If we could share the data in some way, a Redis cluster would be the best choice. But if we shared the data by any heuristics, we have less flexibility as the heuristic has to be followed across. Fewer rules, more control is good for life!

Redis Sentinel works best for us as data lookup is done in just one node, connecting at a given point in time while data is not sharded. This also means that even if multiple nodes are lost, the data is still distributed and present in other nodes. So you have more HA and less chances of loss. Of course, this removed the pros from having a cluster, but our use case is different.

Architecture At 30000 Feet

The diagram below provides a very high-level picture of how our Automate and App Automate dashboards work. Remember the real-time system that we had from the earlier section?


BrowserStack’s real-time Automate and App Automate dashboards.


BrowserStack’s real-time Automate and App Automate dashboards (Large preview)

In our diagram, our main workflow is highlighted with thicker borders. The “automate” section consists of:

  1. Terminals
    Comprised of the pristine versions of Windows, OSX, Android or iOS that you get while testing on BrowserStack.
  2. Hub
    The point of contact for all your Selenium and Appium tests with BrowserStack.

The “user service” section here is our gatekeeper, ensuring data is sent to and saved for the right individual. It is also our security keeper. The “pusher” section incorporates the heart of what we discussed in this article. It consists of the usual suspects including:

  1. Redis
    Our transient storage for messages, where in our case automate logs are temporarily stored.
  2. Publisher
    This is basically the entity that obtains data from the hub. All your request responses are captured by this component which writes to Redis with session_id as the channel.
  3. Subscriber
    This reads data from Redis generated for the session_id. It is also the web server for clients to connect via WebSocket (or HTTP) to get data and then sends it to authenticated clients.

Finally, we have the user’s browser section, representing an authenticated WebSocket connection to ensure session_id logs are sent. This enables the front-end JS to parse and beautify it for users.

Similar to the logs service, we have pusher here that is being used for other product integrations. Instead of session_id, we use another form of ID to represent that channel. This all works out of pusher!

Conclusion (TLDR)

We’ve had considerable success in building out Pub/Sub. To sum up why we built it in-house:

  1. Scales better for our needs;
  2. Cheaper than outsourced services;
  3. Full control over the overall architecture.

Not to mention that JS is the perfect fit for this kind of scenario. Event loop and massive amount of IO is what the problem needs! JavaScript is magic of single pseudo thread. 

Events and Redis as a system keep things simple for developers, as you can obtain data from one source and push it to another via Redis. So we built it.

If the usage fits into your system, I recommend doing the same!

Smashing Editorial
(rb, ra, il)


See original article: 

Building A Pub/Sub Service In-House Using Node.js And Redis

Thumbnail

Meet Smashing Book 6: New Frontiers In Web Design




Meet Smashing Book 6: New Frontiers In Web Design

Vitaly Friedman



Let’s make sense of the front-end and UX madness. Meet Smashing Book 6 with everything from design systems to accessible single-page apps, CSS Custom Properties, Grid, Service Workers, performance patterns, AR/VR, conversational UIs & responsive art direction. And you can add your name into the book, too. About the book ↓.

Smashing Book 6 is dedicated to the challenges and headaches that we are facing today, and how to resolve them. No chit-chat, no theory: only practical, useful advice applicable to your work right away.

Smashing Book 6

Book

$29 $39Print + eBook

Printed, quality hardcover. Free airmail shipping worldwide. Sep 2018.

eBook

$14.90 $19Get the eBook

PDF, ePUB, Kindle. First chapters are already available.

About The Book

With so much happening in front-end and UX these days, it’s getting really hard to stay afloat, isn’t it? New technologies and techniques appear almost daily, and so navigating through it all in regular projects is daunting and time-consuming. Frankly, we just don’t have time to afford betting on a wrong strategy. That’s why we created Smashing Book 6, our shiny new book that explores uncharted territories and seeks to establish a map for the brave new front-end world.

You know the drill: the book isn’t about tools; it’s about workflow, strategy and shortcuts to getting things done well. Respected members of the community explore how to build accessible single-page apps with React or Angular, how to use CSS Grid Layout, CSS Custom Properties and service workers as well as how to load assets on the web in times of HTTP/2 and bloated third-party scripts.

We’ll also examine how to make design systems work in real-life, how to design and build conversational interfaces, AR/VR, building for chatbots and watches and how to bring responsive art-direction back to the web.

Print shipping in late September 2018., hardcover + eBook. 432 pages. Written by Laura Elizabeth, Marcy Sutton, Rachel Andrew, Mike Riethmueller, Harry Roberts, Lyza D. Gardner, Yoav Weiss, Adrian Zumbrunnen, Greg Nudelman, Ada Rose Cannon and Vitaly Friedman. Pre-order the book today.

A look inside the book

Book

$29 $39Print + eBook

Printed, quality hardcover. Free airmail shipping worldwide. Shipping: Sept 2018.

eBook

$14.90 $19Get the eBook

PDF, ePUB, Kindle. First chapters are already available.

Why This Book Is For You

We worked hard to design the book in a way that it doesn’t become outdated quickly. That’s why it’s more focused on strategy rather than tooling. It’s about how we do things, but not necessarily the tools we all use to get there.

Table of Contents

The book contains 11 chapters, with topics ranging from design to front-end. Only practical advice, applicable to your work right away.

  • Making design systems work in real-life
    by Laura Elizabeth
  • Accessibility in times of SPAs
    by Marcy Sutton
  • Production-ready CSS Grid layouts
    by Rachel Andrew
  • Strategic guide to CSS Custom Properties
    by Mike Riethmueller
  • Taming performance bottlenecks
    by Harry Roberts
  • Building an advanced service worker
    by Lyza Gardner
  • Loading assets on the web
    by Yoav Weiss
  • Conversation interface design patterns
    by Adrian Zumbrunnen
  • Building chatbots and designing for watches
    by Greg Nudelman
  • Cross Reality and the web (AR/VR)
    by Ada Rose Cannon
  • Bringing personality back to the web
    by Vitaly Friedman
laura elizabeth
marcy sutton
rachel andrew
mike riethmuller
harry roberts
lyza d gardner
yoav weiss
adrian zumbrunnen
greg nudelman
ada rose edwards
vitaly friedman

From left to right: Laura Elizabeth, Marcy Sutton, Rachel Andrew, Mike Riethmuller, Harry Roberts, Lyza Gardner, Yoav Weiss, Adrian Zumbrunnen, Greg Nudelman, Ada Rose Edwards, and yours truly.

Book

$29 $39Print + eBook

Free airmail shipping worldwide. Sep 2018. Pre-order now and save 25%

eBook

$14.90 $19Get the eBook

PDF, ePUB, Kindle. You can start reading right away.

Download the Sample Chapter

To sneak a peek inside the book, you can download Vitaly’s chapter on bringing personality back to the web (PDF, ca. 18MB). Enjoy!

Goodie: New Frontiers In Web Design Wallpaper

To celebrate the pre-release of the book , Chiara Aliotta designed a set of mobile and desktop wallpapers for you to indulge in. Feel free to download it (ZIP, ca. 1.4MB).

New Frontiers In Web Design Wallpaper
Get ready for the new frontiers in web design with Chiara’s beautiful wallpaper.

About The Designer

Chiara AliottaThe cover was designed by the fantastic Chiara Aliotta. Chiara is an Italian award-winning designer with many years of experience as an art director and brand consultant. She founded the design studio Until Sunday and has directed the overall artistic look and feel of different tech companies and not-for-profit organizations around the world. We’re very happy that she gave Smashing Book 6 that special, magical touch.

Add Your Name To The Book

We kindly appreciate your support, and so as always, we invite you to add your name to the printed book: a double-page spread is reserved for the first 1.000 people. Space is limited, so please don’t wait too long!

Add your name to the book
A double-page spread is reserved for the first 1.000 readers. Add your name to the printed book.

Book

$29 $39Print + eBook

Free airmail shipping worldwide. Sep 2018. Pre-order now and save 25%

eBook

$14.90 $19Get the eBook

PDF, ePUB, Kindle. You can start reading right away.

Smashing Editorial
(cm)


Originally posted here:

Meet Smashing Book 6: New Frontiers In Web Design

Thumbnail

Conversion Wireframing: A Revenue Oriented Design Process

Even though there is growing wisdom around the best product teams involving design, business and engineering heads from the very start, there is, quite often, still a divide between the design process and business needs. As someone who has both a Business and Design background, I have found it useful in client work to draw from both areas from the very start. Here are a few ways you can design web experiences that are not only user-centric but also conversion-centric. But first: What is wire framing? For the non-design heads, wireframes are essentially a schematic or blueprint that define the information…

The post Conversion Wireframing: A Revenue Oriented Design Process appeared first on The Daily Egg.

Read the article:

Conversion Wireframing: A Revenue Oriented Design Process

Thumbnail

Building A Serverless Contact Form For Your Static Site




Building A Serverless Contact Form For Your Static Site

Brian Holt



Static site generators provide a fast and simple alternative to Content Management Systems (CMS) like WordPress. There’s no server or database setup, just a build process and simple HTML, CSS, and JavaScript. Unfortunately, without a server, it’s easy to hit their limits quickly. For instance, in adding a contact form.

With the rise of serverless architecture adding a contact form to your static site doesn’t need to be the reason to switch to a CMS anymore. It’s possible to get the best of both worlds: a static site with a serverless back-end for the contact form (that you don’t need to maintain). Maybe best of all, in low-traffic sites, like portfolios, the high limits of many serverless providers make these services completely free!

In this article, you’ll learn the basics of Amazon Web Services (AWS) Lambda and Simple Email Service (SES) APIs to build your own static site mailer on the Serverless Framework. The full service will take form data submitted from an AJAX request, hit the Lambda endpoint, parse the data to build the SES parameters, send the email address, and return a response for our users. I’ll guide you through getting Serverless set up for the first time through deployment. It should take under an hour to complete, so let’s get started!

The static site form, sending the message to the Lambda endpoint and returning a response to the user.
The static site form, sending the message to the Lambda endpoint and returning a response to the user.

Setting Up

There are minimal prerequisites in getting started with Serverless technology. For our purposes, it’s simply a Node Environment with Yarn, the Serverless Framework, and an AWS account.

Setting Up The Project


The Serverless Framework web site. Useful for installation and documentation.


The Serverless Framework web site. Useful for installation and documentation.

We use Yarn to install the Serverless Framework to a local directory.

  1. Create a new directory to host the project.
  2. Navigate to the directory in your command line interface.
  3. Run yarn init to create a package.json file for this project.
  4. Run yarn add serverless to install the framework locally.
  5. Run yarn serverless create --template aws-nodejs --name static-site-mailer to create a Node service template and name it static-site-mailer.

Our project is setup but we won’t be able to do anything until we set up our AWS services.

Setting Up An Amazon Web Services Account, Credentials, And Simple Email Service


The Amazon Web Services sign up page, which includes a generous free tier, enabling our project to be entirely free.


The Amazon Web Services sign up page, which includes a generous free tier, enabling our project to be entirely free.

The Serverless Framework has recorded a video walk-through for setting up AWS credentials, but I’ve listed the steps here as well.

  1. Sign Up for an AWS account or log in if you already have one.
  2. In the AWS search bar, search for “IAM”.
  3. On the IAM page, click on “Users” on the sidebar, then the “Add user” button.
  4. On the Add user page, give the user a name – something like “serverless” is appropriate. Check “Programmatic access” under Access type then click next.
  5. On the permissions screen, click on the “Attach existing policies directly” tab, search for “AdministratorAccess” in the list, check it, and click next.
  6. On the review screen you should see your user name, with “Programmatic access”, and “AdministratorAccess”, then create the user.
  7. The confirmation screen shows the user “Access key ID” and “Secret access key”, you’ll need these to provide the Serverless Framework with access. In your CLI, type yarn sls config credentials --provider aws --key YOUR_ACCESS_KEY_ID --secret YOUR_SECRET_ACCESS_KEY, replacing YOUR_ACCESS_KEY_ID and YOUR_SECRET_ACCESS_KEY with the keys on the confirmation screen.

Your credentials are configured now, but while we’re in the AWS console let’s set up Simple Email Service.

  1. Click Console Home in the top left corner to go home.
  2. On the home page, in the AWS search bar, search for “Simple Email Service”.
  3. On the SES Home page, click on “Email Addresses” in the sidebar.
  4. On the Email Addresses listing page, click the “Verify a New Email Address” button.
  5. In the dialog window, type your email address then click “Verify This Email Address”.
  6. You’ll receive an email in moments containing a link to verify the address. Click on the link to complete the process.

Now that our accounts are made, let’s take a peek at the Serverless template files.

Setting Up The Serverless Framework

Running serverless create creates two files: handler.js which contains the Lambda function, and serverless.yml which is the configuration file for the entire Serverless Architecture. Within the configuration file, you can specify as many handlers as you’d like, and each one will map to a new function that can interact with other functions. In this project, we’ll only create a single handler, but in a full Serverless Architecture, you’d have several of the various functions of the service.


The default file structure generated from the Serverless Framework containing handler.js and serverless.yml.


The default file structure generated from the Serverless Framework containing handler.js and serverless.yml.

In handler.js, you’ll see a single exported function named hello. This is currently the main (and only) function. It, along with all Node handlers, take three parameters:

  • event
    This can be thought of as the input data for the function.
  • context object
    This contains the runtime information of the Lambda function.
  • callback
    An optional parameter to return information to the caller.
// handler.js

'use strict';

module.exports.hello = (event, context, callback) => 
  const response = 
    statusCode: 200,
    body: JSON.stringify(
      message: 'Go Serverless v1.0! Your function executed successfully!',
      input: event,
    ),
  };

  callback(null, response);
};

At the bottom of hello, there’s a callback. It’s an optional argument to return a response, but if it’s not explicitly called, it will implicitly return with null. The callback takes two parameters:

  • Error error
    For providing error information for when the Lambda itself fails. When the Lambda succeeds, null should be passed into this parameter.
  • Object result
    For providing a response object. It must be JSON.stringify compatible. If there’s a parameter in the error field, this field is ignored.

Our static site will send our form data in the event body and the callback will return a response for our user to see.

In serverless.yml you’ll see the name of the service, provider information, and the functions.

# serverless.yml

service: static-site-mailer

provider:
  name: aws
  runtime: nodejs6.10

functions:
  hello:
    handler: handler.hello

How the function names in serverless.yml map to handler.js.


How the function names in serverless.yml map to handler.js.

Notice the mapping between the hello function and the handler? We can name our file and function anything and as long as it maps to the configuration it will work. Let’s rename our function to staticSiteMailer.

# serverless.yml

functions:
  staticSiteMailer:
    handler: handler.staticSiteMailer
// handler.js

module.exports.staticSiteMailer = (event, context, callback) => 
  ...
;

Lambda functions need permission to interact with other AWS infrastructure. Before we can send an email, we need to allow SES to do so. In serverless.yml, under provider.iamRoleStatements add the permission.

# serverless.yml

provider:
  name: aws
  runtime: nodejs6.10
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "ses:SendEmail"
      Resource: ["*"]

Since we need a URL for our form action, we need to add HTTP events to our function. In serverless.yml we create a path, specify the method as post, and set CORS to true for security.

functions:
  staticSiteMailer:
    handler: handler.staticSiteMailer
    events:
      - http:
          method: post
          path: static-site-mailer
          cors: true

Our updated serverless.yml and handler.js files should look like:

# serverless.yml

service: static-site-mailer

provider:
  name: aws
  runtime: nodejs6.10

functions:
  staticSiteMailer:
    handler: handler.staticSiteMailer
    events:
      - http:
          method: post
          path: static-site-mailer
          cors: true

provider:
  name: aws
  runtime: nodejs6.10
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "ses:SendEmail"
      Resource: ["*"]
// handler.js

'use strict';

module.exports.staticSiteMailer = (event, context, callback) => 
  const response = 
    statusCode: 200,
    body: JSON.stringify(
      message: 'Go Serverless v1.0! Your function executed successfully!',
      input: event,
    ),
  };

  callback(null, response);
};

Our Serverless Architecture is setup, so let’s deploy it and test it. You’ll get a simple JSON response.

yarn sls deploy --verbose
yarn sls invoke --function staticSiteMailer


    "statusCode": 200,
    "body": ""message":"Go Serverless v1.0! Your function executed successfully!","input":}"
}

The return response from invoking our brand new serverless function.


The return response from invoking our brand new serverless function.

Creating The HTML Form

Our Lambda function input and form output need to match, so before we build the function we’ll build the form and capture its output. We keep it simple with name, email, and message fields. We’ll add the form action once we’ve deployed our serverless architecture and got our URL, but we know it will be a POST request so we can add that in. At the end of the form, we add a paragraph tag for displaying response messages to the user which we’ll update on the submission callback.

<form action=" SERVICE URL }" method="POST">
  <label>
    Name
    <input type="text" name="name" required>
  </label>
  <label>
    Email
    <input type="email" name="reply_to" required>
  </label>
  <label>
    Message:
    <textarea name="message" required></textarea>
  </label>
  <button type="submit">Send Message</button>
</form>
<p id="js-form-response"></p>

To capture the output we add a submit handler to the form, turn our form parameters into an object, and send stringified JSON to our Lambda function. In the Lambda function we use JSON.parse() to read our data. Alternatively, you could use jQuery’s Serialize or query-string to send and parse the form parameters as a query string but JSON.stringify() and JSON.parse() are native.

(() => 
  const form = document.querySelector('form');
  const formResponse = document.querySelector('js-form-response');

  form.onsubmit = e => 
    e.preventDefault();

    // Prepare data to send
    const data = ;
    const formElements = Array.from(form);
    formElements.map(input => (data[input.name] = input.value));

    // Log what our lambda function will receive
    console.log(JSON.stringify(data));
  };
})();

Go ahead and submit your form then capture the console output. We’ll use it in our Lambda function next.


Capturing the form data in a console log.


Capturing the form data in a console log.

Invoking Lambda Functions

Especially during development, we need to test our function does what we expect. The Serverless Framework provides the invoke and invoke local command to trigger your function from live and development environments respectively. Both commands require the function name passed through, in our case staticSiteMailer.

yarn sls invoke local --function staticSiteMailer

To pass mock data into our function, create a new file named data.json with the captured console output under a body key within a JSON object. It should look something like:

// data.json


  "body": ""name": "Sender Name","reply_to": "sender@email.com","message": "Sender message""
}

To invoke the function with the local data, pass the --path argument along with the path to the file.

yarn sls invoke local --function staticSiteMailer --path data.json

An updated return response from our serverless function when we pass it JSON data.


An updated return response from our serverless function when we pass it JSON data.

You’ll see a similar response to before, but the input key will contain the event we mocked. Let’s use our mock data to send an email using Simple Email Service!

Sending An Email With Simple Email Service

We’re going to replace the staticSiteMailer function with a call to a private sendEmail function. For now you can comment out or remove the template code and replace it with:

// hander.js

function sendEmail(formData, callback) 
  // Build the SES parameters
  // Send the email


module.exports.staticSiteMailer = (event, context, callback) => 
  const formData = JSON.parse(event.body);

  sendEmail(formData, function(err, data) 
    if (err) 
      console.log(err, err.stack);
     else 
      console.log(data);
    
  });
};

First, we parse the event.body to capture the form data, then we pass it to a private sendEmail function. sendEmail is responsible for sending the email, and the callback function will return a failure or success response with err or data. In our case, we can simply log the error or data since we’ll be replacing this with the Lambda callback in a moment.

Amazon provides a convenient SDK, aws-sdk, for connecting their services with Lambda functions. Many of their services, including SES, are part of it. We add it to the project with yarn add aws-sdk and import it into the top the handler file.

// handler.js

const AWS = require('aws-sdk');
const SES = new AWS.SES();

In our private sendEmail function, we build the SES.sendEmail parameters from the parsed form data and use the callback to return a response to the caller. The parameters require the following as an object:

  • Source
    The email address SES is sending from.
  • ReplyToAddresses
    An array of email addresses added to the reply to the field in the email.
  • Destination
    An object that must contain at least one ToAddresses, CcAddresses, or BccAddresses. Each field takes an array of email addresses that correspond to the to, cc, and bcc fields respectively.
  • Message
    An object which contains the Body and Subject.

Since formData is an object we can call our form fields directly like formData.message, build our parameters, and send it. We pass your SES-verified email to Source and Destination.ToAddresses. As long as the email is verified you can pass anything here, including different email addresses. We pluck our reply_to, message, and name off our formData object to fill in the ReplyToAddresses and Message.Body.Text.Data fields.

// handler.js
function sendEmail(formData, callback) 
  const emailParams = 
    Source: 'your_email@example.com', // SES SENDING EMAIL
    ReplyToAddresses: [formData.reply_to],
    Destination: 
      ToAddresses: ['your_email@example.com'], // SES RECEIVING EMAIL
    ,
    Message: 
      Body: 
        Text: 
          Charset: 'UTF-8',
          Data: `$formData.messagennName: $formData.namenEmail: $formData.reply_to`,
        },
      },
      Subject: 
        Charset: 'UTF-8',
        Data: 'New message from your_site.com',
      ,
    },
  };

  SES.sendEmail(emailParams, callback);
}

SES.sendEmail will send the email and our callback will return a response. Invoking the local function will send an email to your verified address.

yarn sls invoke local --function testMailer --path data.json

The return response from SES.sendEmail when it succeeds.


The return response from SES.sendEmail when it succeeds.

Returning A Response From The Handler

Our function sends an email using the command line, but that’s not how our users will interact with it. We need to return a response to our AJAX form submission. If it fails, we should return an appropriate statusCode as well as the err.message. When it succeeds, the 200 statusCode is sufficient, but we’ll return the mailer response in the body as well. In staticSiteMailer we build our response data and replace our sendEmail callback function with the Lambda callback.

// handler.js

module.exports.staticSiteMailer = (event, context, callback) => 
  const formData = JSON.parse(event.body);

  sendEmail(formData, function(err, data) 
    const response = 
      statusCode: err ? 500 : 200,
      headers: 
        'Content-Type': 'application/json',
        'Access-Control-Allow-Origin': 'https://your-domain.com',
      ,
      body: JSON.stringify(
        message: err ? err.message : data,
      ),
    };

    callback(null, response);
  });
};

Our Lambda callback now returns both success and failure messages from SES.sendEmail. We build the response with checks if err is present so our response is consistent. The Lambda callback function itself passes null in the error argument field and the response as the second. We want to pass errors onwards, but if the Lambda itself fails, its callback will be implicitly called with the error response.

In the headers, you’ll need to replace Access-Control-Allow-Origin with your own domain. This will prevent any other domains from using your service and potentially racking up an AWS bill in your name! And I don’t cover it in this article, but it’s possible to set-up Lambda to use your own domain. You’ll need to have an SSL/TLS certificate uploaded to Amazon. The Serverless Framework team wrote a fantastic tutorial on how to do so.

Invoking the local function will now send an email and return the appropriate response.

yarn sls invoke local --function testMailer --path data.json

The return response from our serverless function, containing the SES.sendEmail return response in the body.


The return response from our serverless function, containing the SES.sendEmail return response in the body.

Calling The Lambda Function From The Form

Our service is complete! To deploy it run yarn sls deploy -v. Once it’s deployed you’ll get a URL that looks something like https://r4nd0mh45h.execute-api.us-east-1.amazonaws.com/dev/static-site-mailer which you can add to the form action. Next, we create the AJAX request and return the response to the user.

(() => 
  const form = document.querySelector('form');
  const formResponse = document.querySelector('js-form-response');

  form.onsubmit = e => 
    e.preventDefault();

    // Prepare data to send
    const data = ;
    const formElements = Array.from(form);
    formElements.map(input => (data[input.name] = input.value));

    // Log what our lambda function will receive
    console.log(JSON.stringify(data));

    // Construct an HTTP request
    var xhr = new XMLHttpRequest();
    xhr.open(form.method, form.action, true);
    xhr.setRequestHeader('Accept', 'application/json; charset=utf-8');
    xhr.setRequestHeader('Content-Type', 'application/json; charset=UTF-8');

    // Send the collected data as JSON
    xhr.send(JSON.stringify(data));

    // Callback function
    xhr.onloadend = response => 
      if (response.target.status === 200) 
        // The form submission was successful
        form.reset();
        formResponse.innerHTML = 'Thanks for the message. I’ll be in touch shortly.';
       else 
        // The form submission failed
        formResponse.innerHTML = 'Something went wrong';
        console.error(JSON.parse(response.target.response).message);
      
    };
  };
})();

In the AJAX callback, we check the status code with response.target.status. If it’s anything other than 200 we can show an error message to the user, otherwise let them know the message was sent. Since our Lambda returns stringified JSON we can parse the body message with JSON.parse(response.target.response).message. It’s especially useful to log the error.

You should be able to submit your form entirely from your static site!

The static site form, sending the message to the Lambda endpoint and returning a response to the user.
The static site form, sending the message to the Lambda endpoint and returning a response to the user.

Next Steps

Adding a contact form to your static is easy with the Serverless Framework and AWS. There’s room for improvement in our code, like adding form validation with a honeypot, preventing AJAX calls for invalid forms and improving the UX if the response, but this is enough to get started. You can see some of these improvements within the static site mailer repo I’ve created. I hope I’ve inspired you to try out Serverless yourself!

Smashing Editorial
(lf, ra, il)


See original article here:

Building A Serverless Contact Form For Your Static Site

Analyzing Your Company’s Social Media Presence With IBM Watson And Node.js




Analyzing Your Company’s Social Media Presence With IBM Watson And Node.js

If you are unfamiliar with Machine Learning (ML) technology, it has existed in science fiction for many years and is finally reaching its maturity in our society. One of the first ML examples I saw as a kid was in Star Trek’s The Next Generation when Lieutenant Tasha Yar trains with her holographic opponent that learns how to fight and better defeat in future battles.

In today’s society, China has developed a “lane robot” that is a guard rail controlled by a computer system that can direct the flow of traffic into different lanes, increasing safety and improving traveling time. This is done automatically based on time of day and how much traffic is flowing in each direction.

Another example is Pittsburg unveiling AI traffic signals that automatically detect traffic patterns and alter the traffic lights on-the-fly. Each light is controlled independently to help reduce both the commuting time and the idling time of cars. According to the article, pilot tests have demonstrated a reduced travel time of 25% and idling time by over 40%. There are, of course, hundreds of other examples of ML technology that make intelligent decisions based on the content it consumes.

To accomplish today’s goal, I am going to demonstrate (using Node.js) how to perform a search with Twitter’s API to retrieve content that will be inputted into the ML algorithm to be analyzed. This way, you’ll be provided with characteristics about the users who wrote that specific content so that you can get a better understanding of your audience. The example application will be written using Node.js as the server.

It is beyond the scope of this article to demonstrate how to write an ML algorithm. Instead, to aid in the analysis, I will demonstrate how to use IBM’s Watson to help you understand the general personality of your social media audience.

What Is IBM Watson?

In 2011, Watson began as a computer system that attempted to index the (entire) Internet. It was originally programmed to answer questions posed in ordinary English. Watson competed and won on the TV show Jeopardy! claiming a $1,000,000 cash prize.

Watson was now a proven success.

With the fame of winning on Jeopardy!, IBM has continued to push Watson’s capabilities. Watson has evolved into an enterprise-level application that is focused on Artificial Intelligence (AI) which you can train to identify what you care about most allowing you to make smarter decisions automatically.

The suite of Watson’s services is divided into six high-level categories:

  1. Conversation
    The services in this category allow you to build intelligent chatbot’s or a virtual customer service agent.
  2. Knowledge
    This category is focused on teaching Watson how to interpret data to unlock hidden value and monitor trends.
  3. Vision
    This service provides the ability to tag content inside an image that is used to train Watson to be able to automatically recognize the same pattern inside of other images.
  4. Speech
    These services provide the ability to convert speech to text and the inverse, text to speech.
  5. Language
    This category is split between translating one language to another as well as interpreting the text to predict what predefined category the text belongs to.
  6. Empathy
    This category is devoted to understanding the content’s tone, personality, and emotional state. Inside this category is a service called “Personality Insights” that will be used in this article to predict the personality characteristics with the social media content we will provide it.

This article will be focusing on understanding the personality of the content that we will fetch from Twitter. However, as you can see, Watson provides many other AI features that you can explore to automate many other processes simply through training and content aggregation.

Personality Insights

Personality Insights will analyze content and help you understand the habits and preferences at an individual level and at scale. This is called the ‘personality profile.’ The profile is split into two high-level groups: Personality characteristics and Consumption preferences. These groups are further broken down into more finite components.

Note: To help understand the high-level concepts (before we deep dive into the results), the Personality Insights documentation provides this helpful summary describing how the profile is inferred from the content you provide it.


IBM Watson’s Big Five Personality Traits


Big Five Personality Traits. Image courtesy: IBM.com. (Large preview)

Personality Characteristics

The Personality Insights service infers personality characteristics based on three primary models:

  • The ‘Big Five’ personality characteristics represent the most widely used model for generally describing how a person engages with the world. The model includes five primary dimensions:
    • Agreeableness
    • Conscientiousness
    • Extraversion
    • Emotional range
    • Openness
      Note: Each dimension has six facets that further characterize an individual according to the dimension.
  • Needs describe which aspects of a product will resonate with a person. The model includes twelve characteristic needs:
    • Excitement
    • Harmony
    • Curiosity
    • Ideal
    • Closeness
    • Self-expression
    • Liberty
    • Love
    • Practicality
    • Stability
    • Challenge
    • Structure
  • Values describe motivating factors that influence a person’s decision making. The model includes five values:
    • Self-transcendence / Helping others
    • Conservation / Tradition
    • Hedonism / Taking pleasure in life
    • Self-enhancement / Achieving success
    • Open to change / Excitement

For more information, see Personality models.

Consumption preferences

Based on the personality characteristics inferred from the input text, the service can also return an indication of the author’s consumption preferences. ‘Consumption preferences’ indicate the author’s likelihood to pursue different products, services, and activities. The service groups the individual preferences into eight categories:

  • Shopping
  • Music
  • Movies
  • Reading and learning
  • Health and activity
  • Volunteering
  • Environmental concern
  • Entrepreneurship

Each category contains from one to as many as a dozen individual preferences.

Note: For more information, see Consumption preferences. For a more in-depth overview of a particular point of interest, I suggest you refer to the Personality Insights documentation.

To be effective, Watson requires a minimum of a hundred words to provide an insight into the consumer’s personality. The more words provided, the better Watson can analyze and determine the consumer’s preference.

This means, if you wish to target individuals, you will need to collect more data than one or two tweets from a specific person. However, if a user writes a product review, blog post, email, or anything else related to your company, this could be analyzed on both an individual level and at scale.

To begin, let’s start by setting up the Personality Insights service to begin analyzing a real-world example.

Configuring The Personality Insights Service

Watson is an enterprise application but they offer a free, limited service. Once you’ve created an account and are logged in, you will need to add the Personality Insight service. IBM offers a Lite plan that is free. The Lite plan is limited to 1,000 API calls per month and is automatically deleted after 30 days — perfect for our demonstration.


Create the Personality Insights Service


Create the Personality Insights Service. (Large preview)

Once the service has been added, we will need to retrieve the service’s credentials to perform API calls against it. From Watson’s Dashboard, your service should be displayed. After you’ve selected the service, you’ll find a link to view the Service credentials in the left-hand menu. You will need to create a new ‘Credential.’ A unique name is required and optional configuration parameters can be defaulted for this login. For now, we will leave the configuration options empty.

After you have created a credential, select the ‘View’ credentials link. This will display the API’s URL, your username, and password required to securely execute API calls. Save these somewhere safe as we will need them in the next step.

Testing The Personality Insights Service

To perform API calls, I am going to use Node.js. If you already have Node.js installed, you can move on to the next step; otherwise, follow the instructions to setup Node.js from the official download page.

To demonstrate how to use the Personality Insights, I am going to create a new Node.js project on my computer. With a command prompt open, navigate to the directory where your Node.js projects will be stored and create your new project:

mkdir watson-sentiments
cd watson-sentiments
npm init

To assist with making the API calls to Watson, I am going to leverage the NPM Package: Watson Developer Cloud Node.js SDK. This package can be installed via the command prompt:

npm install watson-developer-cloud --save

Before making the first call, the PersonalityInsightsV3 object needs to be instantiated with the credentials from the previous section. Begin by creating a new file called index.js that will contain the Node.js code.

Here is an example of configuring the class so it is ready to make API calls:

var PersonalityInsightsV3 = require(’watson-developer-cloud/personality-insights/v3’);
var personality_insights = new PersonalityInsightsV3(
  "url": "https://gateway.watsonplatform.net/personality-insights/api",
  "username": "**************************",
  "password": "*************",
  "version_date": "2017-12-01"
);

The personality_insights variable is what we will use to interact with the API for the Personality Insights service. Let’s review how to execute a call and return a personality profile:

var fs = require(’fs’);

personality_insights.profile(
"contentItems": [
   
         "content": "Some content that contains more than 100 words...",
         "contenttype": "text/plain",
         "created": 1447639154000,
         "id": "666073008692314113",
         "language": "en"
      
   ],
   "consumption_preferences": true
}, (err, response) => 
if (err) throw err;

fs.writeFile("results.txt", JSON.stringify(response, null, 2), function(err) 
if (err) throw err;

console.log("Results were saved!");
);
  });

The profile function accepts an array of contentItems. The ‘content’ item contains the actual content with a few additional properties identifying additional information to help Watson interpret it.

When this is executed, the results are written to a text file (the results are too large to write in the console). The result is an object that contains the following high-level properties:

  • word_count
  • The count of words interpreted
  • processed_language

The language that the content provided, e.g. (en).

  • Personality
    This is an array of the ‘Big Five’ personality characteristics (Openness, Conscientiousness, Extraversion, Agreeableness, and Emotional range). Each characteristic contains an overall percentile for that characteristic (e.g. 0.8100175318417588). To ascertain more detail, there is an array called children that provides more in-depth insight. For example, a child category under ‘Openness’ is ‘Adventurousness’ that contains its own percentile.
  • Needs
    This is an array of the twelve characteristics that define the aspects a person will resonate with a product (Excitement, Harmony, Curiosity, Ideal, Closeness, Self-expression, Liberty, Love, Practicality, Stability, Challenge, and Structure). Each characteristic contains a percentile of how the content was interpreted.
  • Values
    This is an array of the five characteristics that describe motivating factors that influence a person’s decision making (Self-transcendence / Helping others, Conservation / Tradition, Hedonism / Taking pleasure in life, Self-enhancement / Achieving success, and Open to change / Excitement). Each characteristic contains a percentile of how the content was interpreted.
  • Behavior
    This is an array that contains thirty-one elements. Each element provides a percentile of when the content was created. Seven of the elements define the days of the week (Sunday through Saturday). The remaining twenty-four elements define the hours of the day. This helps you understand when customer’s interact with your product.
  • consumption_preferences
    This is an array that contains eight different categories with as much as a twelve child categories providing a percentile of likelihood to pursue different products, services, and activities (Shopping, Music, Movies, Reading and learning, Health and activity, Volunteering, Environmental concern, and Entrepreneurship).
  • Warnings
    This is an array that provides messages if a problem was encountered interpreting the content provided.

Here is a CodePen of the formatted results:

See the Pen Example Watson Results by Jamie Munro (@endyourif) on CodePen.

Configuring Twitter

To search Twitter for relevant tweets, I am going to use the Twitter NPM package. From a console window where the application is hosted, run the following command to install:

npm install twitter --save

Before we can implement the Twitter package, you need to create a Twitter application.


Retrieving Twitter’s Access Tokens


Retrieving Twitter’s Access Tokens. (Large preview)

Once you’ve created your application, you need to retrieve the authorization keys required to perform API calls. With your application created, navigate to the ‘Keys’ and ‘Access Tokens’ page. Since we are not performing API calls against users of Twitter, OAuth integration is not required. Instead, we need only the four following keys:

  1. Consumer Key
  2. Consumer Secret
  3. Access Token
  4. Access Token Secret

The last two keys need to be generated near the bottom of the ‘Keys’ and ‘Access Tokens’ page. With the keys, here is an example of searching for Tweets about #SmashingMagazine:

var Twitter = require(’twitter’);

var client = new Twitter(
  consumer_key: ’*********************’,
  consumer_secret: ’******************’,
  access_token_key: ’******************’,
  access_token_secret: ’****************’
);

client.get(’search/tweets’,  q: ’#SmashingMagazine’ , function(error, tweets, response) 
if(error) throw error;

console.log(tweets);
);

The result of this code will log a list tweets about Smashing Magazine. For the purposes of this demonstration, the following fields are of interest to us:

  1. id
  2. created_at
  3. text
  4. metadata.iso_language_code

These are the fields we will feed Watson.

Integrating Personality Insights With Twitter

With Twitter setup and Watson setup, it’s time to integrate the two together and see the results. To make it interesting, let’s search for #DonaldTrump to see what the world thinks about the President of the United States. Here is the code example to search Twitter, feed the results into Watson, and write the results to a text file:

var fs = require(’fs’);
var Twitter = require(’twitter’);

var client = new Twitter(
  consumer_key: ’*********************’,
  consumer_secret: ’******************’,
  access_token_key: ’******************’,
  access_token_secret: ’****************’
);

var PersonalityInsightsV3 = require(’watson-developer-cloud/personality-insights/v3’);
var personality_insights = new PersonalityInsightsV3(
  "url": "https://gateway.watsonplatform.net/personality-insights/api",
  "username": "**************************",
  "password": "*************",
  "version_date": "2017-12-01"
);

client.get(’search/tweets’,  q: ’#DonaldTrump’ , function(error, tweets, response) 
if(error) throw error;

var contentItems = [];

// Loop through the tweets
for (var i = 0; i < tweets.statuses.length; i++) 
var tweet = tweets.statuses[i];

contentItems.push(
"content": tweet.text,
"contenttype": "text/plain",
"created": new Date(tweet.created_at).getTime(),
"id": tweet.id,
"language": tweet.metadata.iso_language_code
);
}

// Call Watson with the tweets
personality_insights.profile(
"contentItems": contentItems,
"consumption_preferences": true
, (err, response) => 
if (err) throw err;

// Write the results to a file
fs.writeFile("results.txt", JSON.stringify(response, null, 2), function(err) 
if (err) throw err;

console.log("Results were saved!");
);
});
});

Here is another CodePen of the formatted results that I received:

See the Pen Donald Trump Watson Results by Jamie Munro (@endyourif) on CodePen.

What Do The Results Say?

Once we’ve analyzed the ‘Openness’ trait of the ‘Big Five,’ we can infer the following:

  • Emotion is quite low at 13%
  • Imagination is average at 54%
  • Intellect is very high at 96%
  • Authority challenging is also quite high at 87%

The ‘Conscientiousness’ trait at a high-level is average at 46% compared with the ‘Openness’ high-level average of 88%. Whereas ‘Agreeableness’ is very low at only 25%. I guess people on Twitter don’t like to agree with Donald Trump.

Moving on to the ‘Needs.’ The sub-categories of ‘Curiosity’ and ‘Structure’ are in the 60 percentile compared to other categories being below the 10th percentile (Excitement, Harmony, etc.).

And finally, under ‘Values,’ the sub-category that stands out to me as interesting is the ‘Openness’ to ‘Change’ at an abysmal 6%.

Based on when you perform your search, your results may vary as the results are limited to the past seven days from executing the example.

From these results, I would determine that the average person who tweets about Donald Trump is quite intellectual, challenges authority, and is not open to change.

With these results, it would allow you to automatically alter how you would target your content towards your audience to match the results received. You will need to determine what categories are of interest and what percentiles do you wish to target. With this ammunition, you can begin automating.

What Else Can I Do With Watson?

As I mentioned at the beginning of this article, Watson offers many other different services. With these services, you could automate many different parts of common business processes. For example:

  • Building a chat bot that can intelligently answer questions based on a knowledge base of information;
  • Build an application where you dictate what you want written to Watson by using the speech to text functionality;
  • Automatically translate your content into different languages to create a multi-lingual site or knowledge base;
  • Teach Watson how to look for specific patterns in images. This could be used to determine if a logo is embedded into a photo.

This, of course, is a very small subset that my limited imagination can postulate. I’m sure you can think of many other ways to leverage Watson’s immense capabilities.

If you are looking for more examples, IBM has an entire GitHub repository dedicated to their Node.js SDK. The example folder contains over ten sample applications that convert speech to text, text to speech, tone analysis, and visual recognition to name just a few.

Conclusion

Before Watson can runaway with technological growth, resulting in the singularity where Artificial Intelligence destroys mankind, this article demonstrated how you can turn social media content into a powerful understanding of how the people creating the content think. Using the results from Watson, your application can use the categories of interest where the percentile exceeds or is less than a predetermined amount to change how you target your audience.

If you have other interesting uses of Watson or how you are using the Personality Insights, be sure to leave a comment below.

Smashing Editorial
(rb, ra, yk, il)


See more here – 

Analyzing Your Company’s Social Media Presence With IBM Watson And Node.js

Monthly Web Development Update 3/2018: Service Workers, Building A CDN, And Cheating At Design

Service Worker is probably one of the most misrepresented technologies we currently have. When I hear people talking about it, the topic almost always revolves around serving an app when a user is offline. However, Service Worker can do so much more than that, and every week I come across new articles that show how powerful the technology really is.

This month, for example, we can learn how to use Service Worker for cross-tab messaging and to load off requests into the background with the Background Sync API. I think the toolset we now have in our browsers already allows us to build great experiences regardless of the network state. Now it’s up to us to make the experiences so great that users truly love them. And that’s probably the hardest part.

News

Sketch 49
Sketch 49 has arrived, and with it comes the new Prototyping in Sketch feature which lets you see the entire flow in action. (Image credit)

General

  • Ed Ellson examined Chrome’s Background Sync API and the retry strategy it uses to perform a request. By allowing synchronization in the background after a first attempt has failed, the API helps us improve the browsing experience for users who go offline or are on unstable connections.

UI/UX

Cheating At Design
Use color and weight to create visual hierarchy instead of size is only one of the seven practical tips for cheating at design that Adam Wathan and Steve Schoger share. (Image credit)

Security

  • With GraphQL you can query exactly what you want whenever you want. This is amazing for working with an API but also has complex security implications. Instead of asking for legitimate, useful data, a malicious actor could submit an expensive, nested query to overload your server, database, network, or all of these. To prevent this from happening, Max Stoiber shows us how we can secure the GraphQL API in our projects.

Privacy

  • WebKit is introducing the Storage Access API. The new API targets one of the major issues with Safari’s Intelligent Tracking Protection (ITP): Identifying users who are logged in to a first-party service but view content of it embedded on a third party (YouTube videos on a blog, for example). The Storage Access API allows third-party embeds to request access to their first-party cookies when the user interacts with them. A good solution to protect user privacy by default and allow exceptions on request.

Web Performance

  • Janos Pasztor built his own Content Delivery Network because he thinks it can be a better solution than using existing third parties. The code for the CDN of his personal website is now available on Github. A nice web performance article that looks at common solutions from a different angle.
  • A year after Facebook’s announcement to broadly use Cache-Control: Immutable, Paul Calvano examined how widespread its usage is on the web — apart from the few big players. Interesting research and it’s still sad to see that this useful performance tool is used so little. At Colloq, we use it quite a lot, which saves us a lot of traffic and load on our servers and enables us to serve a lot of pages nearly instantly to recurring users.
Global stats of a self-built CDN
Global stats for the custom CDN that Janos Pasztor built. (Image credit)

HTML & SVG

JavaScript

CSS

Accessibility

Accessibility Checklist
The Accessibility Checklist helps build accessibility into your process no matter your role or stage in a project. (Image credit)

Work & Life

  • This week I read an article by Alex Duloz, and his words still stick with me: “When we develop a new application, when we post content on the Internet, whatever we do that people will have access to, we should consider just for a minute if our contribution adds up to the level of dumbness kids/teenagers are exposed to. If it does, we should refrain from going live.” The truth is, most of us, including me, don’t consider this before posting on the Internet. We create funny things, share funny pictures and try to get fame with silly posts. But in reality, we shape society with this. Let’s try to provide more useful resources and make the consumption of this more enjoyable so young people can profit from our knowledge and not only view things we think are funny. “We should always consider how teenagers will use what we release.”
  • The MIT OpenCourseWare released a lot of free audio and video lectures. This is amazing news and makes great content available to broader masses.
  • Jake Knapp says great work requires idealism and cynicism and has strong arguments to back up this theory. An article worth reading.
  • There’s an important article on how unhappiness has grown in America’s population since around the year 2000. It reveals that while income inequality might play a role, the more important aspect is that young people who use a lot of digital media are unhappier than those who use it only up to an hour a day. Interestingly, people who don’t use digital media at all, are unhappy, too, so the outcome of this could be that we should try to use digital media only moderately — at least in our private lives. I bet it’ll make a big difference.
  • Following the theory of Michael Bradley, projects don’t necessarily need a roadmap for success. Instead, he suggests to create a moral compass that points out why the project exists and what its purpose is.

Going Beyond…

We hope you enjoyed this Web Development Update. The next one is scheduled for April 13th. Stay tuned.

Excerpt from – 

Monthly Web Development Update 3/2018: Service Workers, Building A CDN, And Cheating At Design

Thumbnail

Monthly Web Development Update 2/2018: The Grown-Up Web, Branding Details, And Browser Fast Forward

Every profession is a wide field where many people find their very own, custom niches. So are design and web development today. I started building my first website with framesets and HTML4.0, images and a super limited set of CSS, and — oh so fancy — GIFs and inline JavaScript (remember the onclick=”” attribute?) about one and a half decades ago. It took me four days to learn the initial, necessary skills for that.

Original article:

Monthly Web Development Update 2/2018: The Grown-Up Web, Branding Details, And Browser Fast Forward

Native And PWA: Choices, Not Challengers!

It’s hard to tell exactly where the rift between “native” and “web” really started. I feel like it’s one of those things that had been churning just below the surface since the early days of Flash, only to erupt more recently with the rise of mobile platforms. Regardless, developers have squared off across this “great chasm,” lobbing insults at one another in an attempt to bolster their own side.

Link: 

Native And PWA: Choices, Not Challengers!

How To Make A Dynamic Website Become Static Through A Content CDN

Smashing Magazine gave us a little surprise recently: Its website has been completely overhauled, switching away from WordPress to Netlify. One of the several reasons for the move is cost: Netlify allows for a static version of a website, which can be hosted directly on a content delivery network (CDN), reducing the number of web servers that are needed. In addition, because CDNs are located near users, accessing the website becomes faster.

Read more: 

How To Make A Dynamic Website Become Static Through A Content CDN

Your Author Bio Is a Conversion Goldmine

gold mine

Are you treating your author bio as an afterthought? If you’re like most online marketers, the answer is probably yes. Don’t believe me? Just Google ‘author bio optimization’ and see what your search turns up. Not much. Just one or two dated guides. Clearly, digital marketing experts are not seeing the business potential of an optimized author bio. Well, I’ve got news for you. By ignoring your bio, you’re missing out on a lot of business. Your author bio has a lot of marketing potential. It’s an untapped goldmine that can do so much for your business. Would you like…

The post Your Author Bio Is a Conversion Goldmine appeared first on The Daily Egg.

Source – 

Your Author Bio Is a Conversion Goldmine