Tag Archives: architecture

Thumbnail

Building A Pub/Sub Service In-House Using Node.js And Redis




Building A Pub/Sub Service In-House Using Node.js And Redis

Dhimil Gosalia



Today’s world operates in real time. Whether it’s trading stock or ordering food, consumers today expect immediate results. Likewise, we all expect to know things immediately — whether it’s in news or sports. Zero, in other words, is the new hero.

This applies to software developers as well — arguably some of the most impatient people! Before diving into BrowserStack’s story, it would be remiss of me not to provide some background about Pub/Sub. For those of you who are familiar with the basics, feel free to skip the next two paragraphs.

Many applications today rely on real-time data transfer. Let’s look closer at an example: social networks. The likes of Facebook and Twitter generate relevant feeds, and you (via their app) consume it and spy on your friends. They accomplish this with a messaging feature, wherein if a user generates data, it will be posted for others to consume in nothing short of a blink. Any significant delays and users will complain, usage will drop, and if it persists, churn out. The stakes are high, and so are user expectations. So how do services like WhatsApp, Facebook, TD Ameritrade, Wall Street Journal and GrubHub support high volumes of real-time data transfers?

All of them use a similar software architecture at a high level called a “Publish-Subscribe” model, commonly referred to as Pub/Sub.

“In software architecture, publish–subscribe is a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers, but instead categorize published messages into classes without knowledge of which subscribers, if any, there may be. Similarly, subscribers express interest in one or more classes and only receive messages that are of interest, without knowledge of which publishers, if any, there are.“

Wikipedia

Bored by the definition? Back to our story.

At BrowserStack, all of our products support (in one way or another) software with a substantial real-time dependency component — whether its automate tests logs, freshly baked browser screenshots, or 15fps mobile streaming.

In such cases, if a single message drops, a customer can lose information vital for preventing a bug. Therefore, we needed to scale for varied data size requirements. For example, with device logger services at a given point of time, there may be 50MB of data generated under a single message. Sizes like this could crash the browser. Not to mention that BrowserStack’s system would need to scale for additional products in the future.

As the size of data for each message differs from a few bytes to up to 100MB, we needed a scalable solution that could support a multitude of scenarios. In other words, we sought a sword that could cut all cakes. In this article, I will discuss the why, how, and results of building our Pub/Sub service in-house.

Through the lens of BrowserStack’s real-world problem, you will get a deeper understanding of the requirements and process of building your very own Pub/Sub.

Our Need For A Pub/Sub Service

BrowserStack has around 100M+ messages, each of which is somewhere between approximately 2 bytes and 100+ MB. These are passed around the world at any moment, all at different Internet speeds.

The largest generators of these messages, by message size, are our BrowserStack Automate products. Both have real-time dashboards displaying all requests and responses for each command of a user test. So, if someone runs a test with 100 requests where the average request-response size is 10 bytes, this transmits 1×100×10 = 1000 bytes.

Now let’s consider the larger picture as — of course — we don’t run just one test a day. More than approximately 850,000 BrowserStack Automate and App Automate tests are run with BrowserStack each and every day. And yes, we average around 235 request-response per test. Since users can take screenshots or ask for page sources in Selenium, our average request-response size is approximately 220 bytes.

So, going back to our calculator:

850,000×235×220 = 43,945,000,000 bytes (approx.) or only 43.945GB per day

Now let’s talk about BrowserStack Live and App Live. Surely we have Automate as our winner in form of size of data. However, Live products take the lead when it comes to the number of messages passed. For every live test, about 20 messages are passed each minute it turns. We run around 100,000 live tests, which each test averaging around 12 mins meaning:

100,000×12×20 = 24,000,000 messages per day

Now for the awesome and remarkable bit: We build, run, and maintain the application for this called pusher with 6 t1.micro instances of ec2. The cost of running the service? About $70 per month.

Choosing To Build vs. Buying

First things first: As a startup, like most others, we were always excited to build things in-house. But we still evaluated a few services out there. The primary requirements we had were:

  1. Reliability and stability,
  2. High performance, and
  3. Cost-effectiveness.

Let’s leave the cost-effectiveness criteria out, as I can’t think of any external services that cost under $70 a month (tweet me if know you one that does!). So our answer there is obvious.

In terms of reliability and stability, we found companies that provided Pub/Sub as a service with 99.9+ percent uptime SLA, but there were many T&C’s attached. The problem is not as simple as you think, especially when you consider the vast lands of the open Internet that lie between the system and client. Anyone familiar with Internet infrastructure knows stable connectivity is the biggest challenge. Additionally, the amount of data sent depends on traffic. For example, a data pipe that’s at zero for one minute may burst during the next. Services providing adequate reliability during such burst moments are rare (Google and Amazon).

Performance for our project means obtaining and sending data to all listening nodes at near zero latency. At BrowserStack, we utilize cloud services (AWS) along with co-location hosting. However, our publishers and/or subscribers could be placed anywhere. For example, it may involve an AWS application server generating much-needed log data, or terminals (machines where users can securely connect for testing). Coming back to the open Internet issue again, if we were to reduce our risk we would have to ensure our Pub/Sub leveraged the best host services and AWS.

Another essential requirement was the ability to transmit all types of data (Bytes, text, weird media data, etc.). With all considered, it did not make sense to rely on a third-party solution to support our products. In turn, we decided to revive our startup spirit, rolling up our sleeves to code our own solution.

Building Our Solution

Pub/Sub by design means there will be a publisher, generating and sending data, and a Subscriber accepting and processing it. This is similar to a radio: A radio channel broadcasts (publishes) content everywhere within a range. As a subscriber, you can decide whether to tune into that channel and listen (or turn off your radio altogether). 

Unlike the radio analogy where data is free for all and anyone can decide to tune in, in our digital scenario we need authentication which means data generated by the publisher could only be for a single particular client or subscriber.


Basic working of Pub/Sub


Basic working of Pub/Sub (Large preview)

Above is a diagram providing an example of a good Pub/Sub with:

  • Publishers
    Here we have two publishers generating messages based on pre-defined logic. In our radio analogy, these are our radio jockeys creating the content.
  • Topics
    There are two here, meaning there are two types of data. We can say these are our radio channels 1 and 2.
  • Subscribers
    We have three that each read data on a particular topic. One thing to notice is that Subscriber 2 is reading from multiple topics. In our radio analogy, these are the people who are tuned into a radio channel. 

Let’s start understanding the necessary requirements for the service.

  1. An evented component
    This kicks in only when there is something to kick in.
  2. Transient storage
    This keeps data persisted for a short duration so if the subscriber is slow, it still has a window to consume it.
  3. Reducing the latency
    Connecting two entities over a network with minimum hops and distance.

We picked a technology stack that fulfilled the above requirements:

  1. Node.js
    Because why not? Evented, we wouldn’t need heavy data processing, plus it’s easy to onboard.
  2. Redis
    Supports perfectly short-lived data. It has all the capabilities to initiate, update and auto-expire. It also puts less load on the application.

Node.js For Business Logic Connectivity

Node.js is a nearly perfect language when it comes to writing code incorporating IO and events. Our particular given problem had both, making this option the most practical for our needs.

Surely other languages such as Java could be more optimized, or a language like Python offers scalability. However, the cost of starting with these languages is so high that a developer could finish writing code in Node in the same duration. 

To be honest, if the service had a chance of adding more complicated features, we could have looked at other languages or a completed stack. But here it is a marriage made in heaven. Here is our package.json:


  "name": "Pusher",
  "version": "1.0.0",
  "dependencies": 
    "bstack-analytics": "*****", // Hidden for BrowserStack reasons. :)
    "ioredis": "^2.5.0",
    "socket.io": "^1.4.4"
  ,
  "devDependencies": {},
  "scripts": 
    "start": "node server.js"
  
}

Very simply put, we believe in minimalism especially when it comes to writing code. On the other hand, we could have used libraries like Express to write extensible code for this project. However, our startup instincts decided to pass on this and to save it for the next project. Additional tools we used:

  • ioredis
    This is one of the most supported libraries for Redis connectivity with Node.js used by companies including Alibaba.
  • socket.io
    The best library for graceful connectivity and fallback with WebSocket and HTTP.

Redis For Transient Storage

Redis as a service scales is heavily reliable and configurable. Plus there are many reliable managed service providers for Redis, including AWS. Even if you don’t want to use a provider, Redis is easy to get started with.

Let’s break down the configurable part. We started off with the usual master-slave configuration, but Redis also comes with cluster or sentinel modes. Every mode has its own advantages.

If we could share the data in some way, a Redis cluster would be the best choice. But if we shared the data by any heuristics, we have less flexibility as the heuristic has to be followed across. Fewer rules, more control is good for life!

Redis Sentinel works best for us as data lookup is done in just one node, connecting at a given point in time while data is not sharded. This also means that even if multiple nodes are lost, the data is still distributed and present in other nodes. So you have more HA and less chances of loss. Of course, this removed the pros from having a cluster, but our use case is different.

Architecture At 30000 Feet

The diagram below provides a very high-level picture of how our Automate and App Automate dashboards work. Remember the real-time system that we had from the earlier section?


BrowserStack’s real-time Automate and App Automate dashboards.


BrowserStack’s real-time Automate and App Automate dashboards (Large preview)

In our diagram, our main workflow is highlighted with thicker borders. The “automate” section consists of:

  1. Terminals
    Comprised of the pristine versions of Windows, OSX, Android or iOS that you get while testing on BrowserStack.
  2. Hub
    The point of contact for all your Selenium and Appium tests with BrowserStack.

The “user service” section here is our gatekeeper, ensuring data is sent to and saved for the right individual. It is also our security keeper. The “pusher” section incorporates the heart of what we discussed in this article. It consists of the usual suspects including:

  1. Redis
    Our transient storage for messages, where in our case automate logs are temporarily stored.
  2. Publisher
    This is basically the entity that obtains data from the hub. All your request responses are captured by this component which writes to Redis with session_id as the channel.
  3. Subscriber
    This reads data from Redis generated for the session_id. It is also the web server for clients to connect via WebSocket (or HTTP) to get data and then sends it to authenticated clients.

Finally, we have the user’s browser section, representing an authenticated WebSocket connection to ensure session_id logs are sent. This enables the front-end JS to parse and beautify it for users.

Similar to the logs service, we have pusher here that is being used for other product integrations. Instead of session_id, we use another form of ID to represent that channel. This all works out of pusher!

Conclusion (TLDR)

We’ve had considerable success in building out Pub/Sub. To sum up why we built it in-house:

  1. Scales better for our needs;
  2. Cheaper than outsourced services;
  3. Full control over the overall architecture.

Not to mention that JS is the perfect fit for this kind of scenario. Event loop and massive amount of IO is what the problem needs! JavaScript is magic of single pseudo thread. 

Events and Redis as a system keep things simple for developers, as you can obtain data from one source and push it to another via Redis. So we built it.

If the usage fits into your system, I recommend doing the same!

Smashing Editorial
(rb, ra, il)


See original article: 

Building A Pub/Sub Service In-House Using Node.js And Redis

Thumbnail

Building A Serverless Contact Form For Your Static Site




Building A Serverless Contact Form For Your Static Site

Brian Holt



Static site generators provide a fast and simple alternative to Content Management Systems (CMS) like WordPress. There’s no server or database setup, just a build process and simple HTML, CSS, and JavaScript. Unfortunately, without a server, it’s easy to hit their limits quickly. For instance, in adding a contact form.

With the rise of serverless architecture adding a contact form to your static site doesn’t need to be the reason to switch to a CMS anymore. It’s possible to get the best of both worlds: a static site with a serverless back-end for the contact form (that you don’t need to maintain). Maybe best of all, in low-traffic sites, like portfolios, the high limits of many serverless providers make these services completely free!

In this article, you’ll learn the basics of Amazon Web Services (AWS) Lambda and Simple Email Service (SES) APIs to build your own static site mailer on the Serverless Framework. The full service will take form data submitted from an AJAX request, hit the Lambda endpoint, parse the data to build the SES parameters, send the email address, and return a response for our users. I’ll guide you through getting Serverless set up for the first time through deployment. It should take under an hour to complete, so let’s get started!

The static site form, sending the message to the Lambda endpoint and returning a response to the user.
The static site form, sending the message to the Lambda endpoint and returning a response to the user.

Setting Up

There are minimal prerequisites in getting started with Serverless technology. For our purposes, it’s simply a Node Environment with Yarn, the Serverless Framework, and an AWS account.

Setting Up The Project


The Serverless Framework web site. Useful for installation and documentation.


The Serverless Framework web site. Useful for installation and documentation.

We use Yarn to install the Serverless Framework to a local directory.

  1. Create a new directory to host the project.
  2. Navigate to the directory in your command line interface.
  3. Run yarn init to create a package.json file for this project.
  4. Run yarn add serverless to install the framework locally.
  5. Run yarn serverless create --template aws-nodejs --name static-site-mailer to create a Node service template and name it static-site-mailer.

Our project is setup but we won’t be able to do anything until we set up our AWS services.

Setting Up An Amazon Web Services Account, Credentials, And Simple Email Service


The Amazon Web Services sign up page, which includes a generous free tier, enabling our project to be entirely free.


The Amazon Web Services sign up page, which includes a generous free tier, enabling our project to be entirely free.

The Serverless Framework has recorded a video walk-through for setting up AWS credentials, but I’ve listed the steps here as well.

  1. Sign Up for an AWS account or log in if you already have one.
  2. In the AWS search bar, search for “IAM”.
  3. On the IAM page, click on “Users” on the sidebar, then the “Add user” button.
  4. On the Add user page, give the user a name – something like “serverless” is appropriate. Check “Programmatic access” under Access type then click next.
  5. On the permissions screen, click on the “Attach existing policies directly” tab, search for “AdministratorAccess” in the list, check it, and click next.
  6. On the review screen you should see your user name, with “Programmatic access”, and “AdministratorAccess”, then create the user.
  7. The confirmation screen shows the user “Access key ID” and “Secret access key”, you’ll need these to provide the Serverless Framework with access. In your CLI, type yarn sls config credentials --provider aws --key YOUR_ACCESS_KEY_ID --secret YOUR_SECRET_ACCESS_KEY, replacing YOUR_ACCESS_KEY_ID and YOUR_SECRET_ACCESS_KEY with the keys on the confirmation screen.

Your credentials are configured now, but while we’re in the AWS console let’s set up Simple Email Service.

  1. Click Console Home in the top left corner to go home.
  2. On the home page, in the AWS search bar, search for “Simple Email Service”.
  3. On the SES Home page, click on “Email Addresses” in the sidebar.
  4. On the Email Addresses listing page, click the “Verify a New Email Address” button.
  5. In the dialog window, type your email address then click “Verify This Email Address”.
  6. You’ll receive an email in moments containing a link to verify the address. Click on the link to complete the process.

Now that our accounts are made, let’s take a peek at the Serverless template files.

Setting Up The Serverless Framework

Running serverless create creates two files: handler.js which contains the Lambda function, and serverless.yml which is the configuration file for the entire Serverless Architecture. Within the configuration file, you can specify as many handlers as you’d like, and each one will map to a new function that can interact with other functions. In this project, we’ll only create a single handler, but in a full Serverless Architecture, you’d have several of the various functions of the service.


The default file structure generated from the Serverless Framework containing handler.js and serverless.yml.


The default file structure generated from the Serverless Framework containing handler.js and serverless.yml.

In handler.js, you’ll see a single exported function named hello. This is currently the main (and only) function. It, along with all Node handlers, take three parameters:

  • event
    This can be thought of as the input data for the function.
  • context object
    This contains the runtime information of the Lambda function.
  • callback
    An optional parameter to return information to the caller.
// handler.js

'use strict';

module.exports.hello = (event, context, callback) => 
  const response = 
    statusCode: 200,
    body: JSON.stringify(
      message: 'Go Serverless v1.0! Your function executed successfully!',
      input: event,
    ),
  };

  callback(null, response);
};

At the bottom of hello, there’s a callback. It’s an optional argument to return a response, but if it’s not explicitly called, it will implicitly return with null. The callback takes two parameters:

  • Error error
    For providing error information for when the Lambda itself fails. When the Lambda succeeds, null should be passed into this parameter.
  • Object result
    For providing a response object. It must be JSON.stringify compatible. If there’s a parameter in the error field, this field is ignored.

Our static site will send our form data in the event body and the callback will return a response for our user to see.

In serverless.yml you’ll see the name of the service, provider information, and the functions.

# serverless.yml

service: static-site-mailer

provider:
  name: aws
  runtime: nodejs6.10

functions:
  hello:
    handler: handler.hello

How the function names in serverless.yml map to handler.js.


How the function names in serverless.yml map to handler.js.

Notice the mapping between the hello function and the handler? We can name our file and function anything and as long as it maps to the configuration it will work. Let’s rename our function to staticSiteMailer.

# serverless.yml

functions:
  staticSiteMailer:
    handler: handler.staticSiteMailer
// handler.js

module.exports.staticSiteMailer = (event, context, callback) => 
  ...
;

Lambda functions need permission to interact with other AWS infrastructure. Before we can send an email, we need to allow SES to do so. In serverless.yml, under provider.iamRoleStatements add the permission.

# serverless.yml

provider:
  name: aws
  runtime: nodejs6.10
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "ses:SendEmail"
      Resource: ["*"]

Since we need a URL for our form action, we need to add HTTP events to our function. In serverless.yml we create a path, specify the method as post, and set CORS to true for security.

functions:
  staticSiteMailer:
    handler: handler.staticSiteMailer
    events:
      - http:
          method: post
          path: static-site-mailer
          cors: true

Our updated serverless.yml and handler.js files should look like:

# serverless.yml

service: static-site-mailer

provider:
  name: aws
  runtime: nodejs6.10

functions:
  staticSiteMailer:
    handler: handler.staticSiteMailer
    events:
      - http:
          method: post
          path: static-site-mailer
          cors: true

provider:
  name: aws
  runtime: nodejs6.10
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "ses:SendEmail"
      Resource: ["*"]
// handler.js

'use strict';

module.exports.staticSiteMailer = (event, context, callback) => 
  const response = 
    statusCode: 200,
    body: JSON.stringify(
      message: 'Go Serverless v1.0! Your function executed successfully!',
      input: event,
    ),
  };

  callback(null, response);
};

Our Serverless Architecture is setup, so let’s deploy it and test it. You’ll get a simple JSON response.

yarn sls deploy --verbose
yarn sls invoke --function staticSiteMailer


    "statusCode": 200,
    "body": ""message":"Go Serverless v1.0! Your function executed successfully!","input":}"
}

The return response from invoking our brand new serverless function.


The return response from invoking our brand new serverless function.

Creating The HTML Form

Our Lambda function input and form output need to match, so before we build the function we’ll build the form and capture its output. We keep it simple with name, email, and message fields. We’ll add the form action once we’ve deployed our serverless architecture and got our URL, but we know it will be a POST request so we can add that in. At the end of the form, we add a paragraph tag for displaying response messages to the user which we’ll update on the submission callback.

<form action=" SERVICE URL }" method="POST">
  <label>
    Name
    <input type="text" name="name" required>
  </label>
  <label>
    Email
    <input type="email" name="reply_to" required>
  </label>
  <label>
    Message:
    <textarea name="message" required></textarea>
  </label>
  <button type="submit">Send Message</button>
</form>
<p id="js-form-response"></p>

To capture the output we add a submit handler to the form, turn our form parameters into an object, and send stringified JSON to our Lambda function. In the Lambda function we use JSON.parse() to read our data. Alternatively, you could use jQuery’s Serialize or query-string to send and parse the form parameters as a query string but JSON.stringify() and JSON.parse() are native.

(() => 
  const form = document.querySelector('form');
  const formResponse = document.querySelector('js-form-response');

  form.onsubmit = e => 
    e.preventDefault();

    // Prepare data to send
    const data = ;
    const formElements = Array.from(form);
    formElements.map(input => (data[input.name] = input.value));

    // Log what our lambda function will receive
    console.log(JSON.stringify(data));
  };
})();

Go ahead and submit your form then capture the console output. We’ll use it in our Lambda function next.


Capturing the form data in a console log.


Capturing the form data in a console log.

Invoking Lambda Functions

Especially during development, we need to test our function does what we expect. The Serverless Framework provides the invoke and invoke local command to trigger your function from live and development environments respectively. Both commands require the function name passed through, in our case staticSiteMailer.

yarn sls invoke local --function staticSiteMailer

To pass mock data into our function, create a new file named data.json with the captured console output under a body key within a JSON object. It should look something like:

// data.json


  "body": ""name": "Sender Name","reply_to": "sender@email.com","message": "Sender message""
}

To invoke the function with the local data, pass the --path argument along with the path to the file.

yarn sls invoke local --function staticSiteMailer --path data.json

An updated return response from our serverless function when we pass it JSON data.


An updated return response from our serverless function when we pass it JSON data.

You’ll see a similar response to before, but the input key will contain the event we mocked. Let’s use our mock data to send an email using Simple Email Service!

Sending An Email With Simple Email Service

We’re going to replace the staticSiteMailer function with a call to a private sendEmail function. For now you can comment out or remove the template code and replace it with:

// hander.js

function sendEmail(formData, callback) 
  // Build the SES parameters
  // Send the email


module.exports.staticSiteMailer = (event, context, callback) => 
  const formData = JSON.parse(event.body);

  sendEmail(formData, function(err, data) 
    if (err) 
      console.log(err, err.stack);
     else 
      console.log(data);
    
  });
};

First, we parse the event.body to capture the form data, then we pass it to a private sendEmail function. sendEmail is responsible for sending the email, and the callback function will return a failure or success response with err or data. In our case, we can simply log the error or data since we’ll be replacing this with the Lambda callback in a moment.

Amazon provides a convenient SDK, aws-sdk, for connecting their services with Lambda functions. Many of their services, including SES, are part of it. We add it to the project with yarn add aws-sdk and import it into the top the handler file.

// handler.js

const AWS = require('aws-sdk');
const SES = new AWS.SES();

In our private sendEmail function, we build the SES.sendEmail parameters from the parsed form data and use the callback to return a response to the caller. The parameters require the following as an object:

  • Source
    The email address SES is sending from.
  • ReplyToAddresses
    An array of email addresses added to the reply to the field in the email.
  • Destination
    An object that must contain at least one ToAddresses, CcAddresses, or BccAddresses. Each field takes an array of email addresses that correspond to the to, cc, and bcc fields respectively.
  • Message
    An object which contains the Body and Subject.

Since formData is an object we can call our form fields directly like formData.message, build our parameters, and send it. We pass your SES-verified email to Source and Destination.ToAddresses. As long as the email is verified you can pass anything here, including different email addresses. We pluck our reply_to, message, and name off our formData object to fill in the ReplyToAddresses and Message.Body.Text.Data fields.

// handler.js
function sendEmail(formData, callback) 
  const emailParams = 
    Source: 'your_email@example.com', // SES SENDING EMAIL
    ReplyToAddresses: [formData.reply_to],
    Destination: 
      ToAddresses: ['your_email@example.com'], // SES RECEIVING EMAIL
    ,
    Message: 
      Body: 
        Text: 
          Charset: 'UTF-8',
          Data: `$formData.messagennName: $formData.namenEmail: $formData.reply_to`,
        },
      },
      Subject: 
        Charset: 'UTF-8',
        Data: 'New message from your_site.com',
      ,
    },
  };

  SES.sendEmail(emailParams, callback);
}

SES.sendEmail will send the email and our callback will return a response. Invoking the local function will send an email to your verified address.

yarn sls invoke local --function testMailer --path data.json

The return response from SES.sendEmail when it succeeds.


The return response from SES.sendEmail when it succeeds.

Returning A Response From The Handler

Our function sends an email using the command line, but that’s not how our users will interact with it. We need to return a response to our AJAX form submission. If it fails, we should return an appropriate statusCode as well as the err.message. When it succeeds, the 200 statusCode is sufficient, but we’ll return the mailer response in the body as well. In staticSiteMailer we build our response data and replace our sendEmail callback function with the Lambda callback.

// handler.js

module.exports.staticSiteMailer = (event, context, callback) => 
  const formData = JSON.parse(event.body);

  sendEmail(formData, function(err, data) 
    const response = 
      statusCode: err ? 500 : 200,
      headers: 
        'Content-Type': 'application/json',
        'Access-Control-Allow-Origin': 'https://your-domain.com',
      ,
      body: JSON.stringify(
        message: err ? err.message : data,
      ),
    };

    callback(null, response);
  });
};

Our Lambda callback now returns both success and failure messages from SES.sendEmail. We build the response with checks if err is present so our response is consistent. The Lambda callback function itself passes null in the error argument field and the response as the second. We want to pass errors onwards, but if the Lambda itself fails, its callback will be implicitly called with the error response.

In the headers, you’ll need to replace Access-Control-Allow-Origin with your own domain. This will prevent any other domains from using your service and potentially racking up an AWS bill in your name! And I don’t cover it in this article, but it’s possible to set-up Lambda to use your own domain. You’ll need to have an SSL/TLS certificate uploaded to Amazon. The Serverless Framework team wrote a fantastic tutorial on how to do so.

Invoking the local function will now send an email and return the appropriate response.

yarn sls invoke local --function testMailer --path data.json

The return response from our serverless function, containing the SES.sendEmail return response in the body.


The return response from our serverless function, containing the SES.sendEmail return response in the body.

Calling The Lambda Function From The Form

Our service is complete! To deploy it run yarn sls deploy -v. Once it’s deployed you’ll get a URL that looks something like https://r4nd0mh45h.execute-api.us-east-1.amazonaws.com/dev/static-site-mailer which you can add to the form action. Next, we create the AJAX request and return the response to the user.

(() => 
  const form = document.querySelector('form');
  const formResponse = document.querySelector('js-form-response');

  form.onsubmit = e => 
    e.preventDefault();

    // Prepare data to send
    const data = ;
    const formElements = Array.from(form);
    formElements.map(input => (data[input.name] = input.value));

    // Log what our lambda function will receive
    console.log(JSON.stringify(data));

    // Construct an HTTP request
    var xhr = new XMLHttpRequest();
    xhr.open(form.method, form.action, true);
    xhr.setRequestHeader('Accept', 'application/json; charset=utf-8');
    xhr.setRequestHeader('Content-Type', 'application/json; charset=UTF-8');

    // Send the collected data as JSON
    xhr.send(JSON.stringify(data));

    // Callback function
    xhr.onloadend = response => 
      if (response.target.status === 200) 
        // The form submission was successful
        form.reset();
        formResponse.innerHTML = 'Thanks for the message. I’ll be in touch shortly.';
       else 
        // The form submission failed
        formResponse.innerHTML = 'Something went wrong';
        console.error(JSON.parse(response.target.response).message);
      
    };
  };
})();

In the AJAX callback, we check the status code with response.target.status. If it’s anything other than 200 we can show an error message to the user, otherwise let them know the message was sent. Since our Lambda returns stringified JSON we can parse the body message with JSON.parse(response.target.response).message. It’s especially useful to log the error.

You should be able to submit your form entirely from your static site!

The static site form, sending the message to the Lambda endpoint and returning a response to the user.
The static site form, sending the message to the Lambda endpoint and returning a response to the user.

Next Steps

Adding a contact form to your static is easy with the Serverless Framework and AWS. There’s room for improvement in our code, like adding form validation with a honeypot, preventing AJAX calls for invalid forms and improving the UX if the response, but this is enough to get started. You can see some of these improvements within the static site mailer repo I’ve created. I hope I’ve inspired you to try out Serverless yourself!

Smashing Editorial
(lf, ra, il)


See original article here:

Building A Serverless Contact Form For Your Static Site

Why You Should Care About Website Architecture When it Comes to Your SEO

website seo and architecture

Moz’s Whiteboard Friday on IA and SEO I’ve always had a thing for site architecture and designing sites that are both user AND search engine friendly. However, it can be a bit challenging. Especially for those who have been doing SEO for a while. We all want to over-optimize every web page as much as possible. Well, stop doing that! In this video, Rand does a great job of explaining what you need to keep in mind when you’re designing a website that has both a good user experience and SEO. A Few Key Takeaways From The Video: Good information…

The post Why You Should Care About Website Architecture When it Comes to Your SEO appeared first on The Daily Egg.

Continue reading here – 

Why You Should Care About Website Architecture When it Comes to Your SEO

A Brief Overview On Responsive Navigation Patterns

To say that responsive web design has changed our industry would be an understatement at best. We used to ask our clients which resolutions and devices they wanted us to support, but we now know the answer is “as many as possible.” To answer a challenge like this and to handle our increasingly complex world, our industry has exploded with new thinking, patterns and approaches.
In this article, I want to look specifically at the issue of responsive navigation.

Source – 

A Brief Overview On Responsive Navigation Patterns

Thumbnail

Getting Back Into The (Right) Deliverables Business

“Get out of the deliverables business” has become quite a mantra in the lean startup and UX movements. There’s much to love in that sentiment — after all, for every wireframe you make, you’re not shipping code to customers.
But I’m worried that, just like with the concept of a minimum viable product, we’ve taken this sound advice to an extreme that’s actually hurtful to the creation of good products.

See the article here: 

Getting Back Into The (Right) Deliverables Business

Thumbnail

Getting Started With Wearables: How To Plan, Build and Design


If a user of your product is buying a smartwatch tomorrow and your app is not compatible with it or your notifications can’t be triggered from there, you might frustrate them. If you have a website or an app today, it’s time to start planning support for wearable devices. In this article, we’ll review the platforms available today, what we can do on each of them, how to plan the architecture, and how to develop apps or companion services for these new devices.

Developing For Wearables

Do you remember the shoe phone from Get Smart? If you don’t know what I’m talking about, you are probably too young (or I’m too old). (You can Google it now. Just go; I’ll wait here in this tab.) The shoe phone we saw on TV was followed by many other wearable devices on TV, such as the ones on Knight Rider, The Flintstones, James Bond and Dick Tracy. Many years later, we can say that wearable devices are here and ready to use. We, as designers and developers, need to be ready to develop successful experiences for them.

The post Getting Started With Wearables: How To Plan, Build and Design appeared first on Smashing Magazine.

Link:

Getting Started With Wearables: How To Plan, Build and Design

4 Stupid Mistakes You’re STILL Making On Your Landing Page

stupid-mistakes-landing-page-650
Are you still making these landing page mistakes? Image by Brandon Grasley via Flickr.

Believe it or not, many of the world’s most aesthetically beautiful landing pages fail miserably when it comes to conversion.

Why? Because when you focus too much on design and not enough on your customers, it’s easy to lose sight of the bigger picture and fall into common conversion-killing traps.

In this post, I go through four of the worst mistakes you can make on your landing page, with real-world examples. Fixing even one of these mistakes should result in a serious conversion rate improvement – so let’s get started!

1. Not showing the product

Let’s take a look at this landing page for iMenuPro – an app that allows restaurant owners to design menus online:

imenu-pro-650
Click for full-length landing page.

It’s a nice enough page, right? Solid design, pretty engaging content and it even has a bit of personality. But there’s one crucial thing missing: they never show the product.

iMenuPro is a menu designer, yet we never see any actual menus that have been designed with the tool. Believe it or not, this is an incredibly common mistake.

If this seems like a huge oversight to you, it should. Neglecting to show your product is the #1 cardinal sin of landing page design, and here’s why: humans aren’t just visual learners, they’re visual purchasers.


Do you show & tell? If I can’t see myself using your product, I can’t see myself buying it.
Click To Tweet


If I can’t see your product or what it does, how in the world am I supposed to want it? Imagine trying to buy a car that has only been verbally described to you.

The solution

Show your product up-front and clearly. Make it the hero shot of your page.

And when possible, show your product in action.

This technique, called context of use, helps show prospects how your product works and helps them envision themselves using it:

This is precisely the reason that ShamWow has become a household name – they show their product in action with real people, in real situations you can relate to.

Showing and telling will help you convert browsers into customers.

2. Not explaining what you do

It’s all too easy to forget one of the main purposes of your landing page: educating your prospects.

Many prospects who visit your landing page know nothing about you, your company or what it is that you do. It’s your landing page’s job to fill in the blanks. When you don’t do that, you get a page like this:

marketing-genesis-live-650
Click for full-length landing page.

Marketing Genesis is a paid seminar for aspiring marketers – or, at least, that’s what I think it is. They never actually say.

If you carefully read a few hundred words into the text, you’ll eventually infer what Marketing Genesis is, but it takes some effort. They’re assuming that I know something about their business, but I don’t.

They make this same mistake dozens of times throughout this page:

  • The main headline on the page tells me to “Register Now,” but I don’t know what I’m registering for yet.
  • The CTA asks me to click for tickets, but again, what am I getting tickets to?
  • They even assume that I know where the event is taking place (hint: I don’t).

If you’re thinking, “how could someone possibly forget those things on a page?”, you should know that this sort of thing happens with shocking frequency.

When you’re elbow-deep in the goings-on of your own company, it’s easy to forget what it’s like to not know about your company.

The solution

When in doubt, treat your clients as though they know truly nothing about you.

Explain what you do, why you’re better than your competition and how your product can improve your potential customers’ lives.

The people at Webflow do a brilliant job of this – take a look at their homepage:

webflow-650
Click for larger image.

Even though they’re selling a relatively high-tech product, their opening headline tells me exactly what they’re all about in just a few words: “Professional-looking websites without writing code.”

That’s the kind of quick sales pitch we’re looking for.

Note: explaining what you do does not mean telling prospects about everything you do. As we’ll see below, you want to test making your copy as minimal as possible.

3. Using lots of paragraph text

If there’s one immutable truth about your customers, it’s this: whether you’re Apple or a mom-and-pop shop, nobody wants to read the long paragraphs of text on your landing pages.

Take for example this page from Newschool of Architecture and Design in San Diego:

newschool-architecture-design-650
Click for larger image.

They seem like a lovely university, but they fall into a common trap: they’re over-explaining.

In order to get my questions answered, I need to read through at least a few paragraphs of relatively dry copy. I’m willing to bet that many potential students would rather leave the page than put in the effort.

It might feel like your business is too complicated to explain quickly but in reality, even the most complex businesses can be to be boiled down to a series of short, benefit-driven sentences.

If you absolutely need to write a longer page, communicate your unique value proposition up front and don’t write a word more than you have to.

The solution

Be kind to skimmers and impatient users by cutting down on text, focusing on the key points of your service and providing visual examples.

If you routinely have issues with including too much copy, try writing your copy first before even looking at a landing page template.

That way, you’ll be sure to design a page that complements your copy and only includes the words you absolutely need. Not sure what you need? You should test that.

4. Making users choose (or even think)

Many businesses have multiple buyer personas, which makes marketing to them kind of tough.

How do you tailor a landing page to drastically different groups of people while still resonating with your ideal customers? We’ve all heard it before: Try to appeal to everyone and you’ll appeal to no one.

As a solution to this, many companies add a click-through page that asks users to self-select what kind of customer they are. For example, take a look at this landing page by PerfumesForABuck, an ecommerce outlet for cheap fragrances:

PerfumesforaBuck

Before you can see any product, you’re forced to choose between jewelry for men, women and gift baskets. Until you choose, you can’t see anything about the business or their products – and that’s problematic.

When you force users to choose before seeing content, a strange thing happens: many prospects leave and don’t come back.

Forcing choice adds friction – you’re putting extra work on the visitor, and the visitor doesn’t like work. They shouldn’t have to think.

The solution

Even if you have a segmented customer base, you can market to all of them individually without forcing them to make choices. It just takes a little finesse.

If you’re marketing to multiple personas, create separate ad campaigns for each one and drive those separate campaigns to customized landing pages.

Instead of buying clicks for “perfume” in AdWords, buy clicks for “men’s perfume” and send the traffic to a dedicated landing page. This eliminates choice from the equation and helps drive more targeted, valuable traffic to your site.


Don’t make users self-select. Do the heavy lifting with PPC & customized landing pages.
Click To Tweet


Wrapping things up

It’s tempting to run tests on granular stuff such as your call to action and headlines.

But doing so can lead you to lose site of the bigger picture: at the very least, are you explaining what you do and showing people what you have to offer?

If you’ve made one of these mistakes, count yourself lucky. An error like this is a huge opportunity for improvement. And many of the mistakes outlined above are relatively easy to fix.

So fess up. Are you making any of these mistakes? I want to hear in the comments!

Link:  

4 Stupid Mistakes You’re STILL Making On Your Landing Page

How I Built The One Page Scroll Plugin

Scrolling effects have been around in web design for years now, and while many plugins are available to choose from, only a few have the simplicity and light weight that most developers and designers are looking for. Most plugins I’ve seen try to do too many things, which makes it difficult for designers and developers to integrate them in their projects.
Further reading on Smashing: Infinite Scrolling: Let’s Get To The Bottom Of This Get the Scrolling Right Reapplying Hick’s Law of Narrowing Decision Architecture Advanced Navigation With Two Independent Columns Takeaways From Mobile Web Behavior Not long ago, Apple introduced the iPhone 5S, which was accompanied by a presentation website on which visitors were guided down sections of a page and whose messaging was reduced to one key function per section.

Source:  

How I Built The One Page Scroll Plugin

34 Splendid Art Professions Icons For Free

This week we have a beautiful icon freebie for you. BanzaiTokyo designed this art related icon set. Their collection of 34 icons are an homage to the creative professions. The icons are available as PNG files and released under a Creative Commons Attribution License.

Large view.
The Art Professions Icons set would not be out of place in an artist’s portfolio website, be they cook or painter — or even multi-talented.

View original post here: 

34 Splendid Art Professions Icons For Free

Decoupling HTML From CSS

For years, the Web standards community has talked about the separation of concerns. Separate your CSS from your JavaScript from your HTML. We all do that, right? CSS goes into its own file; JavaScript goes in another; HTML is left by itself, nice and clean.
CSS Zen Garden proved that we can alter a design into a myriad of permutations simply by changing the CSS. However, we’ve rarely seen the flip side of this — the side that is more likely to occur in a project: the HTML changes.

View original post here – 

Decoupling HTML From CSS