Testing Functional Mad Libs

Download the src(github)
View the demo

This is the final post in a three-part series. The first post dealt with building a functional foundation and the second post was about building a functional UI. This post will be about testing everything.

Testing

Testing is very important, but sometimes it gets left behind. This can be because it is not clear how or even what to test. Tightly coupled code is a testing nightmare. It is very hard to unwind specific units of code to test. In addition to this mocking can become a huge task where you have to recreate all the resources the application needs.

This is where functional design comes in. By definition, it should be easy to test. There will be many functions that should only rely on what is passed into them. Mocking becomes trivial. Deciding what to test becomes trivial. Overall testing becomes trivial.

In this post, I will not bore you with the details about every test. Testing can be repetitive. I will highlight how I tested, how I mocked, and highlight any interesting parts.

How to test

There are three mostly agreed upon ways to test, unit, integration, and end to end. The definitions for these will change depending on who you ask, but they are as follows. Unit tests are built to test one specific function. This means that no dependencies or mocking should be used. These tests are the simplest. Next up is integration testing. At this point, pieces are starting to be put together, integrated if you will. Integration tests will include unit tested pieces and some mocking. Although mocking is not always required. Then the final testing is end to end. This is where the entire application is built and tests are run. The previous tests usually do not need more than a test runner. End to end will involve more tooling.

Does this sound familiar? It should because this is exactly the same approach to writing functionally. First, you write simple functions. Those functions are then combined to create larger pieces. Then everything is put together to create the application. Each step maps to unit, integration, and end to end.

This means we start with the simplest functions and run them through their paces. Because these are simple functions the tests pretty much write themselves. Here is an excerpt from basic_functions_test.js.

const assert = require('assert');
const {filterFunc, addField, filterMadLib} = require('../src/basic_functions.js');

describe('filterFunc test for pos', () => {
  let verb = {pos: {Verb: true}};
  let noun = {pos: {Noun: true}};
  let adjective = {pos: {Adjective: true}};
  let adverb = {pos: {Adverb: true}};
  it('should match each word type', (done) => {
    assert.equal(filterFunc('Verb', verb), true);
    assert.equal(filterFunc('Noun', noun), true);
    assert.equal(filterFunc('Adjective', adjective), true);
    assert.equal(filterFunc('Adverb', adverb), true);
    done();
  });
  it('should not match different word types', (done) => {
    assert.equal(filterFunc('Verb', noun), false);
    assert.equal(filterFunc('Verb', adjective), false);
    assert.equal(filterFunc('Verb', adverb), false);

    assert.equal(filterFunc('Noun', verb), false);
    assert.equal(filterFunc('Noun', adjective), false);
    assert.equal(filterFunc('Noun', adverb), false);

    assert.equal(filterFunc('Adjective', noun), false);
    assert.equal(filterFunc('Adjective', verb), false);
    assert.equal(filterFunc('Adjective', adverb), false);

    assert.equal(filterFunc('Adverb', noun), false);
    assert.equal(filterFunc('Adverb', adjective), false);
    assert.equal(filterFunc('Adverb', verb), false);
    done();
  });
});

The tests are clear and very simple. The function only does one thing so we know what to test.

Integration testing

We can move to the next step, integration testing. The great part about this is that we have already integrated the functions by composing them. We can see an example of this kind of test in render_functions_test.js.

const assert = require('assert');
const {createRenderElements, enterRenderElements, doneRenderElements} = require('../src/render_functions.js');

let text = "Somebody once told me the world is gonna roll me. I aint the sharpest tool in the shed.";

describe('createRenderElements test', () => {
  it('should should have MadLib on the indexed', (done) => {
    let create = createRenderElements([], text);
    let icreate = createRenderElements([1,3,5], text);

    assert.equal(create.length, 19);
    assert.equal(create[0].text, 'Somebody');
    assert.equal(icreate.length, 19);
    assert.equal(icreate[0].text, 'Somebody');
    assert.equal(icreate[0].MadLib, undefined);
    assert.equal(icreate[1].MadLib, true);
    assert.equal(icreate[3].MadLib, true);
    assert.equal(icreate[5].MadLib, true);
    done();
  });
})

This excerpt makes it clear that these tests are just as easy to write as our unit tests. Remember that each one of the *renderElements functions is composed of up to six different functions. Now some of these are Ramda functions, but the point still stands that this is an integrated test of functionality.

Mocking

Mocking can be trivial if we have built our functions correctly. If our functions only do one thing then it is clear what we need to mock for that function. In our application, the only thing we need to mock is the document object. And even that only needs a few things mocked to work. Looking at what methods are used on document we see we only need to add some child manipulation tools (appendChild, removeChild, and firstChild) and some properties. Here is the document mock object, it is only 17 lines total.

function Element(node) {
  this.nodeName = node;
  this.dataset = {};
  this.children = [];
  this.appendChild = (el) => { this.children.push(el);},
  this.removeChild = (el) => { this.children.splice(this.children.indexOf(el), 1)},
  Object.defineProperties(this, {
    'firstChild': {
      'get': () => { return this.children[0]}
    }
  });
}

module.exports.createElement = (node) => {
  return new Element(node);
}

This was easily done because we kept each function simple and we could then compile what we need the mock object to do. Now here is some code to test our IO functions. The document mock object will need to be used here. We are testing if this function will remove all children and then add our elements back.

const assert = require('assert');
const {render, setAttribute} = require('../src/io_functions.js');
const document = require('./document_mock.js');

describe('render test', () => {
  it('should remove all current elements', (done) => {
    let root = document.createElement('root');
    root.children = [1,2,3,4,5];
    render(root, []).run();
    assert.equal(root.children.length, 0);
    done();
  });

  it('should add new elements in', (done) => {
    let root = document.createElement('root');
    root.children = [1,2,3,4,5];
    render(root, [1,2,3]).run();
    assert.equal(root.children.length, 3);
    assert.equal(root.children[0], 1);
    assert.equal(root.children[1], 2);
    assert.equal(root.children[2], 3);
    done();
  });
})

We can now trust that this code will execute the correct methods on the real document object when it is used in the browser. Now let’s look at using it to create specific elements.

We have two functions that create elements to go in the DOM, spanMap and inputMap. To test these functions we need to pass in the document mock object and then inspect some of the properties. These are not wrapped in an IO monad because it is a simple map. One object to another. Creating an element is a pure function and adding them to the DOM is an impure function. Here is the code.

const assert = require('assert');
const document = require('./document_mock.js');
const {spanMap, inputMap} = require('../src/dom_element_map_functions.js');
const getTerms = require('../src/text_functions.js').getTerms;

let terms = getTerms(" dog eats ");
terms[0].Index = 0;
terms[1].Index = 1;
terms[1].MadLib = true;

describe('spanMap test', () => {
  it('should create spans based on term', (done) => {
    let span = spanMap(document, terms[0]);
    let madSpan = spanMap(document, terms[1]);

    assert.equal(span.nodeName, 'span');
    assert.equal(span.className, 'Noun');
    assert.equal(span.dataset.index, 0);
    assert.equal(span.title, 'Noun');
    assert.equal(span.innerHTML, ' dog ');

    assert.equal(madSpan.className, 'Verb PresentTense MadLib');
    assert.equal(madSpan.dataset.index, 1);
    assert.equal(madSpan.title, 'Verb PresentTense');
    assert.equal(madSpan.innerHTML, 'eats ');
    done();
  });
});

First, we create the element, then look at the properties. I want to highlight the fact that we have tested DOM manipulation and element creation without using a headless browser like PhantomJS. The testing was accomplished with a 17 line mock. I make this point because many times I see way too much mocking because of bad design.

End to End Testing

Well, I don’t have any end to end testing. The only file that needs this is the final index.js. The tests needed for that would mainly fall in the category of making sure events were wired up correctly.

Two paragraphs ago I railed against over mocking for integration testing, but things are different when discussing end to end testing. We want to create an environment as close to what the real world will be. Now it is fine to pull in headless browsers to do testing. Why is it wrong for integration testing? In short, it is code design. Why not use a headless browser for unit tests? If the code requires this level of mocking, then it can be simplified. Again there are always exceptions, but we should be creating simple and pure functions from the start.

Summary

We have looked all the different methods of testing. I think the most important point is that functional code is easy to test. The tenets of functional design fit with testing perfectly. Start off with simple functions that can be easily unit tested. Then combine those functions with higher order functions. The results of the combination can then be used in integration testing. The final step is to test everything assembled with end to end testing. It is easy to know what and how to test everything this way.

I found a functional way to build Express’s middleware

Using functional design is the new cool thing. This is especially true when talking about JavaScript. Functional programming can make complex code much simpler and much shorter. I am going to highlight some code that I wrote for my Packt Publishing video course, The Complete Guide to Node.js.

I want to note first that this is not a perfect apples to apples comparison. It is also not a judgment on the code written for Express. Express was not built to be functional and cannot be faulted for not using functional ideas. I just want to highlight a functional way of accomplishing a similar task.

The code

We are going to specifically look at middleware in Express. We will focus on the implementation of next. Middleware accomplishes the task of taking a bunch of functions that need to run on a request. Some middleware needs to run on every request and others only on specific requests. This means functions that are not the final item in the chain need to continue the chain. This is done with the next function. If you have not used Express middleware I recommend Express’s documentation.

Let’s look at how Express does this on Github. The code we will look at is in lib/router/index.js. We will look at the handle function and focus at the code starting at line 178. Again I will highlight that the Express version does more than just handle the middleware stack.

Here we can see that there are 12 ifs in the while as it loops over each item in the middleware stack. The good thing is that these ifs are not nested, if they were it would be crazy. This makes it a little difficult to really grok what is happening quickly. Go ahead, jump in and see how long it takes you to determine what is happening.

My code

This is not production ready code but it shows a different way of approaching the same problem. Let’s look at the code first and then discuss what is happening.

const url = require('url');
var routes = [];
var registerRoute = (method, url, fn) => {
  routes.push({method: method,
  url: url,
  fn: fn});
};

var routeMatch = (route, url) => {
  return route === url || route === undefined;
};

var methodMatch = (routeMethod, method) => {
  return routeMethod === method || routeMethod === undefined;
};

var isError = (fn) => fn.length === 3;
var isNormal = (fn) => fn.length === 2;

var mapToRouteMatch = (reqUrl, reqMethod) => {
  return (route) => {
    return routeMatch(route.url, reqUrl)
    && methodMatch(route.method, reqMethod);
  }
};

var handleRequest = (req, res) => {
  var matchedRoutes = routes
  .filter((route) => isNormal(route.fn))
  .filter(mapToRouteMatch(url.parse(req.url).pathname, req.method));
  try{
    matchedRoutes.some((route) => route.fn(req, res));
  }catch(e){
    let errorRoutes = routes
    .filter((route) => isError(route.fn))
    .filter(mapToRouteMatch(url.parse(req.url).pathname, req.method));
    errorRoutes.some((route) => route.fn(req, res, e));
  }
};

module.exports.registerRoute = registerRoute;
module.exports.handleRequest = handleRequest;

This is 42 lines that completely implements a routing middleware much like Express. The routes array and registerRoute are the core data of this. In fact, if you wanted you could change registerRoute to use to make it, even more, like Express.

There are then four filtering functions (routeMatch, methodMatch, isError, and isNormal). They are all one line of code so I won’t spend time discussing them. Next there is a higher order function mapToRouteMatch. This takes a URL and a method and then combines the return of the matching functions. This allows us to make routes that match both/either the method and/or the URL. This gives us the flexibility to run a piece of middleware for every request or just one.

One quick aside, the function mapToRouteMatch is really just a partially applied function. The function partially applies the URL and method which returns a new function that will then expect each route. Which the function will get when mapped over the array.

Finally we get to the core handling, handleRequest. Thinking functionally, there is a clear way to get which pieces of middleware to run on this request, filter! We already have functions that can filter down the array to just the functions that have 2 parameters (req and res) and match the current URL and method. After that we just run some over the array. some will continue over each item until one of them returns true. This is perfect if any function is the final function in the chain, just return true.

This is wrapped in a try/catch. If there is an error we catch it and then find each error middleware and execute it with the error. Let’s see how to actually use this now.

Here is a simple server with two endpoints and six total middleware.

const http = require('http'),
      routes = require('./routes.js');

//actual responses
var log = (req, res) => {
  console.log(`${req.method} ${req.url}`);
  return false;
};

var poweredBy = (req, res) => {
  res.setHeader('X-Powered-By', 'ejosh.co/de');
  return false;
};

var index = (req, res) => {
  res.write("<html><head><title>Page" +
"</title><head><body><h1>Our Web Application</h1>" +
"</body></html>");
  res.end();
  return true;
};

var createError = (req, res) => {
  throw new Error('this will always throw');
  return false;
}

var defaultRoute = (req, res) => {
  res.end();
  return true;
};

var errorRoute = (req, res, err) => {
  res.write(err.message);
  res.end();
  return true;
}

routes.registerRoute(undefined, undefined, log);
routes.registerRoute(undefined, undefined, poweredBy);
routes.registerRoute('GET', '/', index);
routes.registerRoute('GET', '/error', createError);
routes.registerRoute(undefined, undefined, errorRoute);
routes.registerRoute(undefined, undefined, defaultRoute);

var server = http.createServer();
server.on('request', routes.handleRequest);
server.listen(8081, '127.0.0.1');

We are using the built-in HTTP server and including the code we just looked at as routes.js. The next six functions should look really familiar if you have ever built an Express middleware. The main difference is that there is no next function. Just return false to continue processing and true to stop processing.

Next is the section where the routes are registered. This is different than Express as it is much more explicit. Every call has to pass all three parameters; method, URL, and function. Passing in undefined means that it will match for every request. Remember order matters.

Finally, it is wired up by starting the HTTP server and setting handleRequest to handle the requests.

As you can see we now have a functioning Express-like router and application in just 90 lines of code. The router does not have all of the features of Express, but hopefully, it is clear where the features can easily be added. For example regular expressions could be added as another function and used in routeMatch. Ultimately the main advantage is that we have simplified finding and running the correct middleware down to two filters and then a map (some is essentially a map). This allows us to use very simple functions for the actual logic portion.

Build Complex Express Sites with Redis and Socket.io

I have another video series ready from Packt Publishing named Build Complex Express Sites with Redis and Socket.io. This course builds on the foundation from the other course I built for Packt Publishing earlier this year, The Complete Guide to Node.js.

It starts off by covering what and why of Redis. Then goes to building Express sites using Redis. After that, the course jumps to using Socket.io to build real-time sites. Just like Redis the course begins simple and covers what Socket.io is. The course finishes up building sites in Express utilizing Socket.io.

The Complete Guide to Node.js Video Series

I have just recently finished creating a video series for Packt Publishing named The Complete Guide to Node.js. The course focuses on taking someone completely new to Node.js and giving them a great foundation to start building real applications. There is a good mix of theory and practical application as the viewer goes through the course. First, an idea is introduced and explained. Then the idea is applied.

While I was waiting on Packt to send me some finalized videos as a preview, I received a twitter endorsement for the course.

Enough talking about the course, here are two videos that demonstrate the content of the course. Enjoy!