Software Development and Engineering

A software engineer needs a place to discuss software development, programming languages, engineering methodology, etc.

Project: API Web Server

I am currently building a simple Node.js based API web server. It’s primarily a proof-of-concept project that will be used when developing the API for another project. I have written similar servers for other projects in the past, this one should be cleaner and easier to use.

At the core of the server is the node http2 module. This provides the HTTP/2 web server, plus fall back to HTTP/1.1 support. HTTP/2 is the latest version of the hypertext transfer protocol and supports multiplexing data transfer connections, allowing multiple concurrent data streams. HTTP/1.1 serializes data transfer, meaning one stream has finish before another can begin. Below is the code that instantiates the http2 server

//Options for the http2 server
const httpServerOptions = {
        key: fs.readFileSync(path.join(__dirname, 'keypath/key.pem')),
        cert: fs.readFileSync(path.join(__dirname, 'keypath/cert.pem'))
};

var http2server;
if (process.env.HTTP2_PLAIN) {
    console.log('Setting up HTTP2 Plain');
    http2server = http2.raw.createServer({}, processRequest);
} else {
    console.log('Setting up HTTP2 TLS');
    http2server = http2.createServer(httpServerOptions, processRequest);
}

processRequest is the callback function that handles the server requests, it takes two objects as parameters, request and response.

function processRequest(request, response) {
//process the request and provide a response ...

Since this is a simple server, it does not need to support many options. My current server implementation supports the following requirements:

  • Passes GET and POST requests to the API endpoints
  • API endpoints as modules
  • Dynamically load modules during development
  • Transfers html, js, css, and txt, files with correct mime types
    • The allows it to be used as a basic file web server
  • Display 404 when file or endpoint cannot be loaded
  • Error logging

This is not a complete web server, but it could eventually be. A complete web server would support multiple HTTP response codes, have a plugin infrastructure, support large numbers of mime types, access logging, and numerous other features. At this point it is good for development and is easy to expand.

Since I have written similar servers in the past I looked at my previous code bases, it wasn’t pretty. I ended up finding duplicated code and signs of obvious confusion. When I coded the new server I made some similar mistakes, but upon reviewing the code I was able to move duplicated code into dedicated functions, then reworked the flow of the requests so that the main method was simpler. In the past I over complicated functions, stuffed too many actions into them. Simplify the methods increase the number of overall methods, but produced code that is easier to read and debug.

I added dedicated methods for displaying 404 messages, writing content and its mime type back to the requester, and wrote a method dedicated to handling POST and GET requests.

function processRequestMethod (request) {
  if (request.method === 'POST') {
    var data='';
    request.on('data', function(chunk) {
      data += chunk;
    });
    request.on('end', function() {
      return qs.parse(data);
    });
    request.on('error', function(e) {
      console.error('ERROR with POST request '+e.message);
    });
  } else {
  //Process as GET
    return qs.parse(url.parse(request.url).query);
  }
}

My Next goal was to dynamically load API modules, to be continued in part 2.

I support code reviews and look forward to them. I would expect other professional software engineers to embrace code reviews. They have many uses, one of the most useful to me is identifying mistakes in code changes. I’ve seen many situations where even a quick informal code review would have caught bugs that would be easily fixed. A user familiar with a code base will look over lines of code and not notice some obvious mistake, it’s part of the human condition. It’s a common issue that writers also have, it’s why editors and peer reviews are crucial.

The argument against code review is ‘does anyone actually perform the review?’ because ‘they take up a large chunk of time’. There are jobs where developers are pushed to pump out code quickly. Based on my limited research and experience this leads to low quality code. High quality code requires either a top notch developer who never makes mistakes, or peer review. QA doesn’t perform code reviews, they just test the input and output. Only other software developers can accurately peer review another’s piece of code. Good companies make time for their employees to perform code reviews, better product makes for better profit.

Some developers are afraid of code reviews because they’re embarrassed about their code. I remember being extremely self conscious about my code only six years ago. Since then I have performed numerous peer reviews and review numerous other pieces of code. I got to see first hand the code that worked well versus code that would cause issues. It also encouraged me to increase my code’s readability and it is why I started using doxygen to generate technical documents. This eased the code reviews for my peers, I was optimizing code for performance and reliability which significantly increased code complexity.

The key to proper code review implementation is a proper code review process, a good discussion for another time.