It is about time that we talk about the HTTP module in Node.js. We introduced the Event Emitter and the Stream module, and understanding them will play a major factor in understanding the HTTP transactions. Thus, it is recommended to have some familiarity with those modules to grasp the idea of the HTTP transactions.

We are going to utilize the http module to build a simple web server, and understand how the request and the response are handled.

HTTPS is a seperate module in Node.js that handles the HTTP transactions over TLS/SSL, and HTTP2 is an experimental module which provides an implementation of the HTTP/2 protocol.

Creating a Server

To create a web server in Node.js, we utilize the createServer method on the http module. The createServer method takes a function; known as the requestListener which is called for each time there is a request made to the server and returns a new instance of the http.Server which is an EventEmitter.

const http = require('http');

const server = http.createServer((request, response) => {
    // Handling request and response here.
});

The request and response that are passed to the requestHandler function are the objects which will help us deal with the transaction.

The Server that is returned from the http.createServer does not explicitly look like an EventEmitter, and that is because the createServer will take care of that behind the scene. We can refactor the previous bit of code to show how precisely the server object is created and how has the listener been added.

const http = require('http');

const server = http.createServer();

server.on('request', (request, response) => {
    // Handling request and response here.
});

The server is listening to the request event which will be triggered/emitted when a request hits the server.

listening to Requests

To serve requests, the listen method needs to be called on the previously created server object. The listen method takes four params, and all are optional; port, hostname, backlog, callback.

listen(port, hostname, backlog, callback);

The port is the port number, and if it is omitted, or the value 0 was used, the operating system will assign a random port to the server. Most likely, we would need to know the port number in order to connect to the server. Therefore, we can use server.address().port after the listening event has been emitted to retrieve the port number.

const server = http.createServer();

server.on('request', (request, response) => {
    // Handling request and response here.
});

server.listen();

server.on('listening', function () {
    console.log('Port number:', server.address().port);
});

If we execute the previous piece of code, we would have the port number the operating system has assigned to the web server.

If the hostname was omitted, the connection would be accepted on the unspecified IPv6 address (::) when IPv6 is available, or the unspecified IPv4 address (0.0.0.0) otherwise.

In most operating systems, listening to the unspecified IPv6 address (::) may cause the net.Server to also listen on the unspecified IPv4 address (0.0.0.0).

backlog is the maximum length of the queue of pending connections. The actual length will be determined by the OS through sysctl settings such as tcp_max_syn_backlog and somaxconn on Linux. The default value of this parameter is 511 (not 512).

callback is an async function that will be called after the server has run.

The listen method may be called multiple times. Each subsequent call will re-open the server using the provided options.

In most cases, we pass the port number that we require and know is not occupied to the listen method.

const server = http.createServer();

const port = 3000;

server.on('request', (request, response) => {
    // Handling request and response here.
});

server.listen(port, function(){
    console.log(`The server is running on port: ${port}`);
});

If we run this and send a request to the server on http://localhost:3000, we would have the request hit the time out. That is because we are not responding to the client yet. We will take a look at how to do that when we discuss the response object on the requestHandler function. But first, let's see what the request object on that function brings to the table.

The request Object

The request object on the requestHandler function is an instance of the http.IncomingMessage which is created by http.Server. The http.IncomingMessage implements the ReadableStream interface, and has events, methods, and properties that will help us process the HTTP request.

The first things that we look at when handling a new HTTP request are the method, the URL, and the headers.

const http = require('http');

const server = http.createServer();

const port = 3000;

server.on('request', (request, response) => {
    console.log('Method:', request.method);
    console.log('URL:', request.url);
    console.log('Headers: %o', request.headers);
});

server.listen(port, function(){
    console.log(`The server is running on port: ${port}`);
});

The output would be something like the following.

Method: GET
URL: /
Headers: { host: 'localhost:3000',
connection: 'keep-alive',
'cache-control': 'max-age=0',
'upgrade-insecure-requests': '1',
'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36',
accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'en-US,en;q=0.8' }

The request.method is a typical HTTP verb such as GET, POST, PUT, PATCH and DELETE. The request.url is the full URL after the protocol, the hostname, and the port. The request.headers is an object of the HTTP request headers in lower-case (notice the output) regardless of how the client actually sent them. This could be problematic, especially that the behavior of sending repeated headers is that they either get overwritten or joined as comma-separated strings. Therefore, the rawHeaders could be utilized as it is the raw headers list exactly as they received. The console.log(request.rawHeaders) would output something like the following.

[ 'Host',
'localhost:3000',
'Connection',
'keep-alive',
'Cache-Control',
'max-age=0',
'User-Agent',
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36',
'Upgrade-Insecure-Requests',
'1',
'Accept',
'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
'Accept-Encoding',
'gzip, deflate, br',
'Accept-Language',
'en-US,en;q=0.8' ]

POST and PUT Request Body

POST or PUT request body that is received and handled using the request object, but is not straight out processable like the headers, the URL, and the method. As mentioned before, the request object is an instance of the http.IncomingMessage which itself ReadableStream interface, which means that the request object could be treated as a stream. And like any stream, we can grab the data right out of the stream by listening to the stream's 'data' and 'end' events.

Each chunk emitted in each 'data' event is a Buffer.

Prior to the introduction of TypedArray in ECMAScript 2015 (ES6), the JavaScript language had no mechanism for reading or manipulating streams of binary data. The Buffer class was introduced as part of the Node.js API to make it possible to interact with octet streams in the context of things like TCP streams and file system operations. - Node.js Docs.

Let's handle the data that is streamed through the request object in the following piece of code.

const http = require('http');

const server = http.createServer();

const port = 3000;

server.on('request', (request, response) => {
    console.log('Method:', request.method);
    console.log('URL:', request.url);
    console.log('Headers: %o', request.headers);

    let data = [];    

    request.on('data', (chunk) => {
        data.push(chunk);
    });
    
    request.on('end', () => {
        data = Buffer.concat(data).toString();
        // at this point, `data` has the entire request body stored in it as a string
    });

});

server.listen(port, function(){
    console.log(`The server is running on port: ${port}`);
});

Supposedly, we know that the data that is streamed is string data. Therefore we are collecting it in an array, then at the 'end', concatenating and stringing it.

In real life, we would not concatenate the body in this way especially if we are receiving data as JSON or HTML/XML, instead we would use an external npm module, or we would have to write and utilize a module for that reason in case we have some special needs. Some of the existing modules that do the job for us out of the box: concat-stream and body.

A good practice, and somewhat mandatory, is to handle errors at least at a simple level. And as the request object is a ReadableStream and an EventEmitter an error presents itself by emitting an 'error' event on the stream. However, if we don't have a listener for that event, the error will be thrown, which could crash your Node.js program. Let's handle that by simply logging the error to the console.

There are better ways of handling errors and those should be utilized. However, for the sake of simplicity, we would stick to this currently.

const http = require('http');

const server = http.createServer();

const port = 3000;

server.on('request', (request, response) => {
    console.log('Method:', request.method);
    console.log('URL:', request.url);
    console.log('Headers: %o', request.headers);

    let data = [];    
    
    request.on('error', (err) => {
        // This prints the error message and stack trace to `stderr`.
        console.error(err.stack);
    });
    
    request.on('data', (chunk) => {
        data.push(chunk);
    });

    request.on('end', () => {
        data = Buffer.concat(data).toString();
        // at this point, `data` has the entire request body stored in it as a string
    });

});

server.listen(port, function(){
    console.log(`The server is running on port: ${port}`);
});

There are more properties and methods to the http.IncomingMessage object that we did not cover here such as httpVersion, trailers and rawTrailers. It is recommended to read through those.

Up till now, we were handling the request object and did not discuss the response object at all. So let's Do that.

The response Object

The response object is an instance of ServerResponse, which implements is the Writable Stream interface.

The response implements, but does not inherit from, the Writable Stream interface.

Response Status Code and Headers

We can set up the response status code using response.statusCode property like the following.

response.statusCode = 400;

If we did not specify the status code of the response, the default would 200.

We can also set the headers of the response using the response.setHeader() method. That method takes two parameters, the first is the name of the header, and the second is the value of that header. If this header already exists in the to-be-sent headers, its value will be replaced. However, we can use an array of strings here to send multiple headers with the same name.

// Sat the 'Content-Type' to 'text/html'
response.setHeader('Content-Type', 'text/html');

// This will overwrite the 'Content-Type' to be 'text/html'
response.setHeader('Content-Type', 'application/json');

// Multiple Values
response.setHeader('Set-Cookie', ['type=ninja', 'language=javascript']);

Another way to send headers would be using response.writeHead() method, which will explicitly write the status and the headers to the response stream without relying on node to send the headers before the sending the body data.

response.writeHead(200, {
  'Content-Type': 'application/json'
});

Response Body

As mentioned before, the response object is a WritableStream. Therefore, we can use the same methods to write the response body.

response.write('Hello World!');
response.end();

// or just simply
response.end('Hello World!');

Just like the request object, the response will emit error events, and we would need to deal with that as well.

So putting everything together we would end up with something like the following.

const http = require('http');

const server = http.createServer();

const port = 3000;

server.on('request', (request, response) => {
    console.log('Method:', request.method);
    console.log('URL:', request.url);
    console.log('Headers: %o', request.headers);

    let data = [];    
    
    request.on('error', (err) => {
        // This prints the error message and stack trace to `stderr`.
        console.error(err.stack);
    });
    
    request.on('data', (chunk) => {
        data.push(chunk);
    });

    request.on('end', () => {
        data = Buffer.concat(data).toString();
        // at this point, `data` has the entire request body stored in it as a string

        response.on('error', (err) => {
            console.error(err);
        });

        response.statusCode = 200;
        response.setHeader('Content-Type', 'application/json');

        const responseBody = {data};
        
        response.end(JSON.stringify(responseBody));
    });

});

server.listen(port, function(){
    console.log(`The server is running on port: ${port}`);
});

Let's intact the example above more. First thing, we can use ES2015's destructuring assignment on the request object to grab the url, the method, and headers.

const http = require('http');

const server = http.createServer();

const port = 3000;

server.on('request', (request, response) => {
    const {method, url, headers} = request;

    console.log('Method:', method);
    console.log('URL:', url);
    console.log('Headers: %o', headers);

    let data = [];    
    
    request.on('error', (err) => {
        // This prints the error message and stack trace to `stderr`.
        console.error(err.stack);
    });
    
    request.on('data', (chunk) => {
        data.push(chunk);
    });

    request.on('end', () => {
        data = Buffer.concat(data).toString();
        // at this point, `data` has the entire request body stored in it as a string

        response.on('error', (err) => {
            console.error(err);
        });

        response.statusCode = 200;
        response.setHeader('Content-Type', 'application/json');

        const responseBody = {data};
        
        response.end(JSON.stringify(responseBody));
    });

});

server.listen(port, function(){
    console.log(`The server is running on port: ${port}`);
});

Here we go. Then we can chain the listeners on the request object. And let's send those as well in the response using ES2015's object enhancement.

const http = require('http');

const server = http.createServer();

const port = 3000;

server.on('request', (request, response) => {
    const {method, url, headers} = request;

    console.log('Method:', method);
    console.log('URL:', url);
    console.log('Headers: %o', headers);

    let data = [];    
    
    request.on('error', (err) => {
        console.error(err.stack);
    }).on('data', (chunk) => {
        data.push(chunk);
    }).on('end', () => {
        data = Buffer.concat(data).toString();

        response.on('error', (err) => {
            console.error(err);
        });

        response.statusCode = 200;
        response.setHeader('Content-Type', 'application/json');

        const responseBody = {method, url, headers, data};
        
        response.end(JSON.stringify(responseBody));
    });

}).listen(port, function(){
    console.log(`The server is running on port: ${port}`);
});

Simple Endpoint

Let's modify the previous example so we can only send requests to a specific endpoint and accept only POST methods.

const http = require('http');

const server = http.createServer();

const port = 3000;

server.on('request', (request, response) => {
    const {method, url, headers} = request;

    if (method == 'POST' && url == '/') {
        console.log('Method:', method);
        console.log('URL:', url);
        console.log('Headers: %o', headers);

        let data = [];    
        
        request.on('error', (err) => {
            console.error(err.stack);
        }).on('data', (chunk) => {
            data.push(chunk);
        }).on('end', () => {
            data = Buffer.concat(data).toString();

            response.on('error', (err) => {
                console.error(err);
            });

            response.statusCode = 200;
            response.setHeader('Content-Type', 'application/json');

            const responseBody = {method, url, headers, data};
            
            response.end(JSON.stringify(responseBody));
        });
    } else {
        response.statusCode = 404;
        response.end();
    }

}).listen(port, function(){
    console.log(`The server is running on port: ${port}`);
});

We just added an if statement there that check the method and URL.

This is a plain, simple routing functionality. We can also use a switch statement for more routes. However, if we need more complex routing, we probably can use some package on npm such as router which is a middleware-style router or a full-blown framework like express which will touch on in later posts.

Handling request and response Errors

We have discussed the need for handling errors more gracefully for both the request and response, so let's do that now.

const http = require('http');

const server = http.createServer();

const port = 3000;

server.on('request', (request, response) => {
    request.on('error', (err) => {
        console.error(err);
        response.statusCode = 400;
        response.end();
    });

    response.on('error', (err) => {
        console.error(err);
    });
    
    const {method, url, headers} = request;

    if (method == 'POST' && url == '/') {
        console.log('Method:', method);
        console.log('URL:', url);
        console.log('Headers: %o', headers);

        let data = [];    
        
        request.on('data', (chunk) => {
            data.push(chunk);
        }).on('end', () => {
            data = Buffer.concat(data).toString();

            response.on('error', (err) => {
                console.error(err);
            });

            response.statusCode = 200;
            response.setHeader('Content-Type', 'application/json');

            const responseBody = {method, url, headers, data};
            
            response.end(JSON.stringify(responseBody));
        });
    } else {
        response.statusCode = 404;
        response.end();
    }

}).listen(port, function(){
    console.log(`The server is running on port: ${port}`);
});

When the request face an error, it will log that error to the console as it did before, and then set the status code of the response to 400 and end the response. This way, at least the client will know that it was a Bad request (the associated HTTP status name for the 400 HTTP status code).

Conclusion

That covered the most basic HTTP requests handling in Node.js. We learned how to:

  • Instantiate an HTTP server with a request handler function, and have it listen on a port.
  • Get headers, URL, method and body data from request objects.
  • Make routing decisions based on URL and/or other data in request objects.
  • Send headers, HTTP status codes and body data via response objects.
  • Handle stream errors in both the request and response streams.

It is highly recommended that you read through the Http module.