Category Archives: NodeJS

primary-node

Primary node election in distributed computing

Primary node can help with coordination of multiple node deployed in a cloud. One of the projects I’ve worked on had a NodeJS worker that ran multiple types of tasks. I wanted to upgrade this setup in order to be easily scalable, have a primary node (or coordinator) that triggers the tasks and continue processing even when some of the nodes fail.

The Checklist

  • All nodes may participate in an “election” to choose the coordinator
  • Support any number of nodes, including 1 node setup
  • Handle node fail (even the coordinator node) without impacting the flow
  • Allow new nodes to join the party
  • Don’t depend on an expensive technology (paid or resource hungry)

Besides the main scope of the solution I also needed to ensure that the election of the coordinator follows these 3 basic principles:

  • Termination: the election process must complete in a finite time
  • Uniqueness: only one node can be coordinator
  • Agreement: all other nodes know who’s the coordinator.

After I established that I have all the scenarios in mind, I started investigating different algorithms and solutions that the market uses (Apache Zookeeper, Port Locking, Ring Networks). However, most of these require a lot of setup or were incompatible in a multi server setup and I also wanted to embrace a KISS approach so continue reading to see the solution.

The Primary Node Election Algorithm

  1. Node generates a random numeric id
  2. Node retrieves a COORDINATOR_ID key from Redis
  3. If key !NULL
    • We have a coordinator
    • Wait Z minutes (e.g. Z = 1 hour)
    • GoTo Step2
  4. If key NULL
    • No coordinator announce
    • Push the id from Step1 in a Redis list
    • Waits X seconds (depending on how long the deployment takes, e.g. 10 seconds)
    • Retrieve all items in the list and extract the highest number
    • If result === node id
      • Current node is primary
      • Set Redis key COORDINATOR_ID with expiry Z+X
      • Do all the hard work 🙂
    • Wait Z minutes
    • GoTo Step2

Downside of this solution is that if the coordinator node fails, it actually takes 2*Z until a new election takes place.

There’s room for improvement so please don’t hesitate to leave a feedback 🙂

OSX Global NPM Module command not found

In case you ended up in a situation where you just installed a global NPM module, but it still throws command not found, here’s what you have to do:

Find out where the global NPM modules are installed by running:

npm prefix -g

Double check that your $PATH does not already contain that value:

echo $PATH

If the value is not included, you must update your etc/paths with the NPM location:

sudo vi /etc/paths

Add the value returned by npm prefix -g preceded by /bin

e.g. /Users/catalinmunteanu/.npm-global/bin

Save the file and exit.

Open a new terminal tab/window and retry the command.

Cheers!

Node n permission denied without sudo

Each time I do a fresh install of the Node Version Management tj/n I end up getting the permission denied when running npm install.

If you also ran into this issue, well, there’s a quick fix.

The issue is caused by the Node Version Management for which the owner is the root.

The two following steps will help you continue in peace 😀

which n

Returns the install location of the Node Version Management package. e.g. /Users/username/n

sudo chown -R $(whoami) <PATH_WHICH_N>

Sets the current user as owner.

You can now install the NPM packages without the power of sudo.

How to send signal to NodeJS process running in Linux

There might be times when you want to enable debugging or just change some NodeJS configuration without restarting the process. Scroll down to learn how to send signals to NodeJS process running on Linux.

linux signal nodejs

One extremely easy way of doing this is to send a signal to the running Node pid.

You can get the process pid running:

lsof i :<PORT>

This lsof command works if your process is running in a Docker container as well.

We can send the linux signal using:

# kill -s <SIGNAL> <PID>
kill -s SIGUSR2 999

As you can see it’s quite easy to send a signal. Checkout the official docs and find out more about how Node treats signals.

Now that we know how to send the signals we have to handle it in our NodeJS service:

function handle(signal) {
    console.log(`Received ${signal}`)
    // take some action depending on the signal received
}
process.on('<SIGNAL>', handle)

That’s all.

You can now do whatever you want when a signal occurs, without restarting the Node process: reload the configuration, enable debugging, logs and so on.

In one of my apps I used the signal to enable debugging of all HTTP requests.

NPM module command not found

Until you properly config your environment you’ll probably run into the command not found error when trying to run globally installed NPM modules.

Solution for Linux and OSX:

Step 1 – check if the path is correct

You need to have /users/<ME>/.npm-global/bin in your $PATH

echo $PATH

If the printed string doesn’t contain the above path, continue to Step 2

Step 2 – add npm global install path

We need to adjust the $PATH in order to contain the path to the NPM global module folder:

export PATH=$PATH:~/.npm-global/bin

Step 3 – reload .bashrc

The .bashrc file is usually only reloaded on startup. Run the bellow command to reload it again after you made the changes from Step 2.

source ~/.bash_profile

That’s all. Your global NPM modules should now work in your terminal.

How to generate X-WSSE Token using NodeJS

Learn how to generate X-WSSE Token and how to authorize requests using X-WSSE header authentication.

If you’re not familiar with X-WSSE Token Authentication and why you should use it, go ahead and read this article that contains the basics of this type of authentication.

In this article I’ll describe how to generate a X-WSSE Token using NodeJS.

const crypto = require('crypto');
 
function getWsseHeader(username, secret) {
 
    const nonce = crypto.randomBytes(16).toString('hex');
    const created = new Date().toISOString();
    const hash = crypto.createHash('sha256').update(nonce + created + secret, 'utf-8').digest('hex');
    const digest = Buffer.from(hash).toString('base64');
     
    return `UsernameToken Username="${username}", PasswordDigest="${digest}", Nonce="${nonce}", Created="${created}"`;
 
}
 
const xwsse = getWsseHeader('CLIENT_ID', 'CLIENT_SECRET');
console.log(xwsse);

That’s it. Check my other X-WSSE Articles and learn how to generate the token using other programming languages.

What is X-WSSE Token Authentication and how does it work

Learn the basics of X-WSSE Token Authentication and how to authorize requests using X-WSSE header authentication.

X-WSSE Token Authentication can be used to authenticate backend-to-backend integrations using client_id and client_secret properties. The main benefit of this type of authentication is that the client_secret value never leaves the backend of the integrating client, and that each token, even if lost, is only valid for 5 minutes.

The X-WSSE Token is a string with the following format, usually a single HTTP header line which is broke down into multiple lines for easier readability:

X-WSSE: UsernameToken
Username="USERNAME",PasswordDigest="ASDFbasEHAPRo395MBANgoaiERJGJHSOSLGUsernameToken Username="68037425-fa69-49da-8715-fa393dc55471", PasswordDigest="OWRkZGRjMjk3ZjhiOGFhZmMzNGIzMjAwMWIyNmNjY2JkMTM2M2E5OGFlMGM2ZDI3OGIzZmQ5ZDAwY2RiODMzZg==", Nonce="ee2e8c783398782fd63af15141a1cb62", Created="2019-03-14T16:17:24.211Z"==",Nonce="b35f7341829e35d89851497a82894f",Created="2019-03-20T12:10:20Z"

I’ll briefly describe each component of the X-WSSE Token:

X-WSSE

The name of the HTTP header that must be present in order to authorize the request.

UsernameToken

Value represents the authentication method of the use X-WSSE Token. Currently X-WSSE only supports UsernameToken type of authentication.

Username

The client_id property that you should generate for each integration of X-WSSE Token.

PasswordDigest

Field specifies the hashed token that will authorize the reuqest. For each request a new hash must be generated. Check my other posts and learn how to generate the X-WSSE Token using different server-side programming languages

Computing the Password Digest

Computing the password digest involves 5 simple steps:

  1. Generate random 16 byte Nonce formatted as 32 hexadecimal characters.
  2. Retrieve the current timestamp in ISO8601 format.
  3. The properties nonce, timestamp, secret should be concatenated in this order.
  4. Compute the SHA256 hash value of the string from #3 and convert it to hexadecimal format
  5. Encode the value from #5 in BASE64 and obtain the PasswordDigest

Nonce

Random value with the purpose to make your request unique so it cannot be replicated by unknown parties. This string is always 16 bytes long and should be represented as a 32 characters long hexadecimal value.

Created

This field contains the current UTC, GMT, ZULU timestamp (YYYY-MM-DDTHH:MM:SS) according to the ISO8601 format. e.g. 2018-05-20T12:51:45+01:00

Now you know what is a X-WSSE Token and the purpose of each of its components so let’s go to the Implementation. Check my other X-WSSE articles and learn more.


Convert seconds to Days, Hours, Minutes, Seconds

Learn how to convert number of seconds into days, minutes, hours and seconds.


That’s it. You can now create a countdown timer, display the difference between two dates or just print  the time until something great happens.

var seconds = 1001110 // 11 days 14:05:10

// compute the number of days
var days = Math.floor(seconds / (3600*24))
seconds -= days*3600*24

// compute the number of hours
var hours = Math.floor(seconds / 3600)
seconds -= hours*3600

// compute the number of minutes
var minutes = Math.floor(seconds / 60)
seconds -= minutes*60

You now have 4 variables with values as follows:

days === 11
hours === 14
minutes === 5
seconds === 10

Bonus! Learn how to pad the values with 0 so that they look pretty 😀

function str_pad_left (string, pad, length) {
     return (new Array(length+1).join(pad)+string).slice(-length)
}

console.log(str_pad_left(minutes, '0', 2)) // "05"

Check this article to learn how to create a countdown timer.

Socket.IO AngularJS Wrapper

I’ve been playing around with Socket.IO and AngularJS. To make things easier I created this service which can be used for transferring data between the client (which could be a web page) and a NodeJS server.

The app is served by NodeJS so the Socket.IO server is listening on same location.

'use strict';

angular
.module('APP')
.factory('$WS', function($rootScope) {
    var self = this;

    self.ready = false;

    self.socket = io.connect(location.host, {
        'connect timeout':       100,
        'reconnection delay':    100
    });

    self.socket.on('init', function() {
        console.log('init');
    });

    self.socket.on('disconnect', function() {
        console.log('socket disconnected');
    });

    self.socket.on('connect', function() {
        console.log('socket connected');
    });

    return {
        ready: function(callback) {
            if(self.ready) {
                callback();
                return;
            }

            self.socket.on('connect', function() {
                $rootScope.$apply(function() {
                    self.ready = true;
                    callback.apply(self.socket);
                });
            });
        },
        on: function(eventName, callback) {
            self.socket.on(eventName, function() {
                var args = arguments;

                $rootScope.$apply(function() {
                    callback.apply(self.socket, args);
                });
            });
        },
        emit: function(eventName, data, callback) {
            self.socket.emit(eventName, data, function() {
                var args = arguments;
                $rootScope.$apply(function() {
                    if(callback) {
                        callback.apply(self.socket, args);
                    }
                });
            });
        }
    };
});

Usage:

angular
.module('APP')
.controller('TestController', ['$scope', '$WS', function($scope, $WS) {

    $scope.messages = [];

    // Connect websocket
    $WS.ready(function() {
    
        // Receive message
        $WS.on('chat/message', function(data) {
            
            // Validate input data
            if(data.message_id) {
                console.log(data);
                $scope.messages.push(data);
            }
        });
    });

    // You can add another ready state listener for the same connection
    // Let's consider we have a method which sends messages
    $scope.send_message = function(msg_title, msg_value) {
        
        $WS.ready(function() {
            // Emit message
            $WS.emit('chat/message', {
                title:    msg_title,
                message:  mgs_value
            });
        });
    };

}]);

 

This is just basic usage.  Will be back with NodeJS & Socket.IO server example.

NodeJS AES256 Encryption/Decryption

In one of my projects I needed to encrypt some data. I decided to use AES256 symmetrical encryption algorithm.

So this is what I’ve come to (aes_encryption.js):

var crypto	= require('crypto'),
    cipher_seed = 'some_random_characters';
 
var encrypt = function(text) {
	var cipher	= crypto.createCipher('aes-256-cbc', cipher_seed),
	    crypted	= cipher.update(text, 'utf8', 'hex');

	crypted += cipher.final('hex');

	return crypted;
};
 
var decrypt = function(text) {
	var decipher  = crypto.createDecipher('aes-256-cbc', cipher_seed),
	    decrypted = decipher.update(text, 'hex', 'utf8');

	decrypted += decipher.final('utf8');

	return decrypted;
};
 
module.exports.encrypt = encrypt;
module.exports.decrypt = decrypt;

Usage:

var AES = require('path_to_aes_encryption.js');

var data = {
    name:    'Catalin',
    surname: 'Munteanu',
    address: 'Romania'
};

// Encryption
var encrypted_data = AES.encrypt(JSON.stringify(data));

// Decryption
var decrypted_data = JSON.parse(AES.decrypt(encrypted_data));

Don’t forget that if the cipher seed is changed or lost all the encrypted data is also lost.

Easy, right?