Dave's page

Programming remote services with ZeroMQ in Node.js

Introduction to network programming with ZeroMQ

ZeroMQ is a messaging and network communications library. For more information on ZeroMQ, see:

Why would we want to use ZeroMQ? There are several reasons:

  • By constructing our application from multiple processes that communicate via ZeroMQ, we gain the ability to make use of multi-CPU and multi-core devices (even when those processes run on a single machine.
  • We can make use of a variety of devices, for example, desktop machines, laptop machines, single board and embedded computers (such as the Raspberry Pi), and set-top boxes.
  • ZeroMQ has support for Lua, Python, Node.js and a number of other languages, too, which means that I can write my code in one of my favorite languages.

While the example code files in this article are written in Node.js, they could have been written in other languages for which there is ZeroMQ support. You can find some of my experiments with using Lua and Python here - microservices-zeromq.zip.

An alternative -- Erlang or Elixir are also interesting and powerful choices for constructing applications out of multiple communicating processes. And, if fact, Erlang is available on the Raspberry Pi under Raspian GNU/Linux.

Setup for ZeroMQ and Node.js

Installation requirements:

  1. You will need to install Node.js.

  2. Install ZeroMQ support for Node.js, or whatever language you prefer to work with. In the case of Node.js, you can do the following:

    $ npm install zeromq

In these examples I'll be using the following machines:

  • quail - The local machine
  • reifywork.com - a remote machine on the WAN (wide area network)
  • magpie - a remote machine on my LAN (local area network)

We'll need access to each remote machine, and back, via SSH (secure shell), in both directions, I believe. And, we'll want that access without manually entering a password. You can use ssh-keygen to generate authentication keys for ssh. You can do a Web search for how-to information on how to generate and install the authentication key on the remote machine.

We'll need to use an SSH tunnel in order to communicate securely with a remote machine on the network. I used the following script to setup and enable that:

#!/usr/bin/bash -x

# Setup an SSH tunnel for reifywork.com.
ssh -f ec2-user@reifywork.com -L 5556:localhost:5556 -N

# Setup an SSH tunnel for magpie.
ssh -f dkuhlman@magpie -L 5557:localhost:5557 -N

The above script enables use to us the following ports to make requests and receive responses to those requests:

  • Port 5556 to make ZeroMQ requests to a process running on remote machine at reifywork.com.
  • Port 5557 to make ZeroMQ requests to a process running on remote machine at named magpie that is on my LAN.

And, we will use port 5555 to make ZeroMQ requests to a process running on the local machine. No tunnel is needed for that.

Here is the Node.js server implementation that we'll run on the machine at reifywork.com:

#!/usr/bin/env node

// Hello World server
// Binds REP socket to tcp://*:5556
// Expects "Hello" from client, replies with "world"

let zmq = require('zeromq');
let count = 0;

async function runServer() {
  const sock = new zmq.Reply();

  await sock.bind('tcp://*:5556');

  for await (const [msg] of sock) {
    count++;
    console.log(count + '. Received ' + ': [' + msg.toString() + ']');
    await sock.send('World');
    // Do some 'work'
  }
}

runServer();

Here is the Node.js server implementation that we'll run on magpie connected to my LAN:

#!/usr/bin/env node

// Hello World server
// Binds REP socket to tcp://*:5555
// Expects "Hello" from client, replies with "world"

//  Hello World server
//  Binds REP socket to tcp://*:5555
//  Expects "Hello" from client, replies with "World"

let zmq = require('zeromq');
let count = 0;

async function runServer() {
  const sock = new zmq.Reply();

  await sock.bind('tcp://*:5557');

  for await (const [msg] of sock) {
    count++;
    console.log(count + '. Received ' + ': [' + msg.toString() + ']');
    await sock.send('World');
    // Do some 'work'
  }
}

runServer();

If you look carefully, you will see that the only difference between these two implementations is that one uses port 5556 and the other uses port 5557.

Here is the Node.js client that we'll use on the local machine to make ZeroMQ requests to the ZeroMQ server process running at reifywork.com:

#!/usr/bin/env node

// Hello World client
// Connects REQ socket to tcp://localhost:5555
// Sends "Hello" to server.

//  Hello World client
const zmq = require('zeromq');     ■ File is a CommonJS module; it may be converted to an ES module.
const argv = process.argv;     ■ NodeJS process global is discouraged in Deno Add `import process from "node:process";`

let max = 10;
if (typeof argv[2] != 'undefined') {
  max = Number(argv[2]);
}

async function runClient() {
  console.log('Connecting to hello world server…');

  //  Socket to talk to server
  const sock = new zmq.Request();
  sock.connect('tcp://localhost:5556');

  for (let i = 0; i < max; i++) {
    console.log('Sending Hello ', i);
    await sock.send('Hello');
    const [result] = await sock.receive();
    console.log('Received ', result.toString(), i);
  }
}

runClient();

And, here is the Node.js client that we'll use on the local machine to make ZeroMQ requests to the ZeroMQ server process running on magpie (connected on my LAN):

#!/usr/bin/env node

// Hello World client
// Connects REQ socket to tcp://localhost:5555
// Sends "Hello" to server.

//  Hello World client
const zmq = require('zeromq');     ■ File is a CommonJS module; it may be converted to an ES module.
const argv = process.argv;     ■ NodeJS process global is discouraged in Deno Add `import process from "node:process";`

let max = 10;
if (typeof argv[2] != 'undefined') {
  max = Number(argv[2]);
}

async function runClient() {
  console.log('Connecting to hello world server…');

  //  Socket to talk to server
  const sock = new zmq.Request();
  sock.connect('tcp://localhost:5557');

  for (let i = 0; i < max; i++) {
    console.log('Sending Hello ', i);
    await sock.send('Hello');
    const [result] = await sock.receive();
    console.log('Received ', result.toString(), i);
  }
}

runClient();

Again, you can see, if you look carefully, that the only difference betwee these two clients is that the fire uses port 5556 and the second uses port 5557.

Installing and using ZeroMQ on the Raspberry Pi

Installing ZeroMQ on the Raspberry Pi is a bit more involved, but turned out to be easy, at least in my case. Just follow the instructions here - https://github.com/anuragvohraec/ZeroMQ-RPi/ or here (it's the same) https://github.com/MonsieurV/ZeroMQ-RPi.

Published

Category

network

Tags

Contact