ZeroMQ microservices across the network

1   Introduction

I've been experimenting with the implementation of microservices using ZeroMQ. What follows is a few notes on what I've been able to do.

I'll give examples written in both Lua and Python. The use of Lua is a bit interesting because Lua is especially useful and convenient for use as an embedded language.

Note that this code originated from the example code in the zguide - https://zguide.zeromq.org/. You can clone the zguide along with example code in multiple languages from https://github.com/imatix/zguide.git.

For similar examples in Node.js, see http://www.reifywork.com/zeromq-python-nodejs.html.

2   A simple first step - Local machine

Here is a simple server in Python - hwserver.py:

#!/usr/bin/env python

"""
synopsis:
    Hello World server in Python
    Binds REP socket to tcp://*:5555
    Expects b"Hello" from client, replies with b"World"
"""

# import time
import zmq


def receive_requests():
    """Receive the requests."""
    context = zmq.Context()
    socket = context.socket(zmq.REP)
    socket.bind("tcp://*:5555")
    # socket.bind("tcp://10.0.1.2:5555")
    count = -1
    while True:
        count += 1
        #  Wait for next request from client
        message = socket.recv()
        print(f"Received request: {message}")
        #  Do some 'work'
        # time.sleep(1)
        #  Send reply back to client
        socket.send_string(f"World {count}")


def main():
    """Main function"""
    receive_requests()


if __name__ == "__main__":
    main()

And, a very similar server written in Lua - hwserver.lua:

#!/usr/bin/env lua5.3

--  Hello World server
--  Binds REP socket to tcp://*:5555
--  Expects "Hello" from client, replies with "World"
--
--  Author: Robert G. Jakabosky <bobby@sharedrealm.com>
--
local zmq = require"zmq"
require"zhelpers"

local context = zmq.init(1)

--  Socket to talk to clients
local socket = context:socket(zmq.REP)
socket:bind("tcp://*:5555")

while true do
    --  Wait for next request from client
    local request = socket:recv()
    print("Received Hello [" .. request .. "]")

    --  Do some 'work'
                -- s_sleep(1000)

    --  Send reply back to client
    socket:send("World")
end
--  We never get here but if we did, this would be how we end
socket:close()
context:term()

And, here is a simple client written in Python:

#!/usr/bin/env python

"""
synopsis:
    Hello World client in Python
    Connects REQ socket to tcp://localhost:5555
    Sends "Hello" to server, expects "World" back
"""

import sys
import zmq


def send_requests(count):
    """Send the requests."""
    context = zmq.Context()
    #  Socket to talk to server
    print("Connecting to hello world server...")
    socket = context.socket(zmq.REQ)
    socket.connect("tcp://localhost:5555")
    for request in range(count):
        print(f"Sending request {request} ...")
        socket.send_string(f"Hello {request}")
        #  Get the reply.
        message = socket.recv()
        print(f"Received reply {request} [ {message} ]")


def main():
    """Main function"""
    args = sys.argv[1:]
    count = 10
    if len(args) == 1:
        count = int(args[0])
    send_requests(count)


if __name__ == "__main__":
    main()

Next, a very similar client written in Lua:

#!/usr/bin/env lua
--
--  Hello World client
--  Connects REQ socket to tcp://localhost:5555
--  Sends "Hello" to server, expects "World" back
--
--  Author: Robert G. Jakabosky <bobby@sharedrealm.com>
--
local zmq = require("zmq")
local fmt = string.format

local context = zmq.init(1)
local count = arg[1] or 10

--  Socket to talk to server
print("Connecting to hello world server...")
local socket = context:socket(zmq.REQ)
socket:connect("tcp://localhost:5555")

for n = 1, count do
  print("Sending Hello " .. n .. " ...")
  socket:send(fmt("Hello %s", n))

  local reply = socket:recv()
  print("Received World " .. n .. " [" .. reply .. "]")
end
socket:close()
context:term()

3   A next, not quite so simple step - a remote machine

In order to use ZeroMQ to make requests to and receive responses from a remote machine, we will need to create an SSH tunnel. It's easy, but it's also a bit non-obvious. Here is a bash shell script and the commands that we'll use:

#!/usr/bin/bash
ssh -f ec2-user@reifywork.com -L 5555:localhost:5555 -N
ssh -f dkuhlman@magpie -L 5556:localhost:5556 -N

Notes:

• You will need to modify this script to reflect your access to your remote machine.
• The above bash script actually creates two (2) SSH tunnels. The first is to a remote machine at reifywork.com out on the WAN (wide area network, i.e. the big Internet) and the second is to the machine magpie on my LAN (local area network).
• In order to use the first tunnel, to reifywork.com, we'll use port 5555. And for the second, to magpie, we'll use port 5556.
• reifywork.com will respond by using port 5555. And, magpie will respond through port 5556.
• So, in this example, on the machine addressable as reifywork.com I'll implement and run a ZeroMQ server app that receives requests and sends responses through port 5555; and on the machine magpie on my LAN I'll implement and run a ZeroMQ server app that receives requests and sends responses through port 5556.
• The above script starts two processes. Use htop or your own process monitor to verify that.
• Because it creates processes, you will need to re-run that script anytime those processes are killed and you need them again. That happens, for example, when you shut down your machine.
• For this to work, you will need access (without a password, I believe) in both directions. You can use ssh-keygen to generate authentication keys for ssh. You can do a Web search for how-to information on how to generate and install the authentication key on the remote machine.