Realtime Web Stats Using Node.js, Socket.IO and Redis


In today’s bigdata analytics world, it is very important and easy to get real-time stats exposed either to end users to have better user experience or to internal dashboards to drive business insights further.

Few example candidates for real-time stats:

  • Current active visitors
  • Currently logged-in users
  • When a friend comes online or goes offline
  • When someone likes or comments on your status
  • Send notification when an event happens to subscribed users
  • How many people are actively playing a particular game, watching online webinar/event, etc
  • Total transactions, purchases, impressions, messages, clicks till day and so on

Two approaches on how this data can be exposed to the browser:

  1. Front-end browser(s) pulling the data from back-end at fixed regular intervals.
  2. Back-end pushing the change to front-end browser(s) as and when the change happens or when desired.

Both has its own advantages and dis-advantages; and in general one need to pick the right solution (push or pull) that fits the performance and scalability needs (how often the data is getting changed, how many clients are connected, etc).

ARCHITECTURE

Here is the simple push architecture using Node.jsRedis and Socket.IO:

realtime stats

Redis is picked as common source as it supports both pub-sub model (used for push mechanism) and regular key look-ups (for pull mechanism). This can also be implemented using any data store or queue system; but as it needs to be real-time and shouldn’t introduce any latency; redis fits the best.

Application can be either regular app that is serving the main content or it can be a event processing module that works on top of event logs in real-time.

How it works:

  • Application publishes the event/counter to a dedicated channel in redis as and when the event happens.
  • Node.js is subscribed to redis channel and gets all the events as and when it happens along with any other subscribers that are subscribed to redis channel.
  • Node.js also has socket.io implementation; upon receiving the event from redis channel, node.js posts the event back to socket.io.
  • Browser (javascript) is connected to node.js server and subscribes to the same channel from socket.io.
  • Browser gets the event as and when application publishes the event to redis (application -> redis -> node.js -> socket.io -> browser).

Working Example

Here is the screenshot of working model where node.js is started from server.js script and client.html is opened in browser. Any event that gets published to ‘realtime’ channel in redis; browser get to see immediately.

realtime-node-redis

Source code

Here is the source code for the above example and can also be browsed from github.

server.js

server.html

JavaScript

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
/*
    Node.js server script
    Required node packages: express, redis, socket.io
*/
const PORT = 3000;
const HOST = ‘localhost’;
var express = require(‘express’),
    http = require(‘http’),
    server = http.createServer(app);
var app = express();
const redis = require(‘redis’);
const client = redis.createClient();
const io = require(‘socket.io’);
if (!module.parent) {
    server.listen(PORT, HOST);
    const socket  = io.listen(server);
    socket.on(‘connection’, function(client) {
        const subscribe = redis.createClient()
        subscribe.subscribe(‘realtime’);
        subscribe.on(“message”, function(channel, message) {
            client.send(message);
        });
        client.on(‘message’, function(msg) {
        });
        client.on(‘disconnect’, function() {
            subscribe.quit();
        });
    });
}

client.html

client.html

Shell

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
<html>
  <head>
    <title>REAL TIME Node.js, Redis and Socket.IO</title>
    <meta http-equiv=”Content-Type” content=”text/html;charset=UTF-8″/>
    <!–Load the AJAX API–>
    <script type=”text/javascript” src=”https://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js”></script&gt;
    <script type=”text/javascript” src=”http://localhost:3000/socket.io/socket.io.js”></script&gt;
    <script type=”text/javascript”>
      var socket = io.connect(‘http://localhost:3000/&#8217;);
      socket.on(‘connect’, function(data){
        setStatus(‘connected’);
        socket.emit(‘subscribe’, {channel:’realtime’});
      });
      socket.on(‘reconnecting’, function(data){
        setStatus(‘reconnecting’);
      });
      socket.on(‘message’, function (data) {
        console.log(‘received a message: ‘, data);
        addMessage(data);
      });
      function addMessage(data) {
          $(‘#online’).html(data);
      }
      function setStatus(msg) {
          console.log(‘Connection Status : ‘ + msg);
      }
    </script>
  </head>
  <body>
    <table>
      <tr style=”width:100%; height: 200px;”>
        <td align=”center” style=”width: 30%; height: 100%; border: 1px solid #000;” >
            <p id=”online” style=”font-size: 100px;”>0</p>
        </td>
     </tr>
  </table>
  </body>
</html>

Copy from http://venublog.com/2013/06/26/realtime-web-stats-using-node-js-socket-io-and-redis/

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s