Jump to content

Photo

Node: How to use socket.io to push realtime events?


  • Please log in to reply
22 replies to this topic

#1
Mo3

Mo3

    HTTP 420 Enhance Your Calm

  • Moderators
  • 2014 posts
  • Joined: 21-July 10
  • LocationBavaria
  • Expertise:PHP, Java, Javascript, Python, Ruby on Rails, Node.js, SQL, MongoDB
I have a little node.js script which connects to Twitter and prints incoming tweets (with a certain keyword in it) in real time (Keyword "Rebecca Black": Rage spam with over 500 tweets per second). Now I'd like to take those tweets and send them to my browser in real time. How?

EDIT: I do have a HTTP server running and socket.io attached to it, but I don't know how to actually send data and how to display them in real time in the client's browser.

Thanks!

Node.js | Ruby on Rails | Python | PHP | Scala
What is necessary to change a person is to change his awareness of himself.


#2
Kyek

Kyek

    Founder of wdR

  • Administrators
  • 5419 posts
  • Joined: 20-February 10
  • LocationPhiladelphia, PA, USA
  • Expertise:HTML, CSS, PHP, Java, Javascript, Node.js, SQL
Cool, this will be mad easy :) Obviously there's a server-side to socket.io, and a client side that runs in the browser and makes the connection. What's cool about websockets is that it works over HTTP -- so assuming you already have an HTTP object set up like this (example taken from nodejs.org):
var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(8124);
Then you can run your websockets connection using that SAME http object :) And no worries if you're using one created by Connect or Express-- they'll all work. So you can attach socket.io to that variable like this:
var io = require('socket.io');
var socket = io.listen(http);
Holy shit that was hard.

And now that you have a socket object, you can listen and respond to events on it. Well, event, not events, because the only one is 'connection'. So if you want to send a success message to the client when it connects, you could do this:
socket.on('connection', function(client){
    client.send('{"success": 1}');
    client.on('message', function(data) {
        console.log('Client just sent:', data); 
    }); 
    client.on('disconnect', function() {
        console.log('Bye client <img src='http://www.webdevrefinery.com/forums/public/style_emoticons/<#EMO_DIR#>/sad.gif' class='bbc_emoticon' alt=':(' />');
    }); 
});
You don't need to put in those client message and disconnect events to do that -- I just wrote them in so you could see how to handle data coming in from the client side, or a client breaking the connection :)

If you want to broadcast a message to EVERYONE connected through that socket, that's REALLY REALLY HARD. This might be rough to understand, but try to get through it:
socket.broadcast('Hi.');
Just take a minute.. grab an iced tea, try to settle down.. you'll figure it out.

So every time a tweet comes in that you want to be live-updated on your site, just do a socket.broadcast() and you can send whatever you want -- JSON, plain text, html, whatever's easiest for you. Whatever you send, your client side code will get it exactly, character-for-character.

Client side code, now! This first step is pretty easy -- you need to include the socket.io javascript. Socket.io configures your server to send that out automatically for a certain URL, so all you need to do is add this to your page:
<script type="text/javascript" src="socket.io/socket.io.js"></script>

And bam, that'll pull in the socket.io stuff. Now all your code has to do is use that library :) And that's really REALLY similar to the back-end code. First we get an active socket:
var socket = new io.Socket('www.yourdomain.com', {port: 80});
Obviously, just fill in your domain there along with the port you're running Node on. If you can connect to your page without putting a port number in the URL bar, it's 80.

Now we attach events to the socket just like we did before. Something like this:
if (socket.connect()) {
    socket.on('message', function(data) {
        // data contains exactly what you sent from the server.
        // You can print it to the page, or JSON.parse it, or
        // whatever you want to do with it.
    });
    socket.on('disconnect'), function() {
        alert('Damn, connection broken.');
        // You could also tell the script to pause for 5 seconds and then reconnect
        // instead of using an error message-- that way you can restart your node
        // server without pissing anyone off
    });
}
else {
    // Couldn't make a connection.  Throw out an error or something.
}

And that's it :) Enjoy!

**Note: If you want your client-side code to be fully non-blocking, you could take socket.connect() out of an if-statement, and, instead, call it after you've added a listener for the 'connect' event. That might be useful if the app could disconnect and connect again, and you need to run some code every time it reconnects. Your call :) There are some examples of that here: http://socket.io

#3
alexdavey

alexdavey

    Bringing XKCD to a post near you

  • Members
  • 848 posts
  • Joined: 05-November 10
  • LocationLondon
  • Expertise:HTML, CSS, PHP, Javascript
After setting up socket.io on the client side, it really isn't that hard, just parse JSON data sent from your server and write it to the screen. What are you having trouble with? Here is an example of how you would do it:

Your socket server:
/***************************  Require modules  ********************************/
var	sys = require('sys'), 
	http = require('http'),
	io = require('socket.io');

/*************************  Start socket server  ******************************/
server = http.createServer(function(request, response) {
response.writeHead(200, {'Content-Type': 'text/html'});
response.end('<h1>Welcome to the socket.io server<h1>');
});

server.listen(port);

var socket = io.listen(server);
socket.on('connection', function(client) {
        // This will only fire when a client has connected
	// Do some stuff when connected to a client
	
	client.send(/* Your twitter data */); // You don't have to put this here.
	// You can also use socket.broadcast() to send to everyone.
				
	client.on('message', function(data) {
		// Do some stuff when you recieve a message
	});
	
	client.on('disconnect', function(client) {
		sys.puts('Client disconnected');
	});
	
});

Your index.html:
<!DOCTYPE html>
<head>
	<link rel="stylesheet" href="css/styles.css" type="text/css">
	<title>Twitter</title>
</head>
<body>
        <div id="textbox"></div>
	<script src="socket-io/socket.io.js"></script>
	
	<script>
	var oldData = [];
        // Connect to your server and assign the result to a variable
	var socket = new io.Socket(window.location.hostname, {port: 3128});

	socket.connect();
	socket.on('connect', function(client){
		// Do some stuff when connected
	});

	socket.on('message', function(data){
		// Do some stuff when you get a message
		oldData += data;
		document.getElementById('textbox').innerHTML = oldData;
	});

	socket.on('disconnect', function(){
		// Do some stuff when disconnected
	}); 
	</script>
</head>
</body>
</html>

And you still need a static server to serve the javascript and HTML files (you combine both servers, but I think it is better (cleaner) this way)
/***************************  Require modules  ********************************/
var sys = require('sys'),
	http = require('http'),
	path = require('path'),
	url = require('url'),
	fs = require('fs');

/*************************  Start static server  ******************************/
		server = http.createServer(function(request, response) {
		    var uri = url.parse(request.url).pathname;
                    // prefix with httpdocs/ (or whatever you want) if you want to store your static files in a separate folder
		    var filename = path.join(process.cwd(), 'httpdocs/' + uri);
			path.exists(filename, function(exists) {
				
				// Rewrite URL if necessary - just some basic rewrite rules
				if (uri.charAt(uri.length -1) === '/') {
						filename += 'index.html';
					} else if(uri.indexOf('.') === -1) {
						filename += '/index.html';
					}
				
				if(!exists) {
					response.writeHead(404, {"Content-Type": "text/plain"});
					response.end("404 Not found\n");
					return;
					}
				else {
					fs.readFile(filename, "binary", function(err, file) {
						if(err) {
							response.writeHead(403, {"Content-Type": "text/plain"});
							response.end(err +"\n");
							return;
						}
	  
						response.writeHead(200);
						response.end(file, "binary");
					});	
				}
			});
		});
		
                server.listen(8080);
		sys.puts('Static file server listening to port ' + 8080);

EDIT: damn you Kyek! :)

#4
Mo3

Mo3

    HTTP 420 Enhance Your Calm

  • Moderators
  • 2014 posts
  • Joined: 21-July 10
  • LocationBavaria
  • Expertise:PHP, Java, Javascript, Python, Ruby on Rails, Node.js, SQL, MongoDB

Cool, this will be mad easy :) Obviously there's a server-side to socket.io, and a client side that runs in the browser and makes the connection. What's cool about websockets is that it works over HTTP -- so assuming you already have an HTTP object set up like this (example taken from nodejs.org):

var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(8124);
Then you can run your websockets connection using that SAME http object :) And no worries if you're using one created by Connect or Express-- they'll all work. So you can attach socket.io to that variable like this:
var io = require('socket.io');
var socket = io.listen(http);
Holy shit that was hard.

And now that you have a socket object, you can listen and respond to events on it. Well, event, not events, because the only one is 'connection'. So if you want to send a success message to the client when it connects, you could do this:
socket.on('connection', function(client){
    client.send('{"success": 1}');
    client.on('message', function(data) {
        console.log('Client just sent:', data); 
    }); 
    client.on('disconnect', function() {
        console.log('Bye client <img src='http://www.webdevrefinery.com/forums/public/style_emoticons/<#EMO_DIR#>/sad.gif' class='bbc_emoticon' alt=':(' />');
    }); 
});
You don't need to put in those client message and disconnect events to do that -- I just wrote them in so you could see how to handle data coming in from the client side, or a client breaking the connection :)

If you want to broadcast a message to EVERYONE connected through that socket, that's REALLY REALLY HARD. This might be rough to understand, but try to get through it:
socket.broadcast('Hi.');
Just take a minute.. grab an iced tea, try to settle down.. you'll figure it out.

So every time a tweet comes in that you want to be live-updated on your site, just do a socket.broadcast() and you can send whatever you want -- JSON, plain text, html, whatever's easiest for you. Whatever you send, your client side code will get it exactly, character-for-character.

Client side code, now! This first step is pretty easy -- you need to include the socket.io javascript. Socket.io configures your server to send that out automatically for a certain URL, so all you need to do is add this to your page:
<script type="text/javascript" src="socket.io/socket.io.js"></script>

And bam, that'll pull in the socket.io stuff. Now all your code has to do is use that library :) And that's really REALLY similar to the back-end code. First we get an active socket:
var socket = new io.Socket('www.yourdomain.com', {port: 80});
Obviously, just fill in your domain there along with the port you're running Node on. If you can connect to your page without putting a port number in the URL bar, it's 80.

Now we attach events to the socket just like we did before. Something like this:
if (socket.connect()) {
    socket.on('message', function(data) {
        // data contains exactly what you sent from the server.
        // You can print it to the page, or JSON.parse it, or
        // whatever you want to do with it.
    });
    socket.on('disconnect'), function() {
        alert('Damn, connection broken.');
        // You could also tell the script to pause for 5 seconds and then reconnect
        // instead of using an error message-- that way you can restart your node
        // server without pissing anyone off
    });
}
else {
    // Couldn't make a connection.  Throw out an error or something.
}

And that's it :) Enjoy!

**Note: If you want your client-side code to be fully non-blocking, you could take socket.connect() out of an if-statement, and, instead, call it after you've added a listener for the 'connect' event. That might be useful if the app could disconnect and connect again, and you need to run some code every time it reconnects. Your call :) There are some examples of that here: http://socket.io


You're awesome.

Client side question: Where do I have to put the code? In a regular html file?

Node.js | Ruby on Rails | Python | PHP | Scala
What is necessary to change a person is to change his awareness of himself.


#5
Kyek

Kyek

    Founder of wdR

  • Administrators
  • 5419 posts
  • Joined: 20-February 10
  • LocationPhiladelphia, PA, USA
  • Expertise:HTML, CSS, PHP, Java, Javascript, Node.js, SQL

You're awesome.

Client side question: Where do I have to put the code? In a regular html file?

See Alex's answer for that one :) Generally what you have is a node webserver where you're serving some sort of static content. If you're using Connect or Express or Geddy or something, serving static content is really really easy. At some point, someone is connecting to your website with a web browser and seeing a page. However that page is served to them, that's where the HTML code goes to include the socket.io.js file :)

#6
Mo3

Mo3

    HTTP 420 Enhance Your Calm

  • Moderators
  • 2014 posts
  • Joined: 21-July 10
  • LocationBavaria
  • Expertise:PHP, Java, Javascript, Python, Ruby on Rails, Node.js, SQL, MongoDB
I think I'm stupid.. just doesn't work :(

var http = require('http'),  
    sys = require('sys'),
    io = require('socket.io')
    path = require('path'),
    url = require('url'),
    fs = require('fs');


                server = http.createServer(function(request, response) {
                    var uri = url.parse(request.url).pathname;
                    // prefix with httpdocs/ (or whatever you want) if you want to store your static files in a separate folder
                    var filename = path.join(process.cwd(), 'httpdocs/' + uri);
                        path.exists(filename, function(exists) {
                                // Rewrite URL if necessary - just some basic rewrite rules
                                if (uri.charAt(uri.length -1) === '/') {
                                                filename += 'index.html';
                                        } else if(uri.indexOf('.') === -1) {
                                                filename += '/index.html';
                                        }
                                
                                if(!exists) {
                                        response.writeHead(404, {"Content-Type": "text/plain"});
                                        response.end("404 Not found\n");
                                        return;
                                        }
                                else {
                                        fs.readFile(filename, "binary", function(err, file) {
                                                if(err) {
                                                        response.writeHead(403, {"Content-Type": "text/plain"});
                                                        response.end(err +"\n");
                                                        return;
                                                }
          
                                                response.writeHead(200);
                                                response.end(file, "binary");
                                        });     
                                }
                        });
                });
                
                server.listen(8080);
                sys.puts('Static file server listening to port ' + 8080);


var socket = io.listen(server);
socket.on('connection', function(client) {
        // This will only fire when a client has connected
        // Do some stuff when connected to a client
        client.on('message', function(data) {
                // Do some stuff when you recieve a message
        });
        
        client.on('disconnect', function(client) {
                sys.puts('Client disconnected');
        });
        
});
//Twitter stuff
    socket.broadcast("@" + tweet.user.screen_name + ": " + tweet.text);

index.html:

<!DOCTYPE html>
<head>
        <title>Twitter</title>
</head>
<body>
        <div id="textbox"></div>
        <script src="socket-io/socket.io.js"></script>
        
        <script>
        var oldData = [];
        // Connect to your server and assign the result to a variable
        var socket = new io.Socket(window.location.hostname, {port: 8080});

        socket.connect();
        socket.on('connect', function(client){
                // Do some stuff when connected
        });

        socket.on('message', function(data){
                // Do some stuff when you get a message
                oldData += data;
                document.getElementById('textbox').innerHTML = oldData;
        });

        socket.on('disconnect', function(){
                // Do some stuff when disconnected
        }); 
        }
        </script>
</head>
</body>
</html>

Node.js | Ruby on Rails | Python | PHP | Scala
What is necessary to change a person is to change his awareness of himself.


#7
alexdavey

alexdavey

    Bringing XKCD to a post near you

  • Members
  • 848 posts
  • Joined: 05-November 10
  • LocationLondon
  • Expertise:HTML, CSS, PHP, Javascript
How do you know your script is not trying to broadcast before anyone has connected? Don't forget node.js is event driven ;)


Try putting it under
socket.on('connect', function(client){

Also, there is a random bracket at the bottom of the index.html javsacript (my fault :()

I have corrected the code, and tested it - it works.

How have you arranged the files? You should have:

server.js
httpdocs/
| index.html
| socket.io/
-| flashsocket.js
-| htmlfile.js
-| jsonp-polling.js
-| socket.io.js
-| websocket.js
-| xhr-multipart.js
-| xhr-polling.js
-| xhr.js
| css/
-| styles.css

#8
Mo3

Mo3

    HTTP 420 Enhance Your Calm

  • Moderators
  • 2014 posts
  • Joined: 21-July 10
  • LocationBavaria
  • Expertise:PHP, Java, Javascript, Python, Ruby on Rails, Node.js, SQL, MongoDB

How do you know your script is not trying to broadcast before anyone has connected? Don't forget node.js is event driven ;)


Try putting it under

socket.on('connect', function(client){

Also, there is a random bracket at the bottom of the index.html javsacript (my fault :()

I have corrected the code, and tested it - it works.

How have you arranged the files? You should have:

server.js
httpdocs/
| index.html
| socket.io/
-| flashsocket.js
-| htmlfile.js
-| jsonp-polling.js
-| socket.io.js
-| websocket.js
-| xhr-multipart.js
-| xhr-polling.js
-| xhr.js
| css/
-| styles.css



Where the hell do you have all those files from? D:

Node.js | Ruby on Rails | Python | PHP | Scala
What is necessary to change a person is to change his awareness of himself.


#9
Kyek

Kyek

    Founder of wdR

  • Administrators
  • 5419 posts
  • Joined: 20-February 10
  • LocationPhiladelphia, PA, USA
  • Expertise:HTML, CSS, PHP, Java, Javascript, Node.js, SQL
Well you might not have the socket.io folder if you installed it with something like npm, and the css/html files may not be there if you're serving those from a traditional webserver. If 100% of what you were doing were contained inside that one project folder for node, that's what it would look like.

Edit: Actually, that might be your problem! Is it the node app serving your static files (html, js, css, etc) or are those coming direct from a webserver like apache or nginx? If you're serving them from a webserver, that's your issue -- the javascript include for socket.io/socket.io.js will be hitting Apache instead of node, and apache has no idea where the hell the file is. In that case, you'd want to change it to something like:
<script type="javascript" src="http://my.hostname.com:8080/socket.io/socket.io.js"></script>

Whatever that URL is, just run your node server and plug the URL into your browser to see if javascript code comes up. If not, something's wrong :)

#10
Mo3

Mo3

    HTTP 420 Enhance Your Calm

  • Moderators
  • 2014 posts
  • Joined: 21-July 10
  • LocationBavaria
  • Expertise:PHP, Java, Javascript, Python, Ruby on Rails, Node.js, SQL, MongoDB

Well you might not have the socket.io folder if you installed it with something like npm, and the css/html files may not be there if you're serving those from a traditional webserver. If 100% of what you were doing were contained inside that one project folder for node, that's what it would look like.


I see, thanks!

Node is up and running, but my HTML file can't find the socket.io.js file. Why is that? I used npm to install socket.io.

Node.js | Ruby on Rails | Python | PHP | Scala
What is necessary to change a person is to change his awareness of himself.


#11
Kyek

Kyek

    Founder of wdR

  • Administrators
  • 5419 posts
  • Joined: 20-February 10
  • LocationPhiladelphia, PA, USA
  • Expertise:HTML, CSS, PHP, Java, Javascript, Node.js, SQL

I see, thanks!

Node is up and running, but my HTML file can't find the socket.io.js file. Why is that? I used npm to install socket.io.

See my edit :)

#12
alexdavey

alexdavey

    Bringing XKCD to a post near you

  • Members
  • 848 posts
  • Joined: 05-November 10
  • LocationLondon
  • Expertise:HTML, CSS, PHP, Javascript
It depends on how you installed socket.io, they are in the socket.io installation, just make sure you link to socket.io.js which is in the same directory as those files.

#13
Mo3

Mo3

    HTTP 420 Enhance Your Calm

  • Moderators
  • 2014 posts
  • Joined: 21-July 10
  • LocationBavaria
  • Expertise:PHP, Java, Javascript, Python, Ruby on Rails, Node.js, SQL, MongoDB

See my edit :)


I did - still no change D:

I put this inside:

<script src="http://localhost:8080/socket-io/socket.io.js"></script>

If I try to access the js file directly, I get a 404.

Node.js | Ruby on Rails | Python | PHP | Scala
What is necessary to change a person is to change his awareness of himself.


#14
Kyek

Kyek

    Founder of wdR

  • Administrators
  • 5419 posts
  • Joined: 20-February 10
  • LocationPhiladelphia, PA, USA
  • Expertise:HTML, CSS, PHP, Java, Javascript, Node.js, SQL
Do socket.io instead of socket-io in that path, and see what happens :)

Edit: Or use their CDN:
<script src="http://cdn.socket.io/stable/socket.io.js"></script>


#15
Mo3

Mo3

    HTTP 420 Enhance Your Calm

  • Moderators
  • 2014 posts
  • Joined: 21-July 10
  • LocationBavaria
  • Expertise:PHP, Java, Javascript, Python, Ruby on Rails, Node.js, SQL, MongoDB

Do socket.io instead of socket-io in that path, and see what happens :)

Edit: Or use their CDN:

<script src="http://cdn.socket.io/stable/socket.io.js"></script>


Alright, changing the - to a . actually worked :) But the script itself still doesn't. There's no connection being made.

Node.js | Ruby on Rails | Python | PHP | Scala
What is necessary to change a person is to change his awareness of himself.


#16
alexdavey

alexdavey

    Bringing XKCD to a post near you

  • Members
  • 848 posts
  • Joined: 05-November 10
  • LocationLondon
  • Expertise:HTML, CSS, PHP, Javascript
Did you remove that random bracket on the index.html file?

#17
Mo3

Mo3

    HTTP 420 Enhance Your Calm

  • Moderators
  • 2014 posts
  • Joined: 21-July 10
  • LocationBavaria
  • Expertise:PHP, Java, Javascript, Python, Ruby on Rails, Node.js, SQL, MongoDB

Did you remove that random bracket on the index.html file?


Aaaaaand here we go :) Connection is (kinda) successful
21 Mar 21:29:06 - socket.io ready - accepting connections
21 Mar 21:29:08 - Initializing client with transport "flashsocket"

crypto.js:97
  return new Hash(hash);
         ^
TypeError: undefined is not a function
    at CALL_NON_FUNCTION_AS_CONSTRUCTOR (native)
    at Object.createHash (crypto.js:97:10)
    at [object Object]._proveReception (/usr/local/lib/node/.npm/socket.io/0.6.16/package/lib/socket.io/transports/websocket.js:128:22)
    at [object Object]._onConnect (/usr/local/lib/node/.npm/socket.io/0.6.16/package/lib/socket.io/transports/websocket.js:96:13)
    at [object Object].<anonymous> (/usr/local/lib/node/.npm/socket.io/0.6.16/package/lib/socket.io/client.js:23:8)
    at [object Object].<anonymous> (/usr/local/lib/node/.npm/socket.io/0.6.16/package/lib/socket.io/transports/websocket.js:9:10)
    at new <anonymous> (/usr/local/lib/node/.npm/socket.io/0.6.16/package/lib/socket.io/transports/flashsocket.js:8:13)
    at [object Object]._onConnection (/usr/local/lib/node/.npm/socket.io/0.6.16/package/lib/socket.io/listener.js:165:74)
    at [object Object].check (/usr/local/lib/node/.npm/socket.io/0.6.16/package/lib/socket.io/listener.js:84:12)
    at Server.<anonymous> (/usr/local/lib/node/.npm/socket.io/0.6.16/package/lib/socket.io/listener.js:40:15)

EDIT: NEVERMIND! Fixed it by re-compiling node with SSL support.

Node.js | Ruby on Rails | Python | PHP | Scala
What is necessary to change a person is to change his awareness of himself.


#18
alexdavey

alexdavey

    Bringing XKCD to a post near you

  • Members
  • 848 posts
  • Joined: 05-November 10
  • LocationLondon
  • Expertise:HTML, CSS, PHP, Javascript
Do you have node installed with open SSL developement libraries? If not, you need it for crypto to work.

reinstall node.js with:
sudo apt-get install libssl-dev 
cd /usr/local/node 
./configure 
make 
make install 

EDIT: This always happens to me... :(

#19
Kyek

Kyek

    Founder of wdR

  • Administrators
  • 5419 posts
  • Joined: 20-February 10
  • LocationPhiladelphia, PA, USA
  • Expertise:HTML, CSS, PHP, Java, Javascript, Node.js, SQL

EDIT: This always happens to me... :(

I lol every time xD

#20
Mo3

Mo3

    HTTP 420 Enhance Your Calm

  • Moderators
  • 2014 posts
  • Joined: 21-July 10
  • LocationBavaria
  • Expertise:PHP, Java, Javascript, Python, Ruby on Rails, Node.js, SQL, MongoDB
Awesome! It works perfectly.

Next up is some sort of tab system. (Yes, I'm trying to clone http://www.monitter.com/, but also I plan on adding location based filtering and other neat stuff)

Node.js | Ruby on Rails | Python | PHP | Scala
What is necessary to change a person is to change his awareness of himself.





2 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users


    Google (1)