First things first, you need node.js, which you can get from github. Install in the usual way
$ git clone git://github.com/ry/node.git $ cd node $ ./configure $ make $ sudo make install
Then we’ll need node.websocket.js which you can also get from github
$ git clone git://github.com/Guille/node.websocket.js.git
This is basically an experimental implementation of the web socket API. You can create simple server side modules and then a client side implementation that opens up a web socket to the server in which you can then exchange data in both directions. There are a few examples included that you can take a look at including a simple echo server and a chat server. Just fire up the server like so
$ node runserver.js
and then open up one of the html files in the test/websocket/ directory in your browser. One catch though, you’ll need a current browser such as Google Chrome. I’d recommend it anyway, as browsing the web with it does run a bit smoother.
You’ll also need curl, which is pretty common on any linux box these days and can be installed through your package manager.
Setting Up the Server
The way we’re going to implement the server is to use the curl command to pull from the twitter stream into a file. Twitter gives you a bunch of JSON objects back which you can then parse and display. We’ll use this file in a moment to send the data over the web socket to the browser. There is some documentation on what you can do with the API but we’ll keep it simple and search for any tweets with ‘nyc’ in them
$ curl -dtrack=nyc http://stream.twitter.com/1/statuses/filter.json -uUSERNAME:PASSWORD > sample.json
In a nutshell this is waiting for the client to establish a connection, and then it creates a child process that tails our file from the curl command above. Anytime the file is written to the “output” listener is invoked, which runs our callback to parse the JSON into objects that we can then use to send a string back to the client with some readable information from the stream.
Lets break it down just a bit more in case you are not familiar with Node. First we are requiring the system and filesystem modules from Node. Now, node.websocket.js basically just uses the node API to implement a server in the websocket.js file. It looks at the request header to see which module to instantiate and then invokes your onData() method when a client sends over data.
Therefore the onData method is the one we need to implement in our module above. We’ll use the process object in Node to create a child process that emits an event called “output” each time the child sends data to stdout. So the addListener call sets up a callback that will be invoked when our file receives more data from the twitter stream. That data comes in the form of JSON objects, one per line. So we split on the lines to create an array of JSON objects, and loop through them. Each time through the loop we’re sending this data back to the client, which is the web browser.
Just make sure the file you pass in is the correct path to the file you are outputting to from the curl command in the monitor_file() function. Then to run the node.websocket.js server you can just invoke it like so
$ node runserver.js
However if you are running the server from another host, you may want to listen on more than just the default of localhost
$ node runserver.js --host=0.0.0.0
Setting Up the Client
This is probably a bit more straightforward. We’re just implementing a few of the functions from the Websocket interface. First we’re instantiating the WebSocket class with the hostname and port that we’re running the server on. The gardenhose in the path is to tell our server that we want to run the gardenhose.js module that we wrote above. The onopen() function is invoked when the socket is opened, and we send over the word “start” which if you recall from above understands the client has connected and to start the child process that runs tail on our file. The onmessage() function is invoked anytime the server is sending data over the web socket, which is the information we want to show on the page, so we append it to the HTML of our hose div. If the server closes the socket then onclose() is invoked, and we display that on the page.
Async I/O – http://en.wikipedia.org/wiki/Asynchronous_I/O
Node.JS – http://nodejs.org
Node.Websocket.JS – http://github.com/guille/node.websocket.js
Twitter Streaming API (Gardenhose) – http://apiwiki.twitter.com/Streaming-API-Documentation
Websocket API – http://dev.w3.org/html5/websockets