December 2016

Introduction

I recently started to work with a company as a freelancer. As I got access to the source code, I had the chance to look around at what technologies were used and the particular syntax that the software was written into. The software uses Socket.io for most of the communication client-server. But, as I was looking at the packages used and the code, one of those packages was socketio-file-upload. With this blog post, I’ll try out this package in order to understand better what it is supposed to do. Let’s jump into it!

What is it?

Here is the description we can read from the github link I gave you in the introduction.

This module provides functionality to upload files from a browser to a Node.JS server that runs Socket.IO. Throughout the process, if their browser supports WebSockets, the user will not submit a single HTTP request. Supports Socket.IO 0.9 and 1.0.

The intended audience are single-page web apps, but other types of Node.JS projects may benefit from this library.

Since version 0.4, this module also supports monitoring file upload progress.

Simply put, you can upload files from the client to the server using sockets. I upload a file, without using AJAX or any HTTP requests and it will be saved by the server, in a directory of my choice. Let’s see that!

Begin by create a new directory, and run npm init

Accept all the defaults and then run:

npm install --save socketio-file-upload express pug socket.io

We will connect the router of the socketio-file-upload to express later.

Let’s create a file called server.js

server.js

const express = require('express');
const app = express();
const httpServer = require('http').Server(app);
const path = require('path');
const siofu = require('socketio-file-upload');
const socketServer = require('socket.io')(httpServer);

app.set('view engine', 'pug');
app.set('views', path.join(__dirname, '/public/views'));
app.use(express.static(path.join(__dirname, '/public')));
app.use(siofu.router);

app.get('/', (req, res) => {
    res.render('index');
});

socketServer.on('connection', socket => {
    console.log('Socket connected');
    socket.emit('data', 'Hello World');
    const uploader = new siofu();
    uploader.dir = path.join(__dirname, '/files');
    uploader.listen(socket);
});

httpServer.listen(3000);

Ok, let me explain a little but this code. Most of it is setting a socket.io server with express and and http server. If you don’t know how to do this, I wrote a blog post on getting started with socket.io.

After all this setup, you can direct your attention to the socketServer connection. You can see that we instantiate a new SocketIo-file-upload instance(named uploader). Then, we tell this instance where we want the file to be upload. In this case, I want my files to go to the /files directory. Finally, this instance needs to listen to the socket that just connected.

Note: Socketio-file-upload will NOT create the directory for you if it doesn’t exist. It will throw an error instead.

That is all we need for the back-end. Let’s code our front-end now. We will use several methods from the socketio-file-upload package. First, in your public/views directory, create a file called index.pug :

index/pug

h1 Hello Index
input(type='file')#fileUpload
button#file_button ClickHere
script(src='/socket.io/socket.io.js')
script(src='/siofu/client.js')
script(src='../js/main.js') 

Nothing crazy here, a file input and a button to play around with a method we’ll see in our js file. Then, we link to our 3 different scripts. One for the socket.io library, one for the socketio-file-upload library (both are in our node_modules directory) and one for our custom script called main.js. Let’s see our custom main.js:

//Connection to the socket library
const socket = io();

//This is some classic socket.io event listening
socket.on('data', data => {
    console.log(data);
});

//Here is our socketio-file-upload code
const uploader = new SocketIOFileUpload(socket);

uploader.listenOnInput(document.getElementById('fileUpload'));

uploader.addEventListener('start', (event)=> {
    event.file.meta.extension = 'csv';
});

document.getElementById('file_button').addEventListener('click', uploader.prompt, false);

Ok, here is how we do things. We create a new SocketIOFileUpload instance, with our socket as a parameter. We can use the listenOnInput method on our input file. As you can see, we pass the DOM element that we want to listen to. When we will choose a file, we will trigger this event.

When we do that, we will go back to our server.js file. Our socketio-file-upload instance on the back-end will receive this file (remember that it listens to our socket). Once it receives this file, it will store it in the directory that we specified.

I added some custom code on the start event of the instance. You can write on the event.file object for example. Here, I decided to add extension : 'csv' to the meta object. You can imagine that the possibilities are infinite when it comes to playing with the object.

Lastly, you might have seen on the index.pug that I added a button. The last two lines on main.js adds an event listener to a click event on this button. I use the instance.prompt method (instance being here uploader). This will trigger the opening of the same window you would have on a input file.

That’s it! Those are just a few methods available on the socketio-file-upload package. To learn more, you can visit the very good github socketio-file-upload page. You can peak at the documentation to learn more.

Note: you may wonder what is the difference between this package and the socket.io streams package. The stream package will be user to read files. We create a stream to read the contents of the file. With this package, we do not read it, we store it in our files directory (in our case). We could of course read the file with a socket stream and then upload it. Or upload it, then read it with a stream. Both those packages could be very complementary in my opinion.

Feel free to share and comment!
Have a nice day!

Read more

Introduction

Yesterday, I received my Pavlok. If you are not familiar with it, Pavlok is a device that is focused on changing your habits with the help of electrical stimulus. The most effective way to use it is to zap yourself when you are doing a bad habit. For example, every time I bit my nails, ZAP!

The Pavlok system tries to break the cue-routine-reward path by making the reward not as pleasant. If your habit is rewarded by an electric shock, your brain will associate this habit with pain or discomfort. Because we are creatures of pleasure, we will most likely stop doing this habit.

But, as a part of the Internet of Things trend, Pavlok also allows you to customise its use and add some tweaks. For example, there is a Chrome extension available. If I spend too much time on Twitter, I will get zapped. I also linked my Pavlok with RescueTime. This way, I get notified when I’m being productive (vibrating), when I could do better (beeping) and when I’m being baaaaaad (ZAAAAP !).

Ok, so this is great. But the even cooler part is this: Pavlok has a developer API that allows us to interact with our devices through code. This is what this post is about: getting started with the Pavlok developer API.

Getting started

We will use Node.JS to achieve this. First things first, we need to register our application, the same way we would do with Twitter in order to use its API. To do this, visit this page. Enter the name of your app and for the callback URL, just enter http://localhost:3000/auth/pavlok/result .

Great, now our app is registered. You will get a Client ID and a Client Secret. We will need them in our app.

Create a file, let’s call it test.js.

First, run npm init. Select the defaults and then run npm install --save pavlok-beta-api-login

Let’s write our code!

const pavlok = require('pavlok-beta-api-login');
const ClientID = "YOUR_CLIENT_ID";
const Secret = "YOUR_SECRET_ID";

// Init your device
pavlok.init(ClientID, Secret, {
        verbose: true,
        port: 3000
});

//login
pavlok.login( (result, code) => {
    if(result){
        console.log('Code is', code);
    }
});

// Your device will beep
pavlok.beep();

//Your device will vibrate
pavlok.vibrate();

//This is a pattern
// (Beep & Vibrate) * 3
pavlok.pattern({
"pattern": ['beep', 'vibrate'],
"count": 3
});

And that is all we need!
Run node test.js and your device will respond to the stimuli. You Pavlok application will also receive notifications with the name of your app! Awesome!!

Note that the code will be different if you host your application. On local, this code will be fine. But if you need to host it, you’ll need to change a few things. You can check the npm pavlok wiki page to get more information on how to do this.
You can get more information about the device on the Pavlok website and more information on the Developer API can be found on the Pavlok Developer API

I’ll post more informations as I keep playing with it. But this is enough to get you started. Have fun!

Feel free to comment and share!
Have a nice day!

Read more

Introduction

In this article, we will see very simply how to use namespaces with socket.io. Namespaces allow us to group sockets based the page the user visits. In our example, we will use three different groups. One group will contain the sockets in the index page ‘/’, another in the ‘/products’ page and another one in the ‘/contact’ page.

First things first, create a directory that will hold our project. Then, run npm init. Accept the defaults and then install our needed dependencies with :

npm install --save express socket.io 

Ok, now create 4 files, server.js, index.html, products.html, contact.html.

Let’s start with our server.js:

const express = require('express');
const http = require('http');

const app = express();
const server = http.Server(app);
const ioServer = require('socket.io')(server);

//Namespaces Socket.io
const indexUsers = ioServer.of('/');
const productsUsers = ioServer.of('/products');
const contactUsers = ioServer.of('/contact');

// Express file rendering
app.get('/', (req,res)=> {
    res.sendFile(__dirname + '/index.html');
});

app.get('/products', (req,res)=> {
    res.sendFile(__dirname + '/products.html');
});

app.get('/contact', (req,res)=> {
    res.sendFile(__dirname + '/contact.html');
});

// socket.io logic

//Index page sockets
indexUsers.on('connection', socket => {
    socket.emit('connectionMessage', 'Welcome to the index page');
});

//Products page sockets
productsUsers.on('connection', socket => {
    socket.emit('connectionMessage', 'Welcome to the products page');
});

//Contact page sockets
contactUsers.on('connection', socket=> {
    socket.emit('connectionMessage', 'Welcome to the contact page');
});

server.listen(3000);

Quick explanation: We begin by initialising our socket.io server like usual. We use express to server our three html files. Nothing crazy so far. But notice that we define three variables with the of() method on our ioServer. This tells socket.io that depending on the connection made in the front-end, we will group sockets differently. We are then able to use the emit event (or the on event) on those namespaces like any other socket. Let’s see how our three html files will look.

They all look very similar, but I wanted to make sure that everything was clear.

index.html

<!DOCTYPE html>
<html>
    <head>
        <title>Index Page</title>
        <script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/1.7.1/socket.io.js"></script>
    </head>
    <body>
        <h1>Welcome to the Index Page</h1>
        <h3 id="message"></h3>
        <script>
            const socket = io('/');

            socket.on('connectionMessage', message => {
                document.getElementById('message').innerHTML = message;
            });
        </script>
    </body>
</html> 

products.html

<!DOCTYPE html>
<html>
    <head>
        <title>Products Page</title>
        <script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/1.7.1/socket.io.js"></script>
    </head>
    <body>
        <h1>Welcome to the Products Page</h1>
        <h3 id="message"></h3>
        <script>
        const socket = io('/products');

        socket.on('connectionMessage', message => {
            document.getElementById('message').innerHTML = message;
        });
        </script>
    </body>
</html>

contact.html

<!DOCTYPE html>
<html>
    <head>
        <title>Contact Page</title>
        <script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/1.7.1/socket.io.js"></script>
    </head>
    <body>
        <h1>Welcome to the Contact Page</h1>
        <h3 id="message"></h3>
        <script>
            const socket = io('/contact');

            socket.on('connectionMessage', message => {
                document.getElementById('message').innerHTML = message;
            });
        </script>
    </body>
</html>

Alright, the structure of those files is exactly the same. Standard html, with a script for the socket.io CDN. Then, two headers, a h1 that tells us on which page we are on, and an h3 with the id of message. In our script, the socket reacts to the connectionMessage event by putting the message in our h3.

Can you spot the difference between those files?
The definition of our socket is different. The argument in our io() function is what creates the namespace. In combination with the variables definition we did in server.js, we created separated groups of sockets that can emit and respond to socket events independently from one another. Pretty cool huh?

And that’s it! This is a very simple beginning, just to make you understand how this thing works. Of course, this opens a lot of different possibilities. Go ahead and experiment!

As always, feel free to comment or share.

Have a nice day!

Read more

Introduction

I recently started to work with a company as a freelancer. As I got access to the source code, I had the chance to look around at what technologies were used and the particular syntax that the software was written into. The software uses Socket.io for most of the communication client-server. But, as I was looking at the packages used and the code, one of those packages was socketio-streams. With this blog post, I’ll give a quick example of how this package works.. Let’s jump into it!

What is it ?

Here is how it is described:

This is the module for bidirectional binary data transfer with Stream API through Socket.IO.

Combining the native Node Stream API with Socket.IO. That sounds promising. One of the advantages of using streams with Sockets is that, if you end having to transfer a lot of data between the client and the server, it can become very slow. Streams, on the other end, listen to a data event, meaning they receive and send data by chunks. Let’s try to build something with it.

We are going to do something very simple, as an example. Let’s say I want users to send my a file and I will receive it and write the content of the file I received into a whole new file.

We will use socket.io-stream to create a new stream on the front-end. That stream will send the data to our back-end who will then write its content to a new file.

First, create a new directory and let’s install what we will need.

npm install express socket.io socket.io-stream

Then, let’s create two files: server.js and client.html

Here is how our server file will look like:

const ss = require('socket.io-stream');
const path = require('path');
const app = require('express')();
const server = require('http').Server(app);
const io = require('socket.io')(server);
const fs = require('fs');

io.on('connection', socket => {
    console.log('connection');
    socket.emit('connected', 'hello World');
    ss(socket).on('file', (stream, data) => {
        stream.pipe(fs.createWriteStream( Date.now() +'.txt'));
    });
});

app.use((req, res) => {
    res.sendFile(path.join(__dirname + '/client.html'));
})

server.listen(3000);

We require our dependencies at the top. We create a server in order to use socket.io. Express will help use send the static html file on the front-end. So, as you can see, socket-stream listen to an event called file, this event will get two different arguments, a stream and the data. We then use the pipe method on that stream to connect it to a writable stream. This new writable stream write our content to a new .txt file. Let’s take a look at the front-end now.

client.html

<!DOCTYPE html>
<html>
    <head>

    </head>
    <body>
        <input type='file' id="file" />
        <script src="https://cdn.socket.io/socket.io-1.4.5.js"></script>
        <script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io-stream/0.9.1/socket.io-stream.js"></script>
        <script>
            const socket = io();
            let fileElem = document.getElementById('file');

            socket.on('connected', data => {
                console.log(data);
            });
            fileElem.onchange = e => {
                let file = e.target.files[0];
                let stream = ss.createStream();
                ss(socket).emit('file', stream, {size: file.size});
                let blobStream = ss.createBlobReadStream(file);
                let size = 0;
                blobStream.on('data', chunk => {
                    size += chunk.length;
                    console.log(Math.floor(size / file.size * 100) + '%');
                });

                blobStream.pipe(stream);
                blobStream.on('end' , ()=> {
                    console.log('done');
                });

            }

        </script>
    </body>
</html>

The html is very simple here, just an input that accepts file. I use the CDN service for socket.io and socket.io-stream. You could use require() if you are configured for that.

So, anytime that our file input changes, meaning that we give a new file, we create a new stream thanks to socket.io-stream. Then, we emit the file event. The event can take a few optional parameters. Here, I give a stream parameter, and the file size. Then, I create a read stream thanks to createBlobReadStream . A Blob is everything that is not Javascript data (like a file). I added a little snippet to display the progress of the upload (with on(‘data’) ). Then, we connect this read stream to the stream we created at the beginning (which is also the stream we gave as an argument in our emit event).

And that is pretty much it for our application. You can try it out by running:

node server.js  

You will see that every time you try to upload a new file, the progress will be shown in the console. And you will see a new file appearing in your directory.

Obviously, there are a lot of possibilities to use this. But, I think it is a nice start.

Feel free to ask questions and share!
Have a nice day!

Read more