Build a Desktop Sharing Application With Nodejs, Electron, and Socket.io | by Wajiha Abid | Jun, 2022

Done in 3 steps

What is a Desktop Sharing Application?

Desktop sharing application is a product that allows remote access and remote collaboration on another person’s desktop through an interface provided on your desktop. A famous example of such software is anyDesk, TeamViewer etc.

There is one more way of screen sharing in which we share our entire or particular window with the other user but it doesn’t involve any action you can only view the screen like zoom, google meet, etc.

  • Node.js
  • Electron.js
  • Socket.io
  • Robot.js

We can use webRTC for screen sharing, so why to consider Socket.io?

We can easily achieve screen sharing by webRTC and all can be achieved by its client APIs but the problem arises that webRTC provides only limited free user communication on stun server and to expand this limit we have to use 3rd party dependency and customization is a bit difficult . So if you have the constraint of no 3rd party dependency and have the requirement of customization then using your own server would be a good fit.

Let’s start building its major functionalities. If you are here to just see the code then you can find the whole code on git. Following will be the basic communication structure.

Step 1: Clients will connect to the socket server through which they’ll communicate with each other.

Step 2: Any client could request to connect to any other client via server. Consider client A wants to connect to client B, so client A will request the server to connect to client B. Server will forward that request to client B.

Step 3: If the client accepts the request, it will be notified to the source client via server. Client B will accept the request of client A and the server will notify to client A. Then client B will start sending the data packets to client A.

This was the basic structure of the application, for a quick overview I have divided the application into 3 parts as follows:

  • How data is shared via socket server for screen sharing?
  • How the operations are performed on a shared screen?
  • Pointer accuracy algorithm.

For data sharing, we have used realtime-communication of client and server via socket.io.

Suppose we have 2 clients A and B, and A will take the remote access from client B so soon as client B will accept the connection request A, client A will start receiving the screenshots of the screen of client B.

So the server will emit the ready-for-dataevent to client B and it will set a continuous interval of 500ms and take screenshots of the screen and send it to the socket server via screen-dataevent as follows:

**client/controller/screen.controller.js//take screen shots and send the data to original screensocket.on(“ready-for-data”, data => {data = JSON.parse(data)
data.status = true
status = true
interval = setInterval(function () {screenshot().then((img) => {
let dimension = sizeOf(img)
var imgStr = new Buffer.from(img).toString(‘base64’);
var obj = {};
obj.room = data.myID;
obj.image = imgStr;
obj.dimension = dimension
//sending data to server
socket.emit(“screen-data”, JSON.stringify(obj));
})
}, 500)
})

Now on the server the data is received and sent to client A via connected-screen-data event as follows:

**server/index.jssocket.on(“screen-data”, function (data) {
data = JSON.parse(data);
var room = data.room;
var imgStr = data.image;
//check if the connection exist of a client A
if (roomMap.has(room)) {
//emitting to client A
socket.broadcast.to(room).emit(‘connected-screen-data’, {
imgStr,
dimension: data.dimension
});}})

Client A will receive this data via connected-screen-data listener and set it to the whole screen canvas. For sending data to the screen Electron’s IPC will be used.

( Inter-process communication (IPC) is the feature of electron through which Renderer and Main process interact)

**client/controller/screen.controller.js//listening to the screen packets
ipcMain.on(“screen-packets”, (e, arg) => {
socket.on(‘connected-screen-data’, data => {if (data.imgStr) {
e.reply(“screen-packets-reply”, data)
}
})
})

So this was all how the screen is shared between 2 clients.

As we can see we are sharing the screen with the help of screenshots then the question arises of how we will perform any action on the shared screen like moving a cursor, clicking anywhere, or typing, etc. So for these operations, we have used a library named Robotjs. It’s a desktop automation library through which we can control the mouse, and keyboard and read the screen.

In our application whenever the remote client (client A) moves the cursor, click, or types on a screen an event is emitted on a server with the respective coordinates, then the server forwards it to the host client where robotjs take those co-ordinates and perform an action.

For example when client A click on the shared screen, ipcRenderer sends the mouse-click event to ipcMain and it sends it to the server.

**client/js/screen.js//when user click on the screen
imgElement.addEventListener(“mouseup”, e => {
ipcRenderer.send(“mouse-click”, {direction:e.which , double:false})
})
**client/controller/screen.controller.js//host screen will send the server event to emit click event to original screen
ipcMain.on(“mouse-click”, (e, arg) => {
socket.emit(“mouse-click”, { remoteID: Creds.remoteID, direction: arg.direction, double: arg.double })})

Now client B will listen the socket mouse-click event and robotjs will perform the action.

**client/controller/screen.controller.js//click the original screen
socket.on(“mouse-click”, function (data) {
let direction = “left”
switch (data.direction) {
case 1:
direction = “left”
break;
case 2:
direction = “middle”
break;
case 3:
direction = “right”
break;
default:
break;
}
//robotjs function to click
robot.mouseClick(direction, data.double);
})

Check out the full documentation of robotjs here for other operations — it’s an amazing desktop control library.

After all this hustle, the major issue which popped up was the pointer accuracy because we haven’t limited the shared screen size so whatever your screen size the shared screen will be the same as that. So to match the different screen pointer accuracy we used the following algorithm:

Image by author

So this is how the pointer accuracy is maintained and it is used under the event named mouse-move .

So these were the major 3 steps of my screen-sharing application. There are a lot more things inside code — like sending the connection request, setting a password and accessing the remote via password, and sharing files bi-directionally.

Till now I haven’t faced any lagging issues as the screen data is shared in form of a base64 string still working on its improvement and open for suggestions. And more features are planned to be added like chat and call.

You can find the whole app code in my GitHub Repository here.

Leave a Comment