Recently, I’ve been working on a project that does a lot of heavy weight image processing. In a nutshell, a user uploads an image and then the website does a whole load of processing on that image and returns with the results. Depending on the size of the image it could take a while for it to load the results. The websites does other stuff as well so the scenario where a few users are concurrently uploading images, waiting for response while a whole load of other set of users are navigating to other parts of the website is very real. The performance was respectable, however, I wanted more.
After a couple of hours of investigating around the issue, I finally settled on using forks for handling this type of load. A fork is a process that is spun off of a parent process. In this case, when a user uploads an image on the website, the node server creates a fork that then does the image processing. Meanwhile, the parent process hooks into the “exit” callback of the forked process so that it knows when the forked process has exited. When this happens, the parent process returns the results to the user.
Now, every node process, no matter how asynchronous, is single threaded. This means that it utilises at most a single core of your CPU. So the fact that the new fork powered architecture creates a fork process to handle the heavy image processing means lower load on the single node process. It also means that the load is spread evenly across cpu cores.
However, here lies the problem. Every time a new customer arrives on the website and uploads a new image, the node server spins up a new process. So, if many customers arrive and upload large images which could take 10-15 seconds to load, it will spin up a new process for each of those customers. Since each process maps to roughly one CPU core, having too many processes doing CPU intensive things will start to slow down everything thats running on that machine. This means that they will slow down each other as they race to get as much CPU time as they can.
This problem also has a simple solution. Limit the number of forks so that they do not exceed a certain number. This number will vary upon usage and circumstances but after doing some benchmarking, in my case, this was 3. This meant that my application should, at any given time run at most 3 forks (processes).
In technical speak, this is called pooling of resources. The idea behind this is that you create a pool of fixed size and then take things in and out of pool. Translating this into what I needed to do with my fork problem, I needed a pool of size 3 to manage my forks so that it will allow me to put at most 3 things in it. If I tried to put more things in it, it should queue those things, wait for current things that are running in it to finish and then automatically start the queued things.
I googled for some time, trying to find a good way to pool my forks. I found a couple of solutions but none were as elegant as I’d like them to be. I did find some good ones but they either were deprecated or weren’t being maintained anymore. So, I got my IDE ready and started programming what will be known as the forkpool module.
The module comprises of two basic things. Forkpool and Forklet . Forklet is a definition of a fork instance. For example, if you wanted to fork a node module named worker.js , your forklet will comprise of three things. First being the location of the module, second being additional environment configuration needed to run the fork and third (optional) being the time within which the fork must complete its execution (if it doesn’t, it gets killed). The third option of specifying timeout was the one I couldn’t find available in any of the existing fork pools and hence decided to create it.
So once you have a forklet , you can use the forkpool to create a fork. Upon creating the forkpool, you can define the size of the pool in its constructor. This value is the maximum number of forks you want running in parallel at any given time.
Once you have a forkpool and forklet objects you can execute forks. If you execute more forks than the capacity, it will queue the rest of the forks up and execute them one by one as the running forks finish their execution.
The module also comes with a event system comprised of a set of events. Every time something happens with the forks, a event is triggered. You can subscribe to events on a specific fork or to all forks across the entire pool. Working within the asynchronous architecture of node, this just made sense. There are various kinds of events that let you know what is happening to a fork. These are scheduled , started , timedout and exited . Event callbacks also come with additional contextual information that provide more information regarding that specific event.
Checkout these links below for more information regarding the module: