Build your own GenServer in 49 lines of code

As you may guess, this article is not about rebuilding the Elixir GenServer. It’s already there and it works great. And that’s what interests me the most: why it works great?

Also, we will not discuss what GenServer does and its generic concepts. If you are not familiar at all with the it, I suggest starting with the docs. They do a great job explaining the basics:

• https://elixir-lang.org/getting-started/mix-otp/genserver.html
• https://hexdocs.pm/elixir/GenServer.html

The problem

I’ve been using Elixir for quite a while now. But I always had some doubts on some specific aspect of the GenServers: the asynchronous requests (handle_cast/2). To be more explicit:

1. is the GenServer state consistent across multiple async requests that take a various amount of time to execute (eg. running an expensive function inside one of the handle_cast/2 implementation)?
2. are the async requests executed in the same order they were cast even if some of the casts take longer than others?

In other words, are GenSeerver async requests susceptible to race conditions?

Let’s find out

My confusion ended when my colleagues pointed me to the Erlang GenServer docs: http://erlang.org/doc/design_principles/des_princ.html#id63247

Among other useful info, you can find also a very basic implementation of the GenServer. I do not have Erlang experience, but I found the code readable enough. So I decided to rebuild it in Elixir, even add some extra features and shed some light on my dilemma.

MyGenserver

MyGenserver, that’s what we’ll call the project:

mix new my_genserver

Before we start, don’t expect that we’ll end up with a fully featured GenServer. There will be many bits missing. But the basic functionality will be there at the end of our little experiment.

start_link / init

The GenServer normally starts with a start_link/3 function. For our experiment we will ignore the options, so it will take only 2 arguments:

We pass a module and arguments to the start_link/2 function. The module will be the one implementing the GenServer behaviour. Let’s call it Example, so we could start the server like this: start_link(Example, :ok).

spawn/3 creates a new process, running a function server_init/2 which does 2 things:

• expects that Example module defines function init/1. It calls it and expects an {:ok, state} response (congratulations! Our Example can now implement the init/1 callback)
• starts a loop/2 function, passing the Example module and the initial state of the server. The loop is the heart of the GenServer and actually answers all my questions above

But let’s not rush into conclusions and continue the implementation.

call / handle_call

The call/2 function takes the MyGenserver pid, and a request. It then sends a message to the pid, with 3 elements tuple:

• the action, :call
• self() which is the calling process
• request - the arguments we want to pass to the server

Inside the loop/2 there is a receive function that listens to those messages and pattern matches on the action. In this case on {:call, _, _}. We then expect that the passed module (Example) will implement a handle_call/3 callback, that returns {reply, response, state}.

Call makes a synchronous server request. The calling process is blocked until it receives a response from MyGenserver. So we send back the response with send(parent_pid, {:response, response}). The call/2 receives and returns the response.

The last thing happening in the loop is to call itself, with the module (Example) and the new MyGenserver state.

cast / handle_cast

As said above, the async server request, cast/2 is the actual reason I tried to rewrite this basic version of the GenServer. By this time however things are quite clear.

There’s nothing special about cast. It is very similar to call, except it’s not sending a response back to the “casting” process, and will not block it.

MyGenserver process mailbox manages the order of the functions execution, in a first in first out manner. And it’s sequentially consumed by the loop function, which also keeps the state consistency.

It doesn’t matter if many async requests are cast to the MyGenserver. And it doesn’t matter if some of those requests are expensive, time-consuming functions. They will be executed in the order they were cast. There will be no race conditions between them.

With the mystery solved, let’s quickly write the remaining callbacks.

stop / terminate

The same pattern as above, with one big difference. The loop is interrupted and the process will die with the specified reason.

handle_info

handle_info callback does not have an associated API function. It catches all unmatched messages sent to MyGenserver and updates the state of the server.

The 49 Lines GenServer

Let’s put all of this together and see how it looks:

So, does it work?

It’s now time to see if MyGenserver works. For this, we’ll write the Example module referred above.

It’s a very basic example. All the requests take a number and add it to the existing state. The state is initialised in the init/1 callback with value 0.

For the purpose of this demo, we also pass an order argument. We’ll use it to visualise the order in which the functions are called and executed (with IO.inspect).

There are 2 types of requests:

• light — supposed to be very fast functions
• expensive — simulate some expensive operations that will take some considerable amount of time

We use a combination of those to test the state consistency and the order of the functions execution.

Testing

handle_call

We start an expensive call and a light call right after it. The test passes, but the IO.inspect used in the example will give us some more insights:

REQUEST ORDER: : 1
PROCESS ORDER: : 1
REQUEST ORDER: : 2
PROCESS ORDER: : 2

Those are synchronous requests. The caller process is blocked until it will receive a response from the MyGenserver. Only after, a new request is sent to the server.

handle_cast

Same thing with the async requests. The test passes but let’s see the output:

REQUEST ORDER: : 1
REQUEST ORDER: : 2
PROCESS ORDER: : 1
PROCESS ORDER: : 2

The casting process doesn’t wait for a response. It immediately sends the second request. But, even if the first request is an expensive operation, the process order remains the same. The execution of the second request will not start until the first one ends.

handle_info

Same as above, but due to the nature of handle_info we do not have a request order. The request is received directly through a process message.

PROCESS ORDER: : 1
PROCESS ORDER: : 2

terminate

Nothing much to add here. It monitors the newly created server so the test PID will receive the exit message from the server. Checks that the process is alive, stop it, checks the correct reason and that the process no longer exists.

combinations

This test is a combination of different requests. What we can see is the process order will always be sequential, no matter the type of request or the duration of the operation. This is exactly what aimed to demonstrate at the beginning of the article.

REQUEST ORDER: : 1
PROCESS ORDER: : 1
REQUEST ORDER: : 2
REQUEST ORDER: : 5
PROCESS ORDER: : 2
REQUEST ORDER: : 6
PROCESS ORDER: : 3
PROCESS ORDER: : 4
PROCESS ORDER: : 5
PROCESS ORDER: : 6

The Ultimate Test

With less than 49 lines of code, we were able to build our own basic GenServer and understand how it works. Within MyGenserver we implemented the server process loop. We proved the order of the requests execution and state consistency with an Example module and tests.

Now, as a final test. To check that our conclusions are correct, replace MyGenserver with the real GenServer in the example file. Don’t forget to use Genserver at the top of the module. Now run the tests again and they will still pass.

You can see the full MyGenserver, Example and tests code on Github.

If you (as myself) had any doubts about casting requests behaviour, I hope this helps. And you will now use GenServer async requests with full confidence.