Elastic Applications in Erlang

April 17, 2014
 |  No Comments

How realistic or useful are elastic applications in general, or specifically in Erlang? To demonstrate, I start with Joe Armstrong’s favorite Erlang program – the universal server:

universal_server() ->
    receive
        {become, F} ->
            F()
    end.

Once spawned, it sits and waits until it gets an instruction to become something. Then it becomes that thing it’s told to be.

First, this is really cool. Second, we should expand a bit:

  • make the wrapping process loop
  • optionally make the internal function loop
  • if internal function receives messages, make it respect an “abort” message which would make it stop executing and return the control back to its parent

So, blah blah – all of the above is only few lines of code in Erlang. I’ll call this module gen_node and the gen_node.erl file is as follows:

-module(gen_node).
-export([start/0]).

start() ->
    receive
        {become, F} ->
            F(),
            start();
        reset ->
            start();
        stop ->
            ok
    end.

and then the way we invoke it would be:

1> c(gen_node).
{ok,gen_node}
2> N = spawn(gen_node, start, []).
<0.40.0>

OK, the process is spawned and waiting, let’s give it something to do. Let’s define 2 functions: one that doubles the given number, and another that squares it. They both communicate via messages:

3> Double = fun Double() ->
    receive
      reset -> ok;
      {From, Args} -> From ! Args+Args, Double()
    end
   end.
#Fun<erl_eval.44.106461118>
4> Square = fun Square() ->
    receive
      reset -> ok;
      {From, Args} -> From ! Args*Args, Square()
    end
   end.
#Fun<erl_eval.44.106461118>

Now, let’s tell the spawned process to become “Double”, then reset it and tell it to become “Square”:

5> N ! {become, Double}.
{become,#Fun<erl_eval.44.106461118>}
6> N ! {self(), 16}.
{<0.33.0>,16}
7> flush().
Shell got 32
ok
8> N ! reset.
reset
9> N ! {become, Square}.
{become,#Fun<erl_eval.44.106461118>}
10> N ! {self(), 16}.
{<0.33.0>,16}
11>; flush().
Shell got 256
ok

So, this is cool because now we have a generic computing node that we can tell to transform into any arbitrary processor and communicate to it via messages.

Now, finally, let’s wrap this up in an OTP application using gen_server so the worker processes are also supervised:

https://github.com/unix1/gen_node

Now the interaction becomes:

1> Double = fun Double() ->
    receive
      {_, reset} -> ok;
      {From, Args} -> From ! Args+Args, Double()
    end
   end.
#Fun<erl_eval.44.106461118>
2> Square = fun Square() ->
    receive
      {_, reset} -> ok;
      {From, Args} -> From ! Args*Args, Square()
    end
   end.
#Fun<erl_eval.44.106461118>
3> application:start(gen_node).
ok
4> {ok, N, _} = gen_node:start_server().
{ok,<0.42.0>,#Ref<0.0.0.45>}
5> gen_node:become(N, Double).
ok
6> gen_node:send(N, 16).
32
7> gen_node:reset(N).
ok
8> gen_node:become(N, Square).
ok
9> gen_node:send(N, 16).
256

It worked!

So, what’s next?

This is obviously just a concept code. To be usable something needs to track the states and types of nodes, and something else to intelligently control what they are doing and how the behavior should adapt. Possibilities could include:

  • predicting application-specific demand and adjusting resources: e.g. queue X is filling up, while queue Y processes are waiting too long – so make an adjustment in real-time
  • defining capacities of systems by giving different weights to each type of operation, or calculating resources needed to make certain computations
  • more fully using existing resources, or distributing load where the right resources are available

Questions: is this applicable and interesting? Boring? Already been done? Do you have any ideas which direction this should take?

Any constructive feedback is fully welcome.

Tags: