Lesson 14: Supervisor Patterns

We have two servers, chat_stats and chat_registry, and so far we start them by hand. If one crashes, it stays down. Lesson 10 solved that with the hand-written keeper process. OTP has a behaviour for the same job. A supervisor starts a list of child processes, links to them, and restarts them when they die.

The Supervisor Module

A supervisor is a callback module with a single callback. init/1 returns how to supervise and what to start. The supervisor behaviour does the rest.

Create src/chat_server_sup.erl:

-module(chat_server_sup).
-behaviour(supervisor).

-export([start_link/0]).
-export([init/1]).

start_link() ->
    supervisor:start_link({local, ?MODULE}, ?MODULE, []).

init([]) ->
    SupFlags = #{
        strategy => one_for_one,
        intensity => 5,
        period => 10
    },
    Children = [
        #{
            id => chat_stats,
            start => {chat_stats, start_link, []},
            restart => permanent,
            shutdown => 5000,
            type => worker
        },
        #{
            id => chat_registry,
            start => {chat_registry, start_link, []},
            restart => permanent,
            shutdown => 5000,
            type => worker
        }
    ],
    {ok, {SupFlags, Children}}.

There is no loop, no receive and no restart logic in this file. It is pure configuration. A supervisor without application code has very few ways to crash, and the process that restarts everything else must be the most reliable one in the system.

Child Specs

Each child is described by a map.

Only id and start are required. The defaults are permanent, 5000 and worker. We spell them out once to see them.

The start function must spawn the child, link it to the calling process and return {ok, Pid}. gen_server:start_link does exactly that, which is why every server we write has a start_link function. The link tells the supervisor that a child died. It also takes the children down when the supervisor dies, so no orphans are left behind.

Restart Types

• permanent children are always restarted. Use it for servers that must exist as long as the application runs.
• transient children are restarted only after an abnormal exit. Exiting with reason normal or shutdown means the job is done. Use it for tasks that should finish once.
• temporary children are never restarted. Use it for processes such as one client connection. Restarting it makes no sense, because its socket is gone.

Starting the Tree From the Application

In lesson 00, chat_server_app:start/2 returned a placeholder process. The application behaviour expects the pid of the top supervisor there.

Update src/chat_server_app.erl:

-module(chat_server_app).
-behaviour(application).

-export([start/2, stop/1]).

start(_StartType, _StartArgs) ->
    chat_server_sup:start_link().

stop(_State) ->
    ok.

rebar3 shell starts the chat_server application, which now starts the supervisor, which starts both servers in list order.

Watching a Restart

$ rebar3 shell
===> Booted chat_server
1> supervisor:which_children(chat_server_sup).
[{chat_registry,<0.184.0>,worker,[chat_registry]},
 {chat_stats,<0.183.0>,worker,[chat_stats]}]
2> chat_stats:increment(messages).
ok
3> chat_registry:register_user(<<"alice">>, self()).
ok

which_children/1 lists each child as id, pid, type and modules. Now kill the registry. kill cannot be trapped, so this simulates the worst kind of crash.

4> exit(whereis(chat_registry), kill).
true
=SUPERVISOR REPORT==== 19-Sep-2026::10:12:37.531241 ===
    supervisor: {local,chat_server_sup}
    errorContext: child_terminated
    reason: killed
    offender: [{pid,<0.184.0>},
               {id,chat_registry},
               ...

5> whereis(chat_registry).
<0.192.0>
6> chat_registry:list_users().
[]
7> chat_stats:get(messages).
1

The supervisor logged the death and started a new registry with a new pid under the same registered name, so callers that use the name never notice. With one_for_one, chat_stats was not touched and still has its counter.

State Is Lost on Restart

The new registry called init/1 and started with an empty map. Alice is gone. A restart does not resume the old process. It starts a fresh one in a known good state.

That is the point. A crash usually means the process met a situation its code did not handle, and the state may be part of the problem. Carrying corrupt state into the new process would make it crash again.

Whether the loss is acceptable depends on the data. Counters in chat_stats are informational. The registry can be rebuilt, because every entry describes a live connection process that can register again. Data that must survive, such as user accounts, belongs outside the process. Lesson 28 adds a database.

Strategies

The strategy flag decides what else restarts when one child dies.

• one_for_one restarts only the dead child. Use it when children are independent, as ours are.
• one_for_all stops every other child and restarts them all. Use it when children cannot work without each other.
• rest_for_one restarts the dead child and every child listed after it. Use it when later children depend on earlier ones, but not the reverse. Children start in list order and stop in reverse order, so put dependencies first.

A fourth strategy, simple_one_for_one, handles many identical children that are started on demand. Lesson 19 uses it for client connections.

Restart Intensity

Restarting fixes rare failures. It cannot fix a child that crashes every time it starts, for example because a port is already in use. Restarting it forever would burn CPU and fill the logs.

intensity and period set a limit. Ours allow at most 5 restarts within 10 seconds. One more and the supervisor gives up. It stops all its children and exits with reason shutdown. A parent supervisor then sees a dead child and applies its own strategy, so the problem moves up the tree until a restart at some level fixes it.

1> Kill = fun() -> exit(whereis(chat_registry), kill), timer:sleep(100) end.
#Fun<erl_eval.43.113135111>
2> [Kill() || _ <- lists:seq(1, 6)].
=SUPERVISOR REPORT==== 19-Sep-2026::10:18:00.962740 ===
    supervisor: {local,chat_server_sup}
    errorContext: shutdown
    reason: reached_max_restart_intensity
    ...
=INFO REPORT==== 19-Sep-2026::10:18:00.965677 ===
    application: chat_server
    exited: shutdown
    type: temporary

[ok,ok,ok,ok,ok,ok]
3> whereis(chat_server_sup).
undefined
4> whereis(chat_stats).
undefined

The transcript leaves out the six child_terminated reports. chat_server_sup is the top of our tree, so when it gave up, the whole application stopped and chat_stats went down with it. application:start(chat_server) brings it back. In a release (lesson 25) the node itself shuts down at this point, so that an outside service manager can restart it.

Compared With keeper From Lesson 10

keeper trapped exits, spawned a linked worker and spawned it again when an 'EXIT' message arrived. A supervisor works the same way inside and adds what we left out.

• A restart limit, so a broken child cannot spin forever
• Several children with a defined start order and reverse stop order
• Strategies for children that depend on each other
• Ordered shutdown. Each child gets exit(Pid, shutdown) and up to its shutdown time before it is killed
• Log reports and introspection with which_children/1

One detail about shutdown. A gen_server’s terminate/2 runs on a supervisor shutdown only if the server called process_flag(trap_exit, true) in init/1. chat_stats does not, so it dies silently.

Exercises

Exercise 1: See the Other Strategies

Change strategy to one_for_all, restart rebar3 shell, kill chat_registry and compare pids. Repeat with rest_for_one, killing first chat_stats and then chat_registry. Set it back to one_for_one afterwards.

% Expected with rest_for_one:
% killing chat_stats     => both pids change
% killing chat_registry  => only the chat_registry pid changes

Exercise 2: A Transient Worker

Write chat_greeter, a gen_server that prints a banner in init/1 and sends itself a message that makes it stop with reason normal. Add it to the supervisor as a transient child and check that it is not restarted. Then make it stop with reason oops and watch what the restart limit does.

% Expected with reason normal:
% supervisor:which_children(chat_server_sup) shows
% {chat_greeter,undefined,worker,[chat_greeter]}

Exercise 3: Terminate on Shutdown

Add process_flag(trap_exit, true) to chat_stats:init/1. Run application:stop(chat_server) in the shell and confirm that the terminate/2 message from lesson 12 appears with reason shutdown.

% Expected output:
% chat_stats stopping (shutdown) with #{}

Key Takeaways

• A supervisor is configuration. init/1 returns supervisor flags and child specs
• Child specs are maps. id and start are required, and start must name a function that links and returns {ok, Pid}
• permanent, transient, temporary mean restart always, restart after abnormal exits, and never restart
• one_for_one, one_for_all, rest_for_one decide which siblings restart with a dead child
• intensity and period cap the restart rate. Past the limit the supervisor stops its children and exits
• A restarted process starts from init/1. State is lost on purpose. Keep durable data outside the process
• The application’s start/2 returns the top supervisor, so starting the application starts the whole tree

Interactive Koans