Subscribe GenStages Under Umbrella

GenStage Under Umbrella — Part 4

This is the last article in the GenStage under Umbrella series, and the shortest. We made quite some progress. We have a working application and tests are passing. GenStages are sending and receiving the expected information across the Umbrella apps.

We miss just one more thing: automate the GenStage subscriptions. As you remember, we manually subscribed the consumers to the producers in our tests, like so:

GenStage.sync_subscribe(MyUkApp.ReceiveConsumer, to: Converter.ReceiveProducerConsumer)

In the last article, we saw why it is not possible to handle the subscriptions in the GenStage init. Basically, we do not have control over the application start order in a “flat umbrella” architecture. So we cannot guarantee that the producer is up when the consumer tries to subscribe.

What can we do about it?

Enter UmbrellaStage (github)

At Fitzdares, we use this GenStage under Umbrella architecture for one of our current Elixir projects. So we’ve been working on this small hex package called UmbrellaStage. Its only function is to subscribe GenStage consumers to the producers.

Under the hood, it’s a Registry that tracks the GenStage processes across all apps. The registry stores also the subscriptions details. As soon as a producer-consumer pair of processes becomes available, it subscribes them.

For the moment the package implements only GenStage.sync_subscribe/3.

Automated Subscriptions

Let’s add UmbrellaStage to our Stockr app. We will add the package in our Shared application mix.exs: {:umbrella_stage, "~> 0.1"}. Then run mix deps.get to install the package.

Now look for all GenStage.sync_subscribe/3 in our tests and delete them. If you will try to run the tests again, they will fail, as the GenStages are no longer communicating with each other.

It’s time to add the UmbrellaStage configuration in each GenStage. It contains information about the GenStage type, as well as the producer names and options. Add the following configurations at the top of the GenStages, right after use GenStage.

The UmbrellaStage hex package will work only with named producers (and producer_consumers).

And finally, the last step. In each GenStage init function call umbrella_sync_subscribe(). That’s it. Now you can rerun the tests and they will pass. The consumers will subscribe to their producers as soon as both starts.

You can find the full code for our Stockr app on GitHub:

Your Turn

Here we are at the end of this long series. I am very interested to hear your opinion or experiences with GenStage in Umbrella projects. Personally, I find it pretty cool. Please leave a comment with your thoughts, possible concerns, ideas, etc.

Also, if you would like to see more features in the UmbrellaStage hex package, use the issues to request it. Or, please feel free to contribute to the project and rise a PR.




elixir dev | | @iac0bs0n

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store


elixir dev | | @iac0bs0n

More from Medium

HashiCorp Vault KV v2 Lookups from Ansible AWX

WireGuard. How it was

Auto Update Desktop Applications

How to install multiple PHP versions on Linux