Subscribe GenStages Under Umbrella
This is the last article in the GenStage under Umbrella series, and the shortest. We made quite some progress. We have a working application and tests are passing. GenStages are sending and receiving the expected information across the Umbrella apps.
We miss just one more thing: automate the GenStage subscriptions. As you remember, we manually subscribed the consumers to the producers in our tests, like so:
GenStage.sync_subscribe(MyUkApp.ReceiveConsumer, to: Converter.ReceiveProducerConsumer)In the last article, we saw why it is not possible to handle the subscriptions in the GenStage init. Basically, we do not have control over the application start order in a “flat umbrella” architecture. So we cannot guarantee that the producer is up when the consumer tries to subscribe.
What can we do about it?
Enter UmbrellaStage (github)
At Fitzdares, we use this GenStage under Umbrella architecture for one of our current Elixir projects. So we’ve been working on this small hex package called UmbrellaStage. Its only function is to subscribe GenStage consumers to the producers.
Under the hood, it’s a Registry that tracks the GenStage processes across all apps. The registry stores also the subscriptions details. As soon as a producer-consumer pair of processes becomes available, it subscribes them.
For the moment the package implements only GenStage.sync_subscribe/3.
Automated Subscriptions
Let’s add UmbrellaStage to our Stockr app. We will add the package in our Shared application mix.exs: {:umbrella_stage, "~> 0.1"}. Then run mix deps.get to install the package.
Now look for all GenStage.sync_subscribe/3 in our tests and delete them. If you will try to run the tests again, they will fail, as the GenStages are no longer communicating with each other.
It’s time to add the UmbrellaStage configuration in each GenStage. It contains information about the GenStage type, as well as the producer names and options. Add the following configurations at the top of the GenStages, right after use GenStage.
The UmbrellaStage hex package will work only with named producers (and producer_consumers).
And finally, the last step. In each GenStage init function call umbrella_sync_subscribe(). That’s it. Now you can rerun the tests and they will pass. The consumers will subscribe to their producers as soon as both starts.
You can find the full code for our Stockr app on GitHub: https://github.com/iacobson/blog_stockr
Your Turn
Here we are at the end of this long series. I am very interested to hear your opinion or experiences with GenStage in Umbrella projects. Personally, I find it pretty cool. Please leave a comment with your thoughts, possible concerns, ideas, etc.
Also, if you would like to see more features in the UmbrellaStage hex package, use the issues to request it. Or, please feel free to contribute to the project and rise a PR.