Piping Phoenix Contexts

8 min readJan 5, 2020

Phoenix contexts are a great way to organize the code. They separate the business logic from the web related logic. They group together schemas and business logic. And provide clear ways of inter-contextual communication.

If you are new to Phoenix, I recommend reading at least the official contexts docs before proceeding.

The main advantage: dealing with groups of related functionality isolated into manageable contexts. But that’s also a source of complexity. Designing a new, or existing but fast-growing app into contexts is no easy thing. There may be interdependencies between the contexts (especially between schemas).

There is good advice on how to manage this in the official docs above. And you will also find many great articles about the organization of the contexts.

There are times yet when implementing a new feature would touch (almost) all the contexts. And still, does not fit in any of them. That’s the case we will experiment in this article.

The Study Case

First of all, this article is not a guide, but a study case. There is no answer that would fit all cases. We rather explore more paths and decide what makes more sense.

Back to our example. Let’s imagine we develop on an online book store application. There is a Phoenix app called book_store with the following structure:

BookStore
  - Account (context)
    - schema:
      - User
    - public functions:
      - get_user(id :: integer()) :: User.t() | nil
  - Library (context)
    - schema:
      - Book
    - public functions:
      - get_books_with_promotion(book_ids :: [integer()]) :: [Book.t()] | nil
  - Shop (context)
    - schema:
      - Order 
      - Item
      - Coupon
    - public functions:
      - get_order(id :: integer()) :: Order.t() | nil
      - apply_discount(Order.t(), discount_value :: integer()) :: {:ok, Order.t()} | {:error, any()}
      - get_coupon_by_code(promo_code :: binary()) :: Coupon.t() | nil
      - use_coupon(Coupon.t()) :: {:ok, Coupon.t()} | {:error, any()}

There are 3 contexts:

Account — managing the shop users, including account creation, login, etc
Library — takes care of the books. There we could also have inventory, author data, reviews, etc
Shop — manages the commercial side. From shopping cart creation, checkout, discounts etc.

To simplify, we deal only the contexts, schemes and functions that impact our study case, and ignore everything else.

Now, there is a new business requirement:

Implement a new discount coupon

with a set of rules:

one coupon can be used only once
a premium user gets double the discount on the coupon
only certain books qualify for this promotion. The books that do not qualify, will not get any discount

Implementation

During the checkout process, there will be a call to the coupon endpoint. This receives the order_id and the promo_code as params. Based on that info, it should (maybe) apply a discount to the order.

The first thought may be: what’s so complicated about that? A with statement could do it:

While this may work to some extent, we can see a few problems:

1. Where would this code live?

Should it be in the controller calling action: coupon?

I would definitely say no. This is a lot of business logic, and a controller should not handle that. The controller should simply deal with the result of this function.

2. In the existing Shop context?

Maybe… We could defdelegate the apply_coupon function to some internal module Shop.Promotion that will not clutter the main context file. But even so, I feel like handling User and Book related logic may not belong to the Shop.

3. A new context?

A newPromotion context that handles marketing related logic may hold this. We could move also the Coupon schema in the new context. This makes more sense to me than point 2. Still, I see it as a mixture of many contexts, not really belonging to a single one.

4. A top-level, above the contexts, pipeline management component?

This approach is not something we see in the Phoenix conventions. But maybe it’s worth exploring. In most Phoenix examples, under the lib/ directory, we will find the equivalent of BookStore and BookStoreWeb. This level may be a good place to add the logic that does not really fit in the contexts.

And a pipeline of various context functions can be what we need now.

Pipeline
  - ApplyCouponBookStore
  - Account
  - Library
  - Shop

While all of the above approaches would work, I would consider 3 and 4. For our study case, we will explore option 4.

2. Cannot support multiple happy paths

This is a shortcoming of a plain with statement that becomes too complex. Let’s have a look at this line:

%Coupon{used: false} = coupon <- get_coupon(promo_code)

We don’t want to continue the pipeline if the coupon was already used, but still, it is not an error. We would just like to continue the checkout process, without a discount. While we could match on %Coupon{} in the else section, the order is not available there to return

else
%Coupon{} -> {:ok, order} #order is not available

Same case when the coupon is not found by its code. The checkout should proceed without any discount.

Similar issue for books_with_promo. If there are no books with a promotion in the current order, we just want to proceed with the checkout. But again, the order would not be available in the else section.

3. Cryptic error messages

Another issue with plain with in complex pipelines is error handling. Maybe we want to log an error when the order cannot be found, but the else will simply receive a nil. Same nil may come from get_coupon/1 or get_user/1. No way to tell between them unless we modify or add new specialized context functions.

With the learnings above, let’s try to build our coupon pipeline.

Inter-contextual Pipelines

Now that we know where we want the pipelines located ( lib/pipeline.ex, lib/pipeline/apply_coupon.ex ) let’s think how we could implement it, in a way that deals with the shortcomings above.

The first thing to consider. There are elixir libraries dealing exactly with that. Those are probably smarter and more complex than what we’re going to build below. However, it will be up to you to evaluate them and see which one suit your business needs. Or decide if you want to build your own.

Our pipeline would still be a with statement, but one that follows some rules. The rules are up to you to define and should suit your needs. However, they need to be generic enough to be implemented by all inter-contextual pipelines in your project. They should also be visible and easy to follow. The Pipeline @moduledoc could be a good place to display them.

The Pipeline Rules

In our case the rules are:

one pipeline per file/module, implementing the Pipeline behaviour
the pipeline has a single entry point (run/1) that takes a map as argument (defined in the behaviour)
the pipeline has a state that is passed across all steps (the step is the equivalent of a pipe, or function, in the with statement)
each step returns one of the following:
-{:ok, state :: map()}
- {:ok, step :: atom(), state :: map()} when there are multiple happy paths
- {:error, step :: atom(), state :: map, error :: any()}
the step is the name of the function
the pipeline always returns {:ok, result :: map()} or {:error, Pipeline.Error.t()}
the pipeline error includes:
- the pipeline module name
- the step
- the current state (changes so far)
- the error itself

Let’s see how the full code looks like.

We define a Pipeline behaviour with a run/1 callback. All the pipelines will implement this behaviour. The Pipeline module is also a good place to standardize the errors for all the pipelines, as per the rules defined above.

Quite a lot of code here, but no point to go in all details. The ApplyCoupon implements the Pipeline behaviour. Defining the types for the initial state and the expected result makes the pipeline easy to understand and use wherever needed.

As we saw, there may be multiple happy paths. The coupon_action type provides a clear indication of what happened to the order and the potential discount. Probably we could do very well without this explicit state and infer it from the returned result. But it may add some clarity for the current exercise.

Let’s check also one of the more complex functions validate_coupon/1. We first retrieve the coupon by the promo_code

if the coupon is present and not used we update the state with the coupon itself and the discount percentage. Return {:ok, state} and the pipeline continues.
if the coupon is found but already used, we update the state with the final discount value which is zero and the coupon action :coupon_already_used. Return {:ok, :validate_coupon, state} and the pipeline stops. As this is an alternate happy path, it does not result in an error. There is no point continuing the current pipeline, as no discount will be applied anyways. We have all information available (the order) to continue the checkout, with zero discount.
if the coupon is not found, we assume that the input code was wrong. The result is similar to the point above, only with a different coupon action :invalid_promo_code
anything else is treated as an error and stops the pipeline as well

The rest of the functions follow similar logic and makes no sense discussing every one of them.

Advantages

standardized input and output. Everything is documented through specs and types. Should be easy to understand and integrate into the existing code
able to handle alternative happy paths without running the whole pipeline
precise error handling. Even if the final error is something cryptic such as nil, you will know exactly from which pipeline it comes, what was the function generating it (the step), as well as the state of the pipe.
no need to change the behaviour of the existing context functions. The steps act as wrappers around them.
simple testing — the Pipeline behaviour makes it easy and safe to mock the pipeline when testing components that make use of it (eg. a controller).

Caveats

Managing local side effects

This refers to persisted mutations. In our case, there are two database updates: applying the order discount and utilizing the coupon. Let’s assume the order is updated, but the coupon update fails. The pipeline will error. We are left out with a discounted order and a still-active coupon that can be applied again.

A quick solution to manage the local side effects would be to wrap the whole pipeline in a transaction. A failing mutation will undo all other mutations in the pipeline and the state of the system will be restored as before the pipeline run.

Another solution is to manage each of those independently. Our error looks like this: {:error, :use_coupon, state, error}. We have all info to handle multiple business decisions:

we can reset the order discount and restart the process (the state has the current order)
we can retry the using the coupon (the state has the coupon as well)

Having detailed errors offers a lot of flexibility.

Managing external side effects

This gets more complicated. The transaction approach would not be enough.

Let’s assume another pipeline managing payments. This makes an external call to our payment provider. On success, it updates the order state.

A failure in the order update should trigger either a retry or a payment cancellation. Again, the state and step in the error will prove very useful.

However the more external and internal side effects are in a pipeline, the more difficult the error management will become. Keeping the pipelines simple and with as few side effects as possible is a thing to keep in mind if you are choosing this approach.

If that is not possible, maybe alternative methods may be considered.

What is your favourite approach dealing with multiple root-level contexts communication? Looking forward to your comments.

Update 15-Feb-2020

Appsignal blog liked the article and sent me some delicious stroopwafels from Amsterdam 😃.

❤️ 💛 💚 💙