How to not process the same intent more than once

(This is part 3 of a long-running series on idempotence. You can go to the beginning.)

State-based idempotence leverages our domain model to prevent duplicate processing, but sometimes we have practical reasons preventing us from using it. For a bank account to know if it has already processed a certain deposit, it would have to keep a running list of all deposits it has processed and search over it.

Sequential idempotence, prevents us from processing a given message more than once at the cost of storing a command’s global_position in any events it causes. We typically rename it to sequence on the events. You would know you’ve processed a Deposit command if your account has a projected sequence number greater than the Deposit command’s global_position.

Suppose that you have two separate command messages each for the same deposit. Perhaps the client of your component sends you a command for the deposit, but for some reason it doesn’t know if you received the command. In this case it sends you a second message for the deposit.

With state-based idempotence, you’re back to keeping an in-memory log of every deposit ID you’ve ever processed, and with sequential idempotence, you can know you’ve processed the first command, but the second command for the same deposit will have a newer and higher global_position.

How to view multiple commands as one command

Any viable message store technology will give us access to optimistic locking, which in the case of messaging means declaring an expected version for a stream when writing to it.

Suppose we have a Deposit command with account_id 123 , deposit_id abc, and global_position 3, written to the stream account:command-123. We could copy make a copy of this command, writing it to the stream accountTransaction-123+abc. The + in the identifier portion of the stream name denotes a compound identifier.

Now, we make one critical modification. We write this with an expected version of -1, which means, “we expect this stream to be empty—only write this message if there is nothing else in the stream.” In Eventide, we’d do so with code similar to the following:

write.initial(deposit, stream_name)

If the stream is indeed empty, we’ll have written a copy of the Deposit command to this new stream with the same data, except the global_position is now 4, let’s say.

At this point your client writes its duplicate command to account:command-123, same account_id and deposit_id, but now with global_position 5, giving us the following messages in the message store:

• Stream name: account-123 has global_positions 3 and 5
• Stream name: accountTransaction-123+abc has global_position 4

Now we process the duplicate command with global_position 5. How do we do that? We construct a stream name whose ID is made up of the account_id and the deposit_id. Well, since the duplicate command will have duplicate data, we produce the same stream name, namely accountTransaction-123+abc.

When we go to write a second command to that stream with an expected version of -1, the message store tells us we have a version mismatch, and we fail to write the duplicate message to the accountTransaction-123+abc stream. Two commands in the account:command category have been collapsed into a single command in the accountTransaction category. If it happened three, four, maybe five times, we’d get the same result.

We call this the Reservation Pattern.

An exercise for the reader

Now that we’ve been able to collapse multiple commands into a single one, could you carry this across the finish line? With the other patterns we have, could you handle this reduced command in an idempotent manner?

Let me know what you come up with, and tomorrow we’ll go over a solution.