What to do when a 3rd-party isn’t idempotent

We were going to talk about TDD in Eventide, but I’m presenting at a meetup tonight, and going over a specific idempotence concern is relevant to that. We’ll hit the TDD probably starting tomorrow.

Tonight is about idempotence, or making sure that whether you send a request to do an operation 1 time or 1,000 times, the system acts as if you only sent it once. If you’ve ever had one of those UIs that says, “Please only click this button once or else you may get double-charged,” then you know you’re dealing with a system that isn’t idempotent.

Incidentally, a point I’m going to make tonight is that if you build systems that way, you’re committing malpractice. If you provide a system where the only thing between harm to your users and them happening to not double click, then you’re not ready to build user interfaces. Come to my presentation this evening, and then you will be, ha!

We’ve talked about various idempotence patterns before, specifically how to make sure that your systems are idempotent. But what happens if a 3rd-party system you rely on isn’t idempotent? From your customers’ perspective, that practically means that your system isn’t idempotent either.

Suppose you’re trying to effect a transfer of some sort, you call the 3rd party, and you get a 500 response.

At what point in carrying out the operation and returning a response to you did the server encounter an error?

Did it fail before doing the operation:

sequence-failed-early.png

Or did it fail after doing the operation but before generating a response?

sequence-failed-late.png

The hard reality? You have no way of knowing. Welcome to distributed computing.

So, since you don’t know, you have a decision to make. Which do we prefer, doing this operation 0 times or doing this operation 2 or more times? The answer to that changes based on what the operation is.

If you’re sending a welcome email, the occasional double send probably won’t tank your company. If we’re talking about a $10 million wire transfer, even the occasional double send is A Big Deal™.

In the cases where we prefer 0 times over 2 times, we likely still need to get the work done 1 time. In those cases, you can make a log of having attempted the operation and get a human involved.

The decision on how to handle this is best made with Development working in conjunction with Product. A whole lot else goes better too when we don’t treat these disciplines as silos but as a unified Product Development team, but that’s the subject of another email.


Like this message? I send out a short email each day to help software development leaders build organizations the deliver value. Join us!


Get the book!

Ready to learn how to build an autonomous, event-sourced microservices-based system? Practical Microservices is the hands-on guidance you've been looking for.

Roll up your sleeves and get ready to build Video Tutorials, the next-gen web-based learning platform. You'll build it as a collection of loosely-coupled autonomous services, developing a message store interface along the way.

When you're done, you'll be ready to contribute to microservices-based projects.

In ebook or in print.