Idempotent API requests: Ruby on Rails

Idempotency is an attribute of designing safe and reliable APIs for clients to use without worrying too much about throttling, retrying or keeping track of HTTP requests. An API call or operation is idempotent if it has the same result no matter how many times it is applied/called.

Idempotency is not only relevant to API design but you can apply the same principles to the critical section of your application where accidentally making duplicate requests are fatal for your users and for your business such as sending emails, processing payments, Internet of Things operations and many more examples just like this. In short, we need to support idempotency and especially via our API interface since we do not have control over clients who are making requests and triggering operations on our system and in turn calling critical sections of our code.

If we are talking in mostly REST API terms, GET, DELETE, PUT, UPDATE are by nature idempotent operations which means does not matter how many times clients make GET or DELETE requests for example, it will not have any side effects but always gives you the same result.

def show
 user = User.find(params[:id])
 ...
end
def update
 user = User.find(params[:id])
 user.update!(name: params[:name])
 ...
end

For example, you have show/update endpoints that find record and updates record respectively, no matter how many times client makes calls to these endpoints with same params, they will get the same response and our application will be in a valid state without creating any side effects. However.

POST requests are not idempotent by nature

def create
 user = User.create!(name: params[:name])
 SendUserConfirmationEmailJob.perform_later(user.id)
 ...
end

More often than not, we have Createendpoints where the logic looks like the aforementioned where clients send data for a new user and our API creates a new record for requested data and queues some additional operations for the newly created user. This is very common. And, now this should already start looking problematic to everyone who had made this far in this article that for the same params in the request payload, your API will not yield the same result since it for duplicate requests with the same user name it will create duplicate records and your application already is broken. I know, this was a very naive example so let's take one more step further and make it a little bit smarter,

def create
 user = User.find_or_create_by(name: params[:name])
 SendUserConfirmationEmailJob.perform_later(user.id)
 ...
end

There are still issues with this design since if the duplicate requests are being processed simultaneously by your Rails application, it will queue up CREATE command to your database and you still could end up with duplicate results. You can solve it by adding UNIQUE INDEX but that could still end up blowing up your app since your database trying to process duplicate requests simultaneously end up throwing a Unique constraint error and ActiveRecord will throw RecordInvalid an exception which you might have to catch in your controller to return better errors to your clients. You might think this can do the job and it might for this simple case but I am pretty sure in real-world our API operations does not look as simple as this.

Read this Twitter thread to get an idea of what could happen if your APIs are not idempotent.

Network is unreliable

We also have to understand that even in the absence of duplicate requests, there are situations where the network could fail after the server has successfully processed the request but the network failed during sending the successful response to clients. In this case, clients are unaware of the request has been processed successfully or not so it makes sense for them to retry until they get acknowledgment from the server so we need to ensure our APIs are resilient and idempotent to handle multiple tries if need be.

Talking to 3rd party APIs

In the world of microservices, your app could be talking to 3rd party services to perform certain operations (processing payments, sending emails etc.) and their APIs could not be idempotent but we need to ensure that we have checks in place so that cascading requests do not end up making unnecessary requests to these external APIs which could result in a fatal mistake for our customer such as charging twice for a single order.

Reducing clients complexity

With idempotent APIs, we also reduce client complexity where they do not have to worry about retrying requests if needed, throttling API calls for a longer period of time and therefore blocking users from doing anything else on the page, etc, etc.

Conclusion

As we see here, there are many reasons why we should think about Idempotency when it comes to designing APIs as they provide guarantee and predictability with your application. Clients are much more relaxed and confident in providing a reliable User Experience.

Coming soon, follow-up blog post on different idempotent request patterns that you can use with your current Rails applications.

Comment below on what do you think about this blog post.