Micro-services Using go-kit: Rate Limiting


In the service world, usually we need to limit the number of requests in order to protect our services does not get overwhelmed.

Sharing is caring!

Overview

Speed Limit

In the previous article, we already add logging functionality into Lorem Service. But it is not a production-ready yet. We still need to add middleware layer into the service in order to leverage its capabilities.

Therefore, in this article I will bring one of the middleware capabilities, which is instrumentation. Specifically, I will talk about rate limiting with service middleware.

Rate Limiting Service

In the service world, usually we need to limit the number of requests, in order to protect our services does not get overwhelmed. Because sometimes a request consume huge number of CPU processes. Or it is greedy in memory consumption. All those things, combine with multiple requests in a period of time, can lead into performance degradation.

Token Bucket Limiter

For this purpose, I am going to add a token-based rate limiter algorithm. In brief, we will have a bucket of tokens, and each request will take a token in order to continue the process. If there is no token left, the request cannot be completed. Also, the bucket will refill the token at a specific interval. More details about this algorithm, please read it on wiki.

For this purpose, there is a Go library from Juju to implement this algorithm. Furthermore, Go-kit has a middleware built-in function for this implementation as well, so both of them will minimize our efforts.

Middlware in Go-kit

In go-kit, middleware endpoint is a function, which takes endpoint.Endpoint as in input and return endpoint.Endpoint as well.

Just for your note, endpoint.Endpoint is also a function:

Step by Step

It is time to implement all the talks into code. Amazingly, you don’t need to make a big change into existing code. Now, I will use existing code, lorem-logging, and enhance its capability by adding rate limiter. Before it begins, we need download required library first.

Step 1: instrument.go

Create file, give it a name instrument.go. Then add NewTokenBucketLimiter function. The function takes token bucket as an argument and return endpoint.Middleware. One note, before continue to pass execution into next endpoint, we will check whether a token is available or not by using TakeAvailable function.

Step 2: main.go

In this step, we need to define the token bucket ratelimit.NewBucket function. This example will refill the token bucket every seconds, up to maximum five tokens. Please take note, this number is not for production use case because it is too small. I use this number for simulation purpose only. Next, add the middleware function into existing endpoint.

Step 3 – Test

That’s all you need to make it run. Very simple, isn’t it? Now it is time to test it. Run the server and make several requests. And at some point, you will get error response which tells you Rate Limit Exceed.

Options

Maybe you feel unsatisfied if the service return error if the token is unavailable. Also, it is pretty harsh. It is better to make a request waits for a while to make a token is available. For this purpose, go-kit provides NewTokenBucketThrottler middleware. To make this work, just replace

Conclusion

Whenever you create a service, do not forget to limit the number of requests that need to be processed. Especially for service that consumes lot of CPUs and/or memory usage. So your service performance is still in an acceptable level.

Yet, I still amazed about how go-kit works. They make it so modular, therefore whenever I want to add another service capabilities it works like a charm.

That’s all from me. You still can see the source code on my github account.

Reference

  • Go Programming Blueprints – Second Edition by Mat Ryer

 

Author: ru rocker

I am a professional software developer with more than 10 years experiences. I am a certified Java Developer (SCJP and SCWCD). However, In the recent months, I have more interest in DevOps and start to become a polyglot developer. Python and Go-lang become my favorite programming languages besides Java.

Leave a Reply

Your email address will not be published. Required fields are marked *