By now it’s pretty well known that a Microservices architecture has many advantages in terms of low coupling, reusability, business agility and distributed cloud ready applications. But at the same time it makes the architecture brittle, as each user action results in the invocation of multiple services. It replaces in-memory calls of monolithic architecture with remote calls across the network, which works well when all services are up and running. But when one or more services are unavailable or exhibiting high latency, it results in a cascading failure across the enterprise. And the retry logic in service clients worsens the situation for the service exhibiting high latency and brings it down completely.
The Circuit breaker pattern helps to prevent such a catastrophic cascading failure across multiple systems.The circuit breaker pattern allows you to build a fault tolerant and resilient system that can survive gracefully when key services are either unavailable or have high latency.
Everything fails, accept it!
The circuit breaker concept is straightforward. The execution of a function is wrapped or associated with a monitor of some kind, which tracks failures. When the number of failures exceeds a predetermined threshold, the breaker is tripped and any further call attempts return an error without executing the wrapped function. After a timeout period, the circuit is put into a half open state to test if the underlying problem still exists. If a single call fails in this half open state, the breaker is once again tripped. However if it succeeds we then reset the circuit breaker back to the normal state of a closed circuit.
As you can see, the circuit breaker has 3 distinct states – Closed, Open, and Half-Open.
When the circuit breaker is in the CLOSED state, all calls go through to the Supplier Microservice, which responds without any latency.
If the Supplier Microservice is experiencing slowness, the circuit breaker receives timeouts for its requests to the it. Once number of timeouts reaches the predetermined threshold, it trips the circuit breaker to the OPEN state. In theOPEN state, all calls are returned with an error by the circuit breaker, without making calls to the Supplier Microservice. This behavior helps the Supplier Microservice recover by reducing its load.
In order for the circuit breaker to know if the Supplier Microservice has recovered, it needs a monitoring and feedback mechanism. It uses this mechanism to make a trial call to the supplier microservice periodically to check if it has recovered. This is called the HALF-OPEN state. If the call to the supplier microservice times out, the circuit breaker remains in the OPEN state. If the call returns success, then the circuit will be switched to CLOSED state.
Example Circuit Breaker using Spring boot:
If you are using Spring Boot, you can use the circuit breaker implementation from Netflix Hystrix fault tolerance library.
|
|
|
@EnableCircuitBreaker |
|
@RestController |
|
@SpringBootApplication |
|
public class CampaignApplication { |
|
|
|
@Autowired |
|
private TrackingService trackingService; |
|
|
|
@Bean |
|
public RestTemplate rest(RestTemplateBuilder builder) { |
|
return builder.build(); |
|
} |
|
|
|
@RequestMapping(“/to-track”) |
|
public String toTrack() { |
|
return trackingService.getStats(); |
|
} |
|
|
|
public static void main(String[] args) { |
|
SpringApplication.run(ReadingApplication.class, args); |
|
} |
|
} |
view rawgistfile1.txt hosted with ❤ by GitHub
“EnableCircuitBreaker” annotation tells Spring Cloud that the Campaign application uses circuit breakers that will enable their monitoring, opening, and closing as per availability of tracking service.
Advantages:
The circuit breaker is valuable for monitoring; any state change in the circuit breaker should be monitored, logged and recovered to ensure availability of services.
Testing of various states of the circuit helps you to add logic for a fault tolerant system. For example, if the product catalog service is not available (the circuit is open), then the UI can fetch the product list from the cache.
The circuit breaker pattern handles downtime and slowness of key services gracefully and help those services recover by reducing load.
Future Reading:
https://martinfowler.com/bliki/CircuitBreaker.html
https://spring.io/guides/gs/circuit-breaker/
