Deep Dive to Hystrix Resiliency Maintenance

In my previous Microservice Tutorial , I have shown How We can use Hystrix as a circuit Breaker and gives our service breathing room to recover itself.

In this tutorial, we will deep dive into Hystrix architecture and will eventually get to know How it manages resiliency in a complex system.

Let assume there are more than 100 Microservices In a complex system and to perform any business functionality more or less it depends on 5-10 underlying Microservices(approximately).

Generally, There are three types of services are found in aforesaid Microservice based architecture

1. Core Services
2. Aggregator service.
3. Edge Service.

Where the Edge service takes the request from UI/client then it forwards to an Aggregator service, Then Aggregator service fans out the request and sends the calls to core-services and Eventually collects responses from Core services and sends back to the Edge Service, Which actually sends the response to UI/Client.

Imagine we have a complex microservice based system where a single incoming request distributed over 10 dependent services, so one request then split to 10 internal requests (Fans out) and goes to individual core service, Now if there are 100 concurrent requests per seconds in a peak time.

So there are 100*10 = 10000 internal requests will be created per second.  Think about the load of the system, Even a minute of delay response from one of the core services can create a bottleneck as in web server(Tomcat, Jetty) there are fixed numbers of Threads in the Thread pool.

and we have 60*100*1=6000 requests are waiting for an individual service for a minute, So We can assume failure is inevitable even if you have 99.99% of uptime for a service-- because when we do a service call we have to depend on two external entities Network and Socket and the sad part is --The control of Network not in developers hand. To make the system resilience we have to deal with this phenomena and have to plan according to it.

Now let see the options we have to deal the above scenario.

Non Blocking request: If Requests are not waiting for the service irrespective of the service is available or not.Only then all request will be either served or rejected. If we follow this then there will be no requests in the waiting queue as if the service is not available or response is getting delayed immediately that request will be rejected and a fallback/default response will be provided to the caller service.  

So request has been short-circuited and the fallback path invoked -- dependent service gets a chance to recover itself. We know Hystrix do this stuff for us. But it is not a very easy task to implement. Internally a complex flow has been maintained by Hystrix to offer Resilient Microservice Architecture.

We will discuss that workflow now.

Deep Dive to Hystrix Workflow :

ThreadPool : Hystrix maintains a Thread pool so that when a service calls to another service --The call will be assigned to one of the thread from that Thread pool, Basically to handle the concurrency It offers Worker Threads. Each Thread has a timeout limit, If the response is not returned within the Thread time out mentioned then the request will be treated as a failure and fallback path will be invoked.

Network and Socket Timeout :  When a Service call to another service, Two things are happening, the data travels through Network and service talks each other via socket. When data travels through Network it has to cross multiple hops so if a hop is slow or down then we can feel network slowness so we have to calculate an average time for a request/response travel time as well as Socket read/write time to calculate thread timeouts in an optimum way.

Hystrix Workflow :  

1. when a service calls a dependent service Hystrix stepped in and checks Hystrix Circuit is open or not if it is open then it returns to the fallback path.
2. If Circuit is not open then  Hystrix checks are all the worker Threads in the pools are in use if so it returns immediately and the fallback path is invoked.
3. If threads are available in pools then it assigns one free thread and waits for the response from dependent service If response time is greater than thread timeout then it again invokes fallback path.
4. If all is well then the actual response is back to the caller service.
5. If for a certain amount of time (default 10 sec) if 50% of the request is failed then Hystrix opens the circuit.
   
   
   
   
   

It is very important to choose Thread pool and Thread time out wisely unless necessary implications are coming into the picture, if Thread pool count is large say it is greater the database connection thread pool  then in spite of the hystrix all the Connection are consumed by the service again if the Thread pools count is less then we can’t serve many request concurrently may be that will cause a performance hit. So choose the Thread pool count based on your system configuration.

Same for Timeouts if the timeout is big then there will more incoming requests in the queue but if Timeout is short then all calls are timeouts even if your service is healthy because the average response time of your service is greater than the Timeout.

Conclusion :  

Nowadays whatever architecture (Microservices/Monolith) you opt for your project, Resiliency is the foremost criteria, it is not an add on feature good to have it is the first class citizen now.

So Resiliency is mandatory -- when you are planning and developing your project also think about Resilience spend time on it, think how you can offer a Resilient Architecture. There are many strategies to achieve resilience like  Request Timeout, Maximum Retries, failing rate etc. So while developing your project Identify the resource intensive areas try to build it in that fashion, so that there is no resource hogging happens. Also, provide a Fallback mechanism so you can avoid cascade failures and eventually achieve a robust system.

Read more

Microservices Communication: Service to service

In the previous microservice tutorial, we have learned How Microservice communicates with the service registry. In this tutorial, we will learn How one microservice communicates with another dependent microservice service via the Service Registry/Eureka Server. This is the second part of Microservice Communication series.

Let see the sequence How One Microservice calls another Microservice using Eureka server.

Registering Service: All Microservices should be registered into Service registry with a Unique name {service-id}, So it can be identified please note that it is an important step as one of the main benefits of Microservice is autoscaling so we can’t rely on Hostname/Ip address so Unique name is important in distributed environment.

Fetching Registry: Before calling the downstream/dependent service Caller service fetch the registry from Eureka server, Registry contains all the active services register into service registry.

Find the Downstream service: Now using the unique service Id caller service get the instance of downstream service.

Resolve Underlying IP address : Please note the Iniques service id act as a Key in service registry but network does not know about it network expects Hostname to call the desired Rest Endpoint on the dependent service like(localhost:8080/employee/{id} or employee.cognizant,com/2 etc) so it is required to resolve the actual hostname of the dependent service Eureka API provides a method for that we just invoke that method to get the Ip address, For a distributed system it is the public IP of Load balancer.

Call the Rest Endpoint: After resolving the IP address using Spring Resttemplate we call the actual Rest endpoint and got the data.

Coding Time :

For this example we need Three Microservices Project

1. Employee Search Service : Which we created earlier for searching Employee information.
2. Eureka Server : Also ,we created this earlier we will reuse that same application
3. Employee Dashboard Service : We will create this module and call the Employee Search service via Eureka server to get Employee information.

Step 1:Create a service called EmployeeSearchSearch.java  where I  insert some employee using static block and using Java 8 Stream after that I add two methods findById and findAll to display Employee information accordingly.

package com.example.EmployeeSearchService.service;

import java.util.Collection;

import java.util.HashMap;

import java.util.Map;

import java.util.stream.Collectors;

import java.util.stream.Stream;

import org.springframework.stereotype.Service;

import com.example.EmployeeSearchService.domain.model.Employee;

@Service

public class EmployeeSearchService {

   private static Map<Long, Employee> EmployeeRepsitory = null;

   static {

      Stream<String> employeeStream = Stream.of("1,Shamik  Mitra,Java,Architect", "2,Samir  Mitra,C++,Manager",

              "3,Swastika  Mitra,AI,Sr.Architect");

      EmployeeRepsitory = employeeStream.map(employeeStr -> {

          String[] info = employeeStr.split(",");

          return createEmployee(new Long(info[0]), info[1], info[2], info[3]);

      }).collect(Collectors.toMap(Employee::getEmployeeId, emp -> emp));

   }

   private static Employee createEmployee(Long id, String name, String practiceArea, String designation) {

      Employee emp = new Employee();

      emp.setEmployeeId(id);

      emp.setName(name);

      emp.setPracticeArea(practiceArea);

      emp.setDesignation(designation);

      emp.setCompanyInfo("Cognizant");

      return emp;

   }

   public Employee findById(Long id) {

      return EmployeeRepsitory.get(id);

   }

   public Collection<Employee> findAll() {

      return EmployeeRepsitory.values();

   }

}

Step 2: Now add a new controller called EmployeeSearchController expose two endpoints by which other services can call findById and findAll method. findById takes Employee Id and returns the Employee Domain Object.

package com.example.EmployeeSearchService.controller;

import java.util.Collection;

import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.cloud.context.config.annotation.RefreshScope;

import org.springframework.web.bind.annotation.PathVariable;

import org.springframework.web.bind.annotation.RequestMapping;

import org.springframework.web.bind.annotation.RestController;

import com.example.EmployeeSearchService.domain.model.Employee;

import com.example.EmployeeSearchService.service.EmployeeSearchService;

@RefreshScope

@RestController

public class EmployeeSearchController {

   

   @Autowired

   EmployeeSearchService employeeSearchService;

   @RequestMapping("/employee/find/{id}")

   public Employee findById(@PathVariable Long id){

      return employeeSearchService.findById(id);

   }

   

   @RequestMapping("/employee/findall")

   public Collection<Employee> findAll(){

      return employeeSearchService.findAll();

   }

}

Employee Domain Object

package com.example.EmployeeSearchService.domain.model;

public class Employee {

   private Long employeeId;

   private String name;

   private String practiceArea;

   private String designation;

   private String companyInfo;

   public Long getEmployeeId() {

      return employeeId;

   }

   public void setEmployeeId(Long employeeId) {

      this.employeeId = employeeId;

   }

   public String getName() {

      return name;

   }

   public void setName(String name) {

      this.name = name;

   }

   public String getPracticeArea() {

      return practiceArea;

   }

   public void setPracticeArea(String practiceArea) {

      this.practiceArea = practiceArea;

   }

   public String getDesignation() {

      return designation;

   }

   public void setDesignation(String designation) {

      this.designation = designation;

   }

   public String getCompanyInfo() {

      return companyInfo;

   }

   public void setCompanyInfo(String companyInfo) {

      this.companyInfo = companyInfo;

   }

   @Override

   public String toString() {

      return "Employee [employeeId=" + employeeId + ", name=" + name + ", practiceArea=" + practiceArea

              + ", designation=" + designation + ", companyInfo=" + companyInfo + "]";

   }

   

   

   

}

Step 3: Create an EmployeeDashBoard application by downloading the template for this, I choose following modules actuator, config client, web, Jersey, EurekaClient.

Now put @EnableDiscoveryClient on top of EmployeeDashBoardApplication class. To treat this module as Eureka Client and add RestTemplate as a Spring Bean.

package com.example.EmployeeDashBoardService;

import org.springframework.boot.SpringApplication;

import org.springframework.boot.autoconfigure.SpringBootApplication;

import org.springframework.boot.web.client.RestTemplateBuilder;

import org.springframework.cloud.client.discovery.EnableDiscoveryClient;

import org.springframework.context.annotation.Bean;

import org.springframework.web.client.RestTemplate;

@EnableDiscoveryClient

@SpringBootApplication

public class EmployeeDashBoardServiceApplication {

   public static void main(String[] args) {

      SpringApplication.run(EmployeeDashBoardServiceApplication.class, args);

   }

   

   @Bean

   public RestTemplate restTemplate(RestTemplateBuilder builder) {

      return builder.build();

   }

}

Also, rename the application.properties to bootstrap properties and write the following properties.

spring.application.name=EmployeeDashBoard

spring.cloud.config.uri=http://localhost:9090

eureka.client.serviceUrl.defaultZone:http://localhost:9091/eureka

server.port=8081

security.basic.enable: false   

management.security.enabled: false

Step 4: Now create a Controller called EmployeeInfoController , and call the  Service Registry then find the EmployeeSerchService by passing the service-id of the Employee Service see (EmpoyeeService-> bootstrap.properties)

Now call IpAdress method to resolve Ip address and call the dependent service using RestTemplate.

package com.example.EmployeeDashBoardService.controller;

import java.nio.file.Path;

import java.util.ArrayList;

import java.util.Collection;

import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.beans.factory.annotation.Value;

import org.springframework.cloud.context.config.annotation.RefreshScope;

import org.springframework.http.HttpMethod;

import org.springframework.web.bind.annotation.PathVariable;

import org.springframework.web.bind.annotation.RequestMapping;

import org.springframework.web.bind.annotation.RestController;

import org.springframework.web.client.RestTemplate;

import com.example.EmployeeDashBoardService.domain.model.EmployeeInfo;

import com.netflix.appinfo.InstanceInfo;

import com.netflix.discovery.EurekaClient;

import com.netflix.discovery.shared.Application;

@RefreshScope

@RestController

public class EmployeeInfoController {

   

    @Autowired

    private RestTemplate restTemplate;

   

    @Autowired

    private EurekaClient eurekaClient;

   

    @Value("${service.employyesearch.serviceId}")

    private String employeeSearchServiceId;

   @RequestMapping("/dashboard/{myself}")

   public EmployeeInfo findme(@PathVariable Long myself){

      Application application = eurekaClient.getApplication(employeeSearchServiceId);

       InstanceInfo instanceInfo = application.getInstances().get(0);

       String url = "http://"+instanceInfo.getIPAddr()+ ":"+instanceInfo.getPort()+"/"+"employee/find/"+myself;

       System.out.println("URL" + url);

       EmployeeInfo emp = restTemplate.getForObject(url, EmployeeInfo.class);

       System.out.println("RESPONSE " + emp);

       return emp;

   }

   

   @RequestMapping("/dashboard/peers")

   public  Collection<EmployeeInfo> findPeers(){

      Application application = eurekaClient.getApplication(employeeSearchServiceId);

       InstanceInfo instanceInfo = application.getInstances().get(0);

       String url = "http://"+instanceInfo.getIPAddr()+ ":"+instanceInfo.getPort()+"/"+"employee/findall";

       System.out.println("URL" + url);

       Collection<EmployeeInfo> list= restTemplate.getForObject(url, Collection.class);

        System.out.println("RESPONSE " + list);

       return list;

   }

}

Then Up the services in following Order

1. Start config server.
2. Start Eureka Server.
3. Start Employee Search Service.
4. Start Employee DashBoard Service.

When all services are Up-- hit http://localhost:9091 in browswer you will see all services are up and Running

Add caption

Then hit the following URL http://localhost:8081/dashboard/2

You will see the Following Output.

{"employeeId":2,"name":"Samir  Mitra","practiceArea":"C++","designation":"Manager","companyInfo":"Cognizant"}

Read more