Function Curry in Java8




One of the important features of a functional program is Function Currying.As we know java8 introduce lambda which brings some functional nature in java programming. We are blessed that by an intelligent use of Lambda we can create Function currying by own. Although Java8 does not have Currying facilities we can create it by ourselves. In this section, I will discuss the detail step How to create Currying in Java 8.



What is a Function Currying?
To understand the function currying let first understand what is a Partial function.

A partial function is the application of a part of a function.

To be very simple, Suppose I have a java method called add which takes two arguments as input then capability of applying one argument is called the Partial function.

Maybe you are astonished in this moment why the hell we need to do that?

Let me tell you, This is a very rich concept and it helps you to write a rich API and reduce the Repeating of yourself (DRY). It actually eliminates the conceptual duplication.

Now I am trying to describe it with a example so you can understand the above paragraph well.

Suppose, I want to write a generic compute function which takes two inputs and takes a BiFunction (Java 8 functional Interface) so I can pass behavior on the fly which will be applied to the two inputs argument.

Example

public  Integer compute(Integer a, Integer b,BiFunction<Integer,Integer,Integer> function){      
       return function.apply(a,b);
    }


By this compute method I can apply add, multiply, division whatever on these passing parameters a and b.

So I will call the Compute function in following way
BiFunction<Integer,Integer,Integer> addFunction =  (a,b)-> a+b;
Integer summation = fc.compute(50, 20, addFunction);
System.out.println("Summation is " + summation);

The output will be 70, as BiFunction called addFunction which do the adding.

Now suppose we have to build another logic, Say I pass a number then a base will automatically apply to it. The base value is  10.

So if the base is 10 and I pass 20 then the output will be 10 + 20 =30(base + actual parameter).

How would you write this function?

Very easy isn’t it, we should write it in the following manner

public Integer baseCompute(Integer value,Function<Integer,Integer> function){
       return function.apply(value);
    }


It takes a value and a Function which takes one Integer as input and return another Integer So that this function takes the passing value apply base to it and return the resultant value.

We call this baseCompute method by the following call,

Function<Integer,Integer> baseFunction =  (value)-> 10 + value;//declare base
      Integer baseResult = fc.baseCompute(20,baseFunction);
      System.out.println("Base computation " + baseResult);


The output will be 30.
So far so good, But if you are a Clean coder you are not happy with it because you observe

BiFunction<Integer,Integer,Integer> addFunction =  (a,b)-> a+b;
Function<Integer,Integer> baseFunction =  (value)-> 10 + value;

The addFunction and baseFunction body are same, so you duplicate your code breaking the DRY principle. So you must search for an option where you can reuse the same addFunction instead of writing a new baseFunction, But the problem is the Structure of addFunction is different from the base function. addFunction takes two parameters and base takes one, But conceptually both are doing same work,

addFunction add two parameters, baseFunction takes one parameter and add base value internally.

Think How you can solve the problem while I taking a short break for breakfast.

We can solve this problem, by fixing a parameter in the bi-function. As we know the base value is 10 so if we apply that value and take another value from caller then we can reuse the BiFunction.

So we could rewrite the baseCompute method like following

 public Integer applyPartialInternalCompute(Integer value,BiFunction<Integer,Integer,Integer> function){
       return function.apply(10,value);
    }


And calls the above method like following

BiFunction<Integer,Integer,Integer> addFunction =  (a,b)-> a+b;
Integer baseInternal = fc.applyPartialInternalCompute(20, addFunction);
 System.out.println("Base Internal " + baseInternal);



Congrats you have written your first partial function, Where you fixed you first parameter and takes a parameter from Caller.

It is the essence of Partial function apply a partial part of the function and returns a new function which can take remaining arguments. So it is consists of two thing



  1. Applying a part.
  2. Return a new function after applying one argument.

Mathametically

f ( x , y ) = x+y
After applying the  first parameter of function
f(2,y)=N(y)=2+y; where N(Y) is a new function returned after currying.


But what we did in our applyPartialInternalCompute method is not returning a new function or the intermediate function after partial applying the first value- Which is base value,  so it is not properly curryed this is very basic version, Let see how can we improve the function so we can get the new intermediate function after applying the first argument base.

public Function<Integer,Integer> applyPartialExternalCompute(Integer base,BiFunction<Integer,Integer,Integer> function){
       return value->function.apply(base,value);
    }


This is a more sophisticated version, It takes a Bi-function and a base value, Then apply the base value to the BiFunction and return a new Function which takes One parameter from the caller and returns another Integer.

So this function works as an adapter, It takes Bi-function applying a parameter or fixing a parameter and returns new Intermediate function to work upon,


BiFunction<Integer,Integer,Integer> = Fixed Base value + Function<Integer,Integer>

Please note that the returning function is looks  like of our baseFunction   Function<Integer,Integer> baseFunction =  (value)-> 10 + value;//declare base

This is called Function Currying, so we can use the BiFunction at fullest, So a BiFunction can take two parameters also it serves any functionality which takes one argument by fixing the first one.

How to call the applyPartialExternalCompute method,

Integer basepartial= fc.applyPartialExternalCompute(10, addFunction).apply(20);
System.out.println("Base External " + basepartial);


Our above solution is very good, and we properly use function Currying to remove conceptual duplication but wait there is still some glitch in the method we have written.

It takes Bi-function and a value and returns a Function, So every time we invoke this method it applies the parameter to BiFunction and return a new Function, But why should we perform two steps every time if we got the readymade intermediate function then we just apply our values,
Suppose there is a function which takes three arguments then we have to write two adapter functions one takes tri-function and return a Bi-function and another takes Bi-function and return a function,

So if parameter values are n we have to write n-1 adaptor methods. Which is again a duplication, So if the Curry function itself generates all intermediate structures then it will be a full proof solution lets implement this,

public   Function<Integer,Function<Integer,Integer>> applyCurryingCompute(BiFunction<Integer,Integer,Integer> function){
       return value->(base->function.apply(base,value));
    }


Yes, we are waiting for this implementation, and it is the Crux of this article, By above Curry function we should have every intermediate function and we can do any functionality by fixing others,

It takes a Bi-Function which returns an Intermediate function which takes an argument and returns a function which again takes another argument and lastly returns an Integer.

Hard to understand,

Then see the following transformation to understand what actually going on

(x,y)->z = x->(y->z)   BiFunction changes it structure after currying and create a new structure
(Integer,Integer)=Integer - > Integer =Integer->( Integer->Integer)

f(y)=x->f(x,y)=z where x->f(x,y) means x applied on Bifunction creates a function of y which is equivalent to Z

So (x->y)->z = x->(y->z).


How to call above applyCurryingCompute method

Integer baseCurry= fc.applyCurryingCompute(addFunction).apply(10).apply(20);
System.out.println("Base Curry " + baseCurry);


The full version of the program

package com.example.function.curry;

import java.util.function.BiFunction;
import java.util.function.Function;

public class FunctionCurrying {
   
    public  Integer compute(Integer a, Integer b,BiFunction<Integer,Integer,Integer> function){      
       return function.apply(a,b);
    }
    public Integer baseCompute(Integer value,Function<Integer,Integer> function){
       return function.apply(value);
    }
   
    public Integer applyPartialInternalCompute(Integer value,BiFunction<Integer,Integer,Integer> function){
       return function.apply(10,value);
    }
    public Function<Integer,Integer> applyPartialExternalCompute(Integer base,BiFunction<Integer,Integer,Integer> function){
       return value->function.apply(base,value);
    }
   
    public   Function<Integer,Function<Integer,Integer>> applyCurryingCompute(BiFunction<Integer,Integer,Integer> function){
       return value->(base->function.apply(base,value));
    }
   
   
    public static void main(String[] args) {
       FunctionCurrying fc = new FunctionCurrying();
       BiFunction<Integer,Integer,Integer> addFunction =  (a,b)-> a+b;
       Function<Integer,Integer> baseFunction =  (value)-> 10 + value;
       Integer summation = fc.compute(50, 20, addFunction);
       System.out.println("Summation is " + summation);
       Integer baseResult = fc.baseCompute(20,baseFunction);
       System.out.println("Base computation " + baseResult);
       Integer baseInternal = fc.applyPartialInternalCompute(20, addFunction);
       System.out.println("Base Internal " + baseInternal);
       Integer basepartial= fc.applyPartialExternalCompute(10, addFunction).apply(20);
       System.out.println("Base External " + basepartial);
       Integer baseCurry= fc.applyCurryingCompute(addFunction).apply(10).apply(20);
       System.out.println("Base Curry " + baseCurry); 
     
     
   }  
   

}




Conclusion: Function Currying is a rich technique, In the functional program we use it often to get rid of conceptual duplication. You can use it in Java 8 now so Please use the concept wisely.

Techniques for reducing Tight Coupling

“Tight Coupling is Bad” How many times you have heard this word from your seniors. Probably many many times.
But why Coupling is bad what are the implications comes if you do tight coupling?
What is actually a Tight coupling?
How we can fight with it.?
In this tutorials, we will dig the answers.

Coupling: In a simple term coupling is when a Class or Interface dependent on another class/interface i.e has a HAS-A relationship.

Example :

Class Vehicle{

private Wheel wheel =new Wheel();

}

In above example, Vehicle is dependent on Wheel, Which means without creating Wheel Object we can’t create Vehicle Object if anything goes wrong while creating Wheel Object vehicle will never be created. Also if we need to test Vehicle, first Wheel Object has to be created then Vehicle can be tested. Without, Wheel Vehicle has no existence. This type of coupling is called Tight Coupling, We know Vehicle must contain Wheel, To make the statement more generic sometimes we have requirements where a class must have to contains other classes to fulfill its purpose, they can't-do thing independently. So Coupling is inevitable, it can not be avoided but by programming technique we can make it pluggable in such a way so that we can reduce the degree of coupling so Dependable and Dependent class/interface can be changed without impacting each other. We called this technique as Loose coupling. I will show you some techniques which reduce the coupling between Objects.

Creation of Objects : Often while we doing coding we direct create the dependable Object instance, either in a init method or in Constructor or supply it through setter/constructor. But it is always risky. Once you have done that, Then you lose the flexibility if requirement changes in future you have to change the Dependent Object to accommodate the change in dependable Object. Let say All Ford cars use MRF Wheel So they are dependent on MRF wheel, Now if they change the mind and want to use Another company’s wheel then they have to change all car's Wheel Object creation so it is against the Open-Close principle.

Ex.
public Class Ford{
MRFWheel wheel;
Ford(MRFWheel wheel){
this.wheel =wheel;// replaced by new JKWheel()
}




}

So the best practices would be If you think dependent object will be changed frequently or may have multiple implementations always create an Interface and use that interface as a reference so anytime you can change the implementation without affecting the dependent class. But if you are sure about the dependent Objects behavior will not change then unnecessary don’t create interface it again against YAGNI and KISS.

IWheel{
//wheel related methods
}

public class Ford{
IWheel wheel;
Ford(IWheel wheel){
this.wheel =wheel;// replaced by new JKWheel()
}




}

Assuming MRFWheel and JKWheel are the subclasses of the IWheel

new Ford(new MRFWheel());
new Ford(new JKWheel())



Use Static factory method for creating Object : While creating an Interface for multiple implementations is good as you can change implementation dynamically. But if the dependable Object implementation is changed then again you have to change the all dependant Object which again breaks the Open/close principle.

Say Now Wheel take Air Pressure as a Constructor arguments then the caller of the Ford car has to change its logic, as multiple cars have dependencies on Wheel every implementation will break due to this change.


new Ford(new MRFWheel(AirPressure pressure));
new Ford(new JKWheel(AirPressure pressure))

The problem is we do not centralize the Object creation so all the caller has a responsibility to create the Objects and when the project grows it is a tedious job to find all references and fix them in case of a change in business logic in dependable Object. We certainly reduce our effort, if we use a Static Factory method to create the instances, So we centralize the creation of dependable object all dependent objects refer static factory method to get the dependable Object.So if any implementation details change it will only affect the Static factory method, Unless the method signature of the Static factory method is changed.

public static IWheel createWheel(WHEEL wheel){
if(WHEEL.mrf.equals(wheel)
new MRFWheel(AirPressure pressure)
}
else if(WHEEL.JK.equals(wheel)
new JKWheel(AirPressure pressure)
}else{
New DumyWheel(AirPressure pressure)
}

calling:

new Ford(WheelFactory.createWheel(WHEEL.mrf)));
new Ford(WheelFactory.createWheel(WHEEL.JK)));



Don’t take dependable Object Responsibility : Often knowingly or unknowingly, caller take the responsibility of dependable Object, which is breaking the encapsulation and it is the most common coding mistake I have seen, Not judging the Cohesion properly and it breaks another principle Tell Don’t Ask. Which increases unnecessary coupling and gradually your code not welcoming any future changes. Let's take a Simple example how we take dependable Object responsibility.  Say Ford car has a method which shows the specifications of the car in very detail manner. So when it shows the Wheels Specifications often we do code like this in Ford Class.



public void FordSpecification(){
//Ford car specific specifications
//then
wheel.getAirPressure();
wheel.getManufactureDate();
wheel.getBrandName();

}

But it has a severe problem if in future Wheel specification is changed if it adds or removes any attribute it has a direct impact on Ford class, So all Ford Car classes have to be changed to incorporate the specification changes of the wheel.

It is wheel specification changes so why Ford class would be the sufferer?
Because while coding we did not understand the Wheel class responsibility, It is wheel class responsibility to provide specification through a specification method which uses by the Ford.

In Wheel class

public void specification(){
wheel.getAirPressure();
wheel.getManufactureDate();
wheel.getBrandname();

}

In Ford Class

public void FordSpecification(){
//Ford car specific specifications
//then
wheel.specification();

}


If wheel specification changes it does not impact the Ford class.


Try to reduce Hide-coupling :

Hide Coupling means from the API, you can’t understand there is and Dependency inside it.

It is a common programming technique where we hide the coupling from the user, Like create necessary Objects inside init method or constructor. I am not saying this is bad but , When you hide a Coupling think very carefully , If the Object is a kind of Utility, Connection pools, Worker thread that is fine but if it is a normal business Object, always provide an option for setting the Object from Outside, So user can set a Dummy or Mock Object while testing, Unless as a developer it is very hard to track down why the Object is not created as user does not aware of hiding coupling



Take the first example again

Public Class Ford{
MRFWheel wheel;
Ford(){
this.wheel =new MRFWheel();
}




}


From the API of Ford, it is impossible to say Ford has a dependency on MRFWheel. You will discover it in runtime if  MRFWheel Object is not created from the stack trace. But if you change the implementation.


Public Class Ford{
IWheel wheel;
Ford(IWheel wheel){
this.wheel =wheel;// replaced by new JKWheel()
}




}

Then we can inject a DummyWheel while unit testing the Ford specific method.


Conclusion : Tight Coupling always creates a Big Ball of Mud. And gradually loses the flexibility to incorporate changes. Always take a closer look for coupling while writing code-- A silly mistake can cost you very much in near future. If you take above best practices most of the time you will be on safer side.



5 best practices for Building and Deployment Microservices

Many organizations who are going to adopt or adopted Microservices culture they often think Microservices means breaking a big business functionality into small independent services, which can be scaled automatically. But this is the half part of the story, Another part is often ignored but the other part is equally important and if you do not implement the other part please go back and sit with your architects as you are not getting the full advantages of Microservice culture. The other part is an efficient way to handle build and Deployment so developer code is always production ready.

Today I will discuss 5 best practices for build and development for Microservices.



  1. Independent Deployment : When an organization adopting a Microservice culture, The first rule has to be set is, each and every Microservices should be independent. Here what I mean by independent is, Each Microservice should have a separate code base, publish/consume well-defined API, Separate persistence layer, Separate build, and deployment script, and separate deployable artifacts.  So Developer of the Microservices own the process end to end, They must have complete control over their services in terms of technical and management, which help them to protect themselves from any external impediments. When you building a Microservice you should know Other Microservices API which you will go to consume, And also publish a API so that another Microservices Consumes your Service by doing that you can become independent as you know the request/response for the services you consume so you can MOCK them and work independently, also as you publish your API , there is well-defined method signature so unless you change the API signature, whatever you can do with your codebase, even you can deploy it thousand times in a day that will not affect others. Also in Management perspective Team must own the build and deployment pipeline, administrator access in each and every environment even production!!! Should have complete control everything related to Administration, If your organization does not allow this then you are not doing Microservice and not get its essence. Sometimes dependency with Other services is unavoidable( Not management related those are in our hand), or to cope future changes you need to revamp your API, But those are the Exception cases, in my previous article I had a talk about how to handle the same, But again that is exception, By rule every service is independent has its sole identity, Team related to this service owns everything they are all in all of this service. This brings the speed of development and as an organization, you can publish features to end user easily.

  1. CI/CD : One of the main causes of slowness in publishing feature in production is dependency between Dev team and Ops team, Dev team holds the technical part and ops holds the environment for it , This causes a barrier like( communication, availability of environment/artifact etc) as they depend on each other, In a competitive market even a delay of one hour matters, This is not the way to adopt Microservice culture, Ideal Microservice environment would be anytime any code from developers environment  can be production ready , If developers code pass all criteria then it can be published to production within no time and there may be a manual intervention to publish it in production but else should be automated, Microservice culture welcomes DevOps culture automatically. Without DevOps Organisation not utilize the power of Microservice fullest. There should be a build pipeline whenever user checks in a code it will trigger a build then artifacts should be in UAT environment where unit testing will be done then it moves to SIT and integration testing will be fired at last it can publish to production with or without manual intervention.

  1. Environment Replication : In Project Management lifecycle, our traditional approach is to create the different environments like UAT environment, SIT environment, PROD environment etc. This is really a good approach but the problem is that environments are not identical, In SIT we have few servers , A Load balancer manages them but in Production the Infrastructure is totally changed there may be a Primary and Secondary pool of servers for Blue/green deployment or they may have different data center based on geographical affinity. So one artifact different Topologies. One artifact tested against multiple environments, as a result,  we can’t predict how the artifacts behave in Production as we do not have production like infrastructure to test on. The most common bug comes from production is how artifacts behave when there is a heavy load. So to identify how exactly code behaves it needs an Identical environment like production with Load testing capability. It can be done if organization adopt IAAS, Where we can spawn machines seamlessly and clone the developer's environment. So our artifacts should not be only jar, war, gems. An Artifacts should contain the whole environment i.e the OS, Application server, persistence layer all ancillary settings. Nowadays we can achieve it through IAAS(Infrastructure As A Service). Also, Docker is a very popular option to spawn a container and by Docker file, we can instruct what should be packaged inside the container. Puppet and Chef also a good option to achieve that. If you want it should be managed by the third party you can go for AWS or Cloud Foundry, BlueMix etc.



  1. Artifact Repository :  Although it comes with CI/CD I want to highlight this feature. Why is an Organization adopting Microservice architecture in terms of Business perspective?
The answers are
  1. They want to release their feature quickly to end user to stay ahead of competitors.
  2. No Downtime which increases reputation.

    To achieve this organization has to have a robust framework where it can store all the artifacts with its version number as if any production issues occur with current release either we can patch the fix and deploy it to production or if it takes time to fix we can easily revert the changes and deploy the old artifacts so user does not have to wait for fix for next release. So it is utmost important to maintains the artifact repository and it should be very easy to deploy in production, No cumbersome processes so that it needs a special expertise, anyone can do this it should be like a command in CLI.

Like deploy <ServiceName><Version ID><Environment> where
Service ID: Logical Name of the Microservice.
Version ID is the Version you want to deploy.

Environment: Which has a logical name like prod1,prod2,sit1 internally it contains all the information of the servers and apply artifacts to all servers.



5. Scrum of Scrums : We know that one Microservice holds one Domain information. So When an organization wants to build a feature which distributed over multiple domains naturally the work should be distributed in multiple Agile teams. Although in Microservice culture each Team is independent as they can mock the other services, but make functionality works fully it needs an Integration.  So it is utmost important every team's Work Item should be in Sync so that In a defined timeframe all teams did their jobs and all the works can be integrated and tested in the Integrated environment. So Organization must have to conduct a Scrum of Scrums where each team's Scrum master and Product owner sit together and decide the priority items so that business functionality published in production according to the announced release.



Conclusion : Breaking a Complex Business domain into small moving parts and each part work independent is just one side of the coin, To make and publish new features in a hassle-free manner we need to focus on the Management structure of the organization also.  

I am curious to know from The organization who adopted or about to adopt microservice culture how they change their Management policy or what are the other management related challenges they faced when they move into Microservice Culture.


5 Best Practices for REST based MIcroservice

In this tutorial, we will discuss 5 best practices by which you can make your Microservice architecture developer friendly so they can manage and track the error easily.

  1. User Agent : It is very important to provide a meaningful name in request header so if any problem like slowness, huge memory access, spike etc occurs developers can understand from which microservice this request is originated. It is the best practice to provide the logical name/{service id } in the User-Agent property in the Request header.
Ex : User-Agent:EmployeeSearchService



  1. API Version Control :  In REST based Microservices architecture, One Microservice access another Microservices resources via API. So API acts as a Facade to other Microservices. The utmost important part is -- written API in a judicious way so that API is not changed frequently because other Microservices consume the same so any changes in the API method signature is breaking other codes, but the Irony is Change is inevitable as we don’t know the future so today's business strategy may become old in a few days so API must be revamped. As an architect the main challenge is how to cope with these changes, Answer is maintaining the Version, For a major changes you can update the API version and provide notice to your consumers that new Version is arrived so please migrate to the new version within a defined timeframe -- In this time frame as an API provider you have to maintain both versions and constantly send high important notice to your consumer to migrate there codebase call to new version and after that period you can decommission your old version.

  1. Correlate ID :We know that  a business functionality in a MIcroservice architecture is distributed over multiple microservices, So one request from a client internally fans out to many separate requests  and it is highly probable that one of the services is slow or down, But as a developer how to track which service is slow in the Microservice forest, So  somehow we need to group the requests logically. So it is always a good practice to generate a Random UUID for a client request and pass that UUID to every internal request so in a log we can track by the UUID and find the call trace accordingly.

  1. ELK Implementation : Microservice is meant for autoscaling, so in a complex business domain it is very hard to manage Log files for Microservices, say In a system there are 50 microservices involved and each has 10 instances so there will be 50*10=500 logs file will be generated so as a developer it is not possible to log into each instance and collect logs for investigate an Issue. So we need a Centralized mechanism where all logs are dumped and we can do some intelligent search over that log like find out Error or a particular exception or search by the host, correlate ID etc. ELK provide the same where E stands for ElasticSearch L is Logstash and K is Kibana. ElasticSearch dumps the logs and provide a fuzzy search capability Logstash used for collect logs from different sources and transform it. Kibana is a Graphical User Interface where a developer can search the logs as there need. Alternatively, you can use Splunk or any other opensource framework for centralizinig and analysis the log .

  1. Resiliency Implementation : As I said earlier in a Microservice architecture -- many microservices are involved to complete a business functionality. So it is very common that one of the services is not responding which will stop the whole flow to handle such scenario Resiliency is utmost important and each and every service should implement resiliency to provide a seamless experience to the end user.When a service is not responding a certain amount of time there should be a fallback path so user are not wait for the response but immediately notified about the internal problem with a default response. It is the essence of a responsive design.




Conclusion: when building a REST-based Microservices think about two perspective
1. User experience
2. Developer perspective.
A User does not like to wait and S/he needs a clear nontechnical message what is going wrong if there is an internal problem occurs so S/he can communicate with help desk properly. On another hand when it's time for support the microservice we need the all important information in the log as the incident is already happened and based on the log we are analyzing it. So always think about support perspective while developing a Microservice so you can track a flow and get sufficient information from a log.

Deep Dive to Distributed Service Registry

In my previous article, I discussed how to maintain Resiliency in Microservice/Distributed Architecture. In this tutorial, I discussed Distributed Service Registry.


What is a Distributed Service Registry ?

In a service registry pattern, all the services are registered in a Registry(A Map Data Structure). If any service needs an instance of an another service it contacts Registry and gets the service instance. Very simple isn’t it But when we deal with a distributed environment it's getting highly complex.

Let’s discuss why it is getting so complex in a distributed environment. When we are talking about the distributed environment we assume that each service is deployed in a separate web server/application server to be very simple each service runs in a separate JVM. per service per JVM. As Service registry holds the service name and its instances in a key/Value pair so we need a centralized system which only maintains this service registry and performs CRUD operations on it and each service contact this registry for getting required services, so far so good. But the problem is when we make service registry centralized i.e a separate service or JVM we bring a Single point of failure in our architecture.  Think about the scenario if the Service registry is unavailable then our application falls down as no service can contact service registry so it not gets the desired service and fails. So makes service registry centralize is a bad idea.

See the following picture and think how can we improve the architecture.

Service Registry
 
One thing we can do to improve the architecture if we maintain multiple instances of  Service registry so if one is unavailable other can serve the purpose then there will be no SPF(Single point of failures).

Distributed Service Registry Algorithm

The idea is like there will be one Leader Node(Service registry) all the CRUD operation applied on this node, and it is leader node duty to distribute the updated state of the registry to another Service registry(peer nodes). Although by reading it seems very easy and robust solution but it is very complex to implement and a new set of challenges comes with this architecture.

I am trying to talk about the challenges one by one

Leader Election: As the Service registries are connected to each other How we can select one node as a Leader also if that node fails basis on which we will elect another leader who will take over the responsibility?

State Synchronization: As there are multiple service registry nodes now the question is if Leader registry got updated by adding a new instance or deleting an instance then how leader node distribute that state to others and how it becomes to know that all nodes are updated with the latest state?


To solve these problems Distributed registry take helps from Algorithm.




Bully Algorithm for Leader Election :  In Bully algorithm each node/Service registry has a process Id, greater process Id will select as the Leader node. Suppose assume Leader node is not available then How Bully Algorithm selects the next leader.

Step 1: When a node discovers that Leader node is unavailable it starts the election by sending a claim message to all other nodes who have greater id than this node.

Step 2:  When greater nodes are received this claim message, they either show interest or not provide the response for the message, If they are interested they send previous node a response to stop as they will take it from here and send claim message to all other nodes which have greater Id than this.

Step 3: If it does not receive any response from the higher node, then it selects itself as Leader node and sends message to all other nodes by saying it is the Leader.

Bully Algorithm

State Synchronization by Consensus: If any instances are added or deleted from the leader node it has to be reflected on other nodes so every service registry got the same value. To decide the correct state of the registry majority of the node has to agree on a certain value/latest value.  And faulty nodes needs to update its state to be sync with other nodes and system will be eventually consistent.

Consensus works on two major criteria

Value proposition : Each node/Service registry propose a value..
Agreement: Majority of the node should agree on the proposed value.

A consensus algorithm works in following way


Termination
    Every node decides some value.
Validity
    If all registry proposes the same state say  X, then all correct nodes decide X as an updated state.
Integrity
    Every correct node decides at most one value, and if it decides some value X then X  must have been proposed by Leader Node.
Agreement
    Every correct process must agree on the same value X.


Conclusion:  To maintains a Highly available and fault tolerant service registry in a distributed system Consensus and Leader election is the main techniques to be followed.