5 great points why you use event source solutions?

The Event Sourcing Pattern

Joe has a habit whenever he did some transaction by his Debit card he used to write them in his Personal Diary so he can track his transactions. Joe is not a technology savvy and not able to check account statements online.

At the month end, his bank SMS him his current balance but he immediately notices a discrepancy between current savings what bank shows and as per his calculation based on the Diary. He immediately calls Bank helpline and arguing about the discrepancy. Then the bank sends him an Account statement with all transactions recorded.

When he is trying to match transaction records with his diary he understood one transaction not in the Diary as that day he was so sick, he thought it to write it next day but somehow forgot.

But the question is what we can extract from this story?

If we look minutely we discover one fact that Bank always stores all the events/transactions happens on the Account. Like Account creation, credit, debit etc. and the current balance is nothing but the outcome of those transactions. So what I meant to say is that account balance is not a fixed column in a database rather than it is a derivative/outcome of the transactions/events what were applied on the Account.We called it Event Sourcing.

Now think what we generally do in software if we credit or debit any amount we just add or subtract that amount from current balance and update the account with new balance right.

So we have the current state but lost the information how that state is achieved some system uses Audit trail still it is not fully deterministic. So anytime anyone challenges the system this is not the desired system state we don’t have any solid proof other than pleaded to them that system can not be wrong. But if you maintain that history or cause of the state changed like Bank then you just give them the History and asking to check -- a solid way to store proofs.

This is a very common story may anyone of us gone through the same and then look the Account statement for doubt clearing.

Technically what is Even Sourcing?

Event Sourcing is a technique by that we store all the changes of application state as a sequence of events. we can rebuild the state anytime from those events, also can query on the events to construct the desired state.

So the two key benefits are

1.we store all the events in a sequence which enables huge opportunities.

2.The state is the derivative of events so we don’t have to maintain state in the database rather we can programmatically derive state based on the event.

Now this opens a new direction that we don’t have to persist state rather we can derive state and it bring many advantages I will talk about 5 such advantages.

1. State Rebuild :  As we stores every event applies on an application object, we can create a blank /initial application object and apply every event in the same sequence it applied will bring the same state, so anywhere any point of time we can rebuild a state from events. So systems must have a mechanism to apply event, Then you can rebuild a state if the state is blown up for some reason. One may argue if your application state derives from millions of events applied on it, so computing all events may take time and also storing all events need a big storage area. but the fact is nowadays memory are really cheap also we can have TB of in memory space so computation is also faster, alternatively, we can store snapshot i.e milestone of the state and apply event and rebuild state from latest snapshot.

2.  Temporal Query : Event sourcing is perfect for Auditors. Business analysis team always want to see the past state so they can compare the growth or loss or any valuable statistical data so they need the flexibility to query the system in all possible way to collect statistical data. So If system has a feature to build the past state by passing parameters then analyst team will be delighted and the System which maintains all the state they can easily rebuild /compute the state by the parameters provide by the analyst team say analyst want to see the Account details for 10th December 2016, by event sourcing we can fetch all events till 10 the December and apply them in sequence to build the state and return the result to analysts -- easy job isn’t it.

Add caption

3. Comparing State : Sometimes in a complex system, you need to know if events were applied in different ways what would be the outcome and how much deviation it cause from the current state say, A bank saving account interest rate is 8% previously it was 8.5. Now if the bank wants to know due to the decrease of the interest what is the actual amount bank benefits so they will replay events of 8.5 percents in all accounts and compare the state with current state to know the actual benefits although it is not very easy to implement but we can.

4. Debugging State : Suppose there is a bug in production system and we need to debug why the bug happens by event sourcing it is very easy like copy the Account in Dev environment then change the Log level to Debug and apply event one by one in the sequence and check the outcome is predicted or not ,if not then  found the Event and check how it applies to change the application state to found the defect.

5. Future Prediction :  In some Business domain it is important task to analysis what will be outcome if we take some business decision, if the outcome is successful they will take the decision,But in a naked eye it is impossible to predict the application state as different services are interlinked with each other and based on one event they can change, dependent services are subscribed to certain events when that event occurs they take action on basis of event value.  say A bank’s stock share worth is 8 INR but bank analysis team predict  within 1 month it will be increased to 12 INR and they have moreover 30K stocks are public so analysis team wants to know what will be the effects of the application state if stock worth is 12 INR so they will run some ad-hoc future events on top of current state  based on two criteria.

Taking per stock as 12 INR

Taking per stock as 8 INR

Then compare two application states to find out what are the effect of this stock value increase for each interlinked services.

Conclusion : Some systems are inherently Event sourced like Version control (GIT), Banking application, Order Tracking application etc. but we can implement the same in general system also.Using Event sourcing you can easily back and forth you application state by replaying events and state cloning into any environment is just a matter of time but the Irony is, This pattern not used broadly in industry.

Read more

3 wise men on Tell Don't ask

A story on Tell Don't ask Principle

John, Doe, and Marcus are three good friends. They have over 20 years of experience Java/JEE stack and working in IBM, Cognizant and TCS respectively. They have immense experience in design pattern and all the new technologies and respected by their colleague for Exceptional insight on the Technology Stack.

They are planning to go for a vacation in the upcoming weekend to enjoy their spare time with lots of Burger, whiskey, and cooking. John has an Outhouse in a village so they planned to go there by driving.

At last, the day came they pack their Beer, Whiskey, and Burger and headed towards John outhouse it was far away from Town. At evening they reached the outhouse and prepare some snacks for their whiskey and sit together in a round table to enjoy Chicken roast and Whiskey.

Suddenly the power cut happens, The room is so dark that no one can see anything, from outside they can hear the Call of Cricket, Marcus put on his mobile flashlight, now they can see each other.

Doe breaking the silence by saying,

“Oh well, this ambiance is perfect for a horror story can anyone share any real life experience?”

Marcus replied in a witty way

“Umm No, I believe all we are from town and busy with IT industry so sorry can not share any horror experience but can share some Java experience which still frightened me”

John and Doe’s Architect instinct flashed with this proposal.

They said, “ Oh yes what would be more good to discuss about something which frightened us an Architect in this ambiance, it is same as Horror Story :).”

Marcus slowly demonstrated the problem.

Marcus: As an Architect when I design a solution for a problem it always frightened me what we encapsulate and what portion we expose to our client program?

John and Doe Nodded their head.

Marcas Continue with his speech,

There are lots of OOPs principle which says how judiciously you can encapsulate your classes or API from outside world.

Take an Example, The  Tell Don’t Ask principle, It says us always tell to Object, in a layman term instruct Object what to do never query for an internal state and take a decision based on that because then you loose control over the object.

Take a simple example suppose I want to write a Parcel Delivery Service and there are two domain Objects Parcel and Customer so how should we design it.

If I write following code fragments

/**

*

*/

package com.example.basic;

/**

* @author Shamik Mitra

*

*/

public class PercelDeliveryService {

   

   

    public void deliverPercel(Long customerId){

       

        Customer cust = customerDao.findById(customerId);

        List<Percel> percelList = percelDao.findByCustomerId(customerId);

       

        for(Percel percel : percelList){

           

            System.out.println("Delivering percel to " + cust.getCustomerAddress());

            //do all the stuff for delivery

        }

       

    }

}

According to Tell Don’t Ask it is a violation and should be avoided. In Parcel Delivery Service I try to fetch or ask Customer Address so I can perform the delivery operation so here I query the internal state of the Customer.

And why it is dangerous?

If later if the delivery functionality change says now it also include an email address or mobile so I have to expose these details so exposing more and more internal state think about the other services they may also use the same email address or Customer address. Now If I want to change the Customer Address return type String to Address Object then I need to change all services where it has been used, so a gigantic task to perform and increases risk factor of breaking functionality. Another point is as internal state is exposed to many services there is always a risk to pollute the internal state by a service and it is hard to detect which service changed the state. In one word I loose the control over my object as I don’t know which services use/modify my Object internal state. Now if my object is used by the Internal services of a monolith application I can search the usage in IDE and refactor them but If the Object is exposed through an API and This API used by other organization then I am gone, It really hurts our company reputation and as an architect, I would be fired.

So I can say

Action: More you Expose Internal state

Outcome:  Increase coupling, Increases Risk, Increase Rigidity.

Now again look the solution I provided,

Doe chuckled and guess which point Marcus trying to make so he interrupted him and start saying.

Doe: So Marcus you want to tell if we follow Tell Don’t Ask principle then there are couple of ways we can refactor the problem

1. make an association between Customer and Parcel . and it would be lazy loaded and deliver method should be in Customer Object so from the service we call deliver then deliver method fetch parcel list and deliver it, so if the Internal return type change from String to Address only “deliver” method should be affected.

Like this,

/*

*

*/

package com.example.basic;

/**

* @author Shamik Mitra

*

*/

public class ParcelDeliveryService {

   

   

    public void deliverParcel(Long customerId){

       

        Customer cust = customerDao.findById(customerId);

        cust.deliver();

    }

}

public class Customer{

    public void deliver(){

        List<Percel> percelList = getPercelList();

      for(Percel percel : percelList){

           

            System.out.println("Delivering percel to " + this.getCustomerAddress());

            //do all the stuff for delivery

        }

       

    }

}

By doing this I maintain Tell Don’t Ask principle properly and decreases the risk of exposing internal state and free to modify my class attributes as all behaviors are tied locally with attributes a nice way to achieve encapsulation.

2. We can create a command Object where we pass the Parcel details and pass the command object to Customer Model, the delivery method extracts the Parcel list and deliver it to the respective customer.

But both policy breaks another principle that is Single Responsibility Principle, SRP(Single Responsibility Principle) says A class has only one reason to change.

But If I think in a reverse way, why we write Services? Because each service does one job like Person Delivery Service responsible for “delivery parcel related “ operations so it maintains SRP and this service only change if there are any changes in Parcel Delivery mechanism and if it breaks other services will not be affected unless other services depend on it.

But according to Tell Don’t Ask all Customer related behaviors should be moved into Customer class so we can tell /Instruct/command Customer class to do a task. So Now Customer class has much responsibility because all Customer-related service code now goes into Customer Model. So Customer has more reason to change so increase the risk factor of failing.

So Now we are back in the same problem risk factor.

If exposing internal state then the risk for modifying attribute if move all behavior into a class then the risk of modifying functionality break the system.

So SRP and Tell Don’t Ask principle contradict in this context.

John: John nodded his head and started with his husky voice, Yes this is really a problem

not only this, If we want to implement a cache in a service or want to implement Aggregation function like Percell delivery rate charge according to distance or Account type, Find most parcels sent to a locality we use Aggregator service where we ask for internal state and compute the result. So often we break Tell Don’t Ask principle. Even as per current trend, Model Object should be lightweight and should not be coupled with each other. If the business needs an information which distributes over multiple models we can write an aggregator service and query each model and compute the result , so we querying internal state. Think about Spring data.

Now If we look in another perspective, according to the Domain-driven Design, In a Parcel Delivery Context (Bounded context) is Customer responsible for delivering the parcel?

Absolutely not, In that context Delivery Boy is responsible for delivering the parcel to the customer. For that Delivery boy needs Parcel and Customer Model, and in that context only Customer name and Address details required and for parcel Id, parcel name will be required so as per DDD we create an Aggregate Model DeliveryBoy where we have two slick Model Customer and Percell because in this context we don’t need customer other details and parcel other details like customer account details, Customer birthdate etc, Context wise model is changed so no one big model for customer where all attribute and behaviour resides rather small slick models based on bounded context and an Aggregate model querying these slick model and perform the work.

By doing this we can mix and match SRP and Tell Don’t ask. For a service perspective we only tell /command DeliveryBoy Aggregate model to do something, so Service maintains SRP and Tell don’t ask, Our Aggregate model also maintain SRP but querying Customer and Parcel Model to do the operation.

Like

/**

*

*/

package com.example.basic;

/**

* @author Shamik Mitra

*

*/

public class ParcelDeliveryService {

   

   

    public void deliverParcel(Long customerId){

       

        DeliveryBoy boy = new DeliveryBoy();

        boy.deliver(customerId);

    }

}

public class DeliveryBoy{

    Customer cust;// Context driven model

    Percel percel;

    public void deliver(Long id){

        //do stuff

//load customer slick model

//load Percel slick model

//Deliver the same by quering

        }

       

    }

}

Marcus joined and says let's take one step further as DDD insists  Microservice, so in Microservice we try to break a monolith using functional decomposition( A function is a Bounded context in DDD) so one service doing one task maintains SRP principle and if we needed and information which distributes over multiple services we create an Aggregator service and querying individual service and do the task so Microservice often breaks Tell Don’t Ask

Doe joined and says So there is no silver bullet and not all principles are good in all context, Based on the context you have to judge which principles you follow sometimes you need to compromise, may for a specific principle viewpoint your code is bad but for a given context it is optimum.

Principles are generic and they are context free but real life solution are based on context so fit principles based on context not the reverse.

In the meantime Light comes, so John said here we are for fun let stop the discussion and concentrate on Whiskey and Roast !!!!

Everyone agreed and change the topic.

Conclusion : As a Narrator, My question to all viewers, what do you think about the talk they did, is there are any points they left off which needs attention while designing?

Read more