java microservices tutorial:spring cloud config server

microservices architecture java: Config Server 

In microservices we create a central spring cloud config server  where all configurable parameters of micro-services are written and it is version controlled, benefit of central config server is if we change a property for a microservice it can reflect on the fly without redeploying the Microservice.

Before Microservices era we create a properties files where we maintain our configurable parameter so if we change the parameter value or add or delete a parameter we generally need to restart the application container moreover think about the environments often we have development, Staging and Prod environment and each has separate Url for different services like JMS, JNDI etc. so also if environment changes we need to pick right properties file like for development we need to load development property file. Spring provides the Profile concept where based on the passing profile Spring load appropriate environments property.

Like if profile value is dev it loads all {}-dev.properties.

But the main problem is , it is tied to the codebase or statically packaged so any changes in properties file need to build and redeployed the application which is violation of Microservice principle  where it clearly stated

build just once and deploy it any environment.

So somehow we need a technology which maintains all properties and if any properties are changed it will pick up the changes and reflect the same without application rebuild or restart.

The answer is Spring Cloud config server.

Spring Cloud config server Architecture

Few important components of config server .

Repository area : Config server stores all Microservices property files in a Version control system so it can be version controlled and this version control system can be git or svn, but most popular are GIT.Also, you can store it in filesystem then your profile should be native.Config server stores each micro-service properties based on the Service ID. we can configure Microservice Service Id by giving a unique value of spring.application.name property in bootstrap.properties file for that Microservice.

Say I have an Employee Payroll Microservice then I can create a ServiceID like following

In Employee Payroll bootstrap.property file.

spring.application.name=EmployeePayroll.

Now in Config server, we have a set of property files for Employee payroll based on environment

like

EmployeePayroll-dev.properties

EmployeePayroll-stg.properties

EmployeePayroll-prod.properties

Based on profile value.

Note that properties file name should be in the following format.

{ServiceID}-{profile}.properties.

If we do not give the profile name in property file then it is considered as default

Here EmployeePayroll-dev is Service ID and dev/stg/prod are environments.

REST Endpoint : Now every microservices need to communicate with Config server so it can resolve the property value, as all property is stored in Config server. Config server publishes rest endpoint through that microservices communicate with them or we can see the properties in the browser.

Actuator :  If any properties for a microservice has been changed that has been refreshed through Actuator refresh rest endpoint. By doing this Microservice got updated property without rebuild the application.

Cloud Bus : This is optional but very useful. think if we need to refresh actuator endpoint manually for each service then it will be a hectic task as a complex business domain has many microservices. In this scenario, cloud bus helps to push the updated properties to all its subscribed Microservices but for trigger this action we need to manually refresh one Microservice endpoint.

I will talk letter about cloud bus.

Load Balancer :  As all the configurable properties are stored in Config server so it is necessary to up before all Microservices started as all Microservices instance querying the config server to resolve it properties when bootstrapped. So Config server should be Highly Available so if we maintain only one Config server Box/Container it can be a Single point of failure Ideally it should be load balanced so that we can run multiple instances of config servers and load balancer pool should have one public address where every microservice communicated. I also talk about Load Balancer in another article elaborately.

Config server Architecture Diagram (Without load balancing and Cloud bus.

Now we will create a config server using Spring Cloud

1. Setting up File Sytem:

Here I will setup a native  Filesystem based config server  I am not using git. So I will create a local file structure but in production environment please create a git remote repository. To set up a local filesystem execute the following commands in Linux environment (same for windows just need to create file using GUI.)

1. mkdir CentralRepo
2. cd CentralRepo
3. touch config.properties
4. vi config.properties
5. Hit insert
6. Write welcome.message=Hello Spring Cloud
7. Hit esc and wq!

2. Coding for Config server:

Implement config server we will use Spring cloud Netflix config server distribution.

Go to https://start.spring.io/

Create a template project using dependencies Actuator and Config server

Page will look like following

Now  hit the Generate Project button so it will download the code template.

After that I extract the code template and then import the project as an Maven project into eclipse.

If you open the pom.xml it will look like following

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>

    <artifactId>MicroserviceCongigServer</artifactId>

    <version>0.0.1-SNAPSHOT</version>

    <packaging>jar</packaging>

    <name>MicroserviceCongigServer</name>

    <description>Demo project for Spring Boot</description>

    <parent>

        <groupId>org.springframework.boot</groupId>

        <artifactId>spring-boot-starter-parent</artifactId>

        <version>1.5.4.RELEASE</version>

        <relativePath/> <!-- lookup parent from repository -->

    </parent>

    <properties>

        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>

        <java.version>1.8</java.version>

        <spring-cloud.version>Dalston.SR1</spring-cloud.version>

    </properties>

    <dependencies>

        <dependency>

            <groupId>org.springframework.boot</groupId>

            <artifactId>spring-boot-starter-actuator</artifactId>

        </dependency>

        <dependency>

            <groupId>org.springframework.cloud</groupId>

            <artifactId>spring-cloud-config-server</artifactId>

        </dependency>

        <dependency>

            <groupId>org.springframework.boot</groupId>

            <artifactId>spring-boot-starter-test</artifactId>

            <scope>test</scope>

        </dependency>

    </dependencies>

    <dependencyManagement>

        <dependencies>

            <dependency>

                <groupId>org.springframework.cloud</groupId>

                <artifactId>spring-cloud-dependencies</artifactId>

                <version>${spring-cloud.version}</version>

                <type>pom</type>

                <scope>import</scope>

            </dependency>

        </dependencies>

    </dependencyManagement>

    <build>

        <plugins>

            <plugin>

                <groupId>org.springframework.boot</groupId>

                <artifactId>spring-boot-maven-plugin</artifactId>

            </plugin>

        </plugins>

    </build>

</project>

Now open the MicroserviceCongigServerApplication.java under com.example.MicroserviceCongigServer package.

Put @EnableConfigServer annotation on top of the class by doing so we tell SPring boot app to treat it as Config server module.

MicroserviceCongigServerApplication.java

package com.example.MicroserviceCongigServer;

import org.springframework.boot.SpringApplication;

import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication

@EnableConfigServer

public class MicroserviceCongigServerApplication {

    public static void main(String[] args) {

        SpringApplication.run(MicroserviceCongigServerApplication.class, args);

    }

}

Now rename the application properties to bootstrap.properties and put the following lines

server.port=9090

spring.cloud.config.server.native.searchLocations=file://${user.home}/CentralRepo/

SPRING_PROFILES_ACTIVE=native

So the config server runs on 9090 port and we put the native file system location so config server gets the properties files from file system rather than git we also tell config server that this is a native profile.

At this point, we are all set to  run the MicroserviceCongigServerApplication.java and it will up the config server

Now test it using http://localhost:9090/config/default/master url

and we can see following response

{"name":"config","profiles":["default"],"label":null,"version":null,"state":null,"propertySources":[{"name":"file:///home/shamik/centralRepo/config.properties","source":{"welcome.message":"Hello Spring Cloud"}}]}

Read more

5 great points why you use event source solutions?

The Event Sourcing Pattern

Joe has a habit whenever he did some transaction by his Debit card he used to write them in his Personal Diary so he can track his transactions. Joe is not a technology savvy and not able to check account statements online.

At the month end, his bank SMS him his current balance but he immediately notices a discrepancy between current savings what bank shows and as per his calculation based on the Diary. He immediately calls Bank helpline and arguing about the discrepancy. Then the bank sends him an Account statement with all transactions recorded.

When he is trying to match transaction records with his diary he understood one transaction not in the Diary as that day he was so sick, he thought it to write it next day but somehow forgot.

But the question is what we can extract from this story?

If we look minutely we discover one fact that Bank always stores all the events/transactions happens on the Account. Like Account creation, credit, debit etc. and the current balance is nothing but the outcome of those transactions. So what I meant to say is that account balance is not a fixed column in a database rather than it is a derivative/outcome of the transactions/events what were applied on the Account.We called it Event Sourcing.

Now think what we generally do in software if we credit or debit any amount we just add or subtract that amount from current balance and update the account with new balance right.

So we have the current state but lost the information how that state is achieved some system uses Audit trail still it is not fully deterministic. So anytime anyone challenges the system this is not the desired system state we don’t have any solid proof other than pleaded to them that system can not be wrong. But if you maintain that history or cause of the state changed like Bank then you just give them the History and asking to check -- a solid way to store proofs.

This is a very common story may anyone of us gone through the same and then look the Account statement for doubt clearing.

Technically what is Even Sourcing?

Event Sourcing is a technique by that we store all the changes of application state as a sequence of events. we can rebuild the state anytime from those events, also can query on the events to construct the desired state.

So the two key benefits are

1.we store all the events in a sequence which enables huge opportunities.

2.The state is the derivative of events so we don’t have to maintain state in the database rather we can programmatically derive state based on the event.

Now this opens a new direction that we don’t have to persist state rather we can derive state and it bring many advantages I will talk about 5 such advantages.

1. State Rebuild :  As we stores every event applies on an application object, we can create a blank /initial application object and apply every event in the same sequence it applied will bring the same state, so anywhere any point of time we can rebuild a state from events. So systems must have a mechanism to apply event, Then you can rebuild a state if the state is blown up for some reason. One may argue if your application state derives from millions of events applied on it, so computing all events may take time and also storing all events need a big storage area. but the fact is nowadays memory are really cheap also we can have TB of in memory space so computation is also faster, alternatively, we can store snapshot i.e milestone of the state and apply event and rebuild state from latest snapshot.

2.  Temporal Query : Event sourcing is perfect for Auditors. Business analysis team always want to see the past state so they can compare the growth or loss or any valuable statistical data so they need the flexibility to query the system in all possible way to collect statistical data. So If system has a feature to build the past state by passing parameters then analyst team will be delighted and the System which maintains all the state they can easily rebuild /compute the state by the parameters provide by the analyst team say analyst want to see the Account details for 10th December 2016, by event sourcing we can fetch all events till 10 the December and apply them in sequence to build the state and return the result to analysts -- easy job isn’t it.

Add caption

3. Comparing State : Sometimes in a complex system, you need to know if events were applied in different ways what would be the outcome and how much deviation it cause from the current state say, A bank saving account interest rate is 8% previously it was 8.5. Now if the bank wants to know due to the decrease of the interest what is the actual amount bank benefits so they will replay events of 8.5 percents in all accounts and compare the state with current state to know the actual benefits although it is not very easy to implement but we can.

4. Debugging State : Suppose there is a bug in production system and we need to debug why the bug happens by event sourcing it is very easy like copy the Account in Dev environment then change the Log level to Debug and apply event one by one in the sequence and check the outcome is predicted or not ,if not then  found the Event and check how it applies to change the application state to found the defect.

5. Future Prediction :  In some Business domain it is important task to analysis what will be outcome if we take some business decision, if the outcome is successful they will take the decision,But in a naked eye it is impossible to predict the application state as different services are interlinked with each other and based on one event they can change, dependent services are subscribed to certain events when that event occurs they take action on basis of event value.  say A bank’s stock share worth is 8 INR but bank analysis team predict  within 1 month it will be increased to 12 INR and they have moreover 30K stocks are public so analysis team wants to know what will be the effects of the application state if stock worth is 12 INR so they will run some ad-hoc future events on top of current state  based on two criteria.

Taking per stock as 12 INR

Taking per stock as 8 INR

Then compare two application states to find out what are the effect of this stock value increase for each interlinked services.

Conclusion : Some systems are inherently Event sourced like Version control (GIT), Banking application, Order Tracking application etc. but we can implement the same in general system also.Using Event sourcing you can easily back and forth you application state by replaying events and state cloning into any environment is just a matter of time but the Irony is, This pattern not used broadly in industry.

Read more