MicroServices with Devops Example



MicroServices with Devops  Example






In this post I show you How to create Micro service application and Manage it through Dev-ops.

To understand MicroServices please look at my blog post 

To understand Devops please look at my blog post


Tools are Used to implement Microservice
1.       Eclipse as Editor
2.       Java as Language
3.       Using REST Service to communicate between Microservices
4.       Maven as build tool
5.       Tomcat as Web server.


Tool are Used to Manage Microservices using Devops
1.       Jenkins as CI (Continous integration server).
2.       GitHub as SCM(Source control Management).


My intent is creating two Micro services


1.       An Order Service

2.       A Billing service

If client order for a product, Order Service project will communicate with Billing service project by REST API. Billing Service returns Product price if exists in the Product Catalog Otherwise It return a Message Product not found.

Order service accept this response and generate a new response to Client.


To Manage these Microservice, I using Devops

So that Two can be deployed independently, and Two projects are hosted in
GitHub. If someone do a change the code and push it in Github a build has been triggered by Jenkins and Once build is completed, Jenkins install the artifact that is the war file in Local Repo (maven Local REPO) then deploy it to Tomcat Server.

No Manual intervention is needed.


I have made it very simple but in real time there will be much more complexity than two simple services and we need Monitoring tool for health check for different projects which is absent in this example.

I have used Java for Two services but real time one could be in java another in PHP or Node JS any language.


Installation:

Java:  Install java from
Set JAVA_HOME in environment variables to java installation path.


Download Tomcat 6 Zip Version and extract it in your local drive that will your Tomcat installation directory.
Maven:
Extract the zip in your local file Set M2_Home as maven Installation path in your local directory.


Go to Git Command prompt after installation
Change Directory to C: Using Cd ../../
Now apply command (Need Internet connection)

after completing

again hit

Import two project in to Eclipse by import from local option.

If you want to create your new repositories, please go to Jenkins setup section and follow the step.

I prefer you create your new repositories to work independently. Unless if you clone it you will be dependent on me. If I gave you permission, then only you can push change on my project.







Jenkins :
 http://mirrors.jenkins-ci.org/war-stable/  (Take stable release. I use 1.65). Download war and deploy it to <Tomcat_installation_path>/webapps. In tomcat console when a message shows
“Jenkins full up and running” go to browser hit


First thing you need to require add plugins in to your local Jenkins Server

So Follow the steps
Go to Manage Jenkins from left panel
Go to manage Plugin
In Available Tab Search  with “git” and check gitplugin and github plugin
Hit install without restart button
Wait until installation finished.


Go to configure system from manage plugin
 






Create an account in GitHub and then add your username and your email Id.


Then Go to new Item create a Maven project.

Create a Job Name as “BuildandDeploy MicroOrderService”

Please see the following Screen shot




 
In the Source code management section set your Git repository URL
If you download source code from Git by download Zip and create your new repository and push the code

Steps are
After Download project from

and
using “ CloneorDownload”  button on GitHub
Extract Zip files

Create a project on Github using “+” button (Create new)
You will get a url like https://github.com/shami83/<projectname>.git


Execute following commands for two projects in your local computer

Go to  Git cmd
Go to the homefolder of the Extract project
a.       git init
b.      git add .
c.       git commit -m “demo”
d.      git push origin master

Please do the steps for two projects that will create two repositories in Git (require internet connection).

In jenkins on repository url put the url


If you clone it then you need my permission to push code to my stream, please send me a mail the at
mitrashamik@gmail.com or add a comment in blog.


Then put your credentials in credential section your username and password of Git repository.


Follow next step



In Build triggers section Check ” BuildWhen a change pushed to Git hub”

And poll SCM

Put value as * * * * * in schedule text-area

This will detect if any changes pushed on GitHub repository. If so this Jenkins target will run

Now Go to Build section
Set Root POM as pom.xml as you can see a workspace folder create under your Jenkins target where Jenkins pull the codebase from Git server and build it locally.

Set Goals and Option as clean install.
Install step is requires as it maintains your artifacts in your local maven repository.


Last I want to deploy it on Tomcat server

To do that follow the following steps

First go to Tomcat installation directory under conf folder
Open tomcat-users.xml in edit mode
Add following lines under <tomcat-users> tag.

<role rolename="tomcat"/>
  <role rolename="role1"/>
  <user username="tomcat" password="tomcat" roles="admin,manager-script,manager-gui"/>
  <user username="both" password="tomcat" roles="tomcat,role1"/>
  <user username="role1" password="tomcat" roles="role1"/>


Then go to Jenkins job configure section

 





In War/EAR files put target\MicroOrderService.war as maven generates artifacts in target directory.


After that Set user name as tomcat in Tomcat6.x section password is tomcat
Tomcat Url is http://localhost:8080

Click save button that will create MicroOrderService job.


To check hit BuildNow button and go to console output, you will see
Jenkins pull code from GitHub then build it using maven install the artifacts then deploy it in your Tomcat Server



Perform Same thing for Billingservice, create a new Job and configure that.


Now to Check just make a change in Source code
Push it to Git using Git Cmd
Steps
git add .
git commit -m ”change”
git push origin master

go to Jenkins dashboard you will see a build is initiated and deployed.






Main Classes in Micro Order Source Code


OrderService.java


package com.microservice.order;
 import java.io.IOException;
import java.io.StringReader;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Unmarshaller;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;

import com.microservice.model.BillingStatus;
import com.microservice.model.OrderStatus;

@Path("/order")
public class OrderService {

    @GET
    @Path("{name}")
    @Produces(MediaType.TEXT_XML)
    public OrderStatus getOrderStatus(@PathParam("name") String name)
            throws ClientProtocolException, IOException {
        System.out.println("Orerd Item name is ==>" + name);
        DefaultHttpClient httpClient = new DefaultHttpClient();
        HttpGet getRequest = new HttpGet(
                "http://localhost:8080/billingservice/billingservice/billing/"
                        + name);

        String response = getResponse(httpClient, getRequest);
        BillingStatus stat = new BillingStatus();
        stat = this.converStringToObject(stat, response);
        System.out.println("Return Object from JAXB "  + stat);

        OrderStatus status = new OrderStatus();
        status.setOrderName(stat.getItem());
        status.setOrderCost(stat.getPrice());
        return status;
    }

    public String getResponse(DefaultHttpClient client, HttpGet request) {

        HttpResponse response = null;
        String apiOutput = null;

        try {

            response = client.execute(request);

            // verify the valid error code first
            int statusCode = response.getStatusLine().getStatusCode();
            if (statusCode != 200) {
                throw new RuntimeException("Failed with HTTP error code : "
                        + statusCode);
            }

            HttpEntity httpEntity = response.getEntity();
            apiOutput = EntityUtils.toString(httpEntity);
           
           

            System.out.println(apiOutput);

        } catch (Exception ex) {
            ex.printStackTrace();
        }

        return apiOutput;

    }

    public <T> T converStringToObject(T t, String res) {

        try {
            JAXBContext jaxbContext = JAXBContext.newInstance(t.getClass());
            Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();

            t = (T) jaxbUnmarshaller.unmarshal(new StringReader(res));
        } catch (JAXBException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        return t;

    }

}




Main Class in Billing service Module


package com.microservice.billing.service;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

import com.microservice.billing.model.BillStatus;
import com.microservice.billing.utility.PriceCatalog;



@Path("/billing")
public class BillingService {
   
   
    @GET
    @Path("{name}")
    @Produces(MediaType.TEXT_XML)
    public BillStatus getOrderStatus(@PathParam("name")String name)
    {
       
        System.out.println("Check Item in Inventory ==>" + name);
        return PriceCatalog.getCatalog().getStatus(name.toLowerCase());
    }

}


PriceCatalog.java

package com.microservice.billing.utility;

import java.util.HashMap;   
import java.util.Map;

import com.microservice.billing.model.BillStatus;

public class PriceCatalog {
   
    private static PriceCatalog calalog=new PriceCatalog();
    private Map<String,String> priceMap=new HashMap<String,String>();
    private PriceCatalog()
    {
        init();
    }
   
    private void init()
    {
        priceMap.put("pen", "10");
        priceMap.put("shirt", "1000");
        priceMap.put("televison", "20000");
        priceMap.put("ac", "30000");
        priceMap.put("smartphone", "14000");
        priceMap.put("radio", "2000");
        priceMap.put("musicbox", "20000");
        priceMap.put("bike", "66000");
        priceMap.put("scooter", "26000");
        priceMap.put("pant", "100");
        priceMap.put("harddisc", "3000");
        // end of map
       
       
    }
   
    public static PriceCatalog getCatalog()
    {
        return calalog;
    }
   
    public BillStatus getStatus(String key)
    {
        BillStatus status= new BillStatus();
        if(key == null || priceMap.get(key)==null)
        {
            status.setItem("Sorry" + key + " Not found in Inventory!!");
            status.setPrice("Not Applicable");
            return status;
        }
        status.setItem(key.toUpperCase());
        String price =  priceMap.get(key) + " Rupees only.";
        status.setPrice(price);
        return status;
    }
   
   
   

}
Please Dowload Code base from


https://github.com/shami83/MicroOrderService
and 
https://github.com/shami83/BillingService
 


Why Microservice?



Why Microservice?



Companies like Netflix, Amazon, and others have adopted the concept of microservices in their products. Microservices are one of the hottest topics in the software industry, and many organizations want to adopt them. Especially helpful is the fact that DevOps can play very well with microservices.
But what is a microservice? Why should  an organization adopt them?
To understand them, let's first take a look at monolithic software.
In monolithic software, we mainly use a three-tier architecture:
  • Presentation layer
  • Business layer
  • Data access layer
Say a traditional web application client (a browser) posts a request. The business tier executes the business logic, the database collects/stores application specific persistence data, and the UI shows the data to the user.
However, there are several problems with this type of system. All code (presentation, business layer, and data access layer) is maintained within the same code base. Although logically we divide the services like JMS Service and Data-Access Service, they are on the same code base and deployed as a single unit.
Even though you created a multi-module project, one module is dependent on another and, moreover, the module needs dependent modules in its class path. Although you use a distributed environment, it runs under single process context
So, in a single process, different services are communicating with each other. To achieve this, all artifacts and their required libraries (jars) are required in each application container.
Say a JMS service want to use the data access layer. The JMS container needs the data access layer jars and the jars upon which the data access layer is dependent (second level dependencies).
In this concept, there are lots of pain points, and the architecture is very rigid in nature.
Here are some of the problems you face with a monolith.

Problem 1

As there is one codebase, it grows gradually. Every programmer, whether it's a UI Developer or a business layer developer, commits in same code base, which becomes very inefficient to manage. Suppose one developer only works in the JMS module, but he has to pull the whole codebase to his local and configure the whole module in order to run it on a local server. Why? He should only concentrate on the JMS module, but the current scenario doesn't allow for that.

Problem 2

As there is one code base and modules are dependent on each other, minimal change in one module needs to generate all artifacts and needs to deploy in each server pool in a distributed environment.
Suppose in a multi-module project that the JMS module and business module are dependent on the data access module. A simple change in the data access module means we need to re-package the JMS module and business module and deploy them in their server pool.

Problem 3

As monolithic software uses a three-tier architecture, three cross-functional teams are involved in developing a feature. Even though a three-tier architecture allows for separation of responsibility, in the long-run, the boundaries are crossed and the layers lose their fluidity and become rigid.
Suppose an inventory management feature has been developed. The UI, business layer, and data access layer have their own jobs. But everyone wants to take control of the main business part so that when defects come up, they can solve them and are not dependent on another layer's developer. Due to this competition, those boundaries end up being crossed, which results in inefficient architecture.

Problem 4

In many projects, I have seen that there is a developer team and another support team. The developer team only develops the project, and after it's released, they hand it over to the support team. I personally don't support this culture. Although some knowledge transfer happens during the handover, it doesn't solve the problem. For critical incidents, the support team has to get help from the developer team, which hurts their credibility.

Problem 5

As our system is monolithic, so is our team management. Often, we create teams base on the tier — UI developers, backend developers, database programmers, etc. They are experts in their domains, but they have little knowledge about other layers. So when there's a critical problem, it encompasses each layer, and the blame game starts. Not only that, but it takes additional time to decide which layer's problem it is and who needs to solve the issue
Netflix and Amazon address these problems with a solution called microservices.
Microservice architecture tells us to break a product or project into independent services so that it can be deployed and managed solely at that level and doesn't depend on other services.
After seeing this definition, an obvious question comes to mind. On what basis do I break down my project into independent services?
Many people have the wrong idea about microservices. Microservices aren't telling you to break your project down based on the tier, such as JMS, UI, logging, etc.
No this is absolutely not. We need to break it down by function. A complete function and its functionality may consist of UI, business, logging, JMS, data access, JNDI lookup service, etc.
The function should not be divisible and not dependent on other functions.
So If the project has Inventory, Order, Billing, Shipping, and UI shopping cart modules, we can break each service down as an independently deployable module. Each has its own maintenance, monitoring, application servers, and database. So with microservices, there is no centralized database — each module has its own database.
And it could be a relational or a NoSQL database. The choice is yours based on the module. It creates a polyglot persistence.
The most important aspect of microservice culture is that whoever develops the service, it is that team's responsibility to manage it. This avoids the handover concept and the problems associated with it.

Microservice Benefits and Shortcomings

Advantages of Microservices on javaonfly
Disadvantages of Microservices on javaonfly

Benefit 1

As in monolithic software, you only develop in one language, say Java, as the code base. But with microservices, as each service is independent and each service is a new project, each service can be developed in any language that is best fits for the requirement.

Benefit 2

The developer is only concentrated on a particular service, so the code base will be very small, and the developer will know the code very well.

Benefit 3

When one service needs to talk with another service, they can talk via API, specifically by a REST service. A REST service is the medium to communicate through, so there is little transformation. Unlike SOA, a microservice message bus is much thinner than an ESB, which does lots of transformation, categorization, and routing.

Benefit 4

There is no centralized database. Each module has its own, so there's data decentralization. You can use NoSQL or a relational database depending on the module, which introduces that polyglot persistence I mentioned before.
A lot of people think SOA and microservices are the same thing. By definition, they look the same, but SOA is used for communicating different systems over an ESB, where the ESB takes a lot of responsibility to manage data, do categorization, etc.
But microservices use a dumb message bus which just transfers the input from one service to another, but its endpoint is smart enough to do the aforementioned tasks. It has a dumb message bus, but smart endpoints.
As microservices communicate through REST, the transformation scope is very small — only one service is dependent on another service via API call.

But Microservices Have Shortcomings, Too

As every functional aspect is an individual service, so in a big project, there are many services. Monitoring these services adds to the overhead.
Not only that, but when there's a service failure, tracking it down can be a painstaking job.
Service calls to one another, so tracing the path and debugging can be difficult, too.
Each service generates a log, so there is no central log monitoring. That's painful stuff, and we need a very good log management system for it.
With microservices, each service communicates through API/remote calls, which have more overhead than with monolithic software's interprocess communication calls.
But in spite of all of those detriments, microservices do real separation of responsibilities.