In late 2013, we began to read more and more about a new development approach - micro services.
We felt, for quite a long time, that our application was getting bigger and bigger and it was more difficult to keep the high velocity of our development cycle.
We have a release cycle of 2 weeks, so within 2 weeks, our development/QA teams must develop the new features, debug the new functionality and make sure nothing is broken.
As our application became VERY big, the latter task became harder and harder.
We realized that micro services were what we need and we decided, as the best practices suggest, to start developing new features as micro services alongside the existing (big) application.
Our application is written in JAVA, using various Spring frameworks for both REST and offline-batch processing (including Spring MVC, Spring Batch, Spring Security and much more).
The application is deployed in a clustered, scalable environment running on Tomcat web servers.
As we are extensively using Spring, it was only natural to choose Spring Boot as our micro services "launcher". Using a predefined template (archetype) of Spring Boot, we could enable quick creation of a new micro service so that developers can focus on the business logic and not on wiring the new deployable project.
Since a micro service, by definition, has its own repository, build, Spring context, internal logic and RESTful API, we can build each service as WAR file and deploy it on our Tomcat servers. In other words, each Tomcat server will deploy multiple WAR files, and each WAR file is a standalone micro service.
Deploying each micro service in its own Tomcat/machine was a less-preferred option, because it would complicate our deployment and scaling logic.
The WAR prototype Maven build included:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
….
</dependencies>
As expected, the development and QA cycles were dramatically improved. Now we only had to test the logic of a single, small application and its API with the 'big' application.
And we were happy & satisfied with our decision...
Two weeks later, the new micro services went to the production environment and then we started to see some issues we didn't anticipate at first.
The micro services approach, particularly if you build each micro service as a standalone deployable WAR file, is great for development and testing but, in our production environment, all micro services were deployed on the same scalable Tomcat servers and that was the problem.
Resource allocation:
Each service in our system (whether it is a micro service or just a piece of code in the application) usually needs resources:
* Database connections
* Threads from a thread pool
When you have a single application, you can do some rough assumption of the load and capacity of each server and, based on that, pre-allocate thread pools and database connection pools.
When deploying multiple (tens...) of WAR files in a single JVM (Tomcat), it is very hard to make those assumptions. In some use cases, all the resources of a single machine can be allocated to a single service and in other use cases, the resource should be spread between several services.
If you allocate each WAR file/each micro service the 'worst case scenario' when it comes to resource utilization, you'll exhaust your external resources (database has a limited number of connections...).
If you under-allocate the resources per service, you may not be able to serve requests in certain scenarios.
Application boot time:
As we are using a scalable environment, we allocate more servers based on the load of requests.
In this case, it is critical that the new servers will be available to serve requests ASAP.
When you deploy multiple WAR files in a Tomcat server, and each WAR file has its own Spring context that needs to initialize, the Tomcat deploys the WAR files one by one, and each service creates a new application context (which again takes time) and starts its own services.
In fact, the time between the Tomcat start time and application availability was increased almost by 10 when using this approach.
So what do we do?
Deploying each micro service in a dedicated Tomcat would solve the thread pool issue but won’t solve the database connection pool issue (and would dramatically complicate the deployment procedure).
Of course, we didn't want to ditch the micro services approach and move back so we decided to use the micro services in a different approach:
Each micro service will be built as a JAR file and Not WAR file.
External resources, such as thread pool and database connections, will be Autowired and injected by the context (which is NOT part of the micro service).
We used Spring boot (again) for this approach and the configuration was:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
<exclusion>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jdbc</artifactId>
<version>3.1.0.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
….
</dependencies>
We created a single WAR project that will be the container of all the micro services (again using Spring Boot).
This new WAR held all the micro services as dependencies so once it had been deployed, it injected the thread pools, database connection pools & all other shared resources to the various JARs / micro services.
So in this approach, we have:
1) A single allocation of resources that will be used among ALL micro services.
2) Single Spring application context that is initialized in boot time and hence system boot time improved dramatically.
To Illustrate:
When each micro service had its own WAR container:
After migrating the WAR projects to JAR projects and creating a shared resources WAR container:
Roy Udassin
