Batch Web Spring Boot Starter Reference Guide

1. What is batch-web-spring-boot-starter?

Java batch is becoming a hot topic in enterprise environments these days, but how do you do it the right way? The project batch-web-spring-boot-starter offers a best practice approach to modern batch architectures, answering the following questions:

How do I actually deploy jobs?
How do I start, stop and monitor them?
How do I integrate them into my companies' infrastructure?
How do I build job artifacts?

It builds on Spring Batch, Spring Boot and JSR-352.

You may want to switch to the Getting Started page. If you want to know about the possibilities for configuring the application, switch to Configuration options.

2. Blog roll

Most information about this project will be covered in blog posts. You find them here:

Enterprise-ready production-ready Java batch applications powered by Spring Boot

spring-boot-starter-batch-web 1.3.0 released

Enterprise Java Batch - A best practice architecture

batch-web-spring-boot-starter 2.0.0 released

3. Getting Started

Create a Spring Boot maven project. Take a look at this example, it’s pretty much just using spring-boot-starter-parent as a parent und adding the Spring Boot build plugin. Then add the dependency to batch-web-spring-boot-starter like in the example. Check for the current version in the Release Notes.

<dependency>
        <groupId>de.codecentric</groupId>
        <artifactId>batch-web-spring-boot-starter</artifactId>
        <version>2.0.0.RELEASE</version>
</dependency>

If you have a database with the Spring Batch meta data tables and your business data, add the connection properties to the application.properties like in this example. If you don’t specify these properties you’ll get an in-memory database for the Spring Batch meta data tables.
Add a simple logback.xml for logging. Here’s an example inheriting from our basic log configuration to support log file separation.
Add a batch job. You may define it in XML and put it into META-INF/spring/batch/jobs (overridable via property batch.config.path.xml) or in JavaConfig and put it into the package spring.batch.jobs (overridable via property batch.config.package.javaconfig). Each XML file or class annotated with @Configuration in the specified locations will get its own child ApplicationContext. Third option is defining a JSR-352 style job in XML and adding it to META-INF/batch-jobs.
Add an entry point to the application, a class with a main method invoking SpringApplication.run(…). Take a look at this example.
Build the application via maven package. Then start the application using java -jar xxx.jar.

Default port is 8080. Take a look at the JavaDoc of these controllers to get to know the endpoints for starting jobs etc.: JobOperationsController and JobMonitoringController.

4. Configuration options

There are two ways to influence batch-web-spring-boot-starter’s behaviour. The first way is to set certain properties, the second way is to add certain components to the ApplicationContext. Let’s take a look at the available properties first. Note that these are only the properties of batch-web-spring-boot-starter, there are more properties from Spring Boot described in the Spring Boot reference documentation.

Property name	Description	Default value
batch.config.path.xml	Location in the classpath where Spring Batch job definitions in XML are picked up.	/META-INF/spring/batch/jobs
batch.config.package.javaconfig	Package where Spring Batch job definitions in JavaConfig are picked up.	spring.batch.jobs
batch.defaultprotocol.enabled	Whether the default job protocol printed into the log is activated.	true
batch.logfileseparation.enabled	Whether writing one log file for each job execution is activated.	true
batch.metrics.enabled	Whether the transaction safe batch metrics framework is activated so that BatchMetrics may be injected and used.	false
batch.metrics.profiling.readprocesswrite.enabled	Readers, Processors and Writers are profiled with RichGauges when set to true.	false
batch.core.pool.size	Core pool size of the thread pool.	5
batch.queue.capacity	Queue capacity of the task executor.	Integer.MAX_VALUE
batch.max.pool.size	Max pool size of the thread pool.	Integer.MAX_VALUE
batch.repository.isolationlevelforcreate	Database isolation level for creating job executions.	Spring Batch’s default
batch.repository.tableprefix	Prefix for Spring Batch meta data tables.	Not set
batch.web.operations.base	Base URL for the operations endpoint.	/batch/operations
batch.web.monitoring.base	Base URL for the monitoring endpoint.	/batch/monitoring

Property name

Description

Default value

batch.config.path.xml

Location in the classpath where Spring Batch job definitions in XML are picked up.

/META-INF/spring/batch/jobs

batch.config.package.javaconfig

Package where Spring Batch job definitions in JavaConfig are picked up.

spring.batch.jobs

batch.defaultprotocol.enabled

Whether the default job protocol printed into the log is activated.

true

batch.logfileseparation.enabled

Whether writing one log file for each job execution is activated.

true

batch.metrics.enabled

Whether the transaction safe batch metrics framework is activated so that BatchMetrics may be injected and used.

false

batch.metrics.profiling.readprocesswrite.enabled

Readers, Processors and Writers are profiled with RichGauges when set to true.

false

batch.core.pool.size

Core pool size of the thread pool.

batch.queue.capacity

Queue capacity of the task executor.

Integer.MAX_VALUE

batch.max.pool.size

Max pool size of the thread pool.

Integer.MAX_VALUE

batch.repository.isolationlevelforcreate

Database isolation level for creating job executions.

Spring Batch’s default

batch.repository.tableprefix

Prefix for Spring Batch meta data tables.

Not set

batch.web.operations.base

Base URL for the operations endpoint.

/batch/operations

batch.web.monitoring.base

Base URL for the monitoring endpoint.

/batch/monitoring

Now let’s take a look at the interfaces / abstract classes you may implement to add those implemented components to the ApplicationContext:

ListenerProvider

MetricsOutputFormatter

Example:

Unresolved directive in index.adoc - include::{sourcedir}/de/codecentric/batch/metrics/MetricsListener.java[tags=contains,indent=0]

AbstractBatchMetricsAspect

JobLogFileNameCreator

5. Initscript Template

This is a simple example for an initscript template to control your Spring Boot batch application:

#!/bin/sh
#
# Simple initscript for a Java application
#

# Verifies the status of the application
verify_status() {
  [...]
}

# Starts the application
start() {
  # First check if the application is already started
  verify_status
  # Start JVM
  java -jar /opt/batch/example-batch/example-batch.jar >> /var/log/example-batch.log 2>&1 &
}

# Stopps the application
stop() {
  [...]
}

# Shows the application status on the console
status() {
  [...]
}

case "$1" in
  start)
    start
    ;;
  stop)
    stop
    ;;
  status)
    status
    ;;
  restart)
    stop
    start
    ;;
  *)
    echo "Usage: $0 {start|stop|restart|status}"
    exit 1
esac

exit 0

6. RPM

The following code snippet may help you to configure the Maven RPM Plugin, which creates an RPM during the build process for you. In this example the RPM will be created as an attached artifact. For more information see the documentation of the plugin.

<plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>rpm-maven-plugin</artifactId>
        <version>2.1-alpha-4</version>
        <extensions>true</extensions>
        <executions>
                <execution>
                        <phase>package</phase>
                        <goals>
                                <goal>attached-rpm</goal>
                        </goals>
                </execution>
        </executions>
        <configuration>
                <name>${project.name}</name>
                <version>${project.version}</version>
                <copyright>2014,codecentric</copyright>
                <distribution>sles11</distribution>
                <group>Application/codecentric</group>
                <packager>jenkins</packager>
                <changelogFile>src/changelog</changelogFile>
                <vendor>sles</vendor>
                <defineStatements>
                        <defineStatement>_unpackaged_files_terminate_build 0</defineStatement>
                </defineStatements>
                <mappings>
                        <mapping>
                                <directory>/opt/batch/${project.name}/</directory>
                                <filemode>755</filemode>
                                <username>batch</username>
                                <groupname>batch</groupname>
                                <sources>
                                        <source>
                                                <destination>${project.name}.jar</destination>
                                                <location>target/${project.artifactId}-${project.version}.jar</location>
                                        </source>
                                </sources>
                        </mapping>
                        <mapping>
                                <directory>/etc/init.d/</directory>
                                <username>root</username>
                                <groupname>root</groupname>
                                <sources>
                                        <source>
                                                <destination>${project.name}</destination>
                                                <location>target/classes/initscript-template</location>
                                        </source>
                                </sources>
                        </mapping>
                </mappings>
                <preremoveScriptlet>
                        <script><![CDATA[ if [ "$1" = "0" ]; then /etc/init.d/example-batch stop; fi]]></script>
                </preremoveScriptlet>
        </configuration>
</plugin>

7. Metrics

In this Starter there are a lot of metrics that will be written during jobruns (see below).

7.1. Configuration

7.1.1. Enable Metrics in the Starter

batch.metrics.enabled=true

7.2. Gauges

7.2.1. Job duration for a specific job name

name: batch.metrics, tags: {context=simpleJob, name=duration}

7.2.2. Step duration for a specific job name

name: batch.metrics, tags: {context=simpleJob.simpleStep, name=duration}

7.2.3. Chunk duration/count for a specific job name

name: batch.metrics, tags: {context=simpleJob.simpleStep, name=chunk.duration}
name: batch.metrics, tags: {context=simpleJob.simpleStep, name=chunk.count}

7.2.4. Item duration/count for a specific job name

name: batch.metrics, tags: {context=simpleJob.simpleStep, name=item.duration}
name: batch.metrics, tags: {context=simpleJob.simpleStep, name=item.count}

7.2.5. Read/Process/Write methods

name: batch.metrics, tags: {context=simpleJob.simpleStep, name=DummyItemReader.read.duration}

7.2.6. Custom

name: batch.metrics, tags: {context=simpleJob.simpleStep, name=ExampleService.callExternalRemoteService.duration}
name: batch.metrics, tags: {context=simpleJob.simpleStep, name=businesscounter}