Spring Batch Example with One Job and Two Steps

Below is an example Spring Batch project with one job with two steps.  Each step has a reader/processor/writer where it reads from the DB, processes db record specific metadata in the processor, and then writes the data records.  A job listener performs a beforeJob and afterJob database record writing and updating for auditing purposes.

The job is querying data to create user notifications, and each of the steps performs a query for each type of data being processed.

Please also read my blog on Spring Batch Decision for running two jobs at different times here.

Environment:
Spring Boot / Spring Boot Starter Batch Version 1.5.6.RELEASE
Oracle Java 8
mysql  Ver 14.14 Distrib 5.7.10, for Win64 (x86_64)

I am using MYSQL, so had to set up the necessary tables in my schema to support Spring Batch -  https://docs.spring.io/spring-batch/trunk/reference/html/metaDataSchema.html.

• They are located in spring-batch-core jar file under package org.springframework.batch.core for many different example DBs.  I used the schema-drop-mysql.sql and schema-mysql.sql as examples since I was using MySql
• Navigate to the spring-batch-core-*.jar.  I am using gradle, so I found it here - C:\Users\jhsu\.gradle\caches\modules-2\files-2.1\org.springframework.batch\spring-batch-core\3.0.8.RELEASE\5116a8aec6959f869cd78e779a153e2d43084097\spring-batch-core-3.0.8.RELEASE.jar\org\springframework\batch\core\

Below is the file structure of the sample project:
README.md
/src/main/java/com/cherryshoe/batch/BatchProcessorApp.java
/src/main/java/com/cherryshoe/batch/job/config/TypeOneJobConfig.java
/src/main/java/com/cherryshoe/batch/job/config/TypeTwoJobConfig.java
/src/main/java/com/cherryshoe/batch/job/config/UserNotificationJobConfig.java
/src/main/java/com/cherryshoe/batch/job/JobNotificationListener.java
/src/main/java/com/cherryshoe/batch/job/UserNotificationJobLauncher.java
/src/main/java/com/cherryshoe/batch/model/CsAuditBatchProcess.java
/src/main/java/com/cherryshoe/batch/model/CsUserNotification.java
/src/main/java/com/cherryshoe/batch/model/dto/CsAndUsersDTO.java
/src/main/java/com/cherryshoe/batch/model/dto/UserInfoDTO.java
/src/main/java/com/cherryshoe/batch/model/dto/UserNotificationDTO.java
/src/main/java/com/cherryshoe/batch/processor/TypeOneProcessor.java
/src/main/java/com/cherryshoe/batch/processor/TypeTwoProcessor.java
/src/main/java/com/cherryshoe/batch/writer/CustomUpdateNotificationWriter.java
/src/main/resources/application.properties
/src/main/resources/logback-spring.xml

Below is a short description of each file followed by the file contents:

Manual and Annotation based myBatis-Spring Configuration

Here are two essentially same examples of configuring myBatis-Spring with annotations or using manual configuration, using a custom maven modules.  You can look at the details on github (https://github.com/cherryshoe/cherryshoe-examples/tree/master/mybatis-spring), but the highlights are called out below.

Directory Structure

/*-custom-database     <-- Maven pom.xml

  /src

    /main/

      /java                   <-- Java code

        /com/

          /cherryshoe

            /database

                          /dao            <-- Dao logic

              /domain         <-- Business domain objects

              /persistence    <-- Mapper interfaces

      /resources              <-- Non java files

        /com

          /cherryshoe

            /database

              /persistence    <-- Mapper XML files

                /spring               <-- Spring files

    /test/

      /java                   <-- Java code

        /com/

          /cherryshoe

            /database

                          /dao            <-- Dao logic tests

      /resources              <-- Non java files

        /database             <-- Sql in memory files (H2)

        /spring               <-- Spring test files

Differences with the manual configuration method:
LogsDao.java

• No @Service for LogsDao class
• No @Autowired for LogsMapper
• Add in setter method for LogsMapper

  public void setVwDocManCtsMapper(VwDocManCtsMapper vwDocManCtsMapper) {
      this.vwDocManCtsMapper = vwDocManCtsMapper;
  }
  

spring-custom-database.xml and spring-custom-database-test.xml

• No <context:component-scan> to enable component scanning
• No <context:annotation-config> to enable use of autowiring
• No org.mybatis.spring.mapper.MapperScannerConfigurer to scan for mappers and let them be autowired
• Add in spring beans for the LogsMapper and LogsDao classes.  These can then also be used when importing this module into another maven module.

  <bean id="logsMapper" class="org.mybatis.spring.mapper.MapperFactoryBean">
    <property name="mapperInterface" value="com.cherryshoe.database.persistence.LogsMapper" />
    <property name="sqlSessionFactory" ref="sqlSessionFactory" />
  </bean>
  
  <bean id="logsDao" class="com.cherryshoe.database.dao.LogsDao" >
      <property name="logsMapper" ref="logsMapper" />
  </bean>
  

Spring Managed Alfresco Custom Activiti Java Delegates

I'm using Alfresco 4's activiti workflow engine.  I recently needed to make a change to have activiti call an object managed by Spring instead of a class that is called during execution.  Couple of reasons for this:

1. A new enhancement was necessary to access a custom database table, so I needed to inject a DAO bean into the activiti serviceTask.
2. Refactoring of the code base was needed.  Having Spring manage the java delegate service task versus instantiating new objects for each process execution is always a better way to go, if the application is already Spring managed (which Alfresco is).  
1. i.e. I needed access to the DAO bean and alfresco available spring beans.  
2. NOTE:  You now have to make sure your class is thread safe though!
3. Another side-effect is it's a little bit easier to write junit tests when using Spring dependency injection than instantiating new objects inside methods (take a look at one of my older posts about that).  

For a tutorial on Alfresco's advanced workflows with activiti, take a look at Jeff Pott's tutorial here.  This blog will only discuss what was refactored to have Spring manage the activiti engine java delegates.

I wanted to piggy-back off of the activiti workflow engine that is already embedded in Alfresco 4, so decided not define our own activiti engine manually.  The Alfresco Summit 2013 had a great video tutorial, which helped immensely to refactor the "Old Method" to the "New Method", described below.

Example:
For our example, we'll use a simple activiti workflow that defines two service tasks, CherryJavaDelegate and ShoeJavaDelegate (The abstract AbstractCherryShoeDelegate is the parent).  The "Old Method" does NOT have spring managing the activiti service task java delegates.  The "New Method" has spring manage and inject the activiti service task java delegates, and also adds an enhancement for both service tasks to write to a database table.

Old Method:

1. Notice that the cherryshoebpmn.xml example below is defining the serviceTask's to use the "activiti:class" attribute; this will have activiti instantiate a new object for each process execution:



<process id="cherryshoeProcess" name="Cherry Shoe Process" isExecutable="true">
    ...
    <serviceTask id="cherryTask" name="Insert Cherry Task" activiti:class="com.cherryshoe.activiti.delegate.CherryJavaDelegate"></serviceTask>
    
    <serviceTask id="shoeTask" name="Insert Shoe Task" activiti:class="com.cherryshoe.activiti.delegate.ShoeJavaDelegate"></serviceTask>
    ...
</process>



2. Since we have multiple service tasks that need access to the same activiti engine java delegate, we defined an abstract class that defined some of the functionality.  The specific concrete classes would provide / override any functionality not defined in the abstract class. 



...
import org.activiti.engine.delegate.JavaDelegate;
...
public abstract class AbstractCherryShoeDelegate implements JavaDelegate {
...
    @Override
    public void execute(DelegateExecution execution) throws Exception {
    ...
    }
...
}

public class CherryJavaDelegate extends AbstractCherryShoeDelegate {
...
...
}

New Method:
Here's a summary of all that had to happen to have Spring inject the java delegate Alfresco 4 custom activiti service tasks (tested with Alfresco 4.1.5) and to write to database tables via injecting DAO beans.

1. The abstract AbstractCherryShoeDelegate class extends activiti engine's BaseJavaDelegate
2. There are class load order issues where custom spring beans will not get registered.  Set up depends-on relationship with the activitiBeanRegistry for the AbstractCherryShoeDelegate abstract parent
3. The following must be kept intact:
• In the spring configuration file, 
• Abstract AbstractCherryShoeDelegate class defines parent="baseJavaDelegate" abstract="true" depends-on="activitiBeanRegistry"
• For each concrete Java Delegate:
• The concrete bean id MUST to match the class name, which in term matches the activiti:delegateExpression on the bpmn20 configuration xml file 
• NOTE: Reading this alfresco forum looks like the activitiBeanRegistry registers the bean by classname, not by bean id, so likely this is not a requirement
• The parent attribute MUST be defined as an attribute

Details below:

Spring 3.2 @ControllerAdvice to handle Controller Exceptions

My Java/Spring web application has controllers that either return ModelAndView to a jsp page or return json for ajax calls.  Exceptions can occur at any time, and I noticed that on ajax success the data returned was json, but on ajax error the data returned was regular text; all ajax calls should return the same kind of data, namely json.  There had to be a way to solve this problem generically!

Spring 3.2 to the rescue - it introduced a new annotation called @ControllerAdvice (http://docs.spring.io/spring/docs/3.2.x/spring-framework-reference/html/new-in-3.2.html#new-in-3.2-webmvc-controller-advice)  that defines methods that applies to all @RequestMapping rest url methods, and in particular to help with exceptions with @ExceptionHandler.  We can use this to have a generic exception handling solution to have all ajax calls return json, and all ModelAndView return html.

• @ControllerAdvice allows you to define ONE controller class to handle ALL exceptions that could occur (no specific exceptions defined needed)
• You can control the HTTP status code returned
• You can control the json message returned by grabbing the exception message from the specific exception that occurred

Here's how you do it:

1. Your controllers won't change, the code will still continue to throw any number of specific exceptions:

@Controller
public class ExceptionController {
    
    @RequestMapping(value = "/randomException", method = RequestMethod.GET)
    public String randomException(Authentication auth, HttpServletRequest request) throws Exception {    
       
        throw new NumberFormatException(" " +
                "Test ControllerAdvice randomException[" + "NumberFormatException" + "]");

        [...]
    }

    @RequestMapping(value = "/mavException", method = RequestMethod.GET)   
    public ModelAndView modelAndViewException(Authentication auth, HttpServletRequest request) throws Exception {  
     
        throw new UnexpectedRollbackException("Test ControllerAdvice Exception for mav");
        [...]
    }
}

2. Write ONE new Controller, this will act as your ControllerAdvice controller. Notice:
• the class is annotated with @ControllerAdvice
• @ExceptionHandler is annotated above your generic "handleException" function
• handleException function takes generic Exception e as a parameter
• it checks the accept header to see how the controller function was invoked (via ajax or via form submit; randomException or mavException, respectively).  *Update*  We have to check for null first in case the request doesn't specify a response type expected.  If the request doesn't specify it, then simply return text/html.
• It returns json or html

@ControllerAdvice
public class CherryShoeControllerAdvice {

    /*
     * Handles JSON and HTML
     */
    @ExceptionHandler
    @ResponseBody
    @ResponseStatus(HttpStatus.BAD_REQUEST)
    public String handleException(HttpServletRequest request, HttpServletResponse response, Exception e) throws IOException {    
        String acceptHeader = request.getHeader("Accept");
       
        // If Accept header exists, check if it expects a response of type json, otherwise just return text/html
        // Use apache commons lang3 to escape json values
        if(acceptHeader.contains("application/json")) {
            // return as JSON
            String jsonString = 
                    "{\"success\": false, \"message\": \"" + StringEscapeUtils.escapeJson(e.getMessage()) + "\" }";
        
            System.out.println("In handleGeneric" + e.getMessage());
            return jsonString;
        } else {
            //return as HTML
            response.setContentType("text/html");
            return response.toString();
        }
    }

}

BTW - if you wanted to specify each type of exception going to a separate @ExceptionHandler you could.

BTW - The main cons for using pre-Spring 3.2 @ExceptionHandler (http://docs.spring.io/spring/docs/3.1.x/spring-framework-reference/html/mvc.html#mvc-exceptionhandlers) by itself are you have to:

• Define an ExceptionHandler for EACH controller (or have each controller inherit from one common base)
• Define EACH exception type to be handled – Or have each controller throw the specific type of exception expected

Active Directory Role Based Access within an Alfresco Web Script

The Alfresco project I work on uses LDAP for authentication against Active Directory (AD), users and groups are synched from AD, with Kerberos protocol for Single Sign On (SSO); Alfresco runs on Apache Tomcat.  I won’t go into detail on how to configure this, my focus today is retrieving a SSO logged-in user’s AD groups to determine role based access within an Alfresco Web Script in order to access the web script’s functionality.

First, you’ll need to determine which AD Groups and Users will by synched with Alfresco.  This will be configured via Alfresco’s Authentication Subsystem.

Second, determine which AD groups will have access to this functionality, i.e. “AllBloggersGroup and SomeGroup”.  Make sure the user that you want access to this web script belongs to this group, i.e. “cherryshoe” belongs to group “AllBloggersGroup”.

Third, inject Alfresco’s Authentication Service Spring Bean into the service, and define the allowed roles.

<bean id="com.blogspot.cherryshoe.ExampleService"
class="com.blogspot.cherryshoe.ExampleService"
parent="webscript">
    <property name="authenticationService" ref="AuthenticationService"/>        
    <property name="allowedRoles" value="AllBloggersGroup,SomeGroup"/>
</bean>

Fourth, you’ll need to run authorityService.getAuthoritiesForUser as an admin user, since calling it requires admin credentials.  This article helped me solve the problem of insufficient permissions (running your code as a different user with alfresco blog).

private AuthenticationService authenticationService;
    
// holds the allowed roles
private String allowedRoles;
private List<String> allowedRoleList;

public void setAuthenticationService(AuthenticationService authenticationService) {
    this.authenticationService = authenticationService;
}

public void setAllowedRoles(String allowedRoles) {
    this.allowedRoles = allowedRoles;

    // set up value of allowedRoleList, get rid of any whitespace
    allowedRoleList = Arrays.asList(allowedRoles.split("\\s*,\\s*"));
}

public boolean isUserAuthorized(final String userName) {
    // someFunctionStart
    Set<String> authorizedAuthRoleSet = org.alfresco.repo.security.authentication.AuthenticationUtil
            .runAs(new org.alfresco.repo.security.authentication.AuthenticationUtil.RunAsWork<Set<String>>() {
                @Override
                public Set<String> doWork() throws Exception {
                    // executes the following lines as admin user
                    Set<String> auths = authorityService
                            .getAuthoritiesForUser(userName);
                    return auths;
                }
            }, "admin");

    boolean isUserRoleAuthorized = false;

    for (String currAuthRole : authorizedAuthRoleSet) {
        for (String currRole : allowedRoleList) {
            if (currAuthRole.contains(currRole)) {
                isUserRoleAuthorized = true;
                break;
            }
        }
        if (isUserRoleAuthorized)
            break;
    }

    return isUserRoleAuthorized;
    // someFunctionEnd
}

NOTE:  Make sure that the LDAP synchronization occurs without any errors, or the call to getAuthoritiesForUser won’t work.  My current theory for this is because all users / groups synched cannot have any errors, or the call won’t work for any specific user.

Lastly, invoke the web service using an already authenticated user using Internet Explorer 9 web client using Kerberos for Single Sign On (adding Alfresco as trusted domain).  The web script is invoked without passing alf_ticket (since authentication is automatically passed for you using SSO) and using  /alfresco/wcservice in the base web URL (versus /alfresco/service).
i.e.  If user “cherryshoe” is already logged into Alfresco with IE 9 web client, then invoke URL http://127.0.0.1:8080/alfresco/wcservice/com/blogspot/cherryshoe/exampleService to use cherryshoe’s user credentials.

HOW TO: Make file uploads work with WizardPro jQuery Library and Spring MVC 3.0

My web application front-end code is built using HTML5, CSS3, and jQuery, using the Spring MVC 3.0 Framework to get back-end data back to the browser. We use the WizardPro jQuery library to aid the user in creating objects in the webapp (WizardPro is a great little jQuery library that enables you to create an object in a few steps).

Here is where my heartache comes in – WizardPro’s default form submit uses XMLHttpRequest. Don’t get me wrong, most of the time the default behavior works great! Specifically, it doesn’t work is when you need file uploads in your web app, therefore requiring an input type of file as a form control. Luckily, Spring MVC 3.0 fileupload support was available in our toolkit already (it provides a MultipartResolver for use with Apache Commons FileUpload). To be able to make file uploads work with WizardPro and Spring MVC file upload, you must do the following:

1.  For the Spring MultipartHttpServletRequest to work, set WizardPro’s  defaultAjaxRequest attribute to false when the wizard is instantiated.

$("#wizard").wizardPro({
defaultAjaxRequest: false
,...
});

2.  WizardPro uses a default form class called .defaultRequest, this makes sure the plugin handles the default form ajax requests submits and validation.  Make a copy of the .defaultRequest CSS classes and name it uniquely, i.e. .defaultRequest2.  In the form tag, set the class to this new class name, so the class will not bind wizardPro to use default ajax submits.  Instead WizardPro will use normal HTML form submits with the file input form control for file uploads.

BAD:

<form class=”defaultRequest” action=”someUrl” method=”post” enctype=”application/x-www-form-urlencoded”>

GOOD:

<form class=”defaultRequest2” action=”someUrl” method=”post” enctype=”multipart/form-data“>

3.  Set the encode type to multipart/form-data in the form tag, MultipartHttpServletRequest filter expects this.  Example in number 2 above.

When you follow the above steps, along with setting up Spring MVC’s multipartResolver in the servlet context spring configuration file, ensuring the form action URL and the controller @RequestMapping match, etc, then WizardPro and file uploads work great together!

P.S.  You may ask me, “Why not use an HTML5/AJAX supported file upload library?”.  Uploadrr is a great little library that does this, and I’ve tested it using Firefox 3 and Chrome 10, with both @RequestParam using MultipartFile and also with the HttpServletRequest Input Stream to get the file byte data.  Unfortunately, it does not work for our primary browser platform, IE8, which is why (for now at least), we can’t integrate this into the webapp.