Is there a way to resolve OptimisticLockingFailureException in Spring Batch?

I have a Spring Batch configuration as such.

@Bean(BatchJobConstant.WITHDRAW_BATCH_JOB)
    public Job importJob(@Qualifier(BatchJobConstant.CUSTOM_READER) CustomListItemReader<WithdrawDetails> reader,
                         @Qualifier(BatchJobConstant.CUSTOM_WRITER) ItemWriter<WithdrawDetails> writer,
                         @Qualifier(BatchJobConstant.CUSTOM_PROCESSOR) ItemProcessor<WithdrawDetails,
                                 WithdrawDetails> processor,
                         @Qualifier(BatchJobConstant.POOL_TASK_EXECUTOR) TaskExecutor taskExecutor) {
        final Step writeToDatabase = stepBuilderFactory.get(BatchJobConstant.WITHDRAW_BATCH_STEP)
                .<WithdrawDetails, WithdrawDetails>chunk(chunkSize)
                .reader(reader)
                .processor(processor)
                .writer(writer)
                .transactionManager(transactionManager)
                .taskExecutor(taskExecutor)
                .throttleLimit(throttleLimit)
                .build();

        return jobBuilderFactory.get(BatchJobConstant.WITHDRAW_JOB_BUILDER_FACTORY)
                .incrementer(new RunIdIncrementer())
                .start(writeToDatabase)
                .build();
    }

The custom reader is as:

public class CustomListItemReader<T> implements ItemReader<T> {
    private List<T> list;

    public List<T> getList() {
        return list;
    }

    public void setList(List<T> list) {
        log.debug("Set list of size {}", list.size());
        if (AopUtils.isAopProxy(list)) {
            this.list = list;
        } else {
            this.list = new ArrayList<T>(list);
        }
    }

    @Override
    public synchronized T read() {
        log.debug("Inside custom list item reader");
        if (list != null && !list.isEmpty()) {
            log.debug("Inside read not empty");
            T remove = list.remove(0);
            while (remove == null && !list.isEmpty()) {
                remove = list.remove(0);
            }
            return remove;
        }
        return null;
    }
}

The problem I’m having is that when there is a huge number of data. The

org.springframework.dao.OptimisticLockingFailureException: Attempt to update step execution id=XXX with wrong version (X), where current version is Y

exception occurs. I have tried debugging by using various methods:

  1. I checked whether multiple batch jobs were running concurrently on the same database. This was not the case, only a single batch job was running.

  2. I constructed a NoOpProcessor and a NoOpWriter. I adjusted the job and it worked well. From this, I found that the problem was in the processor as the NoOpProcessor was working really well. But, the question I have is, my original processor is as such:

     @Override
     public WithdrawDetails process(WithdrawDetails details) throws Exception {
         try {
               return withdrawService.performMerchantAutoWithdraw(details);
         } catch (Exception e) {
             log.error("Error occurred. Error: {}", e.getMessage());
             details.setWithdrawStatus(WithdrawStatus.FAILED);
             details.setFailedReason(e.getMessage());
         }
         log.info("INSIDE PROCESSOR :{}", details.getId());
         return details;
     }
    

I am catching a generic Exception, and when I checked my logs. I found the INSIDE PROCESSOR log printed every time. So, how is it that when the Spring batch tries to save after the commit interval the Locking Exception occurs? Is it due to the parallel processing?

I tried printing every TRACE log from spring batch. But I only got these

"org.springframework.batch.core.step.FatalStepExecutionException: JobRepository failure forcing rollback
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:464)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:331)
    at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
    at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:273)
    at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:82)
    at org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate$ExecutingRunnable.run(TaskExecutorRepeatTemplate.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.springframework.transaction.UnexpectedRollbackException: Transaction rolled back because it has been marked as rollback-only
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:871)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:708)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:631)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:385)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy297.updateExecutionContext(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor403.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:19"

AND

10:40:19.339 [async-task-exec-11] INFO  com.test.batchJob - INSIDE PROCESSOR :3064
10:40:19.671 [async-task-exec-11] INFO  com.test.batchJob - INSIDE PROCESSOR :3065
10:40:19.871 [async-task-exec-11] INFO  com.test.batchJob - INSIDE PROCESSOR :3066
10:40:20.067 [async-task-exec-11] INFO  com.test.batchJob - INSIDE PROCESSOR :3067
10:40:20.232 [async-task-exec-11] INFO  com.test.batchJob - INSIDE PROCESSOR :3068
10:40:20.325 [async-task-exec-11] INFO  com.test.batchJob - INSIDE PROCESSOR :3070
10:40:20.563 [async-task-exec-11] INFO  com.test.batchJob - INSIDE PROCESSOR :3072
10:40:20.800 [async-task-exec-11] INFO  com.test.batchJob - INSIDE PROCESSOR :3073
10:40:21.100 [async-task-exec-11] INFO  com.test.batchJob - INSIDE PROCESSOR :3074
10:40:21.138 [async-task-exec-11] INFO  com.test.batchJob - INSIDE PROCESSOR :3076
10:40:21.138 [async-task-exec-11] INFO  com.test.batchJob - Updating merchant auto withdraw details.
10:40:21.138 [async-task-exec-11] INFO  com.test.batchJob - IDS FOR WRITER :[3064, 3065, 3066, 3067, 3068, 3070, 3072, 3073, 3074, 3076] 
10:40:21.138 [async-task-exec-11] DEBUG org.springframework.batch.core.step.item.ChunkOrientedTasklet - Inputs not busy, ended: false
10:40:21.187 [async-task-exec-11] DEBUG org.springframework.batch.core.step.tasklet.TaskletStep - Applying contribution: [StepContribution: read=10, written=10, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING]
10:40:21.193 [async-task-exec-11] DEBUG org.springframework.batch.core.step.tasklet.TaskletStep - Saving step execution before commit: StepExecution: id=4615, version=1, name=merchantAutoWithdraw, status=STARTED, exitStatus=EXECUTING, readCount=70, filterCount=0, writeCount=10 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=1, exitDescription=
10:40:21.198 [async-task-exec-11] ERROR org.springframework.batch.core.step.tasklet.TaskletStep - JobRepository failure forcing rollback
org.springframework.dao.OptimisticLockingFailureException: Attempt to update step execution id=4615 with wrong version (1), where current version is 6
    at org.springframework.batch.core.repository.dao.JdbcStepExecutionDao.updateStepExecution(JdbcStepExecutionDao.java:279)
    at org.springframework.batch.core.repository.support.SimpleJobRepository.update(SimpleJobRepository.java:196)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy294.update(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy294.update(Unknown Source)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:457)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:331)
    at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
    at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:273)
    at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:82)
    at org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate$ExecutingRunnable.run(TaskExecutorRepeatTemplate.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
10:40:21.199 [async-task-exec-11] DEBUG org.springframework.batch.core.step.tasklet.TaskletStep - Rollback for RuntimeException: org.springframework.batch.core.step.FatalStepExecutionException: JobRepository failure forcing rollback
10:40:21.200 [async-task-exec-11] INFO  org.springframework.batch.core.step.tasklet.TaskletStep - Commit failed while step execution data was already updated. Reverting to old version.

Leave a Comment