Entries in spring (3)

Sunday
Nov102013

Performance Testing Hibernate Query Approaches

I've known entities are expensive, but wanted to see for myself, so I built this test project to run some benchmarks. The tests aren't complete - just a simple query with no joins, but with a lot of data. The point was to see how much overhead is introduced after we have the query results.

About the Project

This is a simple Maven Spring project that creates an in-memory HSQLDB database, populates it 500,000 records, and then uses several Hibernate query strategies to fetch every one, and report on their average execution times.

Approaches Tested

  1. Using a JpaRepository interface's findAll method to return a list of attached Hibernate entities.

  2. Using Hibernate's StatelessSession interface to return a list of detached Hibernate entities.

  3. Selecting the specific fields of the entity, using Hibernate to return a simple List, and then manually converting that list to a list of detached entities (as DTOs, basically).

  4. Selecting the specific fields of the entity, then using Hibernate's AliasToBeanResultTransformer to build a list of detached entities (as DTOs, basically).

Changing Execution Parameters

By default, the database is loaded with 500,000 records, and each test is repeated in its own transaction 10 times. You can change both of these values in src/main/resources/application.properties.

Running Tests

This big of a database does take up over 256MB of memory, so you might have to increase your heap space. If you run the tests from Maven, you should be fine, since I increase it in the plugin's settings.

Download the sample project and run the following command from inside its directory:

mvn clean test

The test might take several minutes to run. At the end, the test will output the results.

My Results

The results are listed slowest to fastest:

  ---------------------------
  Testing JpaRepository query
  Total # of runs: 10
  JpaRepository avg time: 1073.5ms

  ---------------------------
  Testing stateless session query
  Total # of runs: 10
  Stateless Session avg time: 818.5ms

  ---------------------------
  Testing RowData query
  Total # of runs: 10
  RowData avg time: 317.7ms

  ---------------------------
  Testing ResultTransformer query
  Total # of runs: 10
  ResultTransformer avg time: 311.9ms

The individual times will vary on different systems - the relative performance is what's important.

The StatelessSession query was a little more efficient than returning attached entities, but still pretty slow for this big query. The AliasToBeanResultTransformer and my custom List -> DTO approaches tied as the best performers. I was hoping for this result, but worried that the reflective nature of AliasToBeanResultTransformer might have introduced some overhead. It did not.

Problems? Let me know!

I tried being as careful as I could with these tests:

  • took the average of several test runs
  • turned off Hibernate's second-level cache
  • turned off Hibernate's query cache
  • cleared the entity manager before each run
  • ran each test in its own transaction
  • ignored the first query in the transaction as to avoid any initial performance hit from opening it

I encourage you to download the project, take a look at the code, and try it out for yourself. If you see any issues with my methodology, please let me know, and I'll correct for it.

Friday
Oct182013

Conditionally Run JUnit Integration Tests with Spring

Ideally, your unit test suites require no external dependencies - those should be mocked out. However, sometimes you want extra assurance that your code works with live endpoints via integration tests.

For example, you might want to make sure that your code can successfully navigate your corporate proxy and firewall to make external web requests. For this, we’ll write tests that only run when a command line parameter is defined.

This demonstration assumes that you’re using Spring’s JUnit Runner and Maven for running your tests. The @IfProfileValue annotation will tell the test runner to only include this test suite if our configured ProfileValueSource returns a “true” for “live-external-tests-enabled”.

Let’s verify that this test suite is ignored by default with the maven command:

You’ll see that you now have a skipped test:

Now, try it again, but this time with our test included:

You’ll now see that the test was run:

The @IfProfileValue annotation can also be used above an individual test.

Wednesday
Mar062013

Java: Embedding Spring Batch Admin Into An Existing Application

For my current project, I need to share Spring beans between my front-end web servlets and the Spring Batch jobs that I'll launch from Spring Batch Admin (SBA). This sounds straightforward - configure the SBA servlet to listen on /batch/*, and you should be good to go. However, of course it's not that easy.

URL Path Mismatch

The first problem you run into is bad links in the menus, form posts that go nowhere, and missing CSS files. This is because SBA expects to be deployed on /, not /batch, or /myapp/batch which is where it will go if you deploy it inside another app. You can survive in this mode for the most part by correcting menu URLs after you click them, or use FireBug to change form actions before you submit them - for example, from /batch/files to /myapp/batch/files, but who wants to live like an animal?

Spring Batch Admin 1.2.1 to the Rescue

Spring Batch Admin version 1.2.1 adds the ability to set the base servlet path for all links and forms by overriding the "resourceService" bean. I'm sure there are several ways to successfully accomplish this, but here's what worked for me.

pom.xml

Add the following repository:

<repository>
    <id>repository.springframework.maven.release</id>
    <name>Spring Framework Maven Release Repository</name>
    <url>http://maven.springframework.org/release</url>
</repository>

and the dependencies:

<dependency>
    <groupId>org.springframework.batch</groupId>
    <artifactId>spring-batch-admin-resources</artifactId>
    <version>1.2.1.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework.batch</groupId>
    <artifactId>spring-batch-admin-manager</artifactId>
    <version>1.2.1.RELEASE</version>
</dependency>

web.xml

Configure the Batch Admin Servlet - notice contextConfigLocation:

<servlet>
    <servlet-name>Batch Admin Servlet</servlet-name>
    <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
    <init-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>/WEB-INF/spring/batch-admin/batch-admin-context.xml</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
    <servlet-name>Batch Admin Servlet</servlet-name>
    <url-pattern>/batch/*</url-pattern>
</servlet-mapping>

/WEB-INF/spring/batch-admin/batch-admin-context.xml

This is the context file that's only used by SBA. Keep in mind that any beans defined in your root Spring context will also be available to the jobs that are launched by SBA.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">

    <!-- Spring Batch Admin Context: Additional context for Spring Batch Admin -->
    <import resource="classpath*:/META-INF/spring/batch/servlet/resources/*.xml" />
    <import resource="classpath*:/META-INF/spring/batch/servlet/manager/*.xml" />
    <import resource="classpath*:/META-INF/spring/batch/servlet/override/*.xml" />
    <import resource="classpath*:/META-INF/spring/batch/bootstrap/**/*.xml" />
    <import resource="classpath*:/META-INF/spring/batch/override/**/*.xml" />

    <!-- For Spring Batch -->
    <bean id="resourceService"
        class="org.springframework.batch.admin.web.resources.DefaultResourceService">
        <property name="servletPath" value="/batch" />
    </bean>
</beans>

One Last Gotcha - /myapp/batch

The main "Home" link in the SBA action bar navigates to /myapp/batch. This doesn't work the way I've wired it up, because the SBA servlet is configured to listen to /myapp/batch/, but not /myapp/batch. I tried adding another servlet-mapping to /batch, but the servlet won't answer requests to it.

I'm sure there's a better way to do this via config only - please tell me if you wouldn't mind, but I opted for the more manual way, just so I could move on.

I wired up a servlet in my main web app to listen to /batch, and then have it redirect to /batch/.

web.xml - configuration for a servlet in my main app

Pay attention to the last servlet-mapping. This gives /myapp/batch over to Spring MVC for URL mapping.

<servlet>
    <servlet-name>appServlet</servlet-name>
    <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
    <init-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>/WEB-INF/spring/appServlet/servlet-context.xml</param-value>
    </init-param>
    <async-supported>true</async-supported>
    <load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
    <servlet-name>appServlet</servlet-name>
    <url-pattern>/</url-pattern>
</servlet-mapping>

<!-- LOOK HERE -->
<servlet-mapping>
    <servlet-name>appServlet</servlet-name>
    <url-pattern>/batch</url-pattern>
</servlet-mapping>

HomeController.java

Here's where I redirect /batch to /batch/. Again, I'm sure there's a better way to do this (let me know!). Of course, for this specific code to work, you'll need some dependencies such as spring-webmvc. The bottom line is that I'm just manually listening on /batch and redirecting to /batch/.

@Controller
public class HomeController
{
    /**
     * Redirect the url /batch to /batch/ for the Spring Batch Admin to pick it up.
     * 
     * @return redirect to /batch/
     */
    @RequestMapping(value = "/batch", method = RequestMethod.GET)
    public String redirectBatchToBatchSlash()
    {
        return "redirect:/batch/";
    }
}

Know a Better Way?

Ideally, I'd like to either get Spring Batch Admin to listen to /batch or /myapp/batch, but I'd settle on having a redirect configured in web.xml. Please tell me if you know of a good approach. I moved on with this good-enough solution.