Sunday, April 29, 2012

Camel and beans I

The Camel DSL is very powerful and has different dialects. You can do almost anything when defining a route nesting your business logic directly between the from()...to() statement. But is this what you want to do?

The answer is clearly no unless you apply trivial operations that let's say for the readability reason can be set directly in the route. When you are designing a system you would like to have reusable components implementing atomic business steps that can be easily tested. This makes even more sense if a team is writing the code. you can split the tasks, glue the results together and deliver the system for the user acceptance phase.

Camel has a very handy component called (how fancy) the Bean Component that you can use to modify your Exchange and even send it to endpoints. The steps to create a bean that you can use in Camel are:
  1. Identify the specific job the bean implements
  2. Implement the method(s) within a class
  3. Register the bean in the Camel context
  4. Create tests for the bean
  5. Use the bean within a route
Let's walk through these steps:

1. Some one in your organization want to add an id and a creation date to each order received and then precess the request. The order is a text file that is put in a given directory and the stage of the processing is the order initialization step. 

2. To make you life easy reuse the project created in this post and create a class called OrderHandler as shown below:


This is the implementation of the initialize method:

public class OrderHandler {


private Logger log = Logger.getLogger(OrderHandler.class);

public File initialize(@Body InputStream order) throws IOException {

File destination = FileUtil.createTempFile("init", "tmp");
log.info(destination.getAbsolutePath());

FileUtils.copyInputStreamToFile(order, destination);

Collection<String> lines = new LinkedList<String>();
String orderId = String.valueOf((new Date()).getTime());
lines.add("\n");
lines.add("orderId: " + orderId);
String date = Calendar.getInstance().getTime().toString();
lines.add("date:    " + date);

FileUtils.writeLines(destination, lines , true);

log.debug("business logic executed");
return destination;
}
}

I am using the Apache commons-io library because I don't want to reinvent the wheel especially in I/O. Note the difference between FileUtils (commons-io) and FileUtil (camel). The latter creates a convenient temp file for you in a very convenient way. Inspect the destination variable while debugging the test (point 4) for details. To keep the code focused on the important details I am actually creating an order out of a new date. Although it is a unique value it is not the best thing to do, perhaps a database would be handy here but I want to you focus on the Camel side.

Notice the usage of @Body that at run time will inject the in part of the exchange as an InputStream, the file that has been received from the producer.

3. In order to register the bean in Camel just open your camel-context.xml file and add the bean definition. That's it!


4. Before adding the bean to the route it is always a good idea to create a test for it. This is the test code:

public class OrderHandlerTest {


@Test
public void testInitialize() {
File inFile = new File("target/test-classes/testFile.txt");

InputStream is = null;

try {
is = new FileInputStream(inFile);
OrderHandler oh = new OrderHandler();
File f = oh.initialize(is);

assertNotNull(f);

} catch(Exception e) {
fail(e.toString());
}
}
}

Run the test and check the result that JUnit provides.

5. We are now nesting the bean in the route and it will be very easy as shown below:

public class RouteWithBeans extends RouteBuilder {

/*
* (non-Javadoc)
* @see org.apache.camel.builder.RouteBuilder#configure()
*/
@Override
public void configure() throws Exception {

if (getContext().hasComponent("properties") == null) {
PropertiesComponent pc = new PropertiesComponent();
pc.setLocation("classpath:META-INF/camel.properties");
getContext().addComponent("properties", pc);
}

from("{{in.dir}}").beanRef("orderHandler", "initialize").to(
"{{out.dir}}/?fileName=order.txt");

}


}

Yes, that's it! Rather than implementing a Processor directly in the route or in separate class you can use this approach. Think about it: imagine that you have a service or a module somewhere implementing a big chunk of business logic. If you use a bean you can add a well tested and consistent code reusing what has already been created.
Note that /?fileName=order.txt will be the file name once the order is enriched and moved. That should be revised since you want to have a specific file name for each order. You can use a Simple expression in the File component for that.

You can now run the route within Eclipse directly as shown here. While details about the usage of {{}} the Properties component are here.

You can get the code form github.

Friday, April 20, 2012

Apache Camel first steps II

In the post Apache Camel first steps II you can see how to create a basic camel project using maven and define a simple route. The next step would be using a powerful component the Properties component that allows to use a properties file as its name suggests when defining endpoints.

It is a very handy feature especially during testing or more in general if you want to define endpoint in a file and use its key in the actual route. Combined with the maven filters is even more effective. Let's say you have a route that leverages the jms component but at test time you want to use a simple direct:something endpoint. This is easy to achieve! Follow the few steps below:
  • create an src/filters directory 
  • create a test.properties under src/filters in your maven project
  • add filtering resources in your pom as shown below
 <build>

<!-- setting the filter location and variable -->

<filters>
<filter>src/filters/${filter.file}</filter>
</filters>

<!-- force the filtering of the resources -->
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
  ...
 <build>
  • add the file camel.properties under src/main/resources/META-INF
Let's now add some properties in the files listed above.

test.properties

in.route.param=file:src/foo?noop=true
out.others.route.param=file:target/messages/others
out.uk.route.param=file:target/messages/uk

camel.properties

in.route=${in.route.param}
out.route=${in.route.param}
out.uk.route=${out.uk.route.param}

Run the command mvn compile test-compile -Dfilter.file=test.properties from the command line in your project base dir and check the target/classes/camel.properties file. You should see that all the properties have been set using the filter value. As a matter of fact the -Dfilter.file=test.properties will tell maven to use that file when processing the resources.

So far we didn't do much in camel. Let's use the Properties component then. The first snippet shows the usual way for defining endpoints while the second uses the properties. As you can see the only difference is the usage of {{route}}. Pretty neath!

------ first snippet
        from("file:src/foo?noop=true").
            choice().
                when(xpath("/person/city = 'London'")).to("file:target/messages/uk").
                otherwise().to("file:target/messages/others");

------ second snippet
        from("{{in.route}}").
           choice().
               when(xpath("/person/city = 'London'")).to("{{out.uk.route}}").
               otherwise().to("{{out.route}}");

Now last step is telling camel to use the Properties component defining the location of the file we just created. This is easily done by adding the code below:

if (getContext().hasComponent("properties") == null) {
   PropertiesComponent pc = new PropertiesComponent();
   pc.setLocation("classpath:META-INF/camel.properties");
   getContext().addComponent("properties", pc);
}

When implementing a real world application it is a good idea to divide routes into classes according to some logic equivalence. For instance the PRoperties component and any other initial configuration should be set in one RouteBuilder class.

To use the properties in the test class override the AbstractApplicationContext method as follows:


   @Override
protected AbstractApplicationContext createApplicationContext() {
return new ClassPathXmlApplicationContext(
"classpath:META-INF/spring/camel-context.xml");
}

And override the postProcessTest method:

@Override
protected void postProcessTest() throws Exception {

PropertiesComponent pc = new PropertiesComponent();
pc.setLocation("classpath:META-INF/camel.properties");
context.removeComponent("properties");
context.addComponent("properties", pc);
context.getComponent("properties");
pc.start();
context.setTracing(true);
super.postProcessTest();
}

Last but not least the configuration of the maven bundle plugin: felix. In your pom you need to tell maven how to create the bundle. In this scenario is pretty simple:

<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>2.3.4</version>
<extensions>true</extensions>
<configuration>
<instructions>
<Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
<Require-Bundle>
<!-- no required bundle now -->
</Require-Bundle>
<Include-Resource>
META-INF/spring/camel-context.xml=${basedir}/target/classes/META-INF/spring/camel-context.xml,
META-INF/camel.properties=${basedir}/target/classes/META-INF/camel.properties
</Include-Resource>
<DynamicImport-Package>*</DynamicImport-Package>
</instructions>
</configuration>
</plugin>

Run your test and have fun.

Monday, April 16, 2012

Apache Camel first steps I

Many articles and tutorials exist over the web showing the usage of Apache Camel. The Camel site itself it's a great (of course) source of information and how-tos. Read the first chapter of "Camel in action" (available here) is useful as well. It is a very good way to understand the different aspects that Camel offers, it's definitely worth the price.

When I started working with it I had some difficulties in understanding how to get the best out of it such as the great advantage of using beans to implement the business logic of an app. Once I understood that precious feature I created a workshop to show it to my fellow coworkers.

This post will show a basic route that is going to be extended in the next few posts in order to cover more advanced and interesting features such as the Properties component and the usage of the Bean component. Let's start!

What we are going to see:
  • create a Camel Java DSL project using maven the archetype
  • implement moving files using a route
  • test the route at runtime
Prerequisite:
  • install Eclipse (I am currently using Helios)
  • have m2eclipse plugin installed in Eclipse
Open Eclipse and create a new maven project


Set the base dir of the project then select the maven archetype as show below:


In the next screen define group and artifact id:


Ok now you are set and ready to implement your Camel application. In your project view you should have something similar to:


As you can see I am actually using camel-2.6.0 version. Feel free to use the latest one (2.9.1) since we are going to use features that are compatible with the new version.

Now let's implement the route. It will get an xml file from a directory, check one of its element value  and move the file in some directory based on that value:

        from("file:src/data?noop=true").

            choice().
                when(xpath("/person/city = 'London'")).to("file:target/messages/uk").
                otherwise().to("file:target/messages/others");

The choice..when..otherwise methods work as an if..then..else statement so the code is pretty self explanatory. This is one of the nice features of Camel after all! 

This is the rest of the class code:

/**
 * A Camel Router
 */
public class MyRouteBuilder extends RouteBuilder {

    /**
     * A main() so we can easily run these routing rules in our IDE
     */
    public static void main(String... args) throws Exception {
        Main.main(args);
    }

    /**
     * Let's configure the Camel routing rules using Java code...
     */
    public void configure() {

        getContext().setTracing(true);
       
        // here is a sample which processes the input files
        // (leaving them in place - see the 'noop' flag)
        // then performs content based routing on the message
        // using XPath
        from("file:target/test-classes/data?noop=true").
            choice().
                when(xpath("/person/city = 'London'")).to("file:target/messages/uk").
                otherwise().to("file:target/messages/others");

    }
}

The main method allows to run the route in a runtime environment directly within  Eclipse.
While setting the tracing mode will show what is happening at run time in the Camel environment log. Let's start the route:



This is the output that you should see in the console window:



The environment started and dropping the xml files from the src/data to src/foo will trigger the execution of the route. Note that src/foo will be created by Camel as soon as the framework properly starts. The following is the snapshot of the console after the execution:



Look in target/messages/others and target/messages/uk you have the two xml files moved in one of the two directories based on their content.

Let's create a test of our route. The code shown below tests one of the two endpoints in particular the file moved in target/messages/uk. The most interesting thing is the usage of the @EndpointInject annotation that allows to send content to an endpoint. It is actually an extremely power way of using Camel and in the next post I will show how to take advantage of this technique when implementing routes and business logic.

/**
*
*/
public class MyRouteBuilderTest extends CamelSpringTestSupport {


Logger log = Logger.getLogger(MyRouteBuilderTest.class);

@EndpointInject
ProducerTemplate producer;

@Test
public void testUkFile() {
File inFile = new File("target/test-classes/data/message1.xml");

try {
producer.sendBody("file:target/test-classes/data?noop=true", inFile);
Thread.sleep(1000);
File output = new File("target/messages/uk/" + inFile.getName());
assertTrue(output.exists());
} catch (Exception e) { log.fatal(e);
assertNull(e);

}
}

/*
* (non-Javadoc)
*
* @see
* org.apache.camel.test.CamelSpringTestSupport#createApplicationContext()
*/
@Override
protected AbstractApplicationContext createApplicationContext() {
return new ClassPathXmlApplicationContext(
"classpath:META-INF/spring/camel-context.xml");
}
}