Sunday 24 July 2011

Apache Camel: Email notification when files are placed in FTP folder

I love getting automated notification when events occur rather than having to perform manual checking. As an example is, instead of having to login to FTP and check whether files are uploaded to a particular FTP folder, it will indeed be much more elegant if an email notification can be sent to me automatically to inform me about that fact.


For this article, you can use the Apache Camel project created from my previous blog article. Also, let's enable Apache Camel logging so we can see debug output for ease of understanding what's going on. Verify if you already have log4j.properties in your project (should be in Other Sources -> src/main/resources -> <default package> folder under your project. If it is there, you might want to skip to the FTP and Email section.
To do so, first, ensure that you add dependency for slf4j-log4j12 for the appropriate version of slf4j you have (mine is 1.6.1). You can follow the instructions in my previous article to add the dependency (if you are like me, using NetBeans 7 and Maven).


Next you need to click on the Files tab in your NetBeans project, go to your project and expand to src\main, right-click and select New -> Folder. Create a folder named 'resources' (without quotes, of course).
After that, right-click on the resources folder you have just created, and select New -> Properties File. When the New Properties File dialog appear, enter log4j as the filename. The Folder should be src\main\resources (if you indeed right-clicked on the resources folder). As per screenshot below:




Enter following info into your log4j.properties file:

log4j.rootLogger=TRACE, out

log4j.logger.org.apache.camel=DEBUG

log4j.appender.out=org.apache.log4j.ConsoleAppender
log4j.appender.out.layout=org.apache.log4j.PatternLayout
log4j.appender.out.layout.ConversionPattern=[%30.30t] %-30.30c{1} %-5p %m%n
The file should be accessible from Projects tab in the following navigation under your Project folder: Other Sources -> src/main/resources -> <default package>


Next, add dependencies for both camel-mail and camel-ftp.When done, amend your routing code to:



        context.addRoutes(new RouteBuilder() {
            public void configure() {
                from("ftp://ftp.myftp.com/test?username=myuser&password=mypass&noop=true")
                    .setBody().simple("Hi, new file uploaded to ftp. Name: ${header.CamelFileName}")
                    .setHeader("subject").simple("Received filename: ${header.CamelFileName}")
                    .to("smtps://smtp.gmail.com?username=development.rants@gmail.com&password=******");
            }
        });

The above example (fictitious account info), will check the test folder in ftp.myftp.com for user: myuser. When found, it will send an email to my gmail account development.rants@gmail.com. The sender is of the same email account. An email will be sent for every file found in the ftp account, so for testing purposes, please ensure that you do not have too many files in this folder.


You may also want to modify the call to Thread.sleep in App.java to a longer time, e.g. 60 seconds, as below:


        Thread.sleep(60000);


Next, build and run the project, and voila! You will start receiving emails when files are uploaded into the ftp folder being watched.


For this article, notice that I am using noop=true parameter for the ftp url. That will means the file will be left in that folder as it is. This is useful if you are running the program a few times and wants to be able to test without needing to re-upload files to the ftp folder.


For more options you may want to refer to the Camel FTP component documentation here.


Enjoy!

Wednesday 6 July 2011

Using NetBeans 7.0 to create an Apache Camel project without Spring dependency

NOTE: This article was rewritten to fix the issue of unable to create a new project based on the apache camel quickstart archetype.

Despite the conveniences offered by the Spring Framework, there are times I would rather live without it, especially for simple applications that I need to keep as lightweight as possible. To do this, the steps are below:

1) Create a new Project in NetBeans 7.0, and choose Maven -> Java Application, and click Next.

2) In Step 2 of Project Wizard, enter your project name, and amend the other settings such as Project Location, Group Id, as you deemed fit.

Once done, click Finish, and wait for the Project setup to be completed.

3) Expand your Project from the Project Tab, right-click on Dependencies folder  and select Add Dependency...



4) In the Add Dependency dialog box, enter camel-core in the Query textbox under the Search tab. Once the Search Results appear, expand org.apache.camel: camel-core folder and click on the version you require, or the latest version available (currently latest is version 2.8.0) [bundle]. Central or Local should not matter, as you should only have Local if you have downloaded it before (either directly in your project or indirectly in other projects with camel-core dependency).


     Click on the Add button.

5) Now expand the Source Packages in your project and you should be able to find App.java. Modify the main class to the code below:

    public static void main( String[] args ) throws Exception
    {
        CamelContext context = new DefaultCamelContext();

        // add our route to the CamelContext
        context.addRoutes(new RouteBuilder() {
            public void configure() {
                from("file:src/data?noop=true").
                    choice().
                        when(xpath("/person/city = 'London'")).to("file:target/messages/uk").
                        otherwise().to("file:target/messages/others");
            }
        });

        // start the route and let it do its work
        System.out.println("Starting");
        context.start();
        Thread.sleep(20000);
        System.out.println("Done");
        // stop the CamelContext
        context.stop();
    }

    You will need to add the necessary dependencies.

6) You can now run the application, which will read any files in your src/data folder, and parse the xml, and move the file to either the target/messages/uk or target/messages/others folder depending on whether the person -> city node contains the value 'London' or not.

    If you followed my previous blog entry 'Using NetBeans 7.0 to create a new project using Apache Camel', you should have sample message1.xml and message2.xml under your src/data folder, which you can use for testing purposes.

    
All folders here are relative to your project folder. Until my next blog, Good Luck and Good Bye!

Friday 1 July 2011

Using NetBeans 7.0 to create a new Apache Camel project

Let's get started to develop a simple Java app that uses Apache Camel. As Maven is integrated in NetBeans 7.0, using it for Apache Camel development is a breeze, but I believe you should be able to use any IDE or even command-line for this purpose. The steps are:

1. Fire up your NetBeans 7.0, and click File -> New Project from the Main Menu.

2. Select Maven from the Categories selection and Project from Archetype from the Projects selection, as shown in screenshot below:

 

3. Click on Next, and in the Maven Archetype wizard. Under the Archetypes from remote Maven Repositories, search for camel-archetype-java (The version I am using is 2.7.2. You might be using a never version), and click on the next button. If it doesn't work for you, follow step 4. Otherwise, skip to step 5.


4. If you cannot find camel-archetype-java, you can enter it manually. Click on the 'Add' button in the Maven Archetype wizard page. A Specify Archetype Details dialog will appear. Enter details provided below, as shown in screenshot that follows:

    Group Id: org.apache.camel.archetypes
    Artifact Id: camel-archetype-java
    Version: 2.7.2

    Leave the Repository text box blank. Once done, click on the Ok button.


5. Click next on completion and you will be brought to the Name and Location page. Enter details as screenshot below (your Project Location may be different from mine):


6. Once done, click on the Finish button. It can take some time, especially this is the first time you are using maven, as it will download all the relevant jars, etc.

7. Once done, you can open up the MyRouteBuilder.java file created. That is basically the Route whose rules will be executed. The fragment of route source code is in the configure function as below:

  from("file:src/data?noop=true")
    .choice()
    .when(xpath("/person/city = 'London'"))
      .to("file:target/messages/uk")
    .otherwise()
      .to("file:target/messages/others");

This basically instructs Apache Camel to:

a) monitor for files in src/data folder (which is relative to your project folder. In my case is: d:\java\messaging\myfirst\src\data. noop=true is an option passed to the file: consumer and is used to instruct Apache Camel not to delete the file after it has finished processing.

b) Execute a choice, and using XPATH selector, select the content of the xml files, looking for person -> city element. If it is London, place the contents to target/messages/uk, otherwise, target/messages/others, again relative to your project path.

To run, right-click on the MyRouteBuilder.java in the Projects window, and select Run File. Log messages will appear in the Output tab.

You can always click on the Files tab (which by default, is to the right of the  Projects tab in the left sidebar), and navigate to the folders mentioned above to look at where the message1.xml and message2.xml in src/data are placed into the destination target/messages/uk and target/messages/others folder.

Not bad for a start, right? This is a real simple way to work, especially coupled with the Spring Framework which takes care of instantiating MyRouteBuilder. In the next article, I would like to blog about how to create a cleaner Apache Camel Routes project that will not depend on Spring.

Apache Camel

One particularly interesting open-source project that caught my eyes is Apache Camel. It is an insanely simple and powerful integration framework. An excerpt of the intro from their website:
Apache Camel is a powerful open source integration framework based on known Enterprise Integration Patterns with powerful Bean Integration.
As I do a lot of coding in Java, and my company is mainly developing a PHP-based Content Management System, I am very keen to look for ways to provide a seamless integration and simplify PHP development by allowing most scheduled and batch jobs to be run via Java.

Using Apache Camel, I can easily fetch a file from a folder, and send it via email to a specific destination. It is not to say that PHP cannot do this, but using Java, I can easily allow 10 PHP web-applications to share the same Java engine. The advantage of this is, besides email, sending of SMS, XMPP message can be transparent to PHP as well, thus less config needed, as most of the one-time config can be centralized in the Java backend.

Apache Camel also allows you to easily communicate between servers by leveraging on JMS or sockets, so if you have hundreds of PHP applications strewn across 10 servers, you can have a simple Apache Camel app installed in all 10 servers that will just watch for existence of a file and dump it to a JMS queue, available for pickup by another set of servers that will then send out the Emails or SMS.

Sounds interesting? Continue following my blog as I will add new articles that slowly introduce a few great ways on how Apache Camel can be used.