My thoughts on Steve Jobs, and life in general

    After reading about Steve Jobs in his biography by Walter Isaacson, I am really captivated and touched by his story. To me, he is like my favorite fictional character, Sherlock Holmes, who is a genius at solving crimes, yet as great as the man he is, he is a drug addict, at one moment he is doing something unthinkable that amazes us, yet at another moment he can be just like one of those out on the street, wasting his life away.


    What is more important is that, thru reading the book, I reminded myself that most of the time we, living in a reality full of faults and problems, so much so that we tend to forget, that God has given us unlimited powers, to achieve whatever possible or impossible, limited only by our dreams and believes. It doesn't take an angel, or luck to succeed. It is our visions, our will, our persistence in wanting to succeed, that can bring us to achieve and be labelled a 'success' in the society.


    We have but only one life, ignoring the fact that we can be reborn, which is something that is not a totally proven fact, and not something everyone on earth believes in. And if we do not take this one opportunity to 'make a dent to the universe', it is not just a loss to ourself. It is a loss to mankind in general as well. We who are at least given a chance to try to prove what we can be. Many others do not have the opportunity ... those sperms who have lost their race to the ovum, aborted babies, those who passed away very young for various reasons. That we, given the opportunity, can consider that as a great gift and achievement in itself.


    At this point of time, we can be lost, we can be useless, or even a failure, but that does not mean we do not have a chance to prove to the world and touch the live of others, for, many successful person endure more defeat and made even bigger mistakes compared to us. The Oracle of Omaha, Warren Buffett did not earn his name being right all the time. He earned it, as he rightfully say so, because he made a few right decisions that made him way ahead of the pack.


    In the end, mistakes can be forgiven, wrong decisions can be corrected and wounds can be healed. But if we give up, lose faith and are afraid to make decisions, that will be the biggest mistake of all. For, the fear in us should not be the fear that the sun is not rising tomorrow, or we might not live another day, or we might fail. The real fear is that we ourselves have lost hope and closed the door that leads to the true feelings in our heart, so much so that, the world might have lost a potential hero, without anybody knowing it. Let's not give up and continue to do what we believe in, as success can be just around the corner, much more closer than we think it is. We get out of life what we put into it.

My thoughts on ORM

I used to be fanatical about performance. Every database access must be raw SQL, or smart system generated SQL which are optimized so that I do not have to worry about performance, or at the very least, I know how to tweak the SQL to solve performance issues.

However, the buzz of ORM (Object-Relational Mapping) made me explore a few options such as JDO and JPA (I was using datanucleus implementation at one time as that is what Google AppEngine uses). At first I was quite frustrated due to the difficulties I faced to setup correctly in the initial stages. But after a while, I got more comfortable.

Once, I remember developing a Data Synchronization System early Year 2010, which is a JSP + JDO solution that synchronizes data between two databases based on preset rules (defined in JSON, which is in my universe, the king of complex data structures). I had to port this system to MySQL from Microsoft SQL Server, and my limited knowledge of MySQL really got me worried about how difficult this would be. However, it turns out to be just a snap. Generate the DDL (Data Definition Language), to create the database schema, and that's it. I might face a problem or two, which were so minor I cannot remember them, but all I know is, it is magical. The masking of Database tables as Classes and SQL in some different query language, it makes things so transparent. Not only did I not notice any performance impact, but not needing to maintain separate SQLs or DDLs means I do not have to worry about more modifications in the future if I am to add support for, say PostgreSQL. Amazing!

Fast forward to today, as I am playing with Django Framework for Python, I found that ORM are not only abstractions for database independence, but they actually mean you can play around with code more, to put the validation logic into the classes, as well as modifying them to your heart's content, which feels so much more natural than changing the database design, as well as changing the SQL separately in a program source code. Below is the source code for a simple model, enjoy!


class ServerActionHandler(basemodel.CreateUpdateInfo):
    nameRegEx = re.compile(r'[a-zA-Z][a-zA-Z0-9\.]+',re.DOTALL)
    name = models.CharField(max_length=128,unique=True)
    state = models.CharField(max_length=1,choices=basemodel.DEFAULT_STATE_CHOICES)
    method = models.CharField(max_length=256)
    def clean(self):
        if not self.__class__.nameRegEx.match(self.name):
            raise ValidationError('ServerAction name must start with an alphabet and can only contain alphanumeric and dot (.).')
    def __str__(self):
        return self.name

SubEthaSMTP: Store email as files before sending out

I have written a simple handler for SubEthaSMTP that will save emails to specific folders before sending them out. First the main function to setup stuff (SMTPServer.java)

You can download it from here. (Note: There are three source class in there, with their Class Name properly highlighted).

What it does is to start an SMTP server (standard SMTP Port) in the workstation/server you are running it on.

You will need to configure SMTPMessageHandlerFactory to set the appropriate smtp host, smtp port, smtp user and smtp password to send the email to.

Whenever a mail is received, it will write the contents of the mail in folder: outbox/{fromemailaddr}/{date}/{time}

It will work for normal mail (HTML/plain) and mail with attachments, but not message/rfc822 type.

Also, because I am not able to read BCC headers from the mail received via SMTP, instead of processing the TO, CC, and BCC in the mail content, I will send out the mail to everyone separately, putting them in the TO field for each email sent.

Please help to test out and comment accordingly. Thanks.

Apache Camel and SubEthaSMTP

There are times, having or relying on more than one set of library to do your bidding can be very helpful to streamline and simplify your projects. One example is, although Apache Camel has very powerful integration capabilities, there are times that you might want to do further processing using other libraries to get the benefits of encapsulating changes from your possibly already complicated rules in Apache Camel.

One such example is sending out information to users. You might opt to say, use Apache Camel to send out mails, but instead of to a typical SMTP Server, send it to SubEtha SMTP, which is purely a JAVA library that receives email and pass it to your handler for further processing. It will not be storing or even doing the actual mail delivery. An excerpt from their website (with link to their project)

SubEtha SMTP is a Java library which allows your application to receive SMTP mail with a simple, easy-to-understand API.

Typical interesting scenarios you might want to use this:

• perform filtering on the sender and recipient to weed out inactive or barred users
• log email messages to files or database
• instead of sending mail, lookup recipient preference and optionally send via other channels such as IM (Instant Messaging) or SMS.
• Configure multi-channel send, e.g. Send email to recipient(s), and send notification to them via IM or SMS to inform them about the email header and sender, e.g. You have a mail from development.rants@gmail.com - "Automated Backup for Server BBIGGG completed successfully"
• Analytics or Statistics to determine how many emails are sent to a specific recipient, and how many emails with a particular subject is sent.

 Using SubEtha SMTP is simple, all you need to do is to assign a MessageHandlerFactory to its constructor, which is responsible in creating MessageHandlers to handle incoming mails. A simple example is below:

1: SMTPServer server = new SMTPServer(new MyMessageHandlerFactory());
2: server.setMaxConnections(100);
3: AuthenticationHandlerFactory fact = new
     LoginAuthenticationHandlerFactory( new Validator() );
4: server.setAuthenticationHandlerFactory(fact);
5: server.start();

Line 1 is used to associate your message handler factory (which implements the: org.subethamail.smtp.MessageHandlerFactory interface).

Line 2, which is optional, can be used to set maximum number of connections

Line 3 and 4 are use to setup authentication handlers, in case you would like to authenticate incoming connections.

Line 5 is used to start the server.

In version 3.1.6 of the library, it comes with two higher-level interface and adapter to help to ease construction of your Message Handler, which are the SimpleMessageListener and SmarterMessageListener.

That's all for a start. I'll try to write a more complete example in my next article.

Enjoy processing emails!

Apache Camel: Email notification when files are placed in FTP folder

I love getting automated notification when events occur rather than having to perform manual checking. As an example is, instead of having to login to FTP and check whether files are uploaded to a particular FTP folder, it will indeed be much more elegant if an email notification can be sent to me automatically to inform me about that fact.


For this article, you can use the Apache Camel project created from my previous blog article. Also, let's enable Apache Camel logging so we can see debug output for ease of understanding what's going on. Verify if you already have log4j.properties in your project (should be in Other Sources -> src/main/resources -> <default package> folder under your project. If it is there, you might want to skip to the FTP and Email section.
To do so, first, ensure that you add dependency for slf4j-log4j12 for the appropriate version of slf4j you have (mine is 1.6.1). You can follow the instructions in my previous article to add the dependency (if you are like me, using NetBeans 7 and Maven).


Next you need to click on the Files tab in your NetBeans project, go to your project and expand to src\main, right-click and select New -> Folder. Create a folder named 'resources' (without quotes, of course).
After that, right-click on the resources folder you have just created, and select New -> Properties File. When the New Properties File dialog appear, enter log4j as the filename. The Folder should be src\main\resources (if you indeed right-clicked on the resources folder). As per screenshot below:




Enter following info into your log4j.properties file:

log4j.rootLogger=TRACE, out

log4j.logger.org.apache.camel=DEBUG

log4j.appender.out=org.apache.log4j.ConsoleAppender
log4j.appender.out.layout=org.apache.log4j.PatternLayout
log4j.appender.out.layout.ConversionPattern=[%30.30t] %-30.30c{1} %-5p %m%n

The file should be accessible from Projects tab in the following navigation under your Project folder: Other Sources -> src/main/resources -> <default package>


Next, add dependencies for both camel-mail and camel-ftp.When done, amend your routing code to:



        context.addRoutes(new RouteBuilder() {
            public void configure() {
                from("ftp://ftp.myftp.com/test?username=myuser&password=mypass&noop=true")
                    .setBody().simple("Hi, new file uploaded to ftp. Name: ${header.CamelFileName}")
                    .setHeader("subject").simple("Received filename: ${header.CamelFileName}")
                    .to("smtps://smtp.gmail.com?username=development.rants@gmail.com&password=******");
            }
        });

The above example (fictitious account info), will check the test folder in ftp.myftp.com for user: myuser. When found, it will send an email to my gmail account development.rants@gmail.com. The sender is of the same email account. An email will be sent for every file found in the ftp account, so for testing purposes, please ensure that you do not have too many files in this folder.


You may also want to modify the call to Thread.sleep in App.java to a longer time, e.g. 60 seconds, as below:


        Thread.sleep(60000);


Next, build and run the project, and voila! You will start receiving emails when files are uploaded into the ftp folder being watched.


For this article, notice that I am using noop=true parameter for the ftp url. That will means the file will be left in that folder as it is. This is useful if you are running the program a few times and wants to be able to test without needing to re-upload files to the ftp folder.


For more options you may want to refer to the Camel FTP component documentation here.


Enjoy!

Using NetBeans 7.0 to create an Apache Camel project without Spring dependency

NOTE: This article was rewritten to fix the issue of unable to create a new project based on the apache camel quickstart archetype.

Despite the conveniences offered by the Spring Framework, there are times I would rather live without it, especially for simple applications that I need to keep as lightweight as possible. To do this, the steps are below:

1) Create a new Project in NetBeans 7.0, and choose Maven -> Java Application, and click Next.

2) In Step 2 of Project Wizard, enter your project name, and amend the other settings such as Project Location, Group Id, as you deemed fit.

Once done, click Finish, and wait for the Project setup to be completed.

3) Expand your Project from the Project Tab, right-click on Dependencies folder  and select Add Dependency...

4) In the Add Dependency dialog box, enter camel-core in the Query textbox under the Search tab. Once the Search Results appear, expand org.apache.camel: camel-core folder and click on the version you require, or the latest version available (currently latest is version 2.8.0) [bundle]. Central or Local should not matter, as you should only have Local if you have downloaded it before (either directly in your project or indirectly in other projects with camel-core dependency).

     Click on the Add button.

5) Now expand the Source Packages in your project and you should be able to find App.java. Modify the main class to the code below:

    public static void main( String[] args ) throws Exception
    {
        CamelContext context = new DefaultCamelContext();

        // add our route to the CamelContext
        context.addRoutes(new RouteBuilder() {
            public void configure() {
                from("file:src/data?noop=true").
                    choice().
                        when(xpath("/person/city = 'London'")).to("file:target/messages/uk").
                        otherwise().to("file:target/messages/others");
            }
        });

        // start the route and let it do its work
        System.out.println("Starting");
        context.start();
        Thread.sleep(20000);
        System.out.println("Done");
        // stop the CamelContext
        context.stop();
    }

    You will need to add the necessary dependencies.

6) You can now run the application, which will read any files in your src/data folder, and parse the xml, and move the file to either the target/messages/uk or target/messages/others folder depending on whether the person -> city node contains the value 'London' or not.

    If you followed my previous blog entry 'Using NetBeans 7.0 to create a new project using Apache Camel', you should have sample message1.xml and message2.xml under your src/data folder, which you can use for testing purposes.

    All folders here are relative to your project folder. Until my next blog, Good Luck and Good Bye!

Using NetBeans 7.0 to create a new Apache Camel project

Let's get started to develop a simple Java app that uses Apache Camel. As Maven is integrated in NetBeans 7.0, using it for Apache Camel development is a breeze, but I believe you should be able to use any IDE or even command-line for this purpose. The steps are:

1. Fire up your NetBeans 7.0, and click File -> New Project from the Main Menu.

2. Select Maven from the Categories selection and Project from Archetype from the Projects selection, as shown in screenshot below:

 

3. Click on Next, and in the Maven Archetype wizard. Under the Archetypes from remote Maven Repositories, search for camel-archetype-java (The version I am using is 2.7.2. You might be using a never version), and click on the next button. If it doesn't work for you, follow step 4. Otherwise, skip to step 5.

4. If you cannot find camel-archetype-java, you can enter it manually. Click on the 'Add' button in the Maven Archetype wizard page. A Specify Archetype Details dialog will appear. Enter details provided below, as shown in screenshot that follows:

    Group Id: org.apache.camel.archetypes
    Artifact Id: camel-archetype-java

    Version: 2.7.2

    Leave the Repository text box blank. Once done, click on the Ok button.

5. Click next on completion and you will be brought to the Name and Location page. Enter details as screenshot below (your Project Location may be different from mine):

6. Once done, click on the Finish button. It can take some time, especially this is the first time you are using maven, as it will download all the relevant jars, etc.

7. Once done, you can open up the MyRouteBuilder.java file created. That is basically the Route whose rules will be executed. The fragment of route source code is in the configure function as below:

  from("file:src/data?noop=true")

    .choice()

    .when(xpath("/person/city = 'London'"))

      .to("file:target/messages/uk")

    .otherwise()

      .to("file:target/messages/others");

This basically instructs Apache Camel to:

a) monitor for files in src/data folder (which is relative to your project folder. In my case is: d:\java\messaging\myfirst\src\data. noop=true is an option passed to the file: consumer and is used to instruct Apache Camel not to delete the file after it has finished processing.

b) Execute a choice, and using XPATH selector, select the content of the xml files, looking for person -> city element. If it is London, place the contents to target/messages/uk, otherwise, target/messages/others, again relative to your project path.

To run, right-click on the MyRouteBuilder.java in the Projects window, and select Run File. Log messages will appear in the Output tab.

You can always click on the Files tab (which by default, is to the right of the  Projects tab in the left sidebar), and navigate to the folders mentioned above to look at where the message1.xml and message2.xml in src/data are placed into the destination target/messages/uk and target/messages/others folder.

Not bad for a start, right? This is a real simple way to work, especially coupled with the Spring Framework which takes care of instantiating MyRouteBuilder. In the next article, I would like to blog about how to create a cleaner Apache Camel Routes project that will not depend on Spring.

Apache Camel

One particularly interesting open-source project that caught my eyes is Apache Camel. It is an insanely simple and powerful integration framework. An excerpt of the intro from their website:

Apache Camel is a powerful open source integration framework based on known Enterprise Integration Patterns with powerful Bean Integration.

As I do a lot of coding in Java, and my company is mainly developing a PHP-based Content Management System, I am very keen to look for ways to provide a seamless integration and simplify PHP development by allowing most scheduled and batch jobs to be run via Java.

Using Apache Camel, I can easily fetch a file from a folder, and send it via email to a specific destination. It is not to say that PHP cannot do this, but using Java, I can easily allow 10 PHP web-applications to share the same Java engine. The advantage of this is, besides email, sending of SMS, XMPP message can be transparent to PHP as well, thus less config needed, as most of the one-time config can be centralized in the Java backend.

Apache Camel also allows you to easily communicate between servers by leveraging on JMS or sockets, so if you have hundreds of PHP applications strewn across 10 servers, you can have a simple Apache Camel app installed in all 10 servers that will just watch for existence of a file and dump it to a JMS queue, available for pickup by another set of servers that will then send out the Emails or SMS.

Sounds interesting? Continue following my blog as I will add new articles that slowly introduce a few great ways on how Apache Camel can be used. 

The paths I have taken in Software Development

I first started programming in Apple Basic, before later moving on to DOS environment. I was never a fan of QuickBasic, but I loved Turbo Basic. Next I took a stab on Turbo Pascal, some Assembly language.

During my university years, I started doing web programming and a lot of C++ programming. I used to love Microsoft COM a lot, and I loved DirectX 3. The reference counting stuff is so interesting and at that time I thought that's the greatest gift anyone can bring to the Software Development world.

During my work environment, I started using Java. Initially I hated it so much as it is so slow, we even needed a third-party Symantec JIT to make it run at an acceptable pace. Along came Java 1.2, which makes more sense, and the last piece of puzzle that convinced me to the power of Java is when I was involved with the Java back-end team where they were able to run the same piece of software in HPUX, Solaris and Windows. It wasn't a bed of roses, there are thorns too, but compared to what the C++ team had to do for the cross-platform batch processing code to run across different platforms, it was far more elegant.

So I became an avid Java fan, even until today's Java 6, I am not that convinced that Java UI can make it big, I love doing back-end processing using Java. It is so powerful and expressive enough that many things that seem impossible or complicated in other languages are so beautiful in Java. You can embed a Javascript engine with little effort if you need to, you can have access of many high-quality networking components (think Netty, Mina, Aync Http Client), it is a beautiful environment to work with for systems integration.

In fact, in one of my favourite project, I was able to develop completely on Windows XP 32-bit, and the code was deployed to Solaris 64-bit with no issues at all, not even the need to re-compile! It was a transaction proxy that handles at least 20k transactions a day, and the automated garbage collection is so helpful to make it such a stable system. Touch wood, the system has been running for 3 years continuously without needing a single application or system restart! It really cemented my confidence in Java. It might not be a perfect language, but at least for some purposes, it is The Language.

Welcome to Coders Unite!

Hi there, welcome to Coders Unite blog. I hope this can be a place in the Internet for Intellectual Discussion among coders to improve software development quality.

Over the years, Internet and Open-Source movement has caused a very huge impact on coders' life. Nowadays, there is no need to re-invent the wheel as many open-source projects exists where we can just adapt them to our requirements. As such, sharing knowledge ought to be a more important agenda as it will benefit not just us but the industry in general, as this will increase our awareness of available open-source projects. When usage of these projects increases, it will help to motivate the project owners to further improve upon their projects as well as hopefully introduce more contributors to the projects.