Programming at the speed of thought

In his classic "No Silver Bullets" paper in 1986, Fred Brooks claimed that the "accidents" of software development had been substantially addressed. He used the word "accident" in the Aristotelian sense, meaning tasks not inherently essential to programming like waiting for a program to compile. We were getting close to the being limited only by the essential tasks, where the limit is the human mind. I wonder how he might amend his views had he known how enterprise Java would introduce one bad accident after another to the profession. Fortunately, I think things are turning around.

The contrast between accident and essence came into sharp relief to me recently because of two things that happened one day. The first was when a colleague mentioned to me that he had wasted half a day trying to get Maven to build our application. He wasn't changing the build system: all he wanted to do was check out the latest code, build it, and work on it in Eclipse. This took literally hours to get working. The second thing was that I had been working on a personal project based on Grails.

What might have been

Grails is an example of what developer-centric programming could be. I don't say that it is unique in this regard -- even its name implies that it got its inspiration from elsewhere -- but that it is one example of a philosophy that recognizes there is a human being doing the work. If I may be so bold, I would call it humane development. The whole framework seems designed to make programming a joy. It minimizes the make-work tasks so you can focus on the creative work.

Consider what you need to do to get started with a web app. You might set up a folder structure and write a build script to compile, package, and deploy the application to the app server. You'd need to provision and configure a Tomcat instance. You'd need to set up a database server, configure a database and hook up the database connection parameters. In Grails, all you need is "grails create-app" followed by "grails run-app".

When developing in Grails, you don't build. You don't deploy. There are no build scripts. There is just "run-app". Grails will run your code in-place using its own embedded Tomcat or Jetty. Grails also embeds a RAM-based database server that it instantiates and tears down automatically. You don't need to worry about how to configure an app server nor how to connect to a database: why should you? Code changes take effect as soon as you save the file: Grails detects the changes and hot deploys into your running app. The build/deploy/restart cycle does not exist because your application is running all the time. You edit your code, save, and hit refresh on your browser to see the changes. Even database schema changes are "live": when you change your domain classes, you can have Grails either blow away the old schema (it's just RAM) and create a new one from a bootstrap script or have Hibernate apply a schema update on the fly.

As I said, Grails is not unique in its pampering of the developer. Nor is this philosophy limited to non-Java languages: we see other new frameworks like Play that exhibit a similar philosophy using Java. This kind of experience is only remarkable if you have been trapped in "enterprise Java" for a while.

The great sabotage

I keep saying "enterprise Java" instead of "Java" because the Java platform itself is not inherently unproductive. When Java came out, it eliminated a lot of the accidents of development familiar to us in C/C++ land: memory management, no more SIGSEGV nor core dumps, debugging-friendly stack traces … it was actually a good thing. Then we -- the enterprise Java community -- sabotaged it.

We got the J2EE mentality with the heavy frameworks, the XML configuration files, the bloated app servers, over-architected design patterns, intricate build scripts etc. The Java language itself may have a verbose syntax, but it is the other overhead that saps our productivity. I think the underlying problem is that we forgot about the developer. We're engineers, so we like to focus on the code and architecture. We forgot to ask "how is the developer going to work with this?" I certainly had forgotten to ask, and in the process inflicted my share of pain.

Consider the state of IDEs. When Brooks proclaimed the conquest of accidents, Turbo Pascal was popular. The executable was under 40KB in size. That size included an IDE that lets you edit, compile, run and debug all in one place. It had context sensitive language help. It could compile substantial applications in seconds on a 4.77Mhz Intel 8088 processor. Today, 40KB might be the size of your pom.xml. It takes me 15 minutes to do a build/deploy/start cycle.

It's not that today's IDEs have gotten less advanced: modern IDEs are perfectly capable of compiling and deploying a webapp in seconds. The trouble is that we often sabotage their capabilities by basing our builds on intricate Ant or Maven scripts, so that building from the IDE is impossible. For example, Eclipse can deploy non-Java resources by copying them to the output folder the moment you save the file. But if you rely on Maven's resource filtering -- which replaces placeholders in the text  during the build -- then this Eclipse behavior is no longer useful. When both the IDE and the build tool try to compile the same code, they can conflict. Only one can be in control, so we have to do things the slow way, using Maven.

Consider class hot reloading. Server-side applications can take a long time to load (it doesn't help that class loaders are still generally single-threaded). What if we could just recompile a changed class and push it to a running application? There are projects like DCE VM and Javaleon that do just that. That's great if you are writing a plain Java application, but not very useful for enterprise Java. The problem is that so much configuration happens outside of class loading. Java annotations control web service endpoints, class marshalling, transaction demarcation, database mapping, dependency injection etc. The database named query you want to change is buried in an XML file somewhere. Just reloading a class won't change any of these things: somehow the appropriate framework or container must be instructed to reread the metadata and reconfigure itself. Java frameworks assume these configuration files and annotations are only read at startup.

What could yet be

We (or just myself, at least) have done a fine job of making life miserable for ourselves. I think we should be more selfish. That is, we should not be exclusively code/architecture focused, but rather recognize that a human needs to do this work. After all, it's our productivity that is at stake, and we should be vigilant about guarding it. The technology itself is not inherently unproductive. After all, Grails itself is built on top of Spring, Spring MVC and Hibernate: all mainstream enterprise Java frameworks. We can do better both with our practices and our frameworks.

It would be nice to start a green field project with a more humane framework. But even if we can't, there is the  JRebel option. This product that claims to let you hot-reload your changes most of the time, including annotation and configuration changes for a lot of frameworks. It doesn't address the bloat and complexity of typical enterprise projects, but it does let you eliminate a lot of the build/deploy/restart turnaround time that saps concentration and productivity. I suspect JRebel's name is inspired by the Star Wars movies, and it's an apt reference. I'm thinking of that famous line,

"Help me Obi-Wan Kenobi. You're my only hope"

It's scary that JRebel is the only thing out there that claims to do what it does. It's an expensive commercial product, but it holds great promise for productivity. That is why I have requested an evaluation license to check it out. For the stuff I'm doing, it's my only hope.

Related links

• The pain of enterprise Java development
• Things That Turbo Pascal is Smaller Than
• Hot swapping of Java classes: DCE VM, Javaleon
• ZeroTurnaround's JRebel