OOPs: mangling your database with objects

A developer once showed me a frightening sight: a page and a half of dense SQL, consisting mostly of join clauses. What was frightening was not so much the performance of this query -- it was a utility query, so the long query time was not important -- but the fact that he was actually proud of that query. How did a production DB schema get into such a state? The problem, in a word, was Hibernate. I'm a fan of Hibernate, so I'm not saying that there's something wrong about Hibernate. Rather, Hibernate lets you too easily do things that you should have second thoughts about. To borrow a metaphor, this tool comes with live chainsaws attached.

DVCS vs Subversion smackdown, round 3

That title is probably a little pretentious, since the topic of rival version control systems has probably gone a few zillion more rounds than that. The context here is that in a previous blog post I argued that the recent rise of DVCS (Git, Mercurial) is less due to their essential D (distributed) nature, but the accidental weaknesses of Subversion, the dominant centralized VCS. The weaknesses I mentioned are:

• Poor merging
• Lack of offline commits

Since the Subversion folks are busy rectifying these weaknesses, I questioned whether DVCS is really the wave of the future. That was round 1. Round 2 came when my colleague Pete came galloping to the defense of DVCS, and I would also include the comments in response to my blog post. This is round 3, the response to the responses.

I am a Torch bearer

I am a Torch bearer. More specifically, I am now a proud owner of a BlackBerry Torch 9800. This came as a surprise to me, actually, since I had no plans to buy one. The Torch is an AT&T phone: even after unlocking it, the Torch's 3G radio does not work on my T-Mobile network (wrong frequency). I own a Torch because my MissingLight app became a Regional Selection in RIM's 2010 BlackBerry Super Apps Developer Challenge, and the  prize for that honor is a new Torch. I'm not complaining.

Beware the magic flush

It often surprises me what a good Java profiler can tell me about Hibernate performance issues. Recently I was studying a performance issue that seemed straightforward. There were lots of objects, lots of writes and lots of queries. Preliminary profiling showed one particular Hibernate query dominating the operation. That's not surprising, I thought: it's a frequently made query, and it's probably slow. I'll just find a way to speed it up, maybe with an index.

As it turns out, it was a simple, quick query. There was little hope of speeding that query up much. But further profiling showed that very little time was spent actually doing the DB query. So what was Query.list() doing that was taking so long? It was the implicit flush. The Hibernate session keeps track of objects that you have loaded, looks for modified objects and writes those modified objects to the database when flushed. A final flush happens upon transaction commit. Additionally, Hibernate by default behavior flushes dirty objects to the database before making a query, to ensure that the query is made on data that reflects all updates made to that point. If you have lots of persistent objects loaded, each flush can be expensive because with each query:

• Hibernate has to check all persistent objects in the session for changes 
• Hibernate will write any updated objects to the database. This can be wasteful: 
• The updates may not be complete yet, so these intermediate writes are unnecessary
• Spreading the DB writes across reduces Hibernate's opportunities to batch SQL statements for performance compared to writing everything at the end of the transaction.

The fact that flushes can happen during queries is significant. If you realize that an operation that interleaves object modifications and queries, you might consider the potential for significant speedup just by preventing that interleaving. For example, you could make all the queries up-front before adding, modifying or deleting any persistent objects so the only flush would be the final pre-commit flush.

DVCS for all the wrong reasons

A lot of virtual ink has been expended arguing the superiority of the distributed version control systems (DVCS) like GIT or Mercurial over Subversion, the dominant open source "traditional" VCS. Representing a substantial paradigm switch, DVCS software is hip and adoption is growing. I will argue that DVCS adoption is driven not by the essential superiority of distributed VCS (the D in DVCS) but by the temporary, accidental weaknesses of Subversion.

Why Java needs Oracle

By filing a patent lawsuit against Google over its Java-like implementation, Oracle probably made itself the world's least liked software company. After all, Java itself is an open standard, open source platform that is ill-served by exclusive legal claims, right? Yet perversely, I think this patent lawsuit demonstrates why Java needs Oracle.

MissingLight 2.0 for BlackBerry

I released version 2.0 of my MissingLight BlackBerry app a few weeks ago. This blog entry provides the background for this release.

Oracle vs Google: in search of a silver lining on Android

There is little to cheer about when the stewards of two of the most important application platforms in the industry, Oracle of Java and Google of Android, decide to fight it out in the courts. Oracle's lawsuit threatens both platforms: the publicity and "bad guy" image hurts Java, and the legal uncertainty hurts Android. It seems a lose-lose situation, doesn't it? Nevertheless, I still hope for a positive outcome for the Java world.

Why is it so hard to find a Java developer?

It's that time again. My employer needs another Java software engineer. We are looking for someone familiar with the usual enterprise Java stack: Spring, Hibernate, Tomcat. Pretty vanilla, right? So why is it so hard to find one? The problem is not with the frameworks: these are pretty widely used, and lots of people claim familiarity with them. My problem is the same one I've had for years: it's hard to find candidates who can clearly demonstrate their programming skills.

The rise of the cattle office

When I blogged about the office environment, I contrasted private offices with cubicles. Whatever their relative advantages, the latter suffered from noise, distraction and lack of privacy. Not long afterward, a colleague pointed me to Martin Fowler's article on team rooms that took these disadvantages and multiplied them:

Guess what this made me think of?

Moo.

 

(Photo credit)

Counting down my life

When I wrote TimeForTea for my BlackBerry, I had in mind a countdown timer to let me know when my tea is ready. As it turns out, a countdown timer has a lot more uses than that. I am indebted to my own customers for for their ideas of additional uses of TimeForTea. With room for 9 preset times, you can do a lot more than timing your tea. Looking over the reviews, I found a whole lot of other timing uses, such as:

• Washing cycle
• Dryer cycle (no more wrinkled clothes)
• Lunch break
• Pizza/Chinese takeout
• Kids' timeout
• Kids' video game time

You get the idea. I personally also started using TimeForTea to time when my grill is warmed up. Hey, it's summer.

Real engineers prefer cubicles

Well, maybe not really, but I could never tell from the work environment of software engineers that I see today.

Jim Showalter recently wrote a blog post called Open Cubicles Must Die ranting against you-know-what. I think he made some interesting points, and it's a post worth reading. Jim is hardly the only person who argue that private offices are more productive, but what I find interesting is the seeming lack of desire for private offices from the engineers themselves. Jim's blog post got a lot of page views, but did not really get many comments supporting his position. I personally have not discerned much interest among my software engineer peers, even disregarding cost. People are actually proud of their wide open spaces and are disdainful of individual offices. Have individual offices gone the way of the dodo? Would you prefer your own office if given the choice?

Tips for less painful webapp development on Tomcat

Here are a few things I do to make my life easier when developing Java webapps deployed on Tomcat. These tips cover:

• Rebuilding my application faster
• Tomcat configuration options I like to use
• Running Tomcat
• Hot deploying of modified code

Notes from a BlackBerry Meetup

I went to the first-ever meeting of the New England BlackBerry Developer's group tonight in Cambridge. An official RIM-sponsored group, this Meetup had as speaker Mike Kirkup, Director of Developer Relations at RIM. He covered a number of developer's topics, and his slides will probably be posted on the Meetup page. I won't try to summarize the presentation ("what makes a 'super app' etc"), since the slides should be available elsewhere, but would like to note a few things that I found interesting from the meeting. Since we had an official RIM rep to answer questions, I thought it was a good session.

Don't forget to index your PostgreSQL foreign keys

One thing that surprised me with PostgreSQL after spending a lot of time with MySQL is that setting foreign key constraints do not automatically generate indexes on those columns. PostgreSQL will implicitly generate indexes for primary keys, but not foreign keys. Since those foreign key columns are precisely the ones you tend to use for joins (either explicitly or implicitly by ORMs like Hibernate), omitting indexes could result in a serious performance hit. Creating indexes is easy enough, whether using the usual SQL command or a Hibernate @Index annotation, but where do you start? Well, the whole point of this blog entry is to point you to another blog entry. It provides a handy query to look for foreign keys that could use indexes, ranked by the ones most likely to need an index. Take a look:

Postgresql: Indexes on Foreign Keys

The language wars and the TIOBE index

Recently, I had a bit of fun by writing a modestly titled blog post arguing in favor of Groovy's adoption over Scala. Among other things, I pointed out that the Indeed.com job trends graphs show much stronger Groovy adoption over Scala. One comment there made the interesting observation that the TIOBE Programming Community Index said precisely the opposite. I did not exactly carry out meticulous research so I confess I did not pay attention to this index when I made that blog post. I thought I'd remedy that oversight in this post.

My Groovy can beat up your Scala

This blog entry is inspired by a blog post I read recently bashing Groovy in favor of Scala. I will opportunistically join in the brawl, contributing my 2 cups of kerosene to the flames by coming out in favor of Groovy.  If you are not interested in the language wars, or if you take this sort of thing way too seriously, feel free to skip this post.

I like Groovy. At my last job, I used Groovy almost exclusively as a "better Java", on top of the typical Spring/Hibernate  stack. I did not use Grails at all, except briefly on a minor project. Even though I had a grand time with Groovy, I never thought of Groovy as the heir to the Java language. A new language will eventually replace Java, I figured, but it would not be Groovy. Whatever the merits or demerits of dynamic typing, I thought, the Java community, platform and class libraries were geared towards static typing. Also, Groovy was too slow. Recent developments have made me question my assumptions.

Let's get rid of middle management

Let's get rid of middle management.

No, not you, boss.

I'm talking about middle management in Java code. At one presentation in the New England Software Symposium in Boston last year, some Java-bashing guy (Stuart Halloway) described Java as the language that invented middle management. You have probably seen them: these classes often end with "Manager.java". I'm wondering whether we really need them.

What's new in KBLauncher 1.1.1

I recently released a new version of KBLauncher, my application quick launcher for the BlackBerry. This blog entry serves as my release notes for this app. The rest of you who are here for non-BlackBerry topics can safely move on.

On the breadcrumb metaphor

I've heard the term used in UI work: people refer to "breadcrumbs" when they mean a navigation aid that shows UI path used to arrive at a point in the program. This supposedly prevents a user from feeling lost. It could be a trail of screens or dialogs like Options > Advanced Options > Applications. I don't have an objection to that. I do think, on the other hand, that "breadcrumbs" is a pretty bad term to describe this UI construct.

The most famous example of using bread crumbs to mark a trail resulted in those crumbs being eaten by wild animals and the adventurers nearly becoming dinner themselves. All versions of of the Hansel and Gretel tale that I've read had the children successfully return home when Hansel used pebbles or small stones to mark a trail. This trick failed when Hansel used bread crumbs. With the crumb trail eaten by animals, they end up lost in the woods. They eventually wander to that candy house where they are trapped by a witch who tries to eat them. You know the rest.

So if you say you want to use breadcrumbs, are you really saying that you want to leave the user lost in the woods to be eaten by a cannibalistic witch?

Oops, did you accidentally ship your BlackBerry source code?

If you develop BlackBerry software using RIM's BlackBerry Java plugin for Eclipse and use Subversion for version control, you could end up shipping your source code with your executable if you are not careful. I had hoped this would be solved with the new version (1.1) of the plugin -- still in beta -- but based on my experience with one of the betas this does not appear to be the case.

The problem comes from the plugin's naive handling of non-Java files. Subversion keeps a copy of every version-controlled file in your sandbox. For example, if you have a Foo.java you will also have a copy as .svn/text-base/Foo.java.svn-base. This allows you to do diffs with the base version or revert your changes without even contacting the SVN server. But when RIM's plugin builds your *.cod executable, it assumes those *.svn-base files are resource files to be bundled with your COD file. The result is that if you view your resulting COD file in a text viewer, you will see copies of your source files -- those svn-base files -- embedded in the binary. Good grief.

Until RIM fixes this, one way to work around this is to specifically tell Eclipse to omit those SVN support files from the build. In your project properties, go to Java Build Path, select the Source tab and edit the source folders' "Excluded" entries. You can use wildcards in your exclusion patterns. And of course, make a test build and view it to make sure your source code is really excluded.