Wednesday, January 11, 2006

New Frontiers of Enterprise Java Development

I've been thinking lately about trends in enterprise software development, especially in the Java world. Overall, what I see is a change in priority for enterprise software projects: from struggling not to fail completely, to actually excelling and competing. The latest technology and tools, along with modern "Agile" development methods, enable companies to really compete on quality and functionality, efficiency and productivity. Smart development organizations can not only avoid failure, but deliver better software more efficiently.

In the mid-90s, the ubiquitous distribution and instant maintenance capabilities of Web-based applications pushed them to the forefront. When applications moved beyond brochure-ware, a significant problem arose: Teams developing Web-apps could not readily achieve the enterprise requirements, or qualities, necessary for success: performance, scalability, security, etc. So, most of the effort and money spent by development teams went to achieving the necessary enterprise qualities.

Teams also struggled with process in the 90s, and to a large extent still do. The fast and loose style of early Web development did not work for mission critical apps, and traditional Methodologies did not work for Web teams. Furthermore, I think the frenetic rate of business and technical change finally killed the notion of viability of traditional, waterfall processes, at least for shops that "got it. "

By the late 90s, the major Web application platforms, like J2EE and compliant app. Servers, made it practical to address Enterprise requirements--although not easy or efficient. In addition, the effects of Moore's Law have made it much cheaper to throw more hardware at a solution, reducing the effort involved to handle the scale of enterprise apps. At the same time, Agile project management approaches began to gain visibility, blending the engineering-like discipline with the creative nature of software development.

In 2006, I contend, the development tools and frameworks for Web development have largely caught up to the platforms. Creating a robust, scalable, enterprise app no longer necessitates an army of J2EE coders rolling EJBs by hand. Looking forward, ISVs and IT shops just don't have to focus as much on addressing enterprise requirements*. Nor do teams need to struggle with heavy Methodologies--there are new mountains to climb. It is time for Web application development to grow up and act like "real" product development.

Competing with Agile Project Management:
Today, I think any manager who doesn't at least vaguely acknowledge the benefits of an Iterative or Agile project management approach has been hiding under a rock for several years. Some managers even claim to use these approaches, although few really do it right. Just because a team breaks up the coding for a project into increments, doesn't mean that they are Agile.

To improve productivity with process, management must get away from the compulsion to scope, define, and plan the entire project up front. Rather, they must accept and embrace two essential facts:

  1. Needs and requirements will change constantly during the project; and
  2. User needs, technological pitfalls, and team capabilities will never be adequately or accurately understood until the team has gone through the complete lifecycle with one or more slices of the working system.

Yes, it's a catch-22 situation: Managers have to accept that reality. Managers must embrace change, accept the empirical nature of the software development process, and stop trying to do all the planning and specification work up-front. Companies still trying to use linear, silo'd, paper-heavy methods for Web-based apps are dead, whether they realize if or not. To increase productivity, and reduce costs and time-to-market, managers must truly understand and utilize Agile management. I personally think that Alistair Cockburn's work on Agile development is the most comprehensible. Scrum is perhaps the best overarching management view, and XP provides a system of proven team practices.

Another project management "move" that can greatly reduce development cycle times is rapid prototyping. Simply put, exploring and defining requirements with documents is a relatively poor way of getting the system right. More productive projects user "richer" media for definition and validation, and the richest vehicle is a working prototype. For example, a team can develop a Ruby on RAILs prototype to quickly and cheaply explore and validate the functional requirements. Another powerful option for rapid prototyping and validation is Gorilla Logic.

The working prototype provides much more comprehensible, complete, and unambiguous information than stacks of documentation, which saves a ton of wasted work downstream. If one accepts that an iteration or increment can be thrown away, then prototyping fits very nicely with Agile methods.

Though not strictly a project management move, I also think that automation has tremendous potential to make teams more competitive. That will be the subject for another post. For now, I will simply say that I think that XDoclet is definitely a tool that teams should look at using and extending... and MDA tools like AndroMDA show some potential as well.

Competing with Richer User Interfaces
Along with the powerful distribution and update benefits of the Web came with a catch: The capabilities of a browser-based interface, though often visually attractive, are weak. Things like tree browsers, master-detail views, and fine control over interface widgets have generally been painful to achieve. Teams did not have time to work around these issues, and the user experience suffered.

Now that the technologies are available that help to take care of enterprise issues with much less effort, teams can make there applications better with richer user interfaces. AJAX, a technique for making asynchronous HTTP requests in the javascript for a page, seems to be a very hot technology of late. For applications that require a highly graphical look and feel, along with better usability, Macromedia Flex is strong player. But the next great client platform, I believe, is Eclipse RCP.

*NOTE: I also think that tools Ruby on RAILS, PHP, ColdFusion, etc. might now present a viable challenge to J2EE and .NET in the Enterprise space, with the right hardware.

**NOTE: I really don't have a feel for what is going on in the Microsoft/.NET world. However, to their credit, I think Microsoft has always focused more on productivity than on Enterprise requirements (e.g. VB). How successful they have been on this front is a matter of debate.