What's So Great About Rails?

OK, so now I guess I am becoming a "Rails Guy." At least, I have been doing some Ruby on Rails programming lately and am generally very impressed.

It is still Web programming, there are no major paradigm shifts here. However, Ruby on Rails is an example of how a lot of little things done right can really add up. Ruby is a fine scripting language, improving upon popular languages like Perl and Python. And Rails is an example of getting the little details of Web programming right. Rails is optimized (as I see it) for (1) Productivity and (2) Maintainability. The following are what I really like:

• The clean, simple ActiveRecord ORM layer.
• A full-fledged yet simple MVC that is easy to comprehend and get around in.
• Sensible defaults and very little configur-bation; "convention over configuration."
• A good job of making the most common tasks simple; the 80/20 rule.
• "rake migrate" is the coolest way to manage schemas that I have seen.

So, is Rails the answer to everything? Of course not, you won't find sweeping generalizations here. As always, it depends on *specific* requirements. Ruby and Rails are geared toward fast and simple development and maintenance. Java and JEE technologies generally have not prioritize these factors as highly as Enterprise requirements, e.g. massive scalability, security, etc. And I am not saying that one or the other stack is incapable of solving a kind of problem, rather that it is a matter or priorities, focus, and fit.

So I do like Rails a lot, I think it is probably the best solution for many small-medium sized webapps. But it is not necessarily a replacement for Java, especially in the enterprise. The trick is for decision-makers to weight their needs for big E Enterprise requirements against speed and simplicity.

Productive Editing and Eclipse

I went to a great presentation by Neal Ford last week at the DJUG, entitled the Productive Programmer. Neal is a sharp guy, a good speaker, and a solid thinker on improving productivity. Go ahead and CRTL-Enter nealford in your address bar to check out his site (you'll see what I mean).

The talk was a bit of a wake-up for me. His premise is that the rise of the IDE has made programmers, well, lazy (my words, I don't think he went so far). I realized that I have let some old but valuable skills languish in the past few years. So Neal's talk has inspired me to sharpen the saw back up. For example, I have made the decision that Ruby will be my scripting power tool moving forward (used to be Perl), and I will not shy away from opportunities to explore automation (even if it is initially just exercise).

Another point that resonated with me was the idea of the "perfect" editor. I do think that I am now as productive over all in Eclipse as I used to be in Emacs (with a side trip through NetBeans). But Neal hit on two things that are missing: Multiple cut/paste buffers (kill-loop in Emacs) and editor scripting. Though I am intrigued by the lightweight JEdit, I am determined to stay in Eclipse and make the best of it. I think that surely these two deficiencies can be addressed.

On the multiple buffers front, I quickly came up with an answer that works for me: CLCL. This tool is actually a utility that caches your Windows clipboard. So CRTL-C still works like normal, but you can also reach back in time and paste something from 30 cuts ago. Sounds trivial, but it can change the way you edit.

As for the scripting integration, I haven't looked very far. Prashant Rane's blog led me to Eclipse Monkey, which looks interesting--it's Javascript, which I can defnitely code. I might keep looking, though, for something that provides editor hooks for Ruby, Groovy, or BeanShell. I'll post again if I find the silver bullet.

Shameless Promotion 4: Rapid, Affordable Application Development

I am available to help Clients develop applications cheaply and quickly, applications that help them to reach more customers and reduce costs. I can help clients use the Web to make breakthroughs in marketing, sales, and operations.

I have built dozens of successful applications for the Web and other environments. I can provide hands-on expertise across the software development lifecycle: from strategy and requirements to architecture and design to development and testing. I also provide project management and "general contractor" services, so I can bring the right talent to the table for most any web development project.

The following is a quick summary of the skills and capabilities I provide to help you achieve robust solutions quickly and at affordable rates:

• Experience in online communication, eCommerce, and improving efficiencies.
• Technical lead, architect, and developer experience on dozens of projects, including complex applications for Sheraton Hotels, Travelocity, Budget Rent-A-Car, and the Veteran's Administration.
• Programming experience in Java (5+ years), C (3), VB/ASP (3), Perl (2), and lately Ruby.
• Effective, hands-on skills as a software engineer, architect, and business/systems analyst.
• MBA and a Bachelor's in Computer Science.
• Extensive network of software and Design professionals for general contracting services.
• Very competitive rates, for myself and subcontractors.

My Resume describes my skills in much more detail. I'd love to make your development project a huge success, drop me a line if you are interested.

- John

jtroxel@yahoo.com

Software Success Factor #1: Manage for Agility UPDATED

The ability of an organization to Manage for Agility is one of the most important success factors for modern software development. This factor encompasses the critical role that Management--or the project Client or Sponsor--plays in Iterative Development. In this post, I will attempt to address what this factor entails and how it is accomplished. I will also briefly recap why this management style is superior. I have yet to complete the Overview of the Reference Model, but Agile Management is so important that I wanted to flesh out this success factor first.

What
Iterative development is perhaps the key attribute of modern software success, but without management support it is impossible. To manage for agility, leaders must go against many traditional management styles and ingrained practices. It is a natural for successful business people to try to avoid future change through detailed planning and up-front analysis, but changes are inevitable. Trusting the team and the process to produce the best solution, allowing the scope, the design, and even the process to emerge, is what it means to Manage for Agility.

How
To manage agile projects, sponsors must resist the desire to plan and design the entire project before starting any development. To be agile, management must instead embrace a flexible model that responds to changing requirements and an evolving understanding of the solution. Management must trust the process and the team to produce top quality software, without a relying on a rigid scope.

Often it is the "commercial Model," contracts, SOWS, etc., that most influences the interaction between management and the project team. To manage for agility, the commercial model must provide for evolving scope and design. Put another way, attempting to fix the scope in agreements up-front is simply at odds with an agile style. When using consultants or contractors, a Time-And-Materials approach allows for the most agility, or the agreement can employ separate SOWs created at the start of each iteration. Another approach offered by Kent Beck is the use of "Optional Scope Contracts."
It is also important to understand that agile does not mean that no planning or requirements are done up front. The project still needs a first cut at the inventory of requirements--e.g. a Scrum backlog or a Use-case survey--that is the source from which iteration requirements are selected, refined, and built. An agile project can also develop a detailed, "pro-forma" project plan, but managers should not feel compelled to stick to the original as the team adjusts to accumulating change.

Why
Why should managers buy into this approach, when traditional project management techniques have worked on all kinds of projects, from construction to aerospace engineering? Because software development is fundamentally different from construction or engineering, and treating it like these disciplines magnifies the risk of failure, raises costs, and produces worse software.
Change is unavoidable in Software Development, more so than most any field. Alan Shalloway made the claim, at a recent Agile Denver meeting, that software development is not like building a bridge; it is more like building the first bridge... or the design phase of a bridge project. An analogy is also often drawn between construction and software development. However, many experts claim that software development is essentially all design work, and the only step that is really like construction is the compile/build cycle.
The highly successful Scrum process is based on proven new-product development processes used by Japanese companies. Indeed, the Scrum proponents and many others have claimed, accurately I believe, that software development is very like new product development. If we look at software development as roughly analogous to the design and development of a new kind of car, or a new consumer electronics gadget, it becomes clear that the nature of the process is unpredictable and should be managed accordingly. Agile management uses an empirical process management approach to produce a better product. One need look no further for evidence than the difference between American cars and Japanese, at least until very recently.
Alistair Cockburn expands on the idea of "responding to change over following a plan:"

...traditional process management—by continuous measurement, error identification, and process refinements—strove to drive variations out of processes. This approach assumes that variations are the result of errors. Today, while process problems certainly cause some errors, external environmental changes cause critical variations. Because we cannot eliminate these changes, driving down the cost of responding to them is the only viable strategy. Rather than eliminating rework, the new strategy is to reduce its cost.

[NOTE: I think he means code rework--Big Up-Front planning and design will result in more rework of other work products, assuming the effort is not wasted entirely as these products are abandoned.]

Business needs, priorities, understanding of user needs, expertise with the
technical platform, market forces, and available technologies... All of these
can and usually do change over the life of a project. Agile management rolls
with these changes and capitalizes on them to make the best software possible.
Smart project sponsors can use agile management to build better software for
less money.

The Software Success Reference Model - Overivew

Here, again, is my Reference Model: a 30,000 ft. view of software success. This post will provide an overview of the categories from the model--sort of a taxonomy of success. See my previous post for more information on how I derived this model.

Groupings
Organization and Project Management
Perhaps this should be entitled simply Organization and Management. This grouping deals with factors that are beyond the scope of any project team--organization and management at the highest levels.

Project Team
These factors deal more specifically with the project team involved directly with producing the software.

Iterative Development
This category contains factors that relate to producing software in an iterative, incremental, adaptive manner.

Productive Culture
Factors dealing with the kind of people the Organization employs and how the Organization fosters a culture that promotes productivity.

Productive Practices
Specific practices that help the Project Team develop software efficiently.

Software Engineering (for Large Projects)
Factors necessary for projects with a large number of developers and/or critical consequences of failure.

Factors
Manage for Agility
This factor encompasses the critical role that Management--or the Client--plays in Iterative Development. Leadership must be willing to launch a project without detailed plans or specs, and trust the team and the process. Further, project clients must realize that the agile approach is a Good Thing, allowing the team to respond nimbly to a changing world, and producing more relevant software in less time.
Deliver Value Quickly & Often
Another aspect of Iterative Development, this factor focuses on the ultimate goal of software development: working software. Teams must focus on producing working software from the very first iteration, and keeping implementation as relevant to the needs as possible by avoiding unnecessary interim work products and long time lags.
Continuously Learn & Optimize
The final component of Iterative Development is the concept of continuous improvement. Iterations afford the team with the opportunity to exercise all lifecycle activities in short bursts. This means they can reflect, learn, and optimize their process and practices with each cycle.
Empower the Project Team
Also solely in the Management arena, the client of the project must provide the project team with the tools, resources, and support necessary to succeed. As much as possible, the team must be allowed to organize and direct itself however necessary to deliver the requirements.
Enable Open Communication
Software development is inherently a team design activity. Communicating ideas and building a shared mental model of the solution are critical. Organizations must strive to achieve "osmotic" communication, provide access to experts, and involve users throughout the process.
Demand Quality
Teams must develop a high standard of quality as a norm, and internalize quality validations within the team, rather than "throwing" code "over the wall."
Use More Realistic Specification(s)
Just as a picture is worth a thousand words, the richer, higher-fidelity one can achieve in software specifications the better. Teams should use mock-ups, prototypes, and visual models as much as possible, while still constraining specification to whatever is "minimally sufficient" for the current increment.

Use Automation and Tracking Tools

The value of SCM, task & issue tracking, automated tests, etc. is well established. Just do it.
TODO: Software Engineering factors

Software Development Success Reference Model - Background

What does it take to run a successful software development project? Success does not come easy, but it can be done. The industry has many success stories, and software development thought leaders have identified the key principles that lead to success. But what are the critical success factors, and what principles find common ground among the experts?

About 6 months ago, I started trying to create a set of success factors that were synthesized from the experts. I have since completely revamped my thoughts. What I ultimately want, for my own purposes, is a "Reference Model" (e.g. the OSI 7-Layer Model) that provides some structure with which I can I can think and talk about the various factors that lead to success--independent of any specific methodologies. I hope to be able to say things like "Reflection Workshops in Crystal are one technique to Continuously Learn and Optimize," or "XP provides a system for Iterative Development and Proven Practices, focused on smaller Project Teams."

I have now spent some more time thinking, and included some more popular, respected sources. The following slide is my Reference Model, a 30,000 ft. view of software success.

In future posts, I will provide more details on the individual success factors and how they relate to the work of industry luminaries. So, how did I arrive at this model? I started by taking the most important, highest-level principles and factors from the following excellent resources:

• The Standish Group's CHAOS study [CH1..9]
• The Unified Process or RUP [RUP-1..6]
• The SCRUM meta-process (web, book) [SC-1..4]
• Alistair Cockburn and Crystal methods (web, book, book) [AC-1..7]
• eXtreme Programming (web, web, book) [XP-1..5]
• My own experience on dozens of successful projects [JT-1..6]

There were several steps between extracting the critical success factors from these sources and defining my "nutshell" view, but the following slide captures the start of my reasoning. This is an "affinity diagram" that attempts to consolidate all the factors into logical groupings.



Comments are most welcome.

New Frontiers of Enterprise Java Development

I've been thinking lately about trends in enterprise software development, especially in the Java world. Overall, what I see is a change in priority for enterprise software projects: from struggling not to fail completely, to actually excelling and competing. The latest technology and tools, along with modern "Agile" development methods, enable companies to really compete on quality and functionality, efficiency and productivity. Smart development organizations can not only avoid failure, but deliver better software more efficiently.

Background:
In the mid-90s, the ubiquitous distribution and instant maintenance capabilities of Web-based applications pushed them to the forefront. When applications moved beyond brochure-ware, a significant problem arose: Teams developing Web-apps could not readily achieve the enterprise requirements, or qualities, necessary for success: performance, scalability, security, etc. So, most of the effort and money spent by development teams went to achieving the necessary enterprise qualities.

Teams also struggled with process in the 90s, and to a large extent still do. The fast and loose style of early Web development did not work for mission critical apps, and traditional Methodologies did not work for Web teams. Furthermore, I think the frenetic rate of business and technical change finally killed the notion of viability of traditional, waterfall processes, at least for shops that "got it. "

By the late 90s, the major Web application platforms, like J2EE and compliant app. Servers, made it practical to address Enterprise requirements--although not easy or efficient. In addition, the effects of Moore's Law have made it much cheaper to throw more hardware at a solution, reducing the effort involved to handle the scale of enterprise apps. At the same time, Agile project management approaches began to gain visibility, blending the engineering-like discipline with the creative nature of software development.

In 2006, I contend, the development tools and frameworks for Web development have largely caught up to the platforms. Creating a robust, scalable, enterprise app no longer necessitates an army of J2EE coders rolling EJBs by hand. Looking forward, ISVs and IT shops just don't have to focus as much on addressing enterprise requirements*. Nor do teams need to struggle with heavy Methodologies--there are new mountains to climb. It is time for Web application development to grow up and act like "real" product development.

Competing with Agile Project Management:
Today, I think any manager who doesn't at least vaguely acknowledge the benefits of an Iterative or Agile project management approach has been hiding under a rock for several years. Some managers even claim to use these approaches, although few really do it right. Just because a team breaks up the coding for a project into increments, doesn't mean that they are Agile.

To improve productivity with process, management must get away from the compulsion to scope, define, and plan the entire project up front. Rather, they must accept and embrace two essential facts:

1. Needs and requirements will change constantly during the project; and
2. User needs, technological pitfalls, and team capabilities will never be adequately or accurately understood until the team has gone through the complete lifecycle with one or more slices of the working system.

Yes, it's a catch-22 situation: Managers have to accept that reality. Managers must embrace change, accept the empirical nature of the software development process, and stop trying to do all the planning and specification work up-front. Companies still trying to use linear, silo'd, paper-heavy methods for Web-based apps are dead, whether they realize if or not. To increase productivity, and reduce costs and time-to-market, managers must truly understand and utilize Agile management. I personally think that Alistair Cockburn's work on Agile development is the most comprehensible. Scrum is perhaps the best overarching management view, and XP provides a system of proven team practices.

Another project management "move" that can greatly reduce development cycle times is rapid prototyping. Simply put, exploring and defining requirements with documents is a relatively poor way of getting the system right. More productive projects user "richer" media for definition and validation, and the richest vehicle is a working prototype. For example, a team can develop a Ruby on RAILs prototype to quickly and cheaply explore and validate the functional requirements. Another powerful option for rapid prototyping and validation is Gorilla Logic.

The working prototype provides much more comprehensible, complete, and unambiguous information than stacks of documentation, which saves a ton of wasted work downstream. If one accepts that an iteration or increment can be thrown away, then prototyping fits very nicely with Agile methods.

Though not strictly a project management move, I also think that automation has tremendous potential to make teams more competitive. That will be the subject for another post. For now, I will simply say that I think that XDoclet is definitely a tool that teams should look at using and extending... and MDA tools like AndroMDA show some potential as well.

Competing with Richer User Interfaces
Along with the powerful distribution and update benefits of the Web came with a catch: The capabilities of a browser-based interface, though often visually attractive, are weak. Things like tree browsers, master-detail views, and fine control over interface widgets have generally been painful to achieve. Teams did not have time to work around these issues, and the user experience suffered.

Now that the technologies are available that help to take care of enterprise issues with much less effort, teams can make there applications better with richer user interfaces. AJAX, a technique for making asynchronous HTTP requests in the javascript for a page, seems to be a very hot technology of late. For applications that require a highly graphical look and feel, along with better usability, Macromedia Flex is strong player. But the next great client platform, I believe, is Eclipse RCP.

*NOTE: I also think that tools Ruby on RAILS, PHP, ColdFusion, etc. might now present a viable challenge to J2EE and .NET in the Enterprise space, with the right hardware.

**NOTE: I really don't have a feel for what is going on in the Microsoft/.NET world. However, to their credit, I think Microsoft has always focused more on productivity than on Enterprise requirements (e.g. VB). How successful they have been on this front is a matter of debate.