Lean Cincinnati is Moving !

To get better Blogging facilities Lean Cincinnati is moving to


http://agiledna.com


Please reset your RSS feeds.

-Thanks

Field of Dreams

In the last week, I've talked with two very smart, highly motivated programmers. Both are making plans to leave the Cincinnati area.

Why? Both say that there are no interesting programming projects going on here. All they see is old-style java and .NET projects that are missing the wave of new approaches.

And they feel that they would put their career at risk by staying here.

The questions we have been discussing on a local mailing list (http://groups.yahoo.com/group/grow_cincy_soft/) are:

(1) Why are programmers leaving Cincinnati?
And (2) How can we keep and attract more?

If we conclude that they are leaving because Cincinnati companies are not offering them interesting projects, why is that??

Some say it is because Cincinnati is somehow "Conservative".
That doesn't answer anything really.
Is WalMart (which considers IT as part of their core competency) somehow "Liberal" ??
I don't think so!

It's not a about being "Conservative".
It's about seeing IT as either a competitive advantage or as a cost center to be minimized.

It's a fundamental VALUE choice that each company has to make.

And the question shifts to:
Why don't Cincinnati companies embrace technology as a competitive advantage?

If you see technology as a competitive advantage, and invest in it, the programmers will love to stay and work on the important projects.

ITS THAT SIMPLE.

If you see IT as a cost center to be minimized and out-sourced, then don't be surprised if the good programmers prefer to move away from Cincinnati and work for Google, or ThoughtWorks, or someplace that wants to us IT to MAKE money.

As they said in the movie "Field of Dreams"...
Build it and they will come.

- Mark
http://leansoftwarecincinnati.blogspot.com/

Throughput Accounting

Accounting has different problem solving schools of thought just like software development does.

If you want to measure the effectiveness of a management decision you want to count the costs and benefits of that decision. The traditional way to do this is with Cost Accounting. In the Lean-Thinking approach, Cost accounting blamed for guiding companies toward inefficiencies.

Instead an approach called: Throughput Accounting, measures end-to-end benefits on a system wide basis.

One difference is that with Cost Accounting inventory is valued as a good thing - an asset. The more inventory you have on the books the more your company is worth (on paper anyway).

Using Throughput accounting, Value is only recorded when a product is actually sold. The piles of out of date parts carried in inventory are seen as a liability. The more incomplete work you've got in the process the lower your value (on paper).

Thoughput accounting seems to me to reflect the real word a bit better. Your company sells a product, you record value. Not before.

But there seems to be a struggle in the accounting community between these two camps. Partly, because they are advising management to take different paths toward success.

It reminds me of the struggle between the Data Processing and the Object Oriented approaches to software development. More on that later.

Lean Software Strategies

I just got a Lean Case study published in chapter 25 of
Lean Software Strategies: Proven Techniques for Managers and Developers .




And for a Recap of Lean Software Development see this previous posting.

The Rails Day-After

Well, I didn't give as many "updates from the front" as I intended. But here is my rapid-review of the day of coding.

Wow !

Ok, now more detail...

It was the quickest 18 hour period that I've experienced in a long time. Everything I looked at the clock it jumped another 2 hours. Among the 6 coders we all seemed to feel that way. After an 8 hour day at work we usually feel tired out. After 17 hours in the contest and at 1:00 am, we were all still wishing we had another hour or two, to squeeze a little more functionality in before the deadline. I did not see one single yawn the entire time. As the evening was getting late, we inadvertently missed dinner entirely. We did have plenty of snacks to keep us going.

The Contest

About 120 teams were signed up. 100 competed. This contest was world-wide. More details at railsday.com

The Application

It was a very good start on the app. It was usable and functional. The PhotoDepo sets up user accounts for Photographers, Guests, and Administrators. Photographers can upload Photo's, arrange them into albums and add "tag words" to them. Guests can browse and search the photos by the "tag words". Administrators set up and manage the accounts. And other users can browse without signing on to the application.
We do wish we had another few hours to add some of the cool AJAX features and polish it up a bit more.

Lines Of Code Produced

App Code	445 (not counting the rhtml)
Test Code	524
Total Code	969

That's about 20 lines of code per person per hour

More Railsday Review's

Scott's Review and Jims's Review

RailsDay Milestones

The two programming teams in Cincinnati are Team8 and Team32.
Team8 is doing a food tracking system.
Team32 is doing a photoDepo

The contest rules require at least 12 check-ins.
At this point Team34 has 16 check-ins and Team8 has 32.

Today is Rails Day

Rails day has started. If you would like to check how it's going ask me at AOL Instant Messager Nickname: objwind

Also available for video iChat with Apple iSight.

Show Time !

Tommorow is the first www.RailsDay.com.
A 24-hour contest to code an interesting web application.
Cincinnati has 2 teams entered, and we will be coding in public at the Kenwood towers
The Towers of Kenwood
8044 Montgomery Rd.
Cincinnati, OH

Directions and organizational information are available at:
Rails Day Planning

Stop by and visit the coders in action.

Meta-Data

The Server Side has an article on when to use code Annotations.
Code Annotations are Meta-Data in the code. And the Java community is happy to be getting annotations in Java 5. THe .NET crowd has had them for a while now.

But maybe the emphesis in my title gave away my point. It's Meta-Data. Get it? More Data.

The Object Oriented way to solve the same problem is with objects. Meta-Objects, or Meta-Classes. Java followed .NET down the road of taking the Data-Processing approach.

Annotations may look helpful. And perhaps they are. But they won't solve development problems as cleanly and maintainably as OO could.

We need to look to other languages (like Ruby perhaps) to actually provide Meta-Classes and an Object Oriented approach.

24 hours of code

Lean Software Development produces code on an ongoing basis, rather than in long waits and big chucks of deployment. Using xp www.lifeware.ch delivers code into production every working day. This is made possible by Agile processes like Extreme Programming (XP). And by smart technology choices. LifeWare uses Smalltalk.

Another continuous code delivery technology will be demonstrated
"On June 4th, Ruby on Rails developers all over the globe will have 24 hours to build the best application they can."

See what is possible by tuning in or participating http://railsday.com/

Recap: Lean Software Development

[The links in this recap all point to previous entries. It is a recap after all.]

The Problem is that software projects still deliver an unacceptable level of quality and responsiveness. Part of the reason for this is the industrial-era Scientific Management approach. Part of the reason for this is the progamming culture that grows up around a language.

Lean software development provides information-era Lean management: Reduce Cycle-time and start expecting zero defects.

But the cultural issues and organizational structures remain extraordinarily difficult to deal with. Some hope may be appearing in a new programming language (and more importantly programming culture) Ruby. Which includes in it's libraries and way of development, new technologies that are merely bolted onto J2EE (and .NET).

The promise of applying Lean-Thinking to Software development is available now. It consists of a Lean Business Model, combined with Agile Processes and a deeply Object Oriented environment and culture. All these parts are currently available.

Knocking the competition for a loop, only requires putting them all together.

How hard can that be ?

Looking at Ruby

Jim Weirich is a bit of a local (Cincinnati) legend. He has been modesty and quietly mentioning an up and coming programming language in various local users' groups and forum's for years.

It's a measure of how entrenched a culture can get that even when highly respected community leaders encourage a new idea, the idea still does not really sink in. So, I kept thinking he was right, but not relevant for my own work and interests. And I kept pushing Ruby down the list of things I wanted to look into.

Finally, after I did study up a little on Ruby, I caught up with Jim. I apologized for ignoring his mild suggestions that Ruby was really something, nice to know.

What was it that finally did it?

Initially, in January of 2005, it was seeing the RubyOnRails framework video.

After that, it was the appreciation of the Ruby culture, and the currently forming RubyOnRails sub-culture. More on that later ...

Since Java 1.0

I like Java and J2EE. I work with these often and enjoy the tools and environments. I've been through a number of generations of the hot technology, and I like to play the game of trying to see around the next corner. That means looking hard at the current state of the art. That's what I'll do now ...

The Java language entered mainstream production usage in parallel with the raise of some other increasingly important technologies. One of course was Web development, and Sun was very quick to adapt Java to help address the needs of the web. The web and Java go together very powerfully.

Some other technologies while successful, have been bumpier to integrate. Meta-level programming was added to Java in the form of Reflection with a useful but clumsy API.

Aspect Oriented Programming (AOP) is being added both at runtime with libraries, and at compile time with AspectJ byte modifications. And in a similar way as the Meta-level API, AOP in Java is useful, yet just a bit convoluted.

The most important after-market addition to Java, in my estimation, has been jUnit. And the whole idea of unit testing. The practice of automated programmer testing, is still struggling to become standard operating practice by the typical Java programmers.

• Meta-Level programming
• Aspect Oriented Programming
• Unit Testing

My point is that these three new technologies are available for Java, but not integrated into the programming approach of the typical Java programmer. But that's ok since there are plenty of production Java programs written without these three making money for companies.

Reason #2, Why Java is written in Data Processing style?

Ideas don't thrive unless they fulfill a need. That's simple meme propagation theory.

So even Object Oriented enthusiasts like myself, have to admit that something in the use of Java Data-Processing designs works well, for someone, somewhere.

Objects have been shown to be easier to test, and adapt, and extend. What is it about separating Data and Processing that seems to trump, those good things???

Just because Objects are the best thing for Application development, AND just because the agility Object provide is the best thing for businesses, does not mean that Objects are the best thing for organizational structures. Departments are often separated by concerns not strictly focused on the delivery of Value.

Separating the Database and Application programming into different management hierarchies puts enormous pressure on the organization to produce Data Processing architectures. This is an instance of Conway's Law which states that the shape of the architecture will resemble the shape of the organization.

Is is any wonder that the best flexible, agile software is produced by smaller companies that have not grown so big as to split their people into departments tuned for something other than pure, full-out, software delivery?

Reason #2: the organizational structure encourages the separation of Data and Processing, which results in Data Processing designs.

Reason #1, Why Java is written in Data Processing style?

During my presentation on COBOL Oriented J2EE I talked about the separation of Data and Behavior that plagues so many Java applications. JavaBeans fro data and Transactions scripts for behavior is the opposite of Object Oriented development. This however, is exactly the style advocated by the Sun J2EE blueprints. It is how many programmers designed their web applications.

So why is do much J2EE code written in a Data Processing style? Reason #1, because Sun said it should be.

ROI

Finally, there are two ways to increase the payback (or Return On Investment) on a software project:

• lower the cost or
• increase the value.

Lower costs can be accomplished with offshore development. Understandably, corporate executives are unhappy with paying high dollar rates for poor quality software built locally. Instead they reduce labor costs by moving projects offshore. Offshore programming gets low-cost, & talented programmers to produce the same poor quality software as before. Reducing cost increases payback.

An alternative way to increase payback is to increase project value. This can be realized by savings of time to market, closer understanding of the real customer requirements, and customer goodwill by providing defect free applications. Lean Software Development provides the business case for this approach. Extreme Programming provides the most popular way to implement Lean Software Development.

A closing thought. In the 1980s Japanese Lean Auto Manufacturers established engineering and production facilities in the U.S.. Did they move development off-(Japanese)-shore for lower U.S. labor rates? No way! They now build cars in Ohio and Michigan, in order to be closer to the customer, to ship quicker and to understand more deeply. They moved closer to customers to produce more profitably.

The same occurs in companies that adopt Lean Software Development. Offshore may be less expensive, but when software is business critical, locally built Lean Software Development with short cycle times and low defects are the cost effective, high value option.

Expecting Zero Defects

If you expect zero defects a fundamental change in thinking occurs. Do I mean to say that no single defect will ever occur? No. I mean that we treat every defect as something really, really bad, and figure out how to prevent that class of defect from occurring again.

Lean Manufacturing production lines encourage each assembly worker to stop the line if a defect appears. Paradoxically, lean production lines almost never stop.

In Scientifically Managed production lines, only the manager can stop the line, yet they often crash because defects are not reported creating bigger problems downstream.

Expect zero defects and you realize that the way you schedule, test, program, and release software all needs to change to accomplish your goal. And since 80% of current software project dollars are spent removing defects the goal of Zero Defects becomes financially justifiable.

Extreme Programming has a practice of Test-First coding that is fundamental in achieving the goal of Zero Defects. Test-First coding requires that an automated unit-test is written before the production code and the test-to-code cycle time be 5-15 minutes long. Additionally, automated acceptance testing insures that the correct business value is delivered. Acceptance tests are built directly from the requirements before the production code is written. All tests are built one at a time immediately before each piece of production code. Moving the tests to the front of the process gains efficiencies in providing a tight discipline and direction to the team.

Cycle Time

The software industry finds itself in a situation today similar to automobile manufactures in 1970s. Defects are sapping resources and angering customers. Defects and long cycle times are reducing our ability to respond to new business opportunities. The most heavily marketed solution is to add more bulky process steps and to do more testing at the end. This approach lengthens cycle times. Meanwhile the Lean approach is delivering successful projects using processes with unlikely names like: Scrum, DSDM, and Extreme Programming (XP). Lean Software Development reduces defects and cycle times while delivering a steady stream of incremental business value.

Cycle time for software development is measured in the number of days needed between feature specification and production delivery. This is called: Software In Process (SIP). A shorter cycle indicates a healthier project. A Lean project that deploys to production every 2-weeks has a SIP of 10 working days. Some Lean projects even deploy nightly.

Going to production every 2-weeks may raise fears of introducing new defects. Lets deal with that. In order to achieve short cycle time we need to eliminate defects. Doesn't that sound pretty straightforward?
Start by expecting zero defect software production.

Scientific Project Management

Henry Ford built his early automobiles using scientific management techniques. Every production step was optimized for return on investment. Cars were shipped directly from the factory to the dealer without testing them. The focus was on efficient, not quality production. As the auto industry grew factories added testing and repair stations at the end of the line to catch defects before the customers did. It was a helpful innovation. Quality assurance became focused on that final phase. When defects were found components were disassembled and fixed. The cost of this re-work was high.

Parallels can be found in how most of the software industry builds software today. Programs are either released to the customer for testing, or they are tested as the last step before release to the customer. And much of the current focus of quality assurance is on finding defects at the end. Is it reassuring that we are following the same path as the automotive industry before 1970?

Certainly if we do a good job of testing at the end, many of the defects will be caught.
Testing cars at the end does find the defects. But since a mass-production factory produces in large batches, many cars are built with the same defect before the defect is detected at the end. Consequently many cars often need to be re-worked because of the same defect. When cycle time between error, detection, & correction is long it causes repeated errors to occur before correction. Now that is expensive!

Could this be why 80% of software development costs are spent on defect removal? If we wait until the end of a month-long cycle to test, programmers will introduce similar kinds of defects across the system. Repeated defects in software occur when communication with the customer breaks down causing multiple wrong features to be built; when wrong assumptions are propagated; or when programmers don't share information among themselves and multiple programmers make the same mistake. These defects show the limitations of long cycle times.
That is the approach of Scientific Project Management.

I'll talk about how Lean Manufacturing does it in the next post ...

The Problem

$59.5 Billion is wasted every year! The U.S. gross domestic product loses 0.6% a year because of software defects. Currently, over half of all errors are not found until 'downstream' or during post-sale software use. This occurs even though vendors already spend 80% of development costs on defect removal.

The software industry is becoming increasingly interested in removing defects after the defects are created. A focus on defect removal sounds like a good idea. But what if the defects are not created in the first place? What would that be worth?�� Is it realistic to suggest that software can be built without defects� Teams using Lean Software Development are demonstrating that it can be done. To see what we mean by Lean, let's start with what is not Lean.
We'll save that for the next post ...

Language and Culture

Most J2EE development is in DP style That does not mean that you cannot do Object Oriented J2EE. But what people do with a language creates the language culture. Perhaps some people could build OO programs in a culture of DP, but it will always be fighting against the norm.

You can certainly build OO designs with java. Lot of people do. Often however, requirements are presented in terms of DP vocabulary. This makes building it in OO more difficult for the users to understand.
DP is encouraged when requirements:
* Specify the screens
* Specify the DB schema
* Specify the data fields, flags, switches, and codes
* Specify the data transformations.

Fitting that into an OO mold is another impedance mismatch.

OO requirements would
* talk in Business terms
* Name the concepts involved (Classes)
* Iterate furiously to refine understanding, and reduce concepts to a minimum
* define and place behavior (not data) into and among the classes

This requires business uses to talk in business language. However, I'm noticing that most Business users are now talking in terms of data and processing. Not in their own native business language.
Many Business users speak DP. They expect DP solutions.

So, to really do OO, the upstream expectation need to be broadened. The business users need to speak in their native language of concepts again. This is a switch in culture. And will be difficult.

The first step is for us Java folks to understand that what we have often been delivering is not OO. But at the core of most J2EE systems are data structures and processes, hence DP-style.
Much of the common literature in Java has encouraged this. The J2EE blue prints, Struts, Entity Beans, etc. These all encourage a DP approach.
So to do OO in J2EE, is often swimming against a powerful set of fundamental assumptions, and frameworks. Thats why I'm exploring the Ruby language and culture. The assumptions in Ruby are tilted heavily toward Object Orientation.

I worry that trying to do OO in J2EE will often make one a frustrated outsider.

DP style development

Data Processing style also involves the use of:
* data structures (called Beans) that have only data with gettters/setters.
* Util classes to operate on the Data Structures
* Managers, Services, and Helpers, etc, that do all the real processing.
* And global public constants that are shared by multiple algorithms scattered across the code base

J2EE as Data Processing

I claim that most Java/J2EE applications are written in a Data Processing (DP) style. Rather than an OO style. I'll also say that over 90% of the java being written is DP-oriented.
Since most development is in DP style, is Java really the best choice for building DP programs??? (Years were spent in attempts to use C++ as an everyday OO langauge.) That's why I'm trying to raise the issue of COBOL oriented J2EE.

Java is a mostly-OO language. But the culture that surrounds it is a mostly Data Processing culture. You can't (probably shouldn't) build OO apps in a DP culture. I have nothing against Data Processing. It has a cost/benefit model that may work in some situations. I'm just concerned that an OO language being used in a DP culture is a mismatch.

COBOL oriented J2EE

I gave a talk in Febuary 2005 at the local java group called
COBOL oriented J2EE

The point to the talk is not to say that "COBOL orieinted J2EE " (COJ) is good.
COJ makes development and maintenance harder and more expensive.

However, "COBOL oriented J2EE" (COJ) is the dominant paradigm.
And if we want to make our jobs easier, we need to
understand:

(1) why does COJ exist?
(2) what is Domain Design?
(3) who benefits from COJ?
(4) How can those same people benefit more from OO Domain design?
(5) What small easy steps can we take to more toward OO Domain Design? or How to escape being just a "COJ on the wheel" (couldn't resist :-)

That's a pretty long list to cover in one short presentation.
But I tryed to give some jumping off points for people who
want to go deeper in a particular area.

OO Projects in DP culture

After the presentation below Steve G. asked :
-----------------------------------
When the user (the one who is paying the bills) starts the project off with 'we know what data is available, and we've mocked up what we want the pages to look like and how they should flow, what we need is for someone to get it on the Web'
... you'll never get out of the Data and Processing scenerio. How do you typically address these?
-----------------------------------

He's absolutley right. That's why I say the the Data Processing attitude is deeply dug into
the larger culture. The OO or DP decision is usually made before anyone knows that they
are actually making a decision.

However, you can at least put together a Simple Domain Model and build with that.
The requirements above do push you into talking about data and transformations, where you implemented is still (maybe) under control of the programmer/designer.

It's best, of course, if you can refocus the requirements discussion away from data fileds and transformations and onto business problems and behavior.

That is often quite tricky given that most IT people and most business people deep in their sub-conscious can only think about programs in terms of non-OO, data processing concepts.
That's one reason I also talked about small improvments we can make like eliminating the use of un-encapsulated, public constants.
It's a small thing. But it can move us all to focusing more on Behavior and Nameing.
With any change, I encourge Baby Steps in the OO direction.

Functional Test tools

Many people are realising the need for functional testing.
The seach for these is often limited to commercial screen driven test tools.

While I don't have numbers to back it up, my experience is that the commercial screen driven test tools are nearly as expensive as manual testing. They perform very nicely in demos for management because they simply record the user's actions and replay it later.
What could be less expensive???

Most time they fall down because as soon as something changes someoneneeds to rebuild many of the scripts. This either happens over and over again, or the scripts are abandoned. Mostly, I've seen them abandoned. Plus (or better said 'minus') you have to pay for thesescreen driven tools, lots.

So, whats the alternative? Well it would require upfront work, giving lasting value. The functional testing tools I recommend are
* www.fitnesse.org
* http://wtr.rubyforge.org/

If you want your user's to take ownership of writting the tests, youcan use Fitnesse as the tool for them to enter the tests.
If you want to drive IE via the ActiveX interface in the simplestpossible scripting language I've seen, use Watir.

Either way the tests are more maintainable because they are code notsome binary, proprietary format. And you pay nothing for the tools. One of the items on my personal todo list is to write a Fitnessefixture in Watir. Unless it has already been done.

XP Cincinnati Meeting Notes

Rails - ActiveRecord

ActiveRecord is a database mapping framework that makes a bunch of assumptions about how the Database should be built and then lets you connect that up to a database in the quickest most direct way imaginable while still allowing you to drive your application from a Domain Modeling perspective. (is that sentence too long?)

• Convention over Configuration - Rails uses [XP-practice]coding standards [/XP-practice] to help speed development efforts. For instance if you name database fields according to the standard, the framework will give you extra functionallity. Or you can ignore the standard, and write more code yourself.
• Macros provide a Modeling language. ActiveRecord provides many macros to help with validations and association mapping. like: has_many, belongs_to, validates_uniqness. These really help make the Domain model expressive of your intentions. And they speak a language that clearer than other kind of mapping.
• The challange is to extend these tools to develop Domain language for your application problem space
• Simple Design verse Too Many Pieces. Afterwards we talked about the various pieces if J2EE architecture that we all use to deliver applications. So many issues occur because of the many frameworks and servers that need to fit together to deliver value. Are these peices really part of the solution, or have vendors just been selling us all stuff we don't really, actually need?
• Developing Domain Language, what if we could extend the modeling macros in ActiveRecord to help us develop our own domain specific macros? This would allow domain level rules to be expressed more directly. At this point it's just a question.
• Projects - Let one thousand flowers bloom. In the past we built a single project for xp-cincinnati. We all worked on the same code base. However, this time around we would like to code more between the meetings, too. So we talked about not one, but many projects that we could work on in smaller, non-exclusive groups. The meetings would be a forum to share our work, and brag about our accomplishments, and of course to hang-out afterwards.

---------------------------------------------------

Thanks to all who attended the April meeting, it was great. We had 4 or 5 new people attending this meeting. They said they were attending to hear more about:

• Design-Patterns - and this month we talked about the ActiveRecord pattern as implemented in Rails
• New Stuff - well that sure was new wasn't it?
• Rails - did it too
• XP - the project we reviewed was an XP tracker written in Rails.

Some of the topics we ended up talking about

• Planning and Justifing Big refactorings - Some big refactorings may need to be scheduled. (Think about replacing frameworks). It's hard to explain to the Customer so how do you get it on the schedule. The perfered approach is to try to do the refactoring incrementally. perhaps over a number of iterations or releases. Otherwise, you need to get a good level of trust with the customer to the point where they are willing to accept that most of the Velocity budget will be spent on a refactoring. Big refactorings require some combination of trust and justification and incremental changes.
• Tracking vs. Fixing defects is bug tracking good or counter-productive? Tracking defects is itself work. And work that does not move the product forward. It's best to either fix the code, or reject the defect report as not meaningful. As Yodda told Luke: "Do or Don't Do". Alex and Ed pointed out that some defects don't fit that model, and I wish I had written down Ed's exact formulation of which types of defects that was. When he tells me again I'll update this part of the post.

Why This Blog

Notes on Lean Software Development