Agile Adoption: How to stuff it up
ThoughtWorks ran a thoroughly enjoyable and rather informative pair of sessions cheekily titled Agile Adoption: How to stuff it up in Perth on Tuesday 30 June 2009. My hearty thanks go out to Karin Verloop for the invitation to attend the Quarterly Technology Briefing over a light breakfast at The Parmelia Hilton, and the Executive Round table Luncheon at The Globe.
Martin Fowler, Chief Scientist at ThoughtWorks and one of the original signatories of the Agile Manifesto, was unable to attend but Andy Marks, who specialises in enterprise-level adoption of Lean and Agile concepts, ably led the earlier discussion and along with Keith Dodds, Director of Sales & Marketing, facilitated discussion over the round table luncheon.
The Quarterly Technology Briefing was introduced by Karin Verloop and handed off to Andy Marks, who said he has been looking after the largest agile adoption in Suncorp. The Agile Manifesto states the common ground of lightweight approaches, eg. Scrum and XP. The notable software craftsmen who drafted the manifesto in 2001 at the Utah meeting decided to call it 'Agile' instead of the other possibilities mooted because they felt the word closely reflected their approach.
Andy credited Mike Cohen with the common adoption patterns for Agile:
Here's a basic recipe for stuffing it up:
As opposed to projects reporting green, green, green,... then red one month before delivery - 6 months late; instead early indication, eg. green, green, amber, red... needs a cultural change.
Standup meeting - what I did yesterday, what I am doing today, any blockers. Co-located team, customer embedded, tolerance for failure.
The inspirational ravine of effectiveness-versus-time has a dip as a team moves through learning, change, transition. It is guaranteed to kill adoption if you stomp on people in the ravine.
Outliers by Malcolm Gladwell speaks of the conditions to achieve high level performance - his studies show 10,000h to achieve high-level aptitude, or about 5y at 8h/day. (Note that a journey man plumber, not master, is achieved after 5y.)
The best architectures emerge from Agile design. Don't over-invest in design ('big upfront design'); however the pendulum may swing too far the other way.
Customer focus is important because the impact of technical debt is that the entire organisation, says Kent Beck, can be brought to a standstill under the debt load of unconsolidated implementation. Debt load is lack of technical quality, i.e. refactoring, patterns.
The next step:
In particular, the role of project managers and the governance structures provided by approaches like Prince2, PMBOK and CMMI go largely unremarked, except for a lingering negativity towards such approaches.
[Chatham House Rule applies to the following recollections.]
Surprisingly to me, the organisation with the highest statutory compliance obligation and hence a satisfactory management system was the least inclined towards a traditional project management approach.
To some extent I think their position is a misunderstanding about the purpose of documentation and the intent of the expression in the Agile Manifesto to value, "Working software over comprehensive documentation." The purpose of documentation is knowledge capture and sharing, and they should be so lucky to have it!
The issue of testing in an Agile environment was raised. Like most things Agile, even formal testing using the traditional techniques fits the description. Test specifications can almost entirely replace user and functional requirements because that is precisely what is evaluated during user acceptance, functional and system testing.
Inadequate testing will deliver a fragile product into production every time. Worse still, it will be very difficult to run regression tests, both manual and automated, in order to identify new defects.
Requirements management was the crux of another question: Are changes to requirements assessed and approved by the business first? This is a tricky one because in an Agile approach, a customer representative is supposed to be available and one of the problems is that often they are not empowered to make decisions, or lack the specialist knowledge of a subject matter expert.
Of course, this is the reason for functional requirements but I agree that a user story, written from scratch or enveloping one or few cohesive requirements, can be given to a developer in an Agile team to take 1-2 days.
The Agile business analyst has a similar role to the traditional business analyst and that is to interpret the customer requirements and to act as the customer proxy to the development team. It is not so Agile to interpose someone between the customer and the developer but for complex requirements this is fairly necessary. It was also noted that user stories could be expanded into proper use cases.
Business rules came up for discussion as well and how they can be captured: in user stories? For simple or few rules this may be fine but otherwise it is begging for disaster to scatter business rules across informal user stories and throughout the code base. A little bit of abstraction may be called for using design patterns, eg. policy or delegate, or use a full-blown business rules engine - an architectural decision. The cost of rework can be high unless this decision is made upfront.
Agile in the context of enterprise software development was discussed, involving the integration of legacy and commercial-off-the-shelf (COTS) software and systems. Interface contracts between systems, common messaging formats, architecture of enterprise service bus (ESB) are design elements usually associated with big upfront design (BUFD) but are the only things that matter when developing distributed systems.
It was noted that stubs and mock objects enable the creation of unit tests before launching into full-on integration testing.
Does Agile apply to COTS only development? Yes was the answer, mainly to do with requirements gathering - well, maybe but I'm not sure. The business requirements and scope should be the same regardless of software approach because a project can only be initiated once a problem or gap has been identified by the business.
(The detailed functional requirements, use cases or user stories, in general, are allowed to be a moving target - part of the reason is that many 'discovered requirements' are unknown at the outset.)
It makes a lot of sense to write the evaluation criteria as a test specification so that each function can be ticked-off and compared during the evaluation phase. Then, in place of a traditional software construction there will be several iterations of 'construction by configuration' where the user stories are met at each closure.
Hire these guys if you want to learn to do software development well. The people at ThoughtWorks really know what they are doing and if the marketing hype about Agile gets you down just ignore it.
The values behind the Agile Manifesto are the key to their approach which I would happily classify as Agile-Lean because unnecessary processes and requirements are discarded by applying common sense to software development.
Martin Fowler, Chief Scientist at ThoughtWorks and one of the original signatories of the Agile Manifesto, was unable to attend but Andy Marks, who specialises in enterprise-level adoption of Lean and Agile concepts, ably led the earlier discussion and along with Keith Dodds, Director of Sales & Marketing, facilitated discussion over the round table luncheon.
The Quarterly Technology Briefing was introduced by Karin Verloop and handed off to Andy Marks, who said he has been looking after the largest agile adoption in Suncorp. The Agile Manifesto states the common ground of lightweight approaches, eg. Scrum and XP. The notable software craftsmen who drafted the manifesto in 2001 at the Utah meeting decided to call it 'Agile' instead of the other possibilities mooted because they felt the word closely reflected their approach.
"It is not necessary to change. Survival is optional." said W. Edward Deming, the original thought leader in continuous quality improvement.There are about six root cause issues to cover in this session. ASWEC 2009 on the Gold Coast recognised that "Agile is the new mainstream."
Andy credited Mike Cohen with the common adoption patterns for Agile:
- Scale - from start small to all-in.
- Interest focus - from teaching practices to iterations.
- Visibility - from stealth mode to public display.
Here's a basic recipe for stuffing it up:
- Underestimate the change.
- Assume minimal cultural impact.
- Undervalue experience.
- Learning finishes at school.
- Let others fail first.
- De-emphasise technical design.
As opposed to projects reporting green, green, green,... then red one month before delivery - 6 months late; instead early indication, eg. green, green, amber, red... needs a cultural change.
"He who rejects change is the architect of decay," Harold Wilson.
On organisational culture: "It is very evident that all culture has developed as an initial reaction against adaptation, " Beatrice Winkle. [Can someone help me with this literary allusion?]IT is tailor made to embrace change, especially software developers who can be [ahem] obsessive compulsive (OCD) and show signs of attention deficit disorder (ADD). Making new connections, changing existing connections, 'lusting after change' - you want these people in you team with Agile adoption.
Standup meeting - what I did yesterday, what I am doing today, any blockers. Co-located team, customer embedded, tolerance for failure.
"Failure is the opportunity to begin again more intelligently," Moshe Arens.Often rewarded publicly for 'success' - Herculean efforts, 'by hell or high water'; other projects ('failures?') which have taken risks, lessons are not advertised.
The inspirational ravine of effectiveness-versus-time has a dip as a team moves through learning, change, transition. It is guaranteed to kill adoption if you stomp on people in the ravine.
Outliers by Malcolm Gladwell speaks of the conditions to achieve high level performance - his studies show 10,000h to achieve high-level aptitude, or about 5y at 8h/day. (Note that a journey man plumber, not master, is achieved after 5y.)
"Experience is the name every one gives to their mistakes, " Oscar Wilde.It is a mistake to de-emphasise technical design. The Manifesto implies paying continuous attention to technical excellence and good design enhances agility. [I myself speak of a culture of quality and merciless refactoring.]
The best architectures emerge from Agile design. Don't over-invest in design ('big upfront design'); however the pendulum may swing too far the other way.
Customer focus is important because the impact of technical debt is that the entire organisation, says Kent Beck, can be brought to a standstill under the debt load of unconsolidated implementation. Debt load is lack of technical quality, i.e. refactoring, patterns.
The next step:
- Conduct readiness review.
- Get experience.
- Plan in Agile way.
- Identify a quick win.
- Budget to include learning.
- Publicly reward effort to improve and acknowledge lessons learned.
In particular, the role of project managers and the governance structures provided by approaches like Prince2, PMBOK and CMMI go largely unremarked, except for a lingering negativity towards such approaches.
[Chatham House Rule applies to the following recollections.]
Surprisingly to me, the organisation with the highest statutory compliance obligation and hence a satisfactory management system was the least inclined towards a traditional project management approach.
To some extent I think their position is a misunderstanding about the purpose of documentation and the intent of the expression in the Agile Manifesto to value, "Working software over comprehensive documentation." The purpose of documentation is knowledge capture and sharing, and they should be so lucky to have it!
The issue of testing in an Agile environment was raised. Like most things Agile, even formal testing using the traditional techniques fits the description. Test specifications can almost entirely replace user and functional requirements because that is precisely what is evaluated during user acceptance, functional and system testing.
Inadequate testing will deliver a fragile product into production every time. Worse still, it will be very difficult to run regression tests, both manual and automated, in order to identify new defects.
Requirements management was the crux of another question: Are changes to requirements assessed and approved by the business first? This is a tricky one because in an Agile approach, a customer representative is supposed to be available and one of the problems is that often they are not empowered to make decisions, or lack the specialist knowledge of a subject matter expert.
Of course, this is the reason for functional requirements but I agree that a user story, written from scratch or enveloping one or few cohesive requirements, can be given to a developer in an Agile team to take 1-2 days.
The Agile business analyst has a similar role to the traditional business analyst and that is to interpret the customer requirements and to act as the customer proxy to the development team. It is not so Agile to interpose someone between the customer and the developer but for complex requirements this is fairly necessary. It was also noted that user stories could be expanded into proper use cases.
Business rules came up for discussion as well and how they can be captured: in user stories? For simple or few rules this may be fine but otherwise it is begging for disaster to scatter business rules across informal user stories and throughout the code base. A little bit of abstraction may be called for using design patterns, eg. policy or delegate, or use a full-blown business rules engine - an architectural decision. The cost of rework can be high unless this decision is made upfront.
Agile in the context of enterprise software development was discussed, involving the integration of legacy and commercial-off-the-shelf (COTS) software and systems. Interface contracts between systems, common messaging formats, architecture of enterprise service bus (ESB) are design elements usually associated with big upfront design (BUFD) but are the only things that matter when developing distributed systems.
It was noted that stubs and mock objects enable the creation of unit tests before launching into full-on integration testing.
Does Agile apply to COTS only development? Yes was the answer, mainly to do with requirements gathering - well, maybe but I'm not sure. The business requirements and scope should be the same regardless of software approach because a project can only be initiated once a problem or gap has been identified by the business.
(The detailed functional requirements, use cases or user stories, in general, are allowed to be a moving target - part of the reason is that many 'discovered requirements' are unknown at the outset.)
It makes a lot of sense to write the evaluation criteria as a test specification so that each function can be ticked-off and compared during the evaluation phase. Then, in place of a traditional software construction there will be several iterations of 'construction by configuration' where the user stories are met at each closure.
Hire these guys if you want to learn to do software development well. The people at ThoughtWorks really know what they are doing and if the marketing hype about Agile gets you down just ignore it.
The values behind the Agile Manifesto are the key to their approach which I would happily classify as Agile-Lean because unnecessary processes and requirements are discarded by applying common sense to software development.
posted by Daniel Berinson at 9:53 AM
1 comments
