Fact checking of an Agile development exercise

Hi folks,

A short one, for the sake of keeping my weblog updated.

Recently I have been working with a development team on various sub-projects of a bigger project. The development of some other modules was finished and the new features were being introduced in form of new sub-projects. The team was eager to adopt Agile development. So we decided to do so in one of those sub-projects in order to get them familiar with the concept and hopefully apply it to all the future development projects.
I’ll briefly elaborate the conditions in which those sub-projects have been developed and their outcomes and leave the judgment to you.

Project A:

• Scope is small.

• Requirements are captured in form detail writing; provided by the Product Management team. (Took them one week to finish). This was the usual practice in the entire company. That is, no development starts without a written spec.

• The development team goes through the document (which is at least 50 pages) to find out the areas that are not very clear and clarify them with the PM team. They also estimate the development time for the features requested in the sub-project and negotiate the features with the stakeholders. (They use a lot of buffer in their estimates to mitigate risks)
  

• The development team shares the requirement with the Q.A team and starts development and releases the developed and unit-tested code to the Q.A team whenever a major feature is developed (iteratively) until the sub-project is finished. No change request is accepted when the development is ongoing.

Project B:

• Scope is small.

• Requirements are captured in form of Sketches and Models. (It was ready in three days). The development team and Q.A team had a couple of short meetings with the Product Management team after that, to make sure they understand the models and the priority of the features. They started then designing the overall architecture with the help of Architecture team, finding the reusable components and developing the required base code while the Product Management was working on the detail of some of the high priority features. Q.A team develops test plan and required test cases. These three teams were constantly updating each other during this process.

• PM quickly evaluates the startup development process and the built prototype and come up with a list of change requests. In the meantime, Dev and Q.A teams studies the detail model of the high priority features along with the high level model of the requirements both provided by the PM team to come up with estimates for high priority features and the whole sub-project.

• Teams discuss the changes, the detail model and the estimates.

• Dev team continues development to solidify the architecture and frequently releases the developed and unit-tested code to the Q.A team to integrate and test. When all the bugs are resolved for each build, it will be released to the PM team for evaluation and assessment.

• PM can ask for change before the architecture is frozen which usually happens after the second release. In contrast, the Dev team has the opportunity to negotiate features if any of the early requested changes is beyond the scope and affect the development time or resources.

Outcome of Project A:

• Some features could not make it to the deadline and some were not accepted by the PM as they were misunderstood by the Dev team.

Outcomes of Project B:

• Estimates were ready earlier and more accurate too as they were provided based on the actual development work that the Dev team has done.

• More angles of requirements were clear to Dev and PM teams because of the presence of the Q.A team in all the discussions from the beginning. Amazingly in some cases Q.A team members knew more about requirements than even PM team and that’s because they have been testing the previous sub projects and are more familiar requirements. Besides, they always ask the best questions about requirements and I believe that’s because they think about test cases and scenarios in advance and they need to find out all the possible ways to test the application.

• PM team was very happy about the outcome of the project as all the important and high priority features that could finish in the given time and were agreed upon, were present and entirely tested. They were also happy about the fact that they could start adoption of new features between their customers earlier.

I don’t think I need to explain which project was more successful. However, there are a few things about Agile development that I like to point out here.

• Pure modeling and sketching is not always a suitable way of requirement gathering. If you have distributed teams, challenging stakeholders or very complex requirements then you might want to spend a bit more time digging and documenting.

• If you are using TDD (Test Driven Development), it doesn’t mean that you don’t need the Q.A team or downstream black box testing; no matter how much developers may write unit tests and run white box testing.

• Change adoption doesn’t mean that after every release the stakeholders can come up with change requests. Change can be accepted and handled before the Architecture get stable. So it’s the development team’s responsibility to release the code as frequent as possible in form of a presentable application to the stakeholders and the stakeholders’ responsibility to assess the released code and come up with change requests as early as possible. That means, if you have a stakeholder that has no idea what he wants (and believe me, there are such stakeholders) you need to spend more time building prototypes and exploring requirements before you stabilize the Architecture.

• The project that was chosen for the team to exercise Agile development was relatively small. That’s because it was their first time doing iterative development in Agile manner and I wanted to mitigate the risk that the failure of the project could impose (if it fails).

Don’t forget to leave comment or feedback. I know I haven’t made any update for a long time and I might’ve lost all the readers, but I hope whoever read it find it useful.

I just wanted to write a quick one about the IT Architect Regional Conference SEA which was over yesterday. I will definitely continue writing about all the questions that I got during the two days conference as I've promised.
One of the questions was that how I have become entitled to be a Senior Software Architect at this age.
Well, the quick answer would be that I was lucky enough to find out at early stages of my professional work that Architecture is very much a different profession than software development. If you become a senior developer it doesn't necessarily mean that you can get promoted to become an Architect and I didn't figure that out by myself. Today that forms one the IASA's missions; to tell everyone in the IT industry that Architecture is a profession of its own.
For me it all started by reading a lot of books and articles. This was happening when I was doing development works as well. While I was developing applications, I was thinking about various aspects of the architectural efforts behind it and comparing them with my readings; about its architecture design, about the development process in place to follow and develop the application and mitigate risks, about the chosen technology using which we had to develop that application, and the roadmap that was set in place to realize the benefits of that application.
If you are a developer and want to become a Software Architect, invest on it. As I mentioned being a senior developer doesn't make you a Software Architect.
This doesn't mean that being a Software Architect should keep you away from the details of development works. In fact one of the important things that a software architect should do in "Proper" Iterative and "Incremental" development approaches (which I believe are the right way to develop applications) is to build application's architecture in early stages of development in form of an executable application. A Software Architect can build an application only if he/she owns development expertise and is familiar with technologies.

All your questions and comments are welcomed.

W.C.I.T 2008

Hi

I'll be a delegate in W.C.I.T 2008 (World Congress on Information and Technology) which is being hold in Kuala Lumpur, Malaysia this coming May.

See you all there.

IT Archiect Regional Conference - Malaysia

Hi folks,

I'll be speaking in one of the sessions at the IT Architect Regional Conference of South East Asia in Malaysia. Here is the link to the conference home page:
http://www.iasahome.org/web/itarc/KL

The topic's title in the website is stated as "Managing Changes and Mitigating Risks in Software Development" which is not correct. The topic that I intend to talk about is "Planning Iterative Software Development Projects". I believe Planning is one of the areas that is not broadly explored in the Agile development community and that's why I chose to talk about it.

See you there.

How to deliver real value using Iterative and Incremental Development

Hello folks,

I know, I know. It's been quite a few month. I received not many emails and comments from friends, which I chose not to publish them, and all, in a nutshell, were asking "what the hack I'm doing".
Well, I haven't been doing anything but work. I also have been looking at my weblog every week and thinking what a pity it is that I don't have enough time anymore to update my weblog. Truth be told, even now I'm supposed to be doing something else. Anyway, let's turn our attention to this post.
The benefits of Agile development approaches in which developing software applications iteratively and incrementally is an important and inevitable principal are questionable if deliverables produced during the project do not introduce any business value.
If a development project doesn’t deliver business values incrementally iteration by iteration then it’s executed in a value-neutral setting in which

• requirements are treated as equally important
• code delivery is as important as the delivery of an usable system
• implementing largest number of requirements possible become more important than delivering desired outcome of the project

In another word, the development team iteratively implements the requirements rather than iteratively solve problems. Although project members feel that they are iterating successfully, they never make the transition from iteratively addressing the technical risks to iteratively delivering usable versions of the system that provide realizable business benefit.
This will raise the question that how we exactly have to develop requirements iteratively to secure delivery of the actual desired outcome of the project. To answer this question we need to define the best possible deliverable which assures the stakeholders that development is progressing toward creating a system that is aligned with their actual requirements. That deliverable is nothing but an executable that its features represent a collection of end-to-end behavioral threads of system usage. In a simpler word, the output of each iteration should be an executable that possesses three very important characteristics:

• Should cover a set of the actual “behaviors” of the system that are most important to the stakeholders at that point of time
• Should keep the confirmed “behaviors” that are implemented in the previous iterations
• All its implemented features composing desired “behaviors” should be working integrated smoothly

You might’ve noticed that I put the world behavior in the quotes. What I am trying to emphasize here is that it’s not important how many features you implement. What’s important is that the implemented features form desired behaviors of the system and do something useful for the business. Another word, only those features that are really needed have to be implemented. Statistics data such as CHAOS report, as I’ve pointed out in another article at http://www.iasahome.org/c/portal/layout?p_l_id=PUB.1008.62, show that a high percentage of implemented features in many projects are never used, even in those projects that are declared successful.
Now that we have set the ground rules and defined what it is that our projects iteratively have to deliver, let’s take a look at the ways of doing that.
To achieve the value delivery, we need to have a way to work around the end-to-end behavioral threads (independent threads of system usage) or what is called scenarios. “Credit Check” and “User Registration” are examples of scenarios. They are independent threads of system usage that tell stories about how for example a user registers and what other things might happen along the way.
The obvious question here would be “How to find the best scenarios and in what order they have to be implemented?”
As I mentioned above, scenarios are stories and finding scenarios is about the requirement gathering skills that is far beyond the scope of our discussion here. But as the principals of Iterative development state, scenarios do not have same level of priority. With that in mind, at every iteration we need to find those scenarios that have the following characteristics:

• They are addressing the most important goals of stakeholders at that point of time
• They are addressing the most pernicious risks in the project

Project risks threaten the project's ability to deliver the desired outcomes in a timely and cost-effective manner. Technical risks can be mitigated by choosing scenarios that force the confrontation of the risks. By mapping the risks to scenarios that mitigate them, iterations can be planned to ensure that every iteration reduces project risk and increases the business value realized from the solution.

Relationship between Risks, Scenarios, and Desired Outcomes

So, for example if in our earlier example of user registration, the user information needs to be registered in a LDAP database, the development team need to find risks that are mapped to this scenario, for example limited knowledge of technical team in working with LDAP databases, and rate those risk. If the level of their threat is high then they have to be implemented before other less important risks.
I must mention that addressing risks is not less important than implementing important scenarios for stakeholders. What if a scenario which imposes a big risk is not important when you are in the third iteration and when it comes close to the end of the development a change in business strategy makes it an important scenario? This is a rare case of course. But it can become a counterproductive situation in a project. A smart architect keeps his/her eyes on those scenarios and related business scenarios and as the iterations pass by he/she tries to decide whether the stakeholders’ attention is turning toward those scenarios. Some architects assign a very small team (with One or Two members) to work around some of those risks simultaneously so if their related business scenarios become more important they don’t impose a big danger.
The widest used technique for modeling scenarios is Use Case Modeling. Use cases structure and group requirements by user goals and business value. A use case contains a set of flows, which describe the interaction between the system and its actors. The set of flows provide a map of how the system can be used to meet the goals of the use case and a structured way of describing the set of end-to-end scenarios needed to drive the iteration planning and the development of the software.
A use case captures a set of scenarios, each of which is described by one or more of its flows. All use cases contain a basic flow, which describes the set of normal, "happy day" scenarios. They also contain a number of alternative flows to describe variations, extensions, and exceptions to the basic flow. These can be combined with the basic flow to describe all the other possible scenarios. So, a scenario can be considered to be defined by a use case's basic flow plus zero or more of its alternative flows.
Besides, almost all of the case tools that I have seen out there have a tool for drawing and managing use cases. It seems that it has become a unified way of defining and managing scenarios.

Summary

Nothing but an executable code depicts the actual progress of a development project. But not every executable does that unless it implements end-to-end behavioral threads that deliver most important requirements of business. This doesn’t mean that there shouldn’t be any documentation. It has to be, but as much as it’s required.


-- This post has been published as an article on the ITtoolbox web site as well.

Choosing the Appropriate Software Development Process Framework

Hi folks,

Here is another published article of mine on IASA's web site. I hope can keep it up.
This is the direct link and this is a link through IASA's content repository.

I also figured out that you can not post any comment on the IASA's forum unless you are a member. So please do it here whatever it is.

Overcoming Requirement Challenges with IID

Hi everyone

Recently I've allocated some of my time to write articles to publish on IASA's web site. I thought it might be a good idea to save some times and reuse them as my posts here. That's what we are supposed to do as architects; reusing. Isn't it?
Here is one: Overcoming Requirement Challenges with IID
If it wasn't there as IASA's repository get updated fast then use the direct link.
By the way, I haven't forgotten my promise writing about SOA and SaaS.