It is a given that the corporate landscape is changing dramatically and quickly. Organisations able to respond immediately and effectively to new demands will be the ones to survive and prosper.

It is also a given, at least among non-IT management, that IT moves slowly and cautiously, and even then, may not deliver against the business need. It's not always true, but there has been enough anecdotal evidence to give IT the reputation of failing to react positively to change.

Cloud Storage - Comparison of 30 Services

IT is instead considered the department dragging its feet, restricting lines of business such as marketing from maximising benefits and driving new revenue streams.

Enter the concept of Agile project management.

"Understanding business needs is crucial to the success of any IT function," says Patrick Eltridge, Telstra's CIO. "It sounds fundamental, but sometimes these needs are unspecified, unknown and may evolve and/or change over time. For both business units and IT, this can and has been a source of friction in delivery.

"Agile plays a key role in bringing business and IT groups together to continuously identify business needs and realise value. From my experience, Agile creates a workplace of trust with a strong business orientation."

Agile is a methodology based on producing regular updates, and promotes project changes on the fly. As a solution, it's designed to align with client expectations and work in the current environment, even if that environment doesn't look anything like what you began with.

Often, this means delivering an outcome in iterations, rather than as a finished product. Any uncertainty in this model is then offset by the speed at which these iterations are delivered.

In traditional methods, the IT team often 'disappears' for months at a time, says Eltridge, working on project phases in which the business owner may have little involvement.

"In contrast to order-giving or the 'hand-offs' approach typical in traditional workplace cultures, Agile methods draw the business and IT groups together in a more collaborative relationship," he explains. Stephen Wilkinson is a senior ICT project professional who has worked with large organisations in Queensland, and is currently the infrastructure program director with a leading government statutory authority.

He agrees a closely defined business value is vital, but points out the same value can often be achieved with differing solutions and processes. "Ultimately, the question needs to be answered in the context of, 'Is the business better served by the output of this project or not?'" he says.

Reality check

Project management has had a mixed history to date. While there are professional qualifications in the field, many projects are simply given to the 'next in line' and lack coherency and structure.

According to Wilkinson, projects differ depending on the vendor/customer equation. "While working for a large systems integrator, projects were managed well in the context of what the SI project managers' focus was; namely managing the risk to the integrator balanced with the successful outcome to make margin and encourage repeat business," he says.

"But at the end customer organisation, my experiences have been patchy with regards to successful project management."

Often, end-user organisations use project management skills either in-house or available to them without considering the specialist capabilities required to best ensure success, Wilkinson continues.

This is because many believe a good project manager should be able to manage any type of ICT job.

"My observation has been that a PM has a much greater chance of success if they understand the technology area in which their project is trying to deliver, and by virtue of that understand they should engage the best resources, identify gaps and risk, to contribute to a successful project output," Wilkinson says.

Eltridge claims Agile methodology embeds a 'product owner' from the business within a project and therefore defines a clear level of accountability. "The methodology drives the business to define capability in a way that the business value is measurable in each piece of work," he says.

"They can 'see' exactly what they are getting in real-time. It requires them to work more directly and continuously with the IT delivery teams -- taking more responsibility for design decisions -- and this drives trust in both directions."

The step-based approach also offers underlying control over financial costs. "It is easier to detect and control over-runs, and the iterative cycles and learning loops significantly de-risk overall delivery," Eltridge adds.

Staging projects

With any project, milestones are often regarded as essential, allowing management to assess whether the project is running to schedule. How effective milestones are depends on how closely they stick to original policy. With Agile, the concept of milestones is vital, but they can float as well as be achieved much more regularly.

Wilkinson says there are variations to how milestones are applied in different industries.

"While they are a way of reporting progress from a schedule and critical path perspective, I have a strong view that in ICT projects, schedules are a management tool and are dynamic with the changing requirements, resources competition, and rapidly evolving technology landscape of today.

This differs from the construction industry, where you have repeatable components with clearly understood delivery durations/timeslices that make schedule estimation and execution more predictable.

"ICT projects can rarely be componentised in the same way. As a result, an expectation of delivery against pre-determined milestones is likely to disappoint."

This is where Agile comes in. The starting point is the development of a visible and jointly prioritised list of capabilities to be delivered by the business function and IT, Eltridge says.

Continually updated, this 'common backlog' represents what is in-flight, what work will be finished next, and what the next most valuable thing is.

"This enables rapid changes in priority if business needs change," he says. "Less valuable work can be stopped quickly with little waste if something more valuable is determined.

"The dramatic improvement in early cycle times [typically delivering an initial release in a third to a half of the usual time] has a profound effect on the way the teams collaborate, and as they adjust to the immediacy of faster feedback loops."

Cloud solutions can aid in this process by offering faster provision timeframes and cheaper environment costs that facilitate Agile development and delivery, Wilkinson says.

"It is easier to run multiple development and delivery tracks in an 'as a service' infrastructure model that then lends support to Agile methods."

Roadblocks

That's not to say everything is easy and straightforward with Agile. Last year, Eltridge told CIO that "going Agile" marked a significant business shift for Telstra.

The telco "hasn't always been the easiest company in the planet to work with," he admitted. As a result, he sought to overhaul IT delivery that had been slow, expensive and unreliable. Culturally, Eltridge is also promoting greater collaboration, transparency and risk-taking.

Allowing staff to take risks is important because even if they don't succeed, the experience changes them for the better, he claims. "If you're not prepared to set somebody up for the possibility of failure, you're almost entirely unprepared for the possibility of success."

While people want to reap the potentials of working in a more 'Agile' manner, Wilkinson says many organisations struggle culturally to make the mindshift required to make best use of Agile principles. "The overriding negative perception is the fear of change, and secondly the lack of comfort with changing requirements," he says.

"But to be successful, any application of Agile absolutely needs to be disruptive as the pervading culture, historical approaches and methodologies are not consistent with Agile principles."

Such disruption should not be seen as a negative; organisations have to accept that solutions may not go live in a perfect state, and teams from many areas of the organisation need to work together, Wilkinson adds. That requires a focus on simplicity and time to market, co-location and face-to-face working.

Above all, requirements must be dynamic. "Requirements change and to expect them not to is likely to disappoint and ultimately leads to failure," Wilkinson says.

"Without cultural change, adopting Agile will not succeed. Organisations need to move away from having 'a perfect operation but a dead patient' to an 'alive and kicking patient who may need another round of surgery'."

Beneficial change

If cultural challenges can be overcome, Agile methodologies do provide a light both at the end of the tunnel, as well as along the way. Eltridge cites a comment from a business partner: "I can't believe I fought against this change. I wish we'd done it years ago. Having worked this way now, I can't imagine ever going back."

"The key benefit is delivering useful solutions in a shorter timeframe," Wilkinson says. "Delivering business value in quicker iterations, shortening time to market and above all adapting to changing requirements are the driving factors that realise benefits."

For the CIO, Agile methods provide more transparency into the business benefits delivered and the performance of the individuals and teams, Eltridge says.

"The demand-management practices, and the principle of constraining work in progress, deliver both business and IT with a predictable pace, easier workforce management, and sustainable throughput without the waste in a start/stop workflow of traditional big projects," he says.

"A development team can be kept continuously productive rather than stood down when the phase needs more funding or approval to proceed.

"Both business and IT find themselves learning a more effective way of working, and more often than not, they find that they enjoy it."

For Agile and similar project management methodologies to work, the key is getting all parties to pull together. There should be cooperation, consultation and communication, rather than competition and conflict.

As Wilkinson puts it, "There can be no 'my end of the boat is not sinking' in a successful Agile team."