Introducing Agile Processes into a Waterfall Organisation Chris Cooper-Bland, Senior Architect @ Endava Ltd Agenda    What is Agile – Refresher What makes Organisations Waterfall? Introducing Change     Change the process Change the Organisation What works Summary and Q&A How many of your organisations are using agile currently? How many are planning to? 2 Delivering business value is hard…  “Of the work executed: “Many (possibly most) organisations lose as much as 45% of their total revenues due to costs associated with low quality”   “Some 75 percent of most large-scale J2EE projects fail by missing both time and budget projections …”   Six Sigma Mark Driver, Gartner “64% of features actually delivered are either rarely or never used”  Jim Johnson, Standish Group 3 Brief History of Development Methodologies AGILE e.g. XP (Kent Beck) Methodologies WATERFALL (Royce) Requirements, design implementation, verification & maintenance Waterfall 1960 1970 RUP (Rational) user Incremental, driven, low process RAD Object oriented, (James Martin) iterative, time-boxed, user driven RUP Prototyping, iterative, time-boxed, user driven RAD SPIRAL MODEL (Barry Boehm) V-MODEL (Anon) Iterative Spiral Model Aligns testing to Waterfall development V-Model 1980 4 85 91 98 99 Agile Misconceptions?  Agile means: “letting the programming team do whatever they need to with no project management, and no architecture, allowing a solution to emerge, the programmers will do all the testing necessary with Unit Tests…” 5 What is Agile?  http://www.agilemanifesto.org/ We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more.  Management can tend to value the things on the right over the things on the6 left Agile project management SCRUM Daily Scrum Meeting: 15 minutes Team-Level Planning Each teams member answers 3 questions: 1) What did I do since last meeting? 2) What obstacles are in my way? 3) What will I do before next meeting? Every 24hrs Every Iteration 4-6 weeks Working Software Delivered Prioritised Iteration Requirements Scope Requirements Requirements Requirements Requirements Prioritised Requirements & Features “Backlog” Applying Agile: 7 Continuous integration; continuously monitored progress Agile - XP explained (1) The Values  Communication  Simplicity  Feedback  Courage  Respect (added in the latest version) 8 Agile - XP explained (2) 1. 2. 3. 7. Test First Programming Test First without code The Planning Game - Business Stories - Customer decides, Prog. Implements Small, Frequent Releases 8. 9. 10. - Release early and release often 4. 5. 6. Always use the Simplest design that adds business value System Metaphor - Programmers define a handful of classes and patterns that shape the core business problem and solution - Like a primitive Architecture On-site Customer - Customer has authority to define functionality - encourages face-to-face dialogue 11. 12. 9 Refactoring Restructuring code without changing its functionality - Mainly Simplification Pair Programming Collective Code Ownership Coding Standards - Everyone should use the same coding styles. Continuous Integration - At least a few times a day - All unit tests must pass prior to integration - All functional tests must pass afterwards Forty Hour Week ! - Tired programmers write poor code and make more mistakes Water Fall Organisations       Accounting system – annual accounts, monthly returns Legal and regulatory controls Shareholders – 3 year plans, annual plans Interface with other organisations SLAs Reward System, annual event not immediate 10 Random Example 11 From Northern Constabulary Approach to Change  Models to introduce change into the organisation       Incremental approach Step change Thin threads Scope of change island or wholesale Prerequisites for change Blockers & enablers - timing    Key influencers Other changes Disasters 12 What to change – Best Practices Most Useful          Collaborative working Iterative projects Visual Modelling Risk based prioritisation Requirements Management Change Management Configuration Management Tools Traceability 13 Least Useful Factors for Success  Choose the right project         Size Importance visibility Get buy-in of senior management Communicate to all Use experienced people Don’t trust blindly Common sense – no silver bullets effort = people * environment *size - Walker Royce 14 (process) Changing the process Integrating the method  Advantages     Single place to look Easy alignment People already understand parts Disadvantages    May lead to confusion External staff don’t know it Overlaps and/or gaps 15 Customising the method Standard Method specification Industry Level Industry Standards. Industry Methods development (e.g. (e.g. Iterative, Agile) ISO 12207,SW-CMM) Organisation’s Software Engineering group Project Assurance (monitoring adherence to process and feedback for Software process improvement) Macro-level tailoring Organisational Level Organisational Tailoring of Method Micro-level tailoring Project Level Project defined process documented in some form 16 Changing the organisation     People  Team rewards  Celebrate success  Leadership – management must ask the right questions  Communication, brown bags etc. Understand perceptions of success, what does finished mean?  Quality/stage gates Senior Management control  Estimating and budgeting  Real options  Portfolio planning Structure of the organisation 17 Estimating      This will prove problematic Identify an approach early, keep reviewing it Ideal model is to use a model calibrated with the actuals captured from your organisation, but …. Use model with someone else's metrics Use Industry model     CoComo II Function point analysis Guess Planning game is usually too late to help 18 The Budgeting Problem The Cone of Uncertainty Source McConnell 1998 19 Addressing it Convince management to make a fixed investment to establish costs Source McConnell 1998 20 Application Project Portfolios   Senior management must give authority and control to IT Use a portfolio management approach    Programme portfolio management Define portfolio segments Apportion available budget Revalidate at stage gates, for balance and progress Proven proof of Concepts Stage gate 3 Stage gate 2 Stage gate 1 21 Real Options – a way of thinking      Based on commodity options – right but not an obligation to buy at a point in the future Investment is continued only in favourable conditions, use probability models to predict future likelihood of return Can hedge different investments Allows management to be in control Share holders like it 22 Structure – Skunk Works     Lockhead Martin needed to develop secret projects, outside formal control Formed in June 1943 – Burbank CA 14 rules to ensure efficiency – similar to XP principles Now seen as technique for introducing change – but … 23 Structure - Radical Changes  Change the company structure of the organisation     Create new spin-off company Joint venture Acquisition XP No Change Agility required Waterfall Change the internal structure   Bureaucracy Dynamic Existing Structure New ventures department New project teams 24 Don’t do this – if you want to fail       Use integrated teams Sort out the development environment early Choose tools carefully Enterprise architecture is important for large/long lived systems One person needs to own the process vision with support from many Use partners experienced in the method 25 What Works Well   Configuration and change management, continuous integration Selling the method    Books, presentations etc. Immersion for the project Briefings for the rest of the company  Make them want it too… 26 Summary   The organisation will have to change too More you have to change the harder it is 27 Resources     Craig Larman’s books http://www.bcs-spa.org/ http://www.dsdm.org/ http://www.realoptions.com/ 28 Conclusion & Questions Contact Details Chris Cooper-Bland Senior Architect Chris.cooperbland@endava.com 29