Software process models and agile methods notes — Unit 1

Software is more than just code. It includes the executable programs, associated documentation, and configuration data needed to make these programs operate correctly.

⇒1.1 What is Software Engineering?

Software Engineering is an engineering discipline that is concerned with all aspects of software production from the early stages of system specification through to maintaining the system after it has gone into use.

• Engineering discipline: Using appropriate theories and methods to solve problems, while considering organizational and financial constraints.
• All aspects of software production: Not just coding, but also project management, tool development, testing, and theory formulation.

⇒1.2 Software Crisis

In the late 1960s, hardware capabilities rapidly outpaced software development techniques, leading to the Software Crisis.

• Projects consistently ran over budget and over time.
• Software was of low quality and difficult to maintain.
• Software failed to meet user requirements.
• This crisis led to the birth of 'Software Engineering' as a formal discipline in 1968 at a NATO conference.

Good software must meet the needs of its users while remaining functional and sustainable for the developers.

⇒2.1 Essential Attributes

• Maintainability: Software must evolve to meet changing needs. This is critical because change is inevitable in business environments.
• Dependability & Security: Includes reliability, security, and safety. Dependable software should not cause physical or economic damage in the event of system failure, and malicious users shouldn't be able to access or damage the system.
• Efficiency: Software should not make wasteful use of system resources such as memory and processor cycles. Includes responsiveness and processing time.
• Acceptability: Software must be acceptable to the type of users for which it is designed. This means it must be understandable, usable, and compatible with other systems.

⇒2.2 Software Myths

• Management Myth: 'We already have a book full of standards and procedures for building software. That's all we need.' (Reality: Standards are often outdated and ignored).
• Customer Myth: 'A general statement of objectives is enough to begin writing programs—we can fill in the details later.' (Reality: Poorly defined requirements cause chaos).
• Developer Myth: 'Once we write the program and get it to work, our job is done.' (Reality: Between 60% to 80% of effort is spent after software is delivered to the customer).

A Software Process is a structured set of activities required to develop a software system.

⇒3.1 Fundamental Activities

Regardless of the specific model used, all software processes involve four fundamental activities:

• Software Specification (Requirements): Defining what the system should do and its operational constraints.
• Software Design & Implementation (Development): Defining the organization of the system and implementing the system (coding).
• Software Validation (Testing): Checking that the software does what the customer actually wants.
• Software Evolution (Maintenance): Changing the software in response to changing customer needs.

⇒3.2 Process Models

A Software Process Model is a simplified representation of a software process, presented from a specific perspective. It provides a framework for managing software development.

The Waterfall Model is the classic, original software life-cycle model. It takes a systematic, sequential approach to software development.

⇒4.1 Phases of Waterfall

• Requirements Analysis & Definition: The system's services, constraints, and goals are established by consultation with users.
• System & Software Design: Establishes an overall system architecture and describes fundamental software abstractions.
• Implementation & Unit Testing: The software design is realized as a set of programs. Individual units are tested.
• Integration & System Testing: Individual programs are integrated and tested as a complete system to ensure requirements are met.
• Operation & Maintenance: The system is installed, and errors discovered in practical use are corrected.

⇒4.2 Pros and Cons

To overcome the rigid nature of the Waterfall model, iterative and incremental approaches were developed.

⇒5.1 Incremental Development

The system is broken down into several smaller components (increments). The core features are developed in the first increment, and subsequent increments add new features.

• Advantage: Customers get useful functionality early without waiting for the entire system.
• Advantage: Early increments act as prototypes to help elicit requirements for later increments.
• Disadvantage: Requires strong architectural design so increments can integrate smoothly.

⇒5.2 Iterative Development

The system is developed in repeated cycles (iterations). In each iteration, a working version of the software is produced and evaluated, and improvements are made in the next cycle.

Most modern processes (like Agile) combine both incremental (building piece by piece) and iterative (refining pieces repeatedly) approaches.

Proposed by Barry Boehm, the Spiral Model is a risk-driven process model. It represents the software process as a spiral rather than a sequence of activities.

⇒6.1 The Four Sectors

Each loop in the spiral represents a phase of the software process and consists of four sectors:

• Objective Setting: Specific objectives for that phase are identified. Constraints and alternatives are formulated.
• Risk Assessment and Reduction: Risks are assessed, and activities are put in place to reduce key risks (e.g., building a prototype).
• Development and Validation: A development model is chosen for the system based on the evaluated risks. Code is written and tested.
• Planning: The project is reviewed, and the next phase of the spiral is planned.

⇒6.2 Key Benefit: Risk Management

The Spiral model explicitly addresses project risk (such as using an untested technology or highly volatile requirements). By building prototypes and evaluating risks early in each loop, disastrous failures are avoided. It is typically used for large, expensive, and complicated projects.

A prototype is an initial version of a system used to demonstrate concepts and try out design options. The Prototyping Model is used when requirements are unclear.

⇒7.1 Types of Prototypes

• Throw-away Prototyping: A quick-and-dirty prototype is built to understand requirements. Once requirements are clear, the prototype is discarded, and the real system is built using formal engineering practices.
• Evolutionary Prototyping: The prototype is continually refined and augmented until it eventually becomes the final working system.

⇒7.2 Advantages and Disadvantages

RAD is an incremental software development process model that emphasizes an extremely short development cycle (typically 60 to 90 days).

⇒8.1 Key Features of RAD

• Component-Based Construction: Extensive reuse of existing program components or use of off-the-shelf software elements.
• Time-boxing: Strict time limits are placed on development phases. If a feature falls behind, it is dropped from the current release rather than delaying the schedule.
• Heavy Tooling: Relies heavily on visual development tools, code generators, and specialized IDEs.

⇒8.2 Drawbacks of RAD

While extremely fast, RAD requires highly committed developers and customers. If a system cannot be properly modularized, building the components needed for RAD is impossible. It also struggles with systems requiring high performance tuning, as auto-generated code is rarely optimal.

The V-Model is an extension of the Waterfall model. Instead of moving down in a linear way, the process steps are bent upwards after the coding phase, to form the typical V shape. It emphasizes that testing must be heavily integrated with development.

⇒9.1 Parallel Testing Phases

For every phase in the development cycle, there is a corresponding testing phase.

• Requirements Analysis $\rightarrow$ Acceptance Testing: Test plans are created based on requirements.
• System Design $\rightarrow$ System Testing: Test plans are created based on system architecture.
• Architecture Design $\rightarrow$ Integration Testing: Test plans are created based on module interfaces.
• Module Design $\rightarrow$ Unit Testing: Test plans are created for individual functions/classes.

⇒9.2 Verification vs Validation

• Verification: 'Are we building the product right?' (Checking that software meets its technical specifications).
• Validation: 'Are we building the right product?' (Checking that the software meets the customer's actual business needs).

Traditional plan-driven models (like Waterfall) involve a massive amount of documentation and rigid planning. When business needs change rapidly, these models fail. Agile was born to prioritize adaptability.

⇒10.1 The Agile Manifesto (2001)

Agile practitioners value:

• Individuals and interactions over processes and tools.
• Working software over comprehensive documentation.
• Customer collaboration over contract negotiation.
• Responding to change over following a plan.

⇒10.2 Core Agile Principles

The highest priority is early and continuous delivery of valuable software. Late changes to requirements are welcomed. Business people and developers must work together daily. Progress is measured primarily by working software.

Scrum is the most popular Agile framework. It provides a lightweight structure for teams to manage complex product development iteratively.

⇒11.1 Scrum Roles

• Product Owner: Represents the business/customer. Responsible for maximizing the value of the product and managing the Product Backlog.
• Scrum Master: A servant-leader who ensures the team understands and follows Scrum theory, practices, and rules. They remove impediments block the team.
• Development Team: Cross-functional, self-organizing professionals who do the actual work of delivering a potentially releasable Increment of software.

⇒11.2 Sprints

The heart of Scrum is a Sprint, a time-box of one month or less (usually 2 weeks) during which a 'Done', usable, and potentially releasable product Increment is created. Once a Sprint begins, its duration is fixed and the goal cannot be altered.

⇒12.1 Scrum Artifacts

• Product Backlog: An ordered, evolving list of everything that is known to be needed in the product. Owned by the Product Owner.
• Sprint Backlog: The set of Product Backlog items selected for the current Sprint, plus a plan for delivering the product Increment.
• Increment: The sum of all the Product Backlog items completed during a Sprint and the value of the increments of all previous Sprints.

⇒12.2 Scrum Ceremonies (Events)

• Sprint Planning: The entire team collaborates to decide what can be delivered in the upcoming Sprint and how that work will be achieved.
• Daily Scrum (Stand-up): A 15-minute daily meeting for the Dev Team to synchronize activities and create a plan for the next 24 hours. (What did I do? What will I do? Are there blockers?)
• Sprint Review: Held at the end of the Sprint to inspect the Increment and adapt the Product Backlog if needed. Stakeholders are invited to give feedback.
• Sprint Retrospective: An internal meeting for the Scrum Team to inspect itself and create a plan for improvements to be enacted during the next Sprint.

Extreme Programming (XP) is a highly disciplined Agile methodology focused heavily on software engineering practices and coding quality, whereas Scrum focuses more on project management.

⇒13.1 Core XP Practices

• Pair Programming: All production code is written with two programmers at one machine. One drives (types), the other navigates (reviews dynamically).
• Test-Driven Development (TDD): Automated unit tests are written before the production code is written.
• Continuous Integration (CI): Code is integrated into the mainline and tested multiple times a day.
• Refactoring: Continuous improvement of the design of the code without changing its external behavior.
• Small Releases: Pushing working software to production very frequently (e.g., weekly or daily).
• On-Site Customer: A real customer is heavily involved and sits with the team to answer questions in real-time.

⇒14.1 Kanban

Originating from Toyota's manufacturing processes, Kanban is a visual system for managing work as it moves through a process.

• Visualize the Workflow: Using a Kanban board with columns like 'To Do', 'In Progress', 'Testing', and 'Done'.
• Limit Work In Progress (WIP): A strict limit is placed on the number of items that can be in any specific column at one time. This prevents bottlenecks and context-switching overhead.
• Unlike Scrum, Kanban does not have fixed-length Sprints. Work flows continuously.

⇒14.2 Lean Software Development

Based on Lean Manufacturing principles:

• Eliminate Waste: Remove anything that doesn't add value to the customer (e.g., unnecessary meetings, useless documentation).
• Build Quality In: Use automation and pair programming to prevent defects, rather than finding them later.
• Deliver Fast: The faster you deliver, the faster you get feedback.
• Optimize the Whole: Look at the entire value stream, not just individual developer performance.

CBSE is an approach to software development that relies on the reuse of entities called 'software components'. It shifts the focus from programming software to composing software systems.

⇒15.1 What is a Component?

A component is an independent, deployable implementation of some well-defined functionality. It interacts with the rest of the system strictly through formally defined interfaces.

⇒15.2 The CBSE Process

• Component Specification: Elicit requirements.
• Component Discovery and Selection: Search catalogs (e.g., npm, Maven) for existing components that meet the requirements.
• Component Adaptation: Modify or wrap the component so it integrates with the local system architecture.
• System Assembly: Wire the components together.

⇒15.3 Benefits and Challenges

Benefits: Drastically reduces development time and cost. Increases reliability if using mature, battle-tested components.
Challenges: Component trust (security vulnerabilities in third-party libraries), evolution (the component author updates it in a way that breaks your system).

CMMI is a process improvement framework developed by the Software Engineering Institute (SEI) at Carnegie Mellon. It helps organizations measure the maturity of their software development processes.

⇒16.1 The 5 Maturity Levels

• Level 1: Initial (Ad hoc/Chaotic): Processes are unpredictable, poorly controlled, and reactive. Success depends entirely on individual heroic efforts.
• Level 2: Managed: Basic project management processes are established to track cost, schedule, and functionality. Success is repeatable on similar projects.
• Level 3: Defined: Processes are documented, standardized, and integrated into a standard software process for the entire organization.
• Level 4: Quantitatively Managed: Detailed metrics are collected on process performance and software quality. Everything is measured and statistically controlled.
• Level 5: Optimizing: Continuous process improvement is enabled by quantitative feedback and piloting innovative ideas.

Choosing between an Agile approach and a Plan-Driven (Waterfall-style) approach is one of the most critical decisions an organization makes.

⇒17.1 Comparison

⇒17.2 Hybrid Models

In reality, pure Agile or pure Waterfall rarely exist in large enterprises. Companies use Water-Scrum-Fall: Upfront requirements and architectural planning (Waterfall), execution in 2-week Sprints (Scrum), and rigid, long-term testing and deployment phases (Waterfall).

Created by Rational Software (now IBM), RUP is an adaptable process framework intended to be tailored by the development organizations. It uses UML extensively and is deeply tied to Object-Oriented analysis.

⇒18.1 The Four Phases of RUP

• Inception: Establish the business case for the system and define the project scope. Produce a vision document.
• Elaboration: Develop an understanding of the problem domain and establish a solid architectural framework. Identify major risks.
• Construction: System design, programming, and testing. Parts of the system are developed in parallel and integrated.
• Transition: Transfer the system from development into the hands of the end-users (deployment, training, beta testing).

⇒18.2 Two-Dimensional Model

RUP is unique because it represents the process in two dimensions: Time (the phases above) and Process Workflows (Business Modeling, Requirements, Analysis & Design, Implementation, Test, Deployment). For example, testing happens in all phases, but peaks during Construction.

Cleanroom Software Engineering is a highly formal, mathematically-driven process designed to produce software with a certified level of reliability. The name comes from hardware cleanrooms, where the focus is on defect prevention rather than defect removal.

⇒19.1 Core Philosophies

• Formal Specification: The system is specified using formal mathematical models (like state machines or set theory) rather than natural language.
• Incremental Development: Software is built in small, manageable increments.
• Structured Programming: Strict adherence to structured control constructs to make the code mathematically provable.
• Statistical Quality Control: Testing is not done by the developers. An independent testing team tests the software against the expected operational profile to statistically certify its reliability.

⇒19.2 No Unit Testing by Developers

Remarkably, in pure Cleanroom engineering, developers are often not allowed to compile or execute their code to test it. They must mathematically prove its correctness through formal peer reviews. The code is only compiled and run by the independent certification team.

Software engineers have significant opportunities to do good or cause harm, to enable others to do good or cause harm, or to influence others to do good or cause harm. Professional bodies like the ACM and IEEE have established codes of ethics.

⇒20.1 Core Ethical Principles

• Public Interest: Software engineers shall act consistently with the public interest (safety, health, and welfare).
• Client and Employer: Act in a manner that is in the best interests of the client and employer, consistent with the public interest.
• Product: Ensure that products and related modifications meet the highest professional standards possible.
• Judgment: Maintain integrity and independence in professional judgment.
• Management: Engineering managers must promote an ethical approach to the management of software development.
• Colleagues: Be fair to and supportive of colleagues.

⇒20.2 Whistleblowing

If an engineer discovers that their employer's software poses a severe risk to the public (e.g., a fatal bug in medical software), and management refuses to fix it, the engineer faces an ethical dilemma. Whistleblowing (reporting the issue externally) is considered an ethical obligation under extreme circumstances, though it carries heavy personal risk.