| The User-Developer Convergence: Innovation and Software Systems Development in the Apache Project | ||
|---|---|---|
| Prev | Chapter 2. Software systems development | Next |
What does it mean to apply tried and tested principles of engineering to software systems development? The engineering terminology applied to software systems development is used with different connotations depending on writer and context. The differences are often sublime, and may lead to a confusion of terminology. The Oxford Concise Dictionary defines engineering as "the application of mathematical and scientific principles to practical ends, as in the design construction, and operation of economical and efficient structures, equipment, and systems" [DICT](Allen 1991, p. 388). Traditionally, the engineering disciplines divide work into distinct phases, each phase based upon the results of the preceding phases. Plans are made before building, and planning is often based on a preceding analysis of the task at hand. The charts and diagrams drawn during planning is used in construction.
Three terms are central to software engineering: process model, method and notation/technique. I use the term process model to indicate an overview of the development effort's life-cycle. I will sometimes refer to the process model as simply the model, at other times as the process model. The nuance between the two will be further explored later on. The process model is a description of the development effort's flow of work, which phases it is to be split into, and the activities performed in each phase. The process model describes the entire life-cycle of the development effort, from the inception of the idea to the actual realization of the end product. It encompasses the methods used to analyze, design, and test computer software, the management techniques associated with the control and monitoring of software projects. The process model prescribes what activities is to be done and when in the development project's life cycle to undertake the activity. How to perform the activities is the methods' domain. Object-oriented analysis and design is a method. The Unified Modeling Language is a notation, or technique, that may be used with OOAD. Booch is another technique.
The basic idea of any software engineering process is to first analyse the software requirements. Requirements are often described in natural language. The specifications are then codified into a design. Writes Martin Fowler: "The analogy is [that] design is an engineering drawing" [FOWLER1999](1999, p. 15). The design is often split in two: the top-level design and the detailed design. Top-level design is sometimes called the software architecture. Booch, Jacobson and Rumbaugh calls the architecture design a common vision for the development effort [BOOCH1999](1999). It usually describes the general mechanics of the software system. The detailed design is more of a work plan to base the coding on. Coding, that is building the software system upon the plans laid down earlier in the development effort, is incidentally most commonly disregarded by software engineering.
To give a better view on software engineering, let's have a closer look at two process models, Jackson's software development method and the Unified Software Development Process.
British computer scientists Michael Jackson and John Cameron launched Jackson System Development (JSD) in 1983 with the book Software Design [JACKSON1983]. The method is a continuation of Jackson's earlier work Jackson Structured Programming [JACKSON1975] (Jackson 1975). JSD is a life-cycle process, meaning that it aims at covering the entire development process from mapping the part of the real world which is taken into account when developing the software to implementing the software system. JSD uses a precise notation for specification and implementation. The transition to implementation is not just a detailed re-statement of the specification, but rather the result of applying transformations to the initial specification.
By identifying and linking processes, the software system's architecture is laid down. Processes are linked in one of two ways: by asynchronous data stream connections, or by state vectors. A state vector allows one process to see the internal state of another process without the need to communicate with it. The architecture models "the reality with which the [software] system is concerned, the reality which furnishes its [the software system's] subject matter" [JACKSON1983](Jackson 1983). The goal of modeling is to formalize the relations between real-life processes and processes in the software system.
The focus of design in JSD is the data flow. A good design is one that incorporates this natural flow. Processes themselves are categorized naturally into a few distinct types, an a top-down design approach that targets these types will result in a design that is easy to realize.
Having obtained the design, its implementation must be accomplished in a systemic manner. With a message-based concurrency model within the implementation language this is much easier, as design processes and buffered data flows can all be coded as program processes. Unfortunately, this can lead to a proliferation of processes and a very inefficient implementation.
Most programming languages are not amendable to JSD's transformation techniques. The approach advocated within JSD is a transformation known as inversion. In this, a design process is replaced by a procedure with a single scheduler process controlling the execution of a collection of these procedures; that is, rather than a pipeline of five processes, the scheduler would call five procedures each time a data item appeared (if the five processes were identical then obviously there would only be one procedure, which would be called five times).
JSD is a variation on the waterfall model. Jackson details six phases in developing software, each building on the output of the previous step(s). The first two steps—entity action step and entity structure step—are aimed at capturing the software requirements. Top-level design is the result of the third step, the initial model step. Two aspects of detailed design—user functions and timing—is developed during steps four and five. In the sixth and final step are the plans developed used to build the software. This step is called the implementation step. Once the software is implemented, it is ready to be delivered.
Object-orientation is founded on Kristen Nygaard and Ole Dahl's work on the Simula programming language in the early to late 1960s [HOLMEVIK1995] (Holmevik 1995). As such object orientation has its roots in the programming side of software systems development. During the early 1990s, the object-orientation community saw something which in posterity is dubbed the method wars. By the mid-1990s there were numerous competing object-oriented approaches to software systems development, with no indication of standardizing notation nor techniques. Efforts were made by the independent standardization organization, the Open Management Group, an invitation that was promptly refused by all camps. In an unexpected turn of events, three of the most prominent camps joined up to produce the Unified Software Software Development Process. Declaring victory in the methods war, they started standardizing the way object-oriented software was to be developed [FOWLER1999](Fowler 1999).
The Unified Software Development Process, the unified process for short, is a software engineering method for developing object-oriented software. "A process defines who is doing what when and how to reach that goal" [BOOCH1999](Booch et al. 1999, p. xviii). This is the Unified Software Development Process' creators' goal with their process model. Their process is made up of a number of work flows. A work flow is a sequence of activities that produces a result of observable value. The work flow says something about when to start one or more activities. The activity lays down who is to produce what. The result of an activity is one or more artefacts. An artefact is an ephemeral entity, used to describe just about any tangible product of an activity. It may be a diagram, a document, source code, executables, just to mention a few concrete examples. The worker is responsible for producing the artefact. The worker refers to the roles that define how the individual participants in the development effort should do the work. Examples of workers are system analyst, designer, test designer, to mention a few.
The unified process provides the general framework for a development process. Knowing that no single process fits all sizes, the unified process is centered around the development case. The development case is a tailored process model for an individual project. In fact, one of the unified process' central work flows is the environment work flow. Its purpose is to support the development organization with both processes and tools. The process aspect is taken case of by the process configuration activity, which artifact is the development case.
While independent of technique, the unified process is developed for use with object-oriented development. It uses the Unified Modeling Language for notation. It is used in modeling everything from the development process, to business modeling and the implementation diagrams used in coding the software. The unified process/UML is tightly interwoven, and while the UML is a standard under the Open Management Group's control, it is tailored to fit Rational Corporation's Unified Software Development Process, or the Rational Unified Process as it is also called.
A large part of the unified process focuses on modeling. The process is described as both architecture-centric and use-case driven. Requirements are captured with use-cases. During the requirements work flow, the software system's requirements are captured and described, or using the unified process' terminology: the problem is understood and modeled. The use-case is used to formulate a model of the solution to this problem, it is a graphical representation of a set of requirements. The use-case is a sequence of actions a system performs that yields an observable result of value to an actor. Actor is in this context used to denote a non-specific entity that makes use of the software. The actor may be a user, but it may also be another software system or possibly a hardware system.
The unified process' creators argues the importance of architecture. Architecture helps understanding the software system, it organizes development and fosters reuse. "The software architecture embodies the most significant static and dynamic aspects of the system … Architecture is a view of the whole design with the important characteristics made more visible by leaving details aside" [BOOCH1999](Booch et al. 1999, p. 6). Different kinds of notations are used to model the software system during the analysis and design work flow. In the end the idea is to have as complete drawings as possible to base the implementation on during the construction work flow.
Process models based on the waterfall principle has shown too rigid for software systems development. The unified process solves this by being iterative. This is a common solution to the problems with waterfall processes, and nothing particular to the unified process. Being iterative means that instead of developing the software in one go, the development effort is broken into a succession of small waterfall projects. Upon having completed all stages—or work flows in the unified process—of the waterfall, the process is started all over again from the beginning. Each run is called an iteration. This way software is built incrementally. The advantage of iterative development is that it is presumable more flexible to change. By repeating the requirements gathering, the process is able to cope with a changing environment which the software will be deployed in. It also allows for knowledge created during the development process to be fed back into the development effort. So while each iteration is sequential like waterfall models, the iterative process is different as the work flows are revisited again and again through the development effort's life cycle.
As the development effort passes through a series of iterations, the process model increases and decreases emphasis on different work flows. The analysis and design work flow is more important in early stages of the development effort, while focus on construction increases with each iteration. In the end, when the software is complete focus increases on the deployment work flow, which hasn't had any focus in earlier iterations.
What are the differences and what are the similarities between Jackson Development Process and the Unified Software Development Process? How and in what way do they both belong to the tradition of software engineering?
If engineering is "the application of mathematical and scientific principles to practical ends" [DICT](Allen 1991, p. 388) and software engineering is applying engineering to software systems development, then software engineering's world view is described by Burrel and Morgan as objectivist. Writes Burrel and Morgan:
Neither JSD nor the unified process provides any techniques other than notation to understand "the part of the real world which [is] taken into account when developing programs" [FLOYD1987](Floyd 1987, p. 198). Floyd calls this the referent system, as opposed to the software system which is the software that interfaces with the referent system. Both JSD and the unified process apply classification of the real world into entities and processes/classes and actors. The basic assumption is that it is actually possible to classify and translate social processes, the referent system, into charts and diagrams. The world is classified, and relationships are drawn between the classes—inheritance, aggregation—and then the classes are drawn up in a hierarchy. Subjectivity is not an issue, as the referent system is inherently classifiable.[T]he objectivist applies models and methods derived from the natural sciences to the study of human affairs. The objectivist treats the social world as if it were the natural world. [BURREL1979](Burrel and Morgan 1979, p. 7)
Christiane Floyd draws up a dichotomy between what she calls the product-oriented and process-oriented perspective on software systems development. Software systems development traditionally views "software as a product standing on its own, consisting of a set of programs and related defining texts" [FLOYD1987](Floyd 1987, p. 194). She calls this the product-oriented perspective. From this point of view the world is static, having two states: before and after the software system is introduced. "The process-oriented perspective, on the other hand, views software in connection with human learning, work and communication, taking place in an evolving world with changing needs" [FLOYD1987](Floyd 1987, p. 194). Floyd's argument is quickly summarized by the software development cliche: It's no longer a matter of developing the system right, but developing the right system. To Floyd the product-oriented perspective is concerned with developing the system right, while the process-oriented perspective looks at developing the right system.
It is easy to draw an equality between the product-oriented view and the objectivist perspectives of software engineering. Especially since Floyd says the product-oriented perspective chooses the referent system with a view to successfully develope the software system, and that programs are derived by formalized procedures starting from an abstract specification. The abstract specification being the architecture and models both JSD and the unified process emphasizes. The picture, however, is more nuanced.
The product-oriented view regards the referent system as informationally closed and essentially static, having basically two states: before and after introducing the software system. Methods are generally applicable, well-defined and context independent. The process-oriented view, on the other hand, views software systems development as a process of learning and communication. Herein lies the difference between JSD and the unified process, and the difference between a model and a process model. JSD shares the product-oriented perspective on the referent system. That it is shaped on the waterfall model reflects the view that the referent system has two states: before and after the software system is introduced. The JSD approach to software systems development is a context free description of the work to be undertaken. Iterative software development exhibits the exact opposite view, seeing the development effort as a process of learning and communication. Thereby the use of process model to describe the development effort's life cycle.
This difference isn't unique to JSD and the unified process. It shows a divide that is apparent within software engineering. Many recent additions to software engineering, like the crystal methodologies [COCKBURN2002](Cockburn 2002) and Coad and De Luca's feature-driven development [PALMER2002](Palmer 2002) in addition to the unified process, embrace the learning aspect of the process-oriented perspective by use of iterations. Older development methods, like the Jackson Software Development, are modeled on the waterfall process. Despite their difference in view on the software's role in the real world, both JSD and the unified process are based on the basic principle of software engineering: applying mathematic and scientific methods to building software.