Can Requirements Specifications Be Replaced By Databases?

Abstract:

With the increasing maturity of Requirements Management Tools, and the impact of "Faster, Cheaper, Better" sometimes interpreted as "Get through the Requirements Phase as quickly as you can", it is tempting to do away with formal Requirements Specifications altogether, and manage your requirements set directly via a shared, Configuration-managed database. Is this adequate? What do we lose in the evaluation and review process? Is the database or the paper specification more likely to be read and used?

Are there differences in practice (& efficiency) between internally-generated requirements sets (eg for Commercial product development) and different 'layers' of explicit customer-supplier procurement (e.g. contracting for a User capability; prime System requirement; detailed sub-system subcontract)? What effect does this have on depth of detail and flexibility throughout the development lifecycle - e.g. number of levels of decomposition, rigour of traceability?

The panellists will present their own views, from experience gained in all types of industry and customer / supplier relationship. A moderated debate will ensure the audience can participate and voice their own concerns. The purpose of the panel is to learn from the panellists, and each other, that reliance on a requirements database tool, with fully traceable multi-level decomposition and verification matrices, is no more the answer to our prayers than 1000-page requirements specifications. There are 'best-practice' lessons to be learned from using a little of both worlds!

In particular, the panel will discuss, and invite challenges from the floor, on the following topics:

• Paper versus database as the master version of the requirements
• Use in the review and evaluation process
• Comparative readability and usability
• Treatment of internal and externally-generated requirements
• Decomposition and traceability (whole system vs e.g. software component), and how to treat them in DB and paper environments.

Five 'position papers' are appended below on these topics; the panel itself will present all the topics, but with only four allowed to speak, to give as much time as possible to public debate!

Moderator:

Paul Davies, Thales Sensors, UK - Impartial observer, and Co-Chair for International activities of the INCOSE Requirements Working Group

Mr Alan Arnold, Director Systems Engineering in the Australian Defence Acquisition Organisation - voice of the customer, and representative of the Symposium host nation.

Context
This submission discusses requirements engineering challenges and solutions addressed by a customer of the International Defence Industry. The Australian Defence Material Organisation, together with the Australian Defence Headquarters, manages the definition and acquisition of capital equipment with about AS$2.5Bn expended each year.

Introduction
A number of reports and studies into the management of major capital projects have highlighted problems in the definition and management of requirements and specifications in Australian Defence Projects. Recent initiatives has sought to improve this situation and provide and improved requirements engineering capability for the Australian Defence Organisation.

What is the desired outcome from requirements and specifications?
Efficient and effective communication of requirements to all stakeholders, acquirers and suppliers to underpin the development and delivery of capital equipment on time, within budget and that satisfy the user's real needs.

Who are the stakeholders?
Within Defence, the stakeholders are a diverse community of system users, operators, technical regulatory authorities, training providers, support staff, and managers. Many do not have a technical professional qualification of working background.

Some practical problems

• Non technical professional stakeholders are severely challenged in comprehending detailed engineering specifications.
• Engineering and other technical staff often lack appropriate operational experience to understand the user and operator requirements for a system.
• The nature of the commercial relationship between customer and supplier is a significant risk issue and the specification and management of risk should be appropriate to that relationship.
• Government and organisational policies are significant drivers in requirements engineering.

Some solutions

• A sound Operational Concept Document provides an effective communication vehicle for the diverse stakeholder, acquirer and supplier community.
• The Defence Materiel Organisation has recently trialed and obtained good results from recording and managing requirements document in an integrated database managed by a modern requirements management tools.
• Rapid Application Development techniques involve modelling, development of prototypes, and integrated product and project teams working with defined limits to produce new systems or components within short periods of time. Formal specifications are not used. These techniques have proved effective in the evolutionary acquisition of a combat support system.

Panel Member #2: Mr Richard Moore, Litton TASC, USA - an experienced RWG member with a background in the Contractor side of military requirements.

Introduction
Recent experience on two different projects has shown that both paper specifications (in Word document) and databases (e.g., RDBMS or RM tool) are required to fulfil the systems engineering activities such as requirements definition and test planning (e.g., generate test plans and procedures). In addition, contracting officers require paper specifications for their contract files to meet current directives and regulations (depending on how the contract is written).

Discussion
The use of paper specifications in the review and evaluation process has merit early in a system development with the generation of the operational requirements document and concept of operations document and the initial system specification. However, requirements traceability must be established early in a program to ensure the proper flow down of requirements as requirements are allocated to subsystems to include external and internal interface requirements.

Requirements traceability can be done "manually on paper" for small projects (small set of requirements). Large projects require an automated way to capture the relationships between requirements to reduce the amount of errors in mapping these relationships. Databases provide the ability to quickly map these relationships to ensure the proper flow down of requirements. Databases also allow the systems engineer to sort the requirements in different ways to aid in requirements analysis and to better visualize if there are any missing requirements in lower level specifications. Included within databases are simulation models (e.g., Excel spreadsheet or STATEMATE tool) that allow systems engineers to model the expected behaviour of the requirements set and evaluate the proper flow down of requirements based on this expected behaviour.

The distinction between internally- and externally-generated requirements requires a way of tracking how each of these requirements are controlled and flowed down within the system. Good requirements traceability is required here. Externally generated requirements are originating requirements from the customer, user, and operators in the field. Usually these requirements have established thresholds that must be met or the system will not be accepted in the field. Therefore, complete traceability from concept development to test planning is necessary. Internally generated requirements may be controlled at the system or at lower levels within the program. In addition, these requirements may be derived from externally generated requirements; therefore, a relationship is established that must be traced. Again, this may be done manually through tables in paper specifications but this is very prone to error most of the time.

Requirements Databases allow the systems engineer to validate the "parent-child" relationships consistently throughout systems development, in particular as more and more derived requirements are developed that must be traced to higher-level requirements.

Some practical problems

• Current requirements management tools do not repeatably generate "paper" specifications in the proper format.
• Contracting Officers normally require having system specification in paper for contracting files.
• Government program representatives and configuration control boards are use to dealing with "paper" specifications. They will not easily convert to the "gold copy" in a database without a demonstrated, repeatable process for generating paper products for the review process.

Some solutions

• Current effort on AP233 needs to include "desktop publishing" tools for generating "paper" specifications in the proper format.
• Tools vendors need to look "outside the box" on what the tool development requirements are besides just requirements traceability. They've done good development on the requirements capture process and traceability although there's still work to be done here. However, they need to look at the back end of the requirements process that captures the results of the requirements capture in a form acceptable to the customer.

Panel Member #3: Mr Mark Sampson, SDRC, USA - author / vendor of the SLATE systems engineering tool, and Chair of the INCOSE Modelling and Tools Technical Committee.

Summary Position: Documents don't work, models are too cumbersome; a design database used to generate documents is a good compromise.

Liver and Onions...
At a recent family reunion, we had an opportunity to sit around the dinner table and reminisce about growing up in a large family. Someone asked, "Mum, why did we always have liver and onions on Thursday nights?"."Your Dad really likes liver and onions", Mum replied."I hate liver and onions, I thought you liked it; that was the only reason I ate it", Dad said. "I don't like liver and onions either, maybe it was Ellen who liked it". Ellen was already shaking her head no, "maybe it was your grandmother who liked liver and onions so much?"...it turns out that no one liked liver and onions, yet we had been eating it under the mistaken impression that someone else in the family loved liver and onions...

So who is it that loves lengthy requirements specifications? Or incomprehensible models? Are we all just itching for a change, and if so, what do we want? What is going to give us all the best value for our efforts?

• Study at Texas Instruments into the causes of runaway projects puts the blame on failed systems engineering, failure to stay ahead of development, failure to integrate requirement information into development process
• Reengineering/Hammer suggests putting time/money where the value is-what does the customer value and where do we spend our time on creating that value.
• Systems Engineers spend ~50% of their time generating documents, 25% re-entering the same information in different places, 15% of their time "futzing", and 30% communicating that information (it adds up to greater than 100% because of overtime)
• Do customer's values line up with the time we are spending on developing the document? No.
• What else is wrong with documents?
  • Disconnected document baselines
  • Obsolete by the time they are printed o Document generation outside the development process
  • Don't communicate intent, rationale, etc.
  • Documents are boring, nobody reads them
  • Hide a designs quality
• What does the customer value? The compliant product.
• So in case you've forgotten, documents are not the product of systems engineering, the content of the document is the product of systems engineering.

Models and Methodologies don't stick…

• The modelling people are right when they say "we want the behaviour" and executable specification of the product
• The problem is…
  • How much fidelity is needed to model reality; how much can you afford? (something like 10% of the value to a project effort can consume 50%+ of the resources)
  • Effort in maintaining the model…experience changes your perceptions of reality
  • Obtuse methodologies that make users "tired"
  • Few people speak the language
  • Create their own processes/bureaucracies…creates tool high priests, by-passes design database, side tracks the best and brightest as tool/model maintainers
  • Create psychological barriers around the design…supports group think, ignoring reality, etc.

A better way to produce documents-connected documents. Capture design while capturing interrelationships

• Documentation as a by-product of the design capture process
• Capture design, integrated with requirements…as the design changes, so do the documents that go with it.
• Various models go with the design (also enables cross-discipline system models)
• Other benefits…
  • Known document quality (also means you don't have a stack of paper to hide behind anymore)
  • Focused on customer needs rather than documentation about those needs (complete with CM and other overhead functions that go along with document baselines)
  • Brings requirements to where they can have an impact-disconnected Traceability/Requirements outside of the design process are impotent.
  • Integrates meaning with the design
  • Requirements to have impact on the design must be integrated with design decisions (a carrot in front of every design decision results in built-in compliance).
  • Shorter time to delivery (sail boat race example) o Integrated impact analysis

Summary…
Documentation is a poor approximation of the design through once-removed text describing the design. Models that accurately reflect reality are difficult/costly to build and maintain. Something in between-capturing design while enabling models and generating documents as a by-product - is needed.

Panel Member #4: Dr Michael Winokur, Tel Aviv University & Israel Aircraft Industries - a view from academia and from the non-US/UK/Aus axis.

Introduction
From my experience in applying data base tools to requirements engineering in projects of many sizes, and quite a few of them in the "faster, cheaper, dirtier" environment, I can certainly say that you need both:

• Database tools (with viewing and automatic document generation capabilities) and
• Written specifications, which hopefully come out from the tool automatically as a by product - I managed the development of such a system that actually worked using Statemate - the engineers hated it, but it did the job.

What are the main issues that need attentions?
The optical illusion: without tools and discipline the development process is only "more expensive, slower, messier" . It is only an illusion of a faster or cheaper development cycle if the Requirements Analysis and Specifications activities are not given their proper attention. A recent NASA report about failed missions clearly shows this. The ratio of wasted money versus effort invested in requirements analysis is inversely proportional.

The approach to requirements in "internal" vs. "user contracted" type systems development is an interesting issue, but it is not a central one. Both development types will suffer greatly in quality and cost if requirements are not dealt with properly There really is little difference in the process needs. The problem to tackle here is more related to large customer acquisition bodies, like military or space agencies that require vast amount of paper as part of their project progress tracking procedure. Repeatability of automatic specification from data base tools, obtuse tool operation for document generation and poor quality or readability of automatically generated documents are the problems in this case.

The most interesting issue arises when dealing with methodology-oriented tools (Statemate Magnum, Slate, CORE) vs. the more administration type tools like RTM or Doors. These latter are practical, but to say that they (in themselves) aid the process of requirements engineering is misleading. What is fundamentally the issue is the requirements process itself rather than the paper-based or tool-based support to the process.

Then, what is at the heart of it?
Models and Methodologies for Requirements Analysis and Specification are still at the heart of system development. Yes this is the main issue. The whole activity of "creating content" in systems engineering is about models. The whole issue of engineering as a matter of fact is about modelling.

Would anybody trust a control system developed by engineers who do not know how to model feedback loops and how to analyse their response and stability properties? What is the difference with requirements? Requirement specification, either in a data base or in a document (or in a document generated from a data base) is only the clerical task of storing or documenting the results of the system requirements analysis activities that created the content. In summary:

• Obtuse methodologies: use simpler ones, but use them
• Few people speak the language: create simple models (and use simple language that many people are conversant with, like flowcharts), but take the time and effort required to create them.
• Maintaining the model demands effort: so what?. Developing a system twice (at least) to get it right because we were lazy in maintaining the requirements and design models demands less effort?
• Engineers do not know how to use requirement analysis and modelling methods: train, train, train or you'll loose the train!

Panel Member #5: Mr Peter Lister, Siemens Transportation Systems, UK - representing Commercial product development, and an experienced tool-user.

Introduction
This submission discusses the issues of readability and usability of requirements and the differences between internally and externally generated requirements. There are pros and cons for both a paper based representation of a requirement set and a well structured database of requirement 'objects. Both representations have their place, and indeed most suppliers of Requirement Management tools attempt to satisfy both options by providing documentation features. But are documents, for all their convenience, necessarily the best way to make use of requirements in the development of a system?

Narrative Versus Objects and Relationships
The traditional docu-centric view, where requirements are synonymous with a specification document, still predominates. The requirements document is easily assimilated, disseminated, configured and used by everyone involved in a project. No special training or tools are needed to read and understand the content. But is this really true?

Requirement specifications can often mislead because we assume that a well-presented document does not contain any anomalies. Manually created requirements documents almost invariably contain anomalies, significant omissions and even outright contradictions. These can readily be spotted by carrying out the type of formal requirements analysis that should be applied when developing requirements within a database. Of course the sharp-eyed can spot these problems within the document, but in large documents this is difficult and error prone.

You will note the use of the word 'should' in the previous sentence. There have been too many examples where database requirements management has been poorly executed resulting in requirement obfuscation instead of the desired clarification. The requirements database demands that appropriate techniques are applied to review and analyse requirements as individual objects rather than a narrative. Requirements Management Tools offer unparalleled opportunities for generating large volumes of rubbish if a disciplined approach is not adopted.

So why is a good requirements database superior to a good requirement specification? The answer lies in the ability to create and link to multiple structures within a database. Documents can rarely be more than two (or really one) dimensional. The traditional trace table is at best a selective view of the truth - it relates only two sets of objects but ignores any other sets involved. Much is made of the benefits of downward traceability from the stakeholder requirements to the design requirements, and this is indeed important. But just as important is to link requirements to functional and physical structures of the proposed system.

It is highly instructive to propose an architecture and then try and assign (or allocate if you prefer) requirements to that architecture. How many of the requirements do not have an obvious home? How many parts of the architecture have no requirements? When there are relatively small groups of requirements assigned to each architectural element, how many duplicates can you spot now? How many new requirements seem to be needed to accurately specify the components of the architecture? How many requirements are specifying processes, development tasks, or what the customer has to do rather than actually specifying the system? Of course there may be a logical response for any of these questions, but if there is it out to be recorded (as another link in the database) because otherwise it may represent a hidden assumption.

Having created a requirements database, fully linked to collateral information, then any number of documents can be created from that database containing different views of the system. What all these documents will have in common is that they have been created from a multidimensional view of the system, and therefore are mutually and individually consistent.

Readability & Usability
Most would have to concede that a well-presented document is easier to read than objects in a database. It is possible to 'read' a database if it has good facilities for organising requirements in both defined hierarchies (i.e. as for a document structure) and using reader selected filters and grouping. However, unless you have an easy to use requirements database and a certain level of computer literacy, it is easy to lose one's way or miss a critical item.

The benefit of the database is that should a problem be spotted it is possible to explore the collateral information linked to that requirement, which may explain what is going on. If not, there may be a facility for entering a comment into the database, which is then linked to that requirement. Whilst there may be a price to pay for attaining the skills necessary to exploit the database content, there are also rewards for the diligent user.

There are potential pitfalls in circulating documents generated from the database for review. For a start, the document is frozen on the day that it is printed. The database content is both current, and dynamic - if someone has raised a comment then it is available for others to see. It is common for near identical comments to be received from several sources (each instance having to be recorded and linked to the others), or conversely contradictory comments on the same requirement. Also, when reviewers do not know why an apparently odd requirement is there, it is not unknown for the same comment to be received on the same requirement in different versions of the document. Correlating all these types of comments is hard enough if the document is manually produced - if it has been generated from a database it may be necessary to correct several parts of the database. This can make the customer a little impatient when changing two words in a document takes a week to accomplish. However, having made the change then everyone who uses the database from then on can see what was changed on how it now hangs together.

Internally and Externally Generated Requirements
A requirements database is a living and working entity. Starting from the original customer requirements one works down the system level to the subsystem level and beyond. As this requirement flow-down proceeds, more and more cognisance has to be taken of the emerging system design. It is important to distinguish between requirements that have been elaborated from the customer (or at least the definition of his needs) and those that have been derived from design decisions made by the system developer. As part of the 'Problem Space', the former should be considered if not mandatory then at least worthy of careful consideration and consultation. As part of the 'Solution Space'. the latter may be changed more readily, but the rationale for their existence still needs to be recorded.

There is considerable richness of information to record against both internally and externally generated requirements. It is virtually impossible to record this level of information within a requirement specification, and if the specification is not backed up with the necessary information it can be changed without consideration of the reasoning that brought it into being. A requirements database can retain all this information and present it in context when required.

Conclusion
So the requirements database has greater utility, but potentially less readability. It can record a 'rich picture' that documents cannot match and clearly identify the differences between requirements from different sources. Unfortunately, accessing this 'rich picture' requires both training and tools. It is unrealistic to expect all requirement users to work 'on-line', so the document is a vital by-product of the database.

It seems likely that the document will live with us for the foreseeable future. My position is that it needs to be relegated to a secondary position - an artefact of the database rather than the master information set - to be used and replaced frequently. Anybody carrying out significant design activities should always treat the database - with its up to date information - as the prime source of all information. Requirement change should never be considered the simple modification of a few words in a document - all the related information needs to be considered and changed to suit.