Value of Systems Engineering

View summary of results: ValueSE-INCOSE04.pps [342KB]
View full report: ValueSE-INCOSE04.pdf [330KB]
Download useable slides of Cost and Schedule overruns: ValueSE-CostSchedule.ppt [90KB]
Download useable slide of NASA chart: NASA-Chart.ppt [88KB]

Project 01-03 Phase I Final Results

Data submitted includes 42 valid projects. Most projects expended 5% or less on systems engineering tasks. When factored by the subjective quality of the systems engineering, most projects expended 2% or less on effective systems engineering.

Cost overrun on the reported projects shows a distinct correlation against systems engineering effort (SEE). Two effects are apparent:

• Cost overrun lessens with increasing SEE. Overrun appears to minimize at something greater than 10% SEE, although few data points exist to support a reliable calculation. The solid line is the least-squares trend line for a 2nd order curve.
• Variance in the cost overrun also lessens with increasing SEE. At low SEE, a project has difficulty predicting its overrun, which may be between 0% (actual = planned) and 200% (actual = 3x planned). At 12% SEE, the project cost is more predictable, falling between -20% (actual = .80 x planned) and 41% (actual = 1.41 x planned). The dashed lines are the 90th percentile when assuming a Normal distribution.

Schedule overrun on the reported projects also shows a distinct correlation against systems engineering effort (SEE). Two effects are apparent:

• Schedule overrun lessens with increasing SEE. Overrun appears to minimize at something greater than 10% SEE, although few data points exist to support a reliable calculation. The solid line is the least-squares trend line for a 2nd order curve.
• Variance in the schedule overrun also lessens with increasing SEE. At low SEE, a project has difficulty predicting its overrun, which may be between -35% (actual = .65 x planned) and 300% (actual = 4x planned). At 12% SEE, the project cost is more predictable, falling between -22% (actual = .78 x planned) and 22% (actual = 1.22 x planned). The dashed lines are the 90th percentile when assuming a Normal distribution.

The data available for analysis in this project present several important limitations to the results. Any use of the values herein should be tempered by these limitations.

• The data are self-reported and largely subjective, without checking. Those responding to the data requests may be assumed to be senior engineering personnel by nature of their association with INCOSE; such personnel can be expected to have the kind of data requested. Nonetheless, there have been no quality controls on the submission of data.
• Perceptive influences likely color the data. The underlying hypotheses for this project are well-known and widely accepted. Because of the wide acceptance, respondents can be expected to include a subconscious bias toward supporting the hypotheses. This single fact might have caused much of the correlation observed.
• Systems engineering effort is also self-reported based on the respondents’ individual perceptions of systems engineering. There is no certainty that different respondents had the same perceptions about the scope of work to be included within SEE.
• Respondents come from the population of INCOSE members and others with whom the researchers had contact. This limits the scope of projects included within the data.

Page last modified 25 Sep 04