Legal information

Copyright DB Netz AG, licensed under CC-BY SA 3.0 DE (see full text in CC-BY-SA-3.0-DE)

ARCH.009 Define measures of effectiveness

GoalAchieve shared understanding of the measures that will be used to determine if the system is a success
Requirements met by this process step

ISO 29148 (2018) para 6.2.3.3

ISO 15288 (2015) para 6.4.1.3 a)

Inputs

AMOD-106 Enterprise goals

AMOD-026 Service reliability/ availability targets

Business performance indicators for business units (e. g. DB Starke Schiene)

OutputsAMOD-106 Enterprise goals (modified)
Methodology

The measures of effectiveness indicate how well the railway infrastructure operations unit (wider system of interest) is delivering the business value (the effect) that it is required to. This is not the same as system of interest system performance. We define that later in a different task.

Proposal: split the task in two stages.

First, create a proposal within the organisation team.

Second, take this proposal to business stakeholders (not restricted to only one organisation, could also include stakeholders from external RUs for example), present and discuss, refine with feedback and agree.

The proposal can be created by brainstorming a suitable set of performance indicators given the targets that are set for the organisation to achieve. Reference should be made to organisation's quality management system, particularly any regulations (Richtlinien) that define performance indicators, or define how performance indicators should be created.

It may be possible to use an entirely existing set of effectiveness measures, but this should not be assumed.

Further guidance

The NASA Systems Engineering Handbook defines a measure of effectiveness as:

A measure by which a stakeholder’s expectations are judged in assessing satisfaction with products or systems produced and delivered in accordance with the associated technical effort. The MOE is deemed to be critical to not only the acceptability of the product by the stakeholder but also critical to operational/mission usage. A MOE is typically qualitative in nature or not able to be used directly as a design-to requirement.

These measures must have meaning at the level of stakeholders outside the wider system of interest. This means they must be expressed in terms that are relevant to the stakeholders and their needs - and not in terms of performance of activities internal to the wider system of interest (since stakeholders neither know nor care about how the WSOI delivers, only that it does).

Example: passengers care about journey time, but they do not care how many seconds it takes to set a route for a train.

There already exists a set of measures that are used for quantifying the effectiveness of the railway as it currently operates. These should be elicited from knowledgeable stakeholders and assessed for their suitability as indicators in the future.

Further measures may be identified. New measures may be desirable to quantify effectiveness in areas that have traditionally not been formally monitored. However, the team must be mindful that adding large numbers of new additional measures could indicate that they are thinking on the level of system performance (see ARCH.058 Define measures of performance), rather than operational effectiveness, since the railway infrastructure has broadly the same operational capabilities now as it will in future.

Existing measures may be replaced with new ones where the new ones are considered better suited to future operation.

Tools and non-human resources

DB regulations

DB quality management system, indicators currently in use

CardinalityOne-off with revisions permitted
Completion criteria

The set of measures of effectiveness is agreed between the organisation and the wider business stakeholders.

The set of measures of effectiveness is accepted by stakeholder top management.

Measures of effectiveness are defined sufficiently to assure that they are observable and controllable by the oarganisation.

Step done by (Responsible)Operational concept architect
Design review
Provides input to/assists (Contributes)

Lead system architect

System architect

Operations domain expert

Uses outputs (Informed)None directly