AQAP 2210 software quality assurance

What it is

AQAP-2210 is the NATO Allied Quality Assurance Publication that adds software-specific quality assurance requirements on top of whichever of AQAP-2110 or AQAP-2310 a contract already names. It is never used alone: the publication itself states it "may only be used as a software supplement to AQAP-2110 or AQAP-2310." It brings in AS9115, the software-specific supplement to AS9100, and where it is used alongside AQAP-2110 it substitutes ISO 9001 for every reference AS9115 makes to AS9100. It also converts every "should" in AS9115 into a mandatory "shall," unless the Acquirer agrees otherwise, and states that where a NATO-specific requirement in Chapter 5 conflicts with AS9115, the NATO-specific requirement wins. The current edition is Edition B, Version 1, promulgated August 2022, superseding AQAP-2210 Edition A Version 2.

It has no force by itself. Section 1.3 states that the publication "is primarily intended for use in a contract between two or more parties, in conjunction with AQAP-2110 or AQAP-2310," and that, "when referenced in a contract, this publication shall apply to all the processes necessary for the Supplier to fulfil the contractual requirements." Applicability reaches beyond bespoke development to non-deliverable software, software maintenance, non-developmental and off-the-shelf software, and the software element of firmware, wherever the contract invokes it. A Supplier can also adopt it voluntarily and internally without a contract requiring it, and where the contract and the publication disagree, the contract wins.

Who it binds, and how

The publication does not define Supplier, Acquirer or Government Quality Assurance Representative (GQAR) itself; it borrows them from the documents underneath it: "The applicable definitions of ISO 9000 or AQAP-2110 or AQAP-2310 or AS9115 apply to the terminology used in this publication." Its own Foreword frames the whole document as the customer's requirements of the contractor: "The Acquirer's software quality assurance requirements stated in this document, are based on the experience that quality management of the entire software development process is the key to achieving software quality in complex and mission critical computer systems such as weapon systems, communication systems, and command and control systems." Throughout the requirements chapters it is the Supplier who is instructed to act, with the Acquirer and/or GQAR receiving, reviewing and able to reject the resulting plans.

Because the practical question is never "does this apply to us" but "does the contract name it," the point at which it becomes real is earlier than most Suppliers notice: the Project Software Quality Plan (PSQP) it requires has to address the contractual requirements and be submitted before work starts.

Tailoring by criticality and development category

Only Chapter 5 may be tailored, and even there several sub-clauses are fixed and cannot be deleted, downgraded or modified: General, the PSQP itself, Software Criticality Analysis, organisation roles and responsibilities, Sub-supplier Management, Software Configuration Management, and Infrastructure. Tailoring sizes the requirements to two things. First, criticality, in four categories based on the severity of a system failure's consequences: "A" for software whose failure could cause catastrophic safety or mission-success consequences, down through "B" (critical or hazardous) and "C" (major) to "D" (minor or negligible). Second, development category, in four levels based on the scope of development, test, inspection or control needed: "1" covers all bespoke development or maintenance, and reused or open-source software with significant integration; "4" covers commercial off-the-shelf software with minimal integration. Every tailoring decision has to be agreed between Customer and Supplier, reflected in the contract, and documented in the PSQP with its rationale.

The Project Software Quality Plan

The PSQP is the document through which the Supplier demonstrates compliance. It carries approval signatures, sits under configuration control, and has to "document and maintain traceability of all the requirements from the project process by including a requirement and solution compliance matrix, justifying fulfilment of all the requirements of this publication" - so an auditor expects a live matrix, not a narrative claim. It also covers the criticality analysis method, a quality model derived from ISO/IEC 25010 or an equivalent, and the appropriate scope of operational planning and control, and it has to be reviewed and updated at pre-defined project milestones. The Acquirer and/or GQAR can reject the PSQP or any revision to it.

Configuration management, releases and patches

Software Configuration Management has to maintain integrity and traceability throughout development, prevent uncontrolled changes, and provide planned and released baselines, with procedures for identification, change control, status reporting, audits and coordination of externally developed items. Bidirectional traceability is required between system, software and component requirements, and between requirements, validation tests and test results, retained "in a matrix and/or in a dedicated tool" - with a note that COTS and partially modified software may only allow partial traceability. Every review and delivery needs a software release configuration report naming the software version, delivery media, applicable specifications, build procedure, associated documentation, changed configuration items, deviation and waiver status, and lists of corrected and outstanding faults classified by severity.

Patches get their own, stricter rule: their use "is prohibited except for urgent corrections with agreement of the Acquirer," any patch must stay temporary pending inclusion in a definitive version, and it needs its own patch configuration report covering the impact study, the corrected faults, the installation procedure, and the non-regression tests run against it.

Evaluation, verification, validation and testing

The Supplier has to plan a formal evaluation, verification and validation (EVV) process covering methods, objects, criteria, tools and the documentation it produces, with measures to evaluate the software quality characteristics the requirements or the PSQP specify. People carrying out EVV need "an appropriate level of independence from the person(s) who developed the software product," and the document is explicit that Acquirer or GQAR EVV activity "shall not constitute acceptance, nor shall it in any way replace EVV activities by the Supplier or otherwise relieve the Supplier of its contractual responsibilities." Testing covers item, integration, system and acceptance levels, with controlled test environments and certification of test reports; every piece of software documentation has to be reviewed and approved before issue.

Maintenance and what the document does not cover

Where software maintenance is a contracted requirement after initial delivery, the Supplier provides a Maintenance Plan before the maintenance phase begins, covering the work, the procedures, the records, responsibilities, the configuration management of the item being maintained, fault reporting and resolution, and testing and acceptance of modifications.

AQAP-2210 does not fix a development model, method or lifecycle of its own: it leaves the Supplier to adapt "the required documentation and procedures to the specific development and procurement processes of the project." It names no certification scheme and no accredited body: what it describes throughout is Government Quality Assurance, carried out by the Acquirer or an appointed GQAR directly against the contract, including at Sub-supplier facilities when requested.

Standards it references

• AQAP-2110Background
• AQAP-2310Background
• ISO 9001Background
• AQAP-2000Background
• STANREC 4814Background
• AAP-48Background
• ISO/IEC 27001Background
• AQAP-2105Background
• ISO 31000Background
• AQAP-2070Background