Quality Gates and Build Failure

NDepend Quality Gates (3 minutes)

Introduction

A Quality Gate is a code quality goal.

Such quality goal must be enforced before releasing and eventually, before committing to source control.

A Quality Gate can be seen as a PASS/FAIL criterion for software quality.

A dozen of default Quality Gates are proposed by NDepend related to measures like technical debt amount, code coverage or amount of issues with particular severity.

Notice that special red / yellow / green losange icons shows Quality Gates status: fail / warn / pass.

What is the difference between Quality Gates and Rules?

Both Quality Gates and Rules output problems that should be fixed, so it is legitimate to question about how they differ.

• A Quality Gate outputs a status (Pass, Warn, Fail).

  Typically a Quality Gate must be validated before releasing to production.

• A Rule outputs issues.

  An issue is a code smell that should be fixed to make the code cleaner and avoid potential problems.

  Typically the team can release to production even if some issues are still reported.

Quality Gates operate at a higher level than Rules. Quality Gates can rely on the Rules results (but not the opposite), like for example:

• We don't want new issues since baseline with a severity High
• We want to make sure we fix at least one person-day of technical-debt during each sprint

These are Quality Gates that rely on rules results, but Quality Gates can rely on any other criteria, like for example:

• We want to make sure that all new code written will be tested and covered by tests.

To summarize:

• Quality Gates represent a synthesized way to know if the team can release to production, through a simple PASS / FAIL approach.
• Rules represent a detailed way to assess the quality of a code base through an issue / technical-debt approach.

Quality Gates and Build Failure

Quality Gates can be used to fail the build when certain criteria are not-verified.
When at least one Quality Gate fails, NDepend.Console.exe returns a non-zero value that can be used to fail the build.

Quality Gates and Azure DevOps Build Failure

In the Azure DevOps / TFS extension there is the setting Stop the build when at least one Quality Gate fails.

Creating and Customizing Quality Gates

What makes NDepend unique is that a Quality Gate is a C# LINQ Query that can be easily created, edited and customized. For example if you wish to enforce a certain amount of code coverage through a Quality Gate, you can just write:

// <QualityGate Name="Percentage Code Coverage" Unit="%" />
failif value < 70%
warnif value < 80%
codeBase.PercentageCoverage

Notice also the special clauses failif (mandatory clause) and warnif (optional clause) that are specific to NDepend CQLinq. codebase.PercentageCoverage is actually a C# statement that NDepend evaluates.

Notice the special header if comment that defines the Quality Gate name and unit. The unit here is the string "%". It can be optionally repeated after after the thresholds for readability purpose.

A Quality Gate can also determine a measure based on the diff since baseline. For example the Quality Gate below is ensuring that the overall technical-debt doesn't grow beyond certain thresholds, since the baseline.

Notice that a <Description> tag can be used to describe the Quality Gate for non-developers.

// <QualityGate Name="New Debt since Baseline" Unit="man-days" />
failif value > 2 man-days
warnif value > 0 man-days
let debt = Issues.Sum(i => i.Debt)
let debtInBaseline = IssuesInBaseline.Sum(i => i.Debt)
select (debt - debtInBaseline).ToManDay()

//<Description>
// This Quality Gate fails if the estimated effort to fix new or worsened
// issues (what is called the *New Debt since Baseline*) is higher
// than 2 man-days.
//
// This Quality Gate warns if this estimated effort is positive.
//
// Debt documentation: https://www.ndepend.com/docs/technical-debt#Debt
//</Description>

Finally notice that when defining a Quality Gate threshold, the keyword value must be replaced with the keyword count if the LINQ query is returning rows instead of returning a scalar.
This is useful to make the Quality Gate result more informative when it makes sense.

Exploring Quality Gates Status

The set of Quality Gates status is queryable from C# LINQ queries.
The Dashboard summarizes the Quality Gates status. A single click generates a LINQ query that shows the detailed status for each Quality gate. Quality Gates statuses are also provided in HTML+js report.