Diagnostic Basics Diagnostic Basics Diagnostics

Terms & Techniques

Warnings	compiler messages hinting at potentially problematic runtime behavior / subtle pitfalls	(see below)
Assertions	statements for comparing and reporting expected and actual values of expressions	(see below)
Testing	compare actual and expected behavior of parts or entire program	(see below)
Code Coverage	how much code is actually executed and/or tested	gcov ,…
Static Analysis	finds potential runtime problems like undefined behavior by analyzing the source code	ASAN , UBSAN ,…
Dynamic Analysis	finds potential problems like memory leaks by running the actual program	valgrind ,…
Debugging	step through code at runtime and inspect in-memory values	( next up )
Profiling	find out how much each function/loop/code block contributes to the total running time, memory consumption, …
Micro Benchmarking	small tests that measure the runtime of of a single functions or a block of statements/calls rather than a whole program run

Remember: Use Dedicated Types!

to restrict input parameter values
to ensure validity of intermediate results
to guarantee validity of return values

Goal: compiler as correctness checker – if it compiles, it should be correct

// input guarantee: angle is in radians
Square make_rotated (Square const&,                     Radians angle);
// input: only valid quantity (e.g. > 0)
Gadget duplicate (Gadget const& original,                   Quantity times);
// result guarantee: vector is normalized
UnitVector3d dominant_direction (WindField const&);
// avoid confusion with a good units library
si::kg mass (EllipsoidShell const&,               si::g_cm3 density);
…

Compiler Warnings

Compiler Error = program not compilable
Compiler Warning = program compilable, but there is a problematic piece of code that might lead to runtime bugs

gcc/clang Options

Most Important

-Wall Highly recommended. You should always use at least this. It doesn't really enable all warnings, but rather the most important ones that don't produce too much false positive noise.

-Wextra Enable even more warnings than -Wall. Highly recommended.

-Wpedantic Highly recommended. Issue all warnings demanded by strict ISO C++; reject compiler-specific extensions.

-Wshadow Highly recommended. Issue warnings when variables or type declarations shadow each other.

-Werror Treat all warnings as errors ⇒ any warning will terminate compilation

-fsanitize=undefined,address Enables Undefined Behavior Sanitizer and Address Sanitizer (more on that in the following chapters)

Recommended Set (Production Level)

-Wall -Wextra -Wpedantic -Wshadow -Wconversion -Werror -fsanitize=undefined,address -Wfloat-equal -Wformat-nonliteral -Wformat-security -Wformat-y2k -Wformat=2 -Wimport -Winvalid-pch -Wlogical-op -Wmissing-declarations -Wmissing-field-initializers -Wmissing-format-attribute -Wmissing-include-dirs -Wmissing-noreturn -Wnested-externs -Wpacked -Wpointer-arith -Wredundant-decls -Wstack-protector -Wstrict-null-sentinel -Wswitch-enum -Wundef -Wwrite-strings

Even More… High Performance / Low Memory / Security Performance / Memory / Security

Might be very noisy!

-Wdisabled-optimization -Wpadded -Wsign-conversion -Wsign-promo -Wstrict-aliasing=2 -Wstrict-overflow=5 -Wunused -Wunused-parameter

MS Visual Studio Options MSVC

/W1 Level 1: severe warnings

/W2 Level 2: significant warnings

/W3 Level 3: production level warnings. You should always use at least this. Also the default for newer Visual Studio projects.

/W4 Highly recommended, especially for new projects. Doesn't really enable all warnings, but rather the most important ones that don't produce too much false positive noise.

/Wall Enables even more warnings than level 4; can be a bit too noisy.

/WX Treat all warnings as errors ⇒ any warning will terminate compilation

Assertions

Runtime Assertions

#include <cassert>

assert(bool_expression);

aborts the program if expression yields false

Use cases:

check expected values/conditions at runtime
verify preconditions (input values)
verify invariants (e.g., intermediate states/results)
verify postconditions (output/return values)

Runtime assertions should be deactivated in release builds to avoid any performance impact.

#include <cassert>double sqrt (double x) {  assert( x >= 0 );  …}double r = sqrt(-2.3);

$ g++ … -o runtest test.cpp
$ ./runtest
runtest: test.cpp:3: void sqrt(double): Assertion `x >= 0' failed.
Aborted

Commas must be protected by parentheses

assert is a preprocessor macro (more about them later) and commas would otherwise be interpreted as macro argument separator:

assert( min(1,2) == 1 );  //  ERROR

assert((min(1,2) == 1));  //  OK

How To Add Messages

can be added with a custom macro (there is no standard way):

#define assertmsg(expr, msg) assert(((void)msg, expr))
assertmsg(1+2==2, "1 plus 1 must be 2");

(De-)Activation – g++/clang g++

Assertions are deactivated by defining preprocessor macro NDEBUG, e.g., with compiler switch: g++ -DNDEBUG …

(De-)Activation – MS Visual Studio MSVC

Assertions are explicitly activated

if preprocessor macro _DEBUG is defined, e.g., with compiler switch /D_DEBUG
if compiler switch /MDd is supplied

Assertions are explicitly deactivated, if preprocessor macro NDEBUG is defined; either in the project settings or with compiler switch /DNDEBUG

Static Assertions C++11

static_assert(bool_constexpr, "message");

static_assert(bool_constexpr); C++17

aborts compilation if a compile-time constant expression yields false

…
using index_t = int;
index_t constexpr DIMS = 1;  // oops
void foo () { 
  static_assert(DIMS > 1,                 "DIMS must be at least 2");
  …
}
index_t bar (…) {
  static_assert(
    std::numeric_limits<index_t>::is_integer &&
    std::numeric_limits<index_t>::is_signed, 
    "index type must be a signed integer");
  …
}

$ g++ … test.cpp
test.cpp: In function 'void foo()':
test.cpp:87:19: error: static assertion failed: DIMS must be at least 2
 87 |  static_assert(DIMS > 1, "DIMS must be at least 2");
    |                ~~^~~

Testing

Guidelines

Use Assertions

to check expectations / assumptions that are not already expressible / guaranteed by types:

expected values that are only available at runtime
preconditions (input values)
invariants (e.g., intermediate states/results)
postconditions (output/return values)

Runtime assertions should be deactivated in release builds to avoid any performance impact.

Write Tests

as soon as the basic purpose and interface of a function or type is decided.

faster development: less need for time-consuming logging and debugging sessions
easier performance tuning: one can continuously check if still correct
documentation: expectations/assumptions are written down in code

Use A Testing Framework

More convenient and less error-prone: predefined checks, setup facilities, test runner, etc.

Beginners / Smaller Projects: doctest

very compact and self-documenting style
easy setup: only include a single header
very fast compilation

Larger Projects: Catch2

same basic philosophy as doctest (doctest is modeled after Catch)
value generators for performing same test with different values
micro benchmarking with timer, averaging, etc.
slower to compile and slightly more complicated to set up than doctest

doctest – Simple Test Case

// taken from the doctest tutorial:
#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
#include "doctest.h"
int factorial (int n) {
  if (n <= 1) return n;
  return factorial(n-1) * n;
}
TEST_CASE("testing factorial") {
    CHECK(factorial(0) == 1);
    CHECK(factorial(1) == 1);
    CHECK(factorial(2) == 2);
    CHECK(factorial(3) == 6);
    CHECK(factorial(10) == 3628800);
}

$ g++ … -o runtest test.cpp
$ ./runtest
test.cpp(7) FAILED!
CHECK( factorial(0) == 1 )
with expansion:
CHECK( 0 == 1 )

The test fails, because the implementation of factorial doesn't handle the case of n = 0 properly.

Run this example in Compiler Explorer

doctest – Subcases

#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
#include "doctest.h"
#include <vector>
// taken from the doctest tutorial:
TEST_CASE("vectors can be sized and resized") {
    std::vector<int> v(5);
    REQUIRE(v.size() == 5);
    REQUIRE(v.capacity() >= 5);
    SUBCASE("push_back increases the size") {
        v.push_back(1);
        CHECK(v.size() == 6);
        CHECK(v.capacity() >= 6);
    }
    SUBCASE("reserve increases the capacity") {
        v.reserve(6);
        CHECK(v.size() == 5);
        CHECK(v.capacity() >= 6);
    }
}

For each SUBCASE the TEST_CASE is executed from the start.

As each subcase is executed we know that the size is 5 and the capacity is at least 5. We enforce those requirements with REQUIRE at the top level.

if a CHECK fails: test is marked as fails, but execution continues
if a REQUIRE fails: execution stops

Run this example in Compiler Explorer

Don't Use `cin`/`cout`/`cerr` Directly! Don't Use `cin`/`cout` Directly! I/O Streams

Direct use of global I/O streams makes functions or types hard to test:

void bad_log (State const& s) {   std::cout << … }

In Functions: Pass Streams By Reference Functions

#include <iostream>
#include <string>
#include <sstream>
#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
#include "doctest.h"
struct State { std::string msg; … };

void log (std::ostream& os, State const& s) {   os << s.msg; }

TEST_CASE("State Log") {
  State s {"expected"};
  std::ostringstream oss;
  log(oss, s);
  CHECK(oss.str() == "expected");
}

Class Scope: Store Stream Pointers Classes

But: try to write types that are independent of streams or any other particular I/O method.

#include <iostream>
#include <sstream>
#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
#include "doctest.h"
class Logger {
  std::ostream* os_;
  int count_;
public:
  explicit
  Logger (std::ostream* os):   os_{os}, count_{0} {}
  …
  bool add (std::string_view msg) {
    if (!os_) return false;
    *os_ << count_ <<": "<< msg << '\n';
    ++count_;
    return true;
  }
};

TEST_CASE("Logging") {
  std::ostringstream oss;
  Logger log {&oss};
  log.add("message");
  CHECK(oss.str() == "0: message\n");
}

Exceptions

Beginner's Guide / Diagnostics

Debugging (gdb)

Related …

Testing Frameworks for C++

Last updated: 2022-11-20

First Steps

Input & Output

Custom Types – Part 1

Diagnostics

Standard Library – Part 1

Function Objects

Standard Library – Part 2

Code Organization

Custom Types – Part 2

Generic Programming

Memory Management

Software Design Basics