Next: 8 Profiling of Communication Up: ADAPTOR Profiling Guide Previous: 6 Counting Performance Events Contents

Subsections

7 Tracing of Performance Events and Visualization

7.1 The Idea of Tracing

Instead of the accumulation of all counter values it is also possible to generate a trace file that contains for every entry and exit of a region a trace record containing the following information:

a time stamp;
identification of the region and whether it has been entered or left;
the values of the enabled counters.

Further trace records will be generated that define the regions and the counters that are traced.

Due to the timestamps, a tracefile allows to analyze performance values of regions for different calls during program execution.

7.2 Tracing With ADAPTOR

Tracing is enabled by the following steps:

The program must be instrumented by fadapt to have runtime calls generated when a region or subprogram is entered or left (see Section 3).
Tracing must use a trace interface and a corresponding trace libraray. Currently, only tracing with VampirTrace is supported (see Section 5.3).
```
    adaptor ... [-trace=vt | -trace=vtsp]
```
By default, the generation of a trace file at run-time is disabled. It will only be generated if the flag -trace is used or if the environment variable TRACE is set.
```
                         export TRACE=1
    executable -trace    executable
```
If tracing and performance monitoring is enabled, the runtime system will also generate a trace record for the current values of the performance counters at every entry and exit of a region.
```
    a.out -trace -pm=pm_config
```

7.3 Performance Visualization With VAMPIR

Tracefiles generated by VampirTrace can be displayed by VAMPIR [BHNW01]. Vampir provides a convenient way to graphically analyse runtime event traces produced by MPI or serial applications. Vampir offers the possibility to focus quickly at the appropriate level of detail to analyse application performance issues.

7.3.1 Visualization of Serial Programs

For serial programs, the single processor version of VampirTrace should be linked.

    adaptor ... -o a.out -trace=vtsp -pm=...

When tracing is enabled, a trace record for VampirTrace will be generated at every entry and exit of a region. Enabled performance counters will also be written in the trace records.

    a.out -trace -pm=pm_config

Figure 7 shows the performance counter values related to the regions of the program along a timeline.

**Figure 7:** Visualization of performance counters with VAMPIR.
$\includegraphics[height=84mm]{vampir_counter.eps}$

The ADAPTOR instrumenation also guarantees that VAMPIR finds the corresponding source file if the user wants to see the source code of a region (see Figure 8).

**Figure 8:** Display of source file for an identified activity (region).
$\includegraphics[height=84mm]{vampir_source.eps}$

7.3.2 Visualization for MPI Programs

The main purpose of VAMPIR is the visualization of MPI program. Also for MPI programs all the functionalities can be used together. A little bit attention has been paid to a correct linking step:

    adaptor -hpf -trace=vt -pm=perfctr -dm=mpi

The flag -hpf is currently needed (without explanation here), otherwise there is something missing.
The flag -trace=vt links the ADAPTOR vampir interface and the VampirTrace library. It is very important that this flag is used before the flag that links the MPI library.
The flag -pm=perfctr links the ADAPTOR interface for performance counters and the corresponding library.
The flag -dm=mpi links the ADAPTOR MPI interface and the MPI library.

When running the application without any flags (-pm or -trace), only the Vampir visualization for MPI is enabled (and it is likely that there is the only distinction between user code and MPI calls). By setting the flag -trace the generated trace records for the regions allow to identify regions and MPI calls.

    mpirun -np 2 executable -trace

If performance counting is enabled, there will be additional trace records for the performance counters that can also be visualized.

    mpirun -np 2 executable -trace -pm=pm_config

Figure 9 shows the visualization of a performance counter FP_INS for two processes. The display shows that the floating-point rate is rather different on the two processors.

**Figure 9:** Performance counter values of multiple processes.
$\includegraphics[height=84mm]{counters.eps}$

Next: 8 Profiling of Communication Up: ADAPTOR Profiling Guide Previous: 6 Counting Performance Events Contents

Thomas Brandes 2004-03-19