next up previous contents
Next: 4 Source Code Instrumentation Up: ADAPTOR Profiling Guide Previous: 2 Related Work   Contents

Subsections


3 Source Code Instrumentation for Regions

In contrary to other profiling tools that use debug information, ADAPTOR makes an own source code instrumenation to identify regions in a program for which profiling information can be collected. This kind of instrumentation has been proven less costly and has only very small runtime overhead.

Therefore, the source-to-source translation fadapt has to be applied to the source program(s).

    fadapt -INSTR <source_file>

The option -INSTR itself is an abbreviation for a set of options:

    -instr -prof:SUB -prof:PAR -on:ADP -dsp=no -args=single

Further explanation of all the options for fadapt can be found in the ADAPTOR Users Guide (see [Bra04]).

3.1 Regions

A region is a part of the user program for which profiling information can be collected. Regions must be properly nested, they cannot overlap. Typically, all subprograms can be considered as regions of a program. The following two subroutine calls are used to mark the entry and the exit of a region.

      call DALIB_start_region (region_id, region_name, region_kind,
                               file_name, file_id, line_start)
      ...
      call DALIB end_region (region_id, region_name, line_stop)

A region is characterized by the following items:

If the value of region_id is set to $-1$, the ADAPTOR runtime system will give an unique identification for the region. In this case, the unique identification of the region is given by the file_name (or file_id) and the line_start.

During the execution of the program the runtime system maintains a stack of called regions, one for each process and each thread.

3.2 Subprograms

fadapt generates at every entry and exit of a subprogram a call to the DALIB runtime system. This allows the performance analysis and tracing of subprograms at runtime and can provide the stack of subprogram calls in case of a run time error.

By default, fadapt generates profile calls for every subprogram. Unfortunately, the calls for entering and leaving of regions can still cause a certain overhead that becomes annoying for very small subprograms called within loops. The overhead of the calls might be no longer negligible and trace files could become very large. Therefore it is necessary to provide a mechanism to enable or disable the profiling calls for subprograms.

For the source-to-source translation fadapt there are two flags that disable or enable the generation of profile calls.

     -noprof:SUB   ! switches off the generation of region calls for subprograms
     -prof:SUB     ! switches on the generation of region calls for subprograms

More convenient is the use of ADAPTOR specific directives in the source file that disable the generation of the profile calls (in case of -prof) or enable their generation (in case of -noprof). This directive must appear as a definition statement in the subprogram.

      subroutine SUB1 (...)           function F (...)
!adp$ noprofiling               !adp$ profiling
      ...                             ...
      end subroutine                  end function


3.3 User Regions

Furthermore, ADAPTOR allows to generate profile calls for certain user-defined regions in the program. It provides a directive that must appear before a statement that should be profiled (e.g. a subroutine call or a DO loop) or two directives to be set before and after a block of statements.

!adp$ profile <region_name>
      <stmt>
      ...
!adp$ profile <region_name> begin
      <code lines>
!adp$ end profile <region_name>

All these directives must appear in the execution part of the program. The user-defined name of the region makes it easier to identify performance monitoring information with the corresponding source code region. The region name itself does not have to be unique.

The instrumenation of user regions can be switched on or off by enabling the ADAPTOR specific directives. The default setting depends on the installation but can be seen by the command fadapt -settings.

    fadapt -off:ADP ...     ! disables ADAPTOR directives
    fadapt -on:ADP ...      ! enables ADAPTOR directives

Note: It is mandatory that there are no jumps into or jumps out of the instrumented region. ADAPTOR checks this property during its semantic analysis.

!     correct profiling           !   illegal                        ! illegal
!     jumps within region         !   jump out of region             ! jump into region
 
!adp$ profile LOOP begin           5  continue                   !adp$ profile LOOP begin
   5  continue                  !adp$ profile LOOP begin           5   continue
      B = B + C                       B = B + C                        B = B + C
      A = A - 1                       A = A - 1                        A = A - 1
      if (A > 0) goto 5               if (A > 0) goto 5          !adp$ end profile 
!adp$ end profile               !adp$ end profile                      if (A > 0) goto 5

3.4 Instrumentation of OpenMP Programs

OpenMP programs are instrumented in the same way as serial programs. The runtime system is thread-safe, different threads can enter different regions and profile information is summarzied for the different threads.

For OpenMP programs, there is the additional possibility to instrument the parallel regions of OpenMP programs.

     -noprof:PAR   ! switches off the generation of region calls for parallel regions
     -prof:PAR     ! switches on the generation of region calls for parallel regions

Table 2 shows some performance results for an OpenMP application running two threads on two processors.


Table 2: Performance monitoring of multigrid (class A) for two threads.
  CALLS CALLS WALL_TIME WALL_TIME
  Thread 1 Thread 2 Thread 1 Thread 2
      ms ms
MG 1   12050.209 10647.926
SETUP 2   0.007 0.000
ZERO3 39   1001.228 1000.002
ZERO3_par_1366 39 39 997.180 948.621
ZRAN3 2   2178.360 791.228
ZRAN3_par_1184 2 2 349.018 349.627
COMM3 119   264.275 260.326
COMM3_par_999 119 119 256.794 257.557
NORM2U3 4   228.464 228.194
NORM2U3_par_933 4 4 228.022 228.049
RESID 42   5296.962 5293.624
RESID_par_593 42 42 5079.561 5141.083
MG3P 5   6220.220 6210.890
RPRJ3 35   689.235 687.185
RPRJ3_par_680 35 35 668.683 669.140
PSINV 40   2012.184 2009.854
PSINV_par_522 40 40 1937.589 1938.056
INTERP 35   1057.651 1056.553
INTERP_par_762 35 35 1055.254 1055.685


OpenMP programs can also be compiled by the ADAPTOR system directly by using the translation flag -omp instead of -instr (see also ADAPTOR OpenMP Programmer's Guide [Bra03b]). Instrumentation and profiling works in the same way.

3.5 Instrumentation of MPI Programs

Instrumentation of MPI programs works in the same way as for usual programs. Profile information is collected for the different MPI processes and will be collected at the end of the program.

Attention: The source-to-source translation replaces the calls MPI_INIT and MPI_FINALIZE with corresponding calls to the ADAPTOR runtime system. This is necessary to guarantee that all processes get the same initial information about profiling and that profile information will be collected at the end of the parallel program.

Note: ADAPTOR does not do any profiling for the other MPI routines (e.g. MPI_SEND, MPI_RECV). But of course it is possible to do some own instrumentation for MPI calls.

3.6 Instrumentation of HPF Programs

ADAPTOR instruments also HPF programs when compiled with the ADAPTOR system itself. For HPF translation the translation command -hpf must be used instead of -instr.

A source-to-source translations that only instruments HPF programs is not supported yet.

3.7 Region Information Files

It is possible to give already an unique region id at compile time that avoids the overhead of string comparisons. But in case of separate compilation of multiple files the compiler has to pay attention that the same region id is not given twice. This will be supported by a region information file (rif) that has to be specified for the compilation of all source files.

    fadapt -rif[=<filename>] ...

If only the flag -rif is specified, fadapt takes the file RIF by default as its region information file. Otherwise it takes the file filename for it. For the first compilation, this file will be created automatically. For all further compilations the region information file is read and fadapt will give unique region identifications for the compiled source program that do not conflict with other region identifications that have been given within previous compilations. In the output program the calls will now look like this:

      call DALIB_start_region (22,'GHOST3',0,'hydflo.hpf',0,684)
      ...
      call DALIB_end_region (22,'GHOST3',739)

The region information file of a program using multiple source files looks as follows:

#Files: 30
File: 0 constants.f90
File: 1 DataManagement_Materials2.f90
File: 2 DataManagement_Nodes.f90
File: 3 DataManagement_Elements.f90
File: 4 Process_Management.f90
...
File: 29 remap.f90
#Regions: 157
Region: 1, file=1, lines=81:106, GET_MATERIAL_LIST_ID 0
Region: 2, file=1, lines=137:153, INIT_MATERIAL_LIST 0
...
Region: 156, file=29, lines=1:51, REMAP 0
Region: 157, file=29, lines=36:49, INTERNALNODENR 0

Attention: All source files belonging to one program must be compiled by using the same region information file to guarantee unique region identification. Recompilation of a single source file is possible without recompilation of the other source files.


next up previous contents
Next: 4 Source Code Instrumentation Up: ADAPTOR Profiling Guide Previous: 2 Related Work   Contents
Thomas Brandes 2004-03-19