The Convex Exemplar version has been specially tuned by Convex Computer Corporation to take advantage of the specific architecture of the Exemplar. In particular, most of the collective communication library has been reimplemented using shared memory algorithms; the result is a significant decrease in latency over implementations layered on top of point-to-point functions.
The environment variable MPI_GLOBMEMSIZE, mentioned above, specifies the size of the shared memory region on each hypernode rather than the total amount of shared memory. On the Exemplar, its default value is 16MB.
A Convex-specific environment variable is MPI_TOPOLOGY. If you specify
setenv MPI_TOPOLOGY <i>,<j>,<k>,<l>,...where the sum of the arguments equals the number of processes specified with -np on the mpirun command line, then the specified number of processes is started on each hypernode. Use of this environment variable is optional; the default behavior (keeping the processes on the same hypernode as much as possible) is usually more beneficial.