Global Options

This section describes the global options of the Intel® MPI Library's Hydra process manager. Global options are applied to all arguments sets in the launch command. Argument sets are separated by a colon ':'.

-usize <usize>

Use this option to set MPI_UNIVERSE_SIZE, which is available as an attribute of the MPI_COMM_WORLD.

`<size>`	Define the universe size
`SYSTEM`	Set the size equal to the number of cores passed to mpiexec through the hostfile or the resource manager.
`INFINITE`	Do not limit the size. This is the default value.
<value>	Set the size to a numeric value ≥ 0.

-hostfile <hostfile> or -f <hostfile>

Use this option to specify host names on which to run the application. If a host name is repeated, this name is used only once.

See also the I_MPI_HYDRA_HOST_FILE environment variable for more details.

Note

Use the -perhost, -ppn, -grr, and -rr options to change the process placement on the cluster nodes.

Use the -perhost, -ppn, and -grr options to place consecutive MPI processes on every host using the round robin scheduling.
Use the -rr option to place consecutive MPI processes on different hosts using the round robin scheduling.

-machinefile <machine file> or -machine <machine file>

Use this option to control process placement through a machine file. To define the total number of processes to start, use the -n option. To pin processes within a machine, use the option binding=map in the machine file. For example:

> type machinefile
node0:2 binding=map=0,3
node1:2 binding=map=[2,8]
node0:1 binding=map=8

For details, see the -binding option description.

-genv <ENVVAR> <value>

Use this option to set the <ENVVAR> environment variable to the specified <value> for all MPI processes.

-genvall

Use this option to enable propagation of all environment variables to all MPI processes.

-genvnone

Use this option to suppress propagation of any environment variables to any MPI processes.

-genvexcl <list of env var names>

Use this option to suppress propagation of the listed environment variables to any MPI processes.

-genvlist <list>

Use this option to pass a list of environment variables with their current values. <list> is a comma separated list of environment variables to be sent to all MPI processes.

-pmi-connect <mode>

Use this option to choose the caching mode of process management interface (PMI) message. Possible values for <mode> are:

`<mode>`	The caching mode to be used
`nocache`	Do not cache PMI messages.
`cache`	Cache PMI messages on the local `pmi_proxy` management processes to minimize the number of PMI requests. Cached information is automatically propagated to child management processes.
`lazy-cache`	`cache` mode with on-request propagation of the PMI information.
`alltoall`	Information is automatically exchanged between all `pmi_proxy` before any get request can be done. This is the default mode.

See the I_MPI_HYDRA_PMI_CONNECT environment variable for more details.

-perhost <# of processes >, -ppn <# of processes >, or -grr <# of processes>

Use this option to place the specified number of consecutive MPI processes on every host in the group using round robin scheduling. See the I_MPI_PERHOST environment variable for more details.

Note

When running under a job scheduler, these options are ignored by default. To be able to control process placement with these options, disable the I_MPI_JOB_RESPECT_PROCESS_PLACEMENT variable.

-rr

Use this option to place consecutive MPI processes on different hosts using the round robin scheduling. This option is equivalent to "-perhost 1". See the I_MPI_PERHOST environment variable for more details.

-trace-pt2pt

Use this option to collect the information about point-to-point operations using Intel® Trace Analyzer and Collector. The option requires that your application be linked against the Intel® Trace Collector profiling library.

-trace-collectives

Use this option to collect the information about collective operations using Intel® Trace Analyzer and Collector. The option requires that your application be linked against the Intel® Trace Collector profiling library.

Note

Use the -trace-pt2pt and -trace-collectives to reduce the size of the resulting trace file or the number of message checker reports. These options work with both statically and dynamically linked applications.

-configfile <filename>

Use this option to specify the file <filename> that contains the command-line options. Blank lines and lines that start with '#' as the first character are ignored.

-branch-count <num>

Use this option to restrict the number of child management processes launched by the Hydra process manager, or by each pmi_proxy management process.

See the I_MPI_HYDRA_BRANCH_COUNT environment variable for more details.

-pmi-aggregate or -pmi-noaggregate

Use this option to switch on or off, respectively, the aggregation of the PMI requests. The default value is -pmi-aggregate, which means the aggregation is enabled by default.

See the I_MPI_HYDRA_PMI_AGGREGATE environment variable for more details.

-nolocal

Use this option to avoid running the <executable> on the host where mpiexec is launched. You can use this option on clusters that deploy a dedicated master node for starting the MPI jobs and a set of dedicated compute nodes for running the actual MPI processes.

-hosts <nodelist>

Use this option to specify a particular <nodelist> on which the MPI processes should be run. For example, the following command runs the executable a.out on the hosts host1 and host2:

> mpiexec -n 2 -ppn 1 -hosts host1,host2 test.exe

Note

If <nodelist> contains only one node, this option is interpreted as a local option. See Local Options for details.

-iface <interface>

Use this option to choose the appropriate network interface. For example, if the IP emulation of your InfiniBand* network is configured to ib0, you can use the following command.

> mpiexec -n 2 -iface ib0 test.exe

See the I_MPI_HYDRA_IFACE environment variable for more details.

-l, -prepend-rank

Use this option to insert the MPI process rank at the beginning of all lines written to the standard output.

-s <spec>

Use this option to direct standard input to the specified MPI processes.

Arguments

`<spec>`	Define MPI process ranks
`all`	Use all processes.
`<l>,<m>,<n>`	Specify an exact list and use processes `<l>`, `<m>` and `<n>` only. The default value is zero.
`<k>,<l>-<m>,<n>`	Specify a range and use processes `<k>`, `<l>` through `<m>`, and `<n>`.

-noconf

Use this option to disable processing of the mpiexec.hydra configuration files.

-ordered-output

Use this option to avoid intermingling of data output from the MPI processes. This option affects both the standard output and the standard error streams.

Note

When using this option, end the last output line of each process with the end-of-line '\n' character. Otherwise the application may stop responding.

-path <directory>

Use this option to specify the path to the executable file.

-version or -V

Use this option to display the version of the Intel® MPI Library.

-info

Use this option to display build information of the Intel® MPI Library. When this option is used, the other command line arguments are ignored.

-delegate

Use this option to enable the domain-based authorization with the delegation ability. See User Authorization for details.

-impersonate

Use this option to enable the limited domain-based authorization. You will not be able to open files on remote machines or access mapped network drives. See User Authorization for details.

-localhost

Use this option to explicitly specify the local host name for the launching node.

-localroot

Use this option to launch the root process directly from mpiexec if the host is local. You can use this option to launch GUI applications. The interactive process should be launched before any other process in a job. For example:

> mpiexec -n 1 -host <host2> -localroot interactive.exe : -n 1 -host <host1> background.exe

-localonly

Use this option to run an application on the local node only. If you use this option only for the local node, the Hydra service is not required.

-register

Use this option to encrypt the user name and password to the registry.

-remove

Use this option to delete the encrypted credentials from the registry.

-validate

Validate the encrypted credentials for the current host.

-whoami

Use this option to print the current user name.

-map <drive:\\host\share>

Use this option to create network mapped drive on nodes before starting executable. Network drive will be automatically removed after the job completion.

-mapall

Use this option to request creation of all user created network mapped drives on nodes before starting executable. Network drives will be automatically removed after the job completion.

-logon

Use this option to force the prompt for user credentials.

-noprompt

Use this option to suppress the prompt for user credentials.

-port/-p

Use this option to specify the port that the service is listening on.

-verbose or -v

Use this option to print debug information from mpiexec, such as:

Service processes arguments
Environment variables and arguments passed to start an application
PMI requests/responses during a job life cycle

See the I_MPI_HYDRA_DEBUG environment variable for more details.

-print-rank-map

Use this option to print out the MPI rank mapping.

-print-all-exitcodes

Use this option to print the exit codes of all processes.

`<arg>`	String parameter
`ssh`	Use secure shell. This is the default value.
`rsh`	Use remote shell.
`pdsh`	Use parallel distributed shell.
`pbsdsh`	Use Torque* and PBS* `pbsdsh` command.
`slurm`	Use SLURM* `srun` command.
`ll`	Use LoadLeveler* `llspawn.stdio` command.
`lsf`	Use LSF `blaunch` command.
`sge`	Use Univa* Grid Engine* `qrsh` command.

-binding

Use this option to pin or bind MPI processes to a particular processor and avoid undesired process migration. In the following syntax, the quotes may be omitted for a one-member list. Each parameter corresponds to a single pinning property.

Note

This option is related to the family of I_MPI_PIN environment variables, which have higher priority than the -binding option. Hence, if any of these variables are set, the option is ignored.

This option is supported on both Intel® and non-Intel microprocessors, but it may perform additional optimizations for Intel microprocessors than it performs for non-Intel microprocessors.

Syntax

-binding "<parameter>=<value>[;<parameter>=<value> ...]"

Parameters

Parameter
`pin`	Pinning switch
Values
`enable \| yes \| on \| 1`	Turn on the pinning property. This is the default value
`disable \| no \| off \| 0`	Turn off the pinning property

Parameter
`cell`	Pinning resolution
Values
`unit`	Basic processor unit (logical CPU)
`core`	Processor core in multi-core system

Parameter
`map`	Process mapping
Values
`spread`	The processes are mapped consecutively to separate processor cells. Thus, the processes do not share the common resources of the adjacent cells.
`scatter`	The processes are mapped to separate processor cells. Adjacent processes are mapped upon the cells that are the most remote in the multi-core topology.
`bunch`	The processes are mapped to separate processor cells by #processes/#sockets processes per socket. Each socket processor portion is a set of the cells that are the closest in the multi-core topology.
`p0,p1,...,pn`	The processes are mapped upon the separate processors according to the processor specification on the `p0`,`p1`,...,`pn` list: the `i`^th process is mapped upon the processor `pi`, where `pi` takes one of the following values: processor number like n range of processor numbers like n-m `-1` for no pinning of the corresponding process
`[m0,m1,...,mn]`	The `i`^th process is mapped upon the processor subset defined by `m_i` hexadecimal mask using the following rule: The `j`^th processor is included into the subset `m_i` if the `j`^th bit of `m_i` equals `1`.

Parameter
`domain`	Processor domain set on a node
Values
`cell`	Each domain of the set is a single processor cell (unit or core).
`core`	Each domain of the set consists of the processor cells that share a particular core.
`cache1`	Each domain of the set consists of the processor cells that share a particular level 1 cache.
`cache2`	Each domain of the set consists of the processor cells that share a particular level 2 cache.
`cache3`	Each domain of the set consists of the processor cells that share a particular level 3 cache.
`cache`	The set elements of which are the largest domains among cache1, cache2, and cache3
`socket`	Each domain of the set consists of the processor cells that are located on a particular socket.
`node`	All processor cells on a node are arranged into a single domain.
`<size>[:<layout>]`	Each domain of the set consists of `<size>` processor cells. `<size>` may have the following values: `auto` - domain size = #cells/#processes `omp` - domain size = `OMP_NUM_THREADS` environment variable value positive integer - exact value of the domain size Note Domain size is limited by the number of processor cores on the node. Each member location inside the domain is defined by the optional `<layout>` parameter value: `compact` - as close with others as possible in the multi-core topology `scatter` - as far away from others as possible in the multi-core topology `range` - by BIOS numbering of the processors If `<layout>` parameter is omitted, `compact` is assumed as the value of `<layout>`

Parameter
`order`	Linear ordering of the domains
Values
`compact`	Order the domain set so that adjacent domains are the closest in the multi-core topology
`scatter`	Order the domain set so that adjacent domains are the most remote in the multi-core topology
`range`	Order the domain set according to the BIOS processor numbering

Parameter
`offset`	Domain list offset
Values
`<n>`	Integer number of the starting domain among the linear ordered domains. This domain gets number zero. The numbers of other domains will be cyclically shifted.