[DFTB-Plus-User] dftb+.mpi as spawned process

Alex A. Schmidt aas at ufsm.br
Sun Dec 21 22:27:38 CET 2014


Dear Bálint,

I am developing this fortran mpi genetic algorithm code to find the optimal
configuration
of clusters of atoms. So far, serial dftb+ (evoked by "call system") has
been used quite
successfully as an external app to evaluate the cluster energy. I am now
testing dftb+.mpi
with the same purpose.

The proper way to call an mpi app from another mpi app would be through
"mpi_comm_spawn".
According to a few tests I made, this does work for dftb+.mpi but not
entirely. It seems
dftb+.mpi is not, for lack of a better term, "spawn ready" meaning that
dftb+.mpi should
be aware it is being called from mpi_comm_spawn and, in that case, be able
to match a call
like "call mpi_comm_disconnect(intercomm,error)" from the spawning process.
This call is
necessary to hold the spawning process until the spawnee processes are
finished and further
action can be made by the spawing process. In my case, it would be reading
the output files
from dftb+.mpi.

As far as I know, all that is necessary for dftb+.mpi to become "spawn
ready" would be to
add something like

   integer :: mpi_comm_parent

   call mpi_comm_get_parent(mpi_comm_parent,error0)

   if (mpi_comm_parent.ne.mpi_comm_null) then
       call mpi_comm_disconnect(mpi_comm_parent,error0)
   endif

right before "call mpi_finalize(error0)". Please, note that testing the
value of mpi_comm_parent
as above make the call to mpi_comm_disconnect safe in case dftb+.mpi is
called from "mpirun".

Well, I did add the lines above to "mpifx_finalize.F90" file and the
behaviour of dftb+.mpi
went as expected.

Besides that, there is problem with stdout of dftb+.mpi if it is called
from mpi_comm_spawn.
Uncontrollable output from spawnee processes would be a nuisance since it
would get mixed
with the output of the spawning processes.

I have discussed i/o redirection for spawn processes with the opempi guys
and, at the moment
(and for the foreseeable future), there seems to be no way to specify i/o
redirection for spawnee
processes, i.e., adding args like " > /dev/null" to mpi_comm_spawn does not
work. If I understood
well, i/o redirection is handled by the shell of the fork processes and not
by the args passed to the
app mpirun will execute on the spawnee side...

I notice that dftb+.mpi subroutine "mpiglobal_init()" do a test on the rank
of processes and stdout
is redirected unconditionally to /dev/null in case the process is not
master. Altering the test to

        call mpi_comm_get_parent(mpi_comm_parent,error0)

    if (mympi%master.and.mpi_comm_parent.eq.mpi_comm_null) then
      stdout = output_unit
    else
      stdout = 1
      open(stdout, file="/dev/null", action="write")
    end if

do get stdout suppressed for the master rank if dftb+.mpi is a spawnee
process. But I still
get messages like

[gaia:21165] [[31462,1],0] FORKING HNP: orted --hnp --set-sid --report-uri
16 --singleton-died-pipe 17 -mca state_novm_select 1 -mca ess_base_jobid
2061893632

Note: The following floating-point exceptions are signalling:
IEEE_OVERFLOW_FLAG IEEE_UNDERFLOW_FLAG IEEE_DENORMAL

which I don't know where they come from.

May I suggest adding the above changes (or something of the sort) to
dftb+.mpi ?
It would certainly interest other dftb+.mpi users as they might use
mpi_comm_spawn as well.

Best regards,

Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.zfn.uni-bremen.de/pipermail/dftb-plus-user/attachments/20141221/7066147e/attachment.html>


More information about the DFTB-Plus-User mailing list