[DFTB-Plus-User] mpiprocs and ompthreads setting

[Ben Hourahine]

Hello Zhang,

yes, both of these can change calculation speed (both wall clock and
also cpu time used). Have a look at

https://dftbplus-recipes.readthedocs.io/en/stable/parallel/index.html

for some discussion about openMP (most of the theory is relevant for MPI
parallel as well)

OMP_NUM_THREADS is the shell variable for setting number of threads,
while ompthreads is an instruction to the queueing system to set that
variable as well as  management tasks over job scheduling. There are
some more pbs examples at

https://www2.cisl.ucar.edu/resources/computational-systems/cheyenne/running-jobs/pbs-pro-job-script-examples

including a pure openMP parallel calculation. Depending on how you have
compiled DFTB+, it may be parallelised with openMP, MPI or both.
Generally, mixed openMP and MPI does not give good performance.

For 3x10^4 atoms you may get some memory limitations, as that will
require ~1Tb of memory if you use an eigensolver. If your system is
suitable you might want to explore using one of the alternative solvers
from ELSI (these are only available for MPI parallel code and need to be
enabled at compile time).

Regards

Ben


On 19/06/2020 13:53, jsxz wrote:
> Hi All,
>
> May I ask how to set the mpipros and ompthreads in the submission
> script? Do these two parameters affect the calculation speed?
> Is the ompthreads the same as OMP_NUM_THREADS?
> My system is about 30000 atoms.  I plan to calculate the molecular
> dynamics. The simulation time is aout 10 ps. 
> The documentation below is my submission script. The select is the
> number of nodes. The number of CPU per node is 24.
>
> #!/bin/bash
> #PBS -N dftb
> #PBS -q normal
> #PBS -P 13101xxx
> #PBS -l select=30:ncpus=24:mpiprocs=12:ompthreads=24:mem=96gb
>
> #PBS -l walltime=24:00:00
> #PBS -j oe
>
>
> module load composerxe/2016.1.150
> module load intelmpi
> cd "$PBS_O_WORKDIR"
> export OMP_NUM_THREADS=12
>
> /home/users/xz/software/DFTB/18.2/bin/dftb+ <  dftb_in.hsd > out
>
> Thanks a lot.
>
> Best Regards,
> Chao Zhang
>
>
>  
>
>
> _______________________________________________
> DFTB-Plus-User mailing list
> DFTB-Plus-User at mailman.zfn.uni-bremen.de
> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.zfn.uni-bremen.de%2Fcgi-bin%2Fmailman%2Flistinfo%2Fdftb-plus-user&data=02%7C01%7Cbenjamin.hourahine%40strath.ac.uk%7C4ba8cf7c18584f7e39d308d8145462e4%7C631e0763153347eba5cd0457bee5944e%7C0%7C0%7C637281700548282867&sdata=uOmltkQ7UwTbGCYNaI0SwneGDi6Wch8c2TlYBQSdrxc%3D&reserved=0

-- 
        Dr.  B.  Hourahine,  Senior  lecturer
            SUPA, Department  of  Physics,
             University  of  Strathclyde,
              John  Anderson  Building,
          107 Rottenrow, Glasgow G4 0NG, UK.                    
  +44 141 548 2325, benjamin.hourahine at strath.ac.uk

The  Department is  a partner  in SUPA,  the Scottish
            Universities Physics Alliance

The University  of Strathclyde  is a  charitable body,
       registered in Scotland, number SC015263

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.zfn.uni-bremen.de/pipermail/dftb-plus-user/attachments/20200622/9d5476df/attachment.htm>