[DFTB-Plus-User] DFTB-Plus-User Digest, Vol 58, Issue 11

David Furman sirok4 at gmail.com
Sat Jun 22 18:34:08 CEST 2019 

Hi Ben,

1. I use libblas-dev 3.6.0 (This  implementation is the Fortran 77
reference implementation found  at netlib)
2. I run each instance from the same shell, 'echo $OMP_NUM_THREADS' yields:
1
3. Yes it is the correct context. I try to run a Replica-Exchange MD using
4 replicas. As far as I know, i-pi distributes the replicas to clients
(DFTB+ in this case) so that I can run 4 concurrent replicas. Since I have
a relatively large system in terms of number of atoms, each replica takes a
significant amount of time. Hence I was thinking of parallelizing each
replica on 2 or more MPI cores (i.e. parallelize each DFTB+ instance). But,
then I encounter this weird "bottleneck".
Regarding your suggestion: I'm not sure how running 8 serial instances is
of help in my case.

I appreciate your help!



On Sat, Jun 22, 2019 at 2:54 PM <
dftb-plus-user-request at mailman.zfn.uni-bremen.de> wrote:

> Send DFTB-Plus-User mailing list submissions to
>         dftb-plus-user at mailman.zfn.uni-bremen.de
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> https://mailman.zfn.uni-bremen.de/cgi-bin/mailman/listinfo/dftb-plus-user
>
> or, via email, send a message with subject or body 'help' to
>         dftb-plus-user-request at mailman.zfn.uni-bremen.de
>
> You can reach the person managing the list at
>         dftb-plus-user-owner at mailman.zfn.uni-bremen.de
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of DFTB-Plus-User digest..."
>
>
> Today's Topics:
>
>    1. Possible bug in DFTB+ with MPI (David Furman)
>    2. Re: problem with installation (Farzad Molani)
>    3. Re: Possible bug in DFTB+ with MPI (Ben Hourahine)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 22 Jun 2019 14:41:01 +0100
> From: David Furman <sirok4 at gmail.com>
> To: dftb-plus-user at mailman.zfn.uni-bremen.de
> Subject: [DFTB-Plus-User] Possible bug in DFTB+ with MPI
> Message-ID:
>         <CADtBuo94A729tJFsTW5ev69sEs4qVNM=
> qOXkjMKFTS8_f1N8pg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
>  Hi all,
> I have a question concerning the running of DFTB+ with MPI. There seems to
> be a problem when running more than one DFTB+ instance (with MPI
> parallelization) on the same system.
>
> When I run 4 instances (separate folders) of dftb+ with 2 cores each (i.e.
> 8 cores in total):
> mpirun.openmpi -np 2 dftb+, each %CPU usage drops to 50%.
>
> This is the 'top' output:
> ========================================================
>  35426 user    20   0  423384  40404  18248 R  50.5  0.0   0:37.69 dftb+
>  35434 user    20   0  423080  39720  18248 R  50.2  0.0   0:05.70 dftb+
>  35396 user    20   0  422044  38660  17644 R  49.8  0.0   1:21.35 dftb+
>  35402 user    20   0  423384  40460  18304 R  49.8  0.0   1:09.54 dftb+
>  35403 user    20   0  422044  38872  17848 R  49.8  0.0   1:16.15 dftb+
>  35427 user    20   0  422044  39048  18032 R  49.8  0.0   0:32.93 dftb+
>  35435 user    20   0  421732  38164  17716 R  49.8  0.0   0:09.07 dftb+
>  35395 user    20   0  423384  40360  18204 R  49.5  0.0   1:13.18 dftb+
> =========================================================
>
> Whereas, when I run one instance with 8 cores, the efficiency is 100% as
> expected:
>
> =========================================================
>  35837 user    20   0  441868  35196  19112 R 100.3  0.0   0:04.90 dftb+
>  35828 user    20   0  442580  35676  19120 R 100.0  0.0   0:04.83 dftb+
>  35829 user    20   0  442544  36024  19432 R 100.0  0.0   0:04.91 dftb+
>  35830 user    20   0  441912  35356  19360 R 100.0  0.0   0:04.91 dftb+
>  35831 user    20   0  441688  35048  19304 R 100.0  0.0   0:04.90 dftb+
>  35833 user    20   0  441904  35656  19500 R 100.0  0.0   0:04.91 dftb+
>  35841 user    20   0  441516  34824  19076 R 100.0  0.0   0:04.89 dftb+
>  35843 user    20   0  441224  34456  18868 R  99.7  0.0   0:04.89 dftb+
> =========================================================
>
> I run both cases with OMP_NUM_THREADS=1.
> with open-mpi 1.10.2 and gcc 5.4.0.
>
> Could anyone give a hint about what is wrong?
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mailman.zfn.uni-bremen.de/pipermail/dftb-plus-user/attachments/20190622/96b0d502/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Sat, 22 Jun 2019 21:42:46 +0800
> From: Farzad Molani <f.molani at gmail.com>
> To: "User list for DFTB+ related questions"
>         <dftb-plus-user at mailman.zfn.uni-bremen.de>
> Subject: Re: [DFTB-Plus-User] problem with installation
> Message-ID:
>         <CAJY0Q+obk-e8iUOJ_986DS4tWpWG4kAZ99a=
> 6qNstswS0myifg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> I know,  but I don't know how can I unstall arpack?
> The package downloaded and now I need help to install it.
> Please note that, I don't permit to login as a root.
>
> On Saturday, June 22, 2019, RICCARDO ROZZA <
> rozza.riccardo at studium.unict.it>
> wrote:
>
> > Dear Farzad,
> >
> > It seems that you need to install the arpack library (ld: cannot find
> > -larpack).
> > All the best,
> >
> > Riccardo.
> >
> > On 22 Jun 2019, at 10:01, Farzad Molani <f.molani at gmail.com> wrote:
> >
> >
> > Dear DFTB users,
> > I want to install the newest version of DFTB+. The source and executable
> > files from http://www.dftbplus.org/download/dftb-stable/  have been
> > downloaded and untarred. I try to install the software based on
> > instructions on INSTALL.rst, but I got the following error after the
> make.
> > .
> > .
> > .
> > .
> > w.o dispuffdata.o inputdata.o pmlocalisation.o repcont.o repspline.o
> > reppoly.o xlbomd.o extlagrangian.o linresp.o linrespgrad.o qm.o
> > linrespcommon.o arpack.o eigensolver.o typegeometry.o simplemixer.o
> > numderivs2.o potentials.o thirdorder.o charges.o dftbplusu.o geoopt.o
> > steepdesc.o gdiis.o orbitalequiv.o broydenmixer.o stress.o etemp.o
> > hermite.o factorial.o intrinsicpr.o mixer.o andersonmixer.o
> > elecconstraints.o angmomentum.o eigenvects.o parser.o hsdutils2.o
> > unitconversion.o oldcompat.o typegeometryhsd.o oldskdata.o
> densitymatrix.o
> > populations.o repulsive.o spinorbit.o forces.o  -L/home/fmolani/DFTB+/
> > dftbplus-18.2/_build/external/xmlf90 -lxmlf90  -L/home/fmolani/DFTB+/
> > dftbplus-18.2/_build/external/fsockets -lfsockets
> > -L/apps/intel/2015/composer_xe_2015.3.187/mkl/lib/intel64
> > -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core  -larpack -liomp5
> > ld: cannot find -larpack
> > make[1]: *** [dftb+] Error 1
> > make[1]: Leaving directory `/home/fmolani/DFTB+/dftbplus-
> > 18.2/_build/prog/dftb+'
> > make: *** [dftb+] Error 2
> >
> > I would appreciate to help me to solve the problem.
> > Best regards, Farzad
> > --
> > Farzad Molani
> > Postdoctoral Research Fellow
> > School of Pharmaceutical Science and Technology
> > Tianjin University
> > Building 24, Room A205
> > 92 Weijin Road, Nankai District
> > <
> https://www.google.com/maps/search/92+Weijin+Road,+Nankai+District+%0D%0A+P.+R.+China?entry=gmail&source=g
> >
> > P. R. China
> > <
> https://www.google.com/maps/search/92+Weijin+Road,+Nankai+District+%0D%0A+P.+R.+China?entry=gmail&source=g
> >
> > Tel.: +8617526526420
> > Email: f.molani at gmail.com and f.molani at tju.edu.cn
> > <fmolani at dena.kntu.ac.ir>
> >
> >
> > --
> > Farzad Molani
> > Postdoctoral Research Fellow
> > School of Pharmaceutical Science and Technology
> > Tianjin University
> > Building 24, Room A205
> > 92 Weijin Road, Nankai District
> > <
> https://www.google.com/maps/search/92+Weijin+Road,+Nankai+District+%0D%0A+P.+R.+China?entry=gmail&source=g
> >
> > P. R. China
> > <
> https://www.google.com/maps/search/92+Weijin+Road,+Nankai+District+%0D%0A+P.+R.+China?entry=gmail&source=g
> >
> > Tel.: +8617526526420
> > Email: f.molani at gmail.com and f.molani at tju.edu.cn
> > <fmolani at dena.kntu.ac.ir>
> > _______________________________________________
> > DFTB-Plus-User mailing list
> > DFTB-Plus-User at mailman.zfn.uni-bremen.de
> >
> https://mailman.zfn.uni-bremen.de/cgi-bin/mailman/listinfo/dftb-plus-user
> >
> >
> >
>
> --
> Farzad Molani
> Postdoctoral Research Fellow
> School of Pharmaceutical Science and Technology
> Tianjin University
> Building 24, Room A205
> 92 Weijin Road, Nankai District
> P. R. China
> Tel.: +8617526526420
> Email: f.molani at gmail.com and f.molani at tju.edu.cn <fmolani at dena.kntu.ac.ir
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mailman.zfn.uni-bremen.de/pipermail/dftb-plus-user/attachments/20190622/aae9eaba/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 3
> Date: Sat, 22 Jun 2019 14:54:40 +0100
> From: Ben Hourahine <benjamin.hourahine at strath.ac.uk>
> To: dftb-plus-user at mailman.zfn.uni-bremen.de
> Subject: Re: [DFTB-Plus-User] Possible bug in DFTB+ with MPI
> Message-ID: <16da7e6a-92dd-2184-7a98-067e751cdb14 at strath.ac.uk>
> Content-Type: text/plain; charset="utf-8"
>
> Hello David,
>
> thoughts so far:
>
> 1) Which blas are you using? Sometimes openBlas steals extra threads, is
> your top output shown with thread reporting enabled (H)?
>
> 2) Are you definitely getting the OMP_NUM_THREADS shell variable to each
> of the instances?
>
> 3) If this is related to the i-PI mailing list question, is there any
> reason not to run 8 serial instances with 1 thread each? The MPI
> parallelism overhead on just one process is probably not worth it.
>
> Regards
>
> Ben
>
> On 22/06/2019 14:41, David Furman wrote:
> > Hi all,
> > I have a question concerning the running of DFTB+ with MPI. There
> > seems to be a problem when running more than one DFTB+ instance (with
> > MPI parallelization) on the same system.
> >
> > When I run 4 instances (separate folders) of dftb+ with 2 cores each
> > (i.e. 8 cores in total):
> > mpirun.openmpi -np 2 dftb+, each %CPU usage drops to 50%.
> >
> > This is the 'top' output:
> > ========================================================
> >  35426 user    20   0  423384  40404  18248 R  50.5  0.0   0:37.69 dftb+
> >  35434 user    20   0  423080  39720  18248 R  50.2  0.0   0:05.70 dftb+
> >  35396 user    20   0  422044  38660  17644 R  49.8  0.0   1:21.35 dftb+
> >  35402 user    20   0  423384  40460  18304 R  49.8  0.0   1:09.54 dftb+
> >  35403 user    20   0  422044  38872  17848 R  49.8  0.0   1:16.15 dftb+
> >  35427 user    20   0  422044  39048  18032 R  49.8  0.0   0:32.93 dftb+
> >  35435 user    20   0  421732  38164  17716 R  49.8  0.0   0:09.07 dftb+
> >  35395 user    20   0  423384  40360  18204 R  49.5  0.0   1:13.18 dftb+
> > =========================================================
> >
> > Whereas, when I run one instance with 8 cores, the efficiency is 100%
> > as expected:
> >
> > =========================================================
> >  35837 user    20   0  441868  35196  19112 R 100.3  0.0   0:04.90 dftb+
> >  35828 user    20   0  442580  35676  19120 R 100.0  0.0   0:04.83 dftb+
> >  35829 user    20   0  442544  36024  19432 R 100.0  0.0   0:04.91 dftb+
> >  35830 user    20   0  441912  35356  19360 R 100.0  0.0   0:04.91 dftb+
> >  35831 user    20   0  441688  35048  19304 R 100.0  0.0   0:04.90 dftb+
> >  35833 user    20   0  441904  35656  19500 R 100.0  0.0   0:04.91 dftb+
> >  35841 user    20   0  441516  34824  19076 R 100.0  0.0   0:04.89 dftb+
> >  35843 user    20   0  441224  34456  18868 R  99.7  0.0   0:04.89 dftb+
> > =========================================================
> >
> > I run both cases with OMP_NUM_THREADS=1.
> > with open-mpi 1.10.2 and gcc 5.4.0.
> >
> > Could anyone give a hint about what is wrong?
> >
> >
> > _______________________________________________
> > DFTB-Plus-User mailing list
> > DFTB-Plus-User at mailman.zfn.uni-bremen.de
> >
> https://mailman.zfn.uni-bremen.de/cgi-bin/mailman/listinfo/dftb-plus-user
>
> --
>         Dr.  B.  Hourahine,  Senior  lecturer
>             SUPA, Department  of  Physics,
>              University  of  Strathclyde,
>               John  Anderson  Building,
>           107 Rottenrow, Glasgow G4 0NG, UK.
>   +44 141 548 2325, benjamin.hourahine at strath.ac.uk
>
> The  Department is  a partner  in SUPA,  the Scottish
>             Universities Physics Alliance
>
> The University  of Strathclyde  is a  charitable body,
>        registered in Scotland, number SC015263
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mailman.zfn.uni-bremen.de/pipermail/dftb-plus-user/attachments/20190622/bf90b2fb/attachment.html
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> DFTB-Plus-User mailing list
> DFTB-Plus-User at mailman.zfn.uni-bremen.de
> https://mailman.zfn.uni-bremen.de/cgi-bin/mailman/listinfo/dftb-plus-user
>
> ------------------------------
>
> End of DFTB-Plus-User Digest, Vol 58, Issue 11
> **********************************************
>


-- 

David Furman, PhD, MRSC
Herchel Smith Research Fellow
Department of Chemistry
University of Cambridge
Lensfield Road
CB2 1EW
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.zfn.uni-bremen.de/pipermail/dftb-plus-user/attachments/20190622/4a05b9ea/attachment-0001.html>

More information about the DFTB-Plus-User mailing list