[DFTB-Plus-User] Max Number of Atoms
Benjamin Hourahine
benjamin.hourahine at strath.ac.uk
Thu Oct 9 11:30:58 CEST 2008
Hello Marcelo,
> In principle I can set the stack size to
> unlimited, but I'm not sure how it will affect my calculations. After
> setting it to unlimited, do I have to recompile dftb+? Also, when I set this
> new limit, it gets back to its original value (8192KB) by the time I
> reconnect to the machine.
I would suggest setting it as a large but finite value (say between 1-10 Gb
in your case). It is nothing to do with compilation of a code, its an
enviromental variable in unix, which controls part of the memory layout
at run time when the resulting binary is started. You do not need to recompile.
In the case of codes compiled with ifort, the resulting binary places intermediate
data during many fortran matrix operations into the stack, but the binary will fail if
there is not sufficient space to do this. If that is the problem, then increasing
the stack size and restarting the same copy of DFTB+ would work.
The reason it the stack size is re-setting every time you log in, is that on login
the shell that you are using reads various configuration files which set variables
including this value. Unless you over-ride this in your local settings file
(probably ~/.bashrc or ~/.cshrc) you will always get the default stacksize on login.
> However, since my system was
> made of a molecule over a surface (which is remained frozen during the
> calculation) I could workaround my initial problem on calculating this
> structure by dramatically reducing the number of atoms pertaining to the
> surface (which was taken very big in order to check the limits of our
> hardware available).
I don't know any details of your system, but if you are interested in a molecule
above a surface, might I suggest making a periodic model of the surface, in the
shape of a slab, instead of the current cluster of atoms. This might be a more
reasonable geometry for your system, and if you want to look at the molecule on a
macroscopic sized surface. This may be able to give you good results with fewer atoms,
as you then only have a surface one dimension, this geometry is very common in
surface science simulations. The drawback, is that if the model is too small in the
periodic directions, the molecule can interact with images of itself.
>Regarding the eigensolver, as I new user of DFTB+, I have a question: is the
>standard eigensolver the same as divide and conquer? The manual mentions
>three flags for eigensolvers:
No,
Standard = LAPACK dsygv/zhegv (standard QR based eigensolution method)
DivideAndConquer = LAPACK dsygvd/zhegvd
RelativelyRobust = LAPACK routines put together to use a relatively robust solver.
if you do not request a specific eigensolver, you get DivideAndConquer by default.
> I've been using the Standard one so far, but I tested all these three flags
> in a similar system having 1336 (but only 314 were free to move) atoms and
> all of the flags used used 2.7 - 3.0 % of a total of 16Gb memory, so I
> couldn't notice any difference between those eigensolvers related to memory
> usage, however I haven't tested them by comparing total simulation time,
> yet.
Then, you will be using dsygv if you specifically request
Standard {}
The DivideAndConquer solver requires an additional workspace which is similar in size to
the eigenproblem it is solving (but is faster). Hence if you watch the memory use with
top carefully, you will see the same kind of memory use for any of the eigensolvers
for most of the run, but the DivideAndConquer will suddently double the memory
use for (fairly) short time periods.
RelativelyRobust is also fast and does not require a large workspace, but I find it less
stable than Standard or DivideAndConquer.
Cheers
Ben
On Wed, Oct 8, 2008 at 05:59, Benjamin Hourahine <
benjamin.hourahine at strath.ac.uk> wrote:
> Hello Marcelo,
>
> Have you ruled out the stack size issue with ifort, which I also mailed
> you about?
>
> The probable reason for your discrepancy in estimating the memory is that
> both the Hamiltonian and overlap matrices have to be held in memory
> when using the LAPACK generalized eigensolvers. Incidentally, are you
> sure that you aren't using the divide and conquer eigensolver?
>
> Cheers
>
> Ben
>
> Dr. B. Hourahine, Department of Physics, SUPA, University of
> Strathclyde,
> John Anderson Building, 107 Rottenrow, Glasgow G4 0NG, United Kingdom
> Ph +44 141 548 2325 FAX +44 141 552 2891
> benjamin.hourahine at strath.ac.uk
>
> The University of Strathclyde is a Scottish registered charitable body,
> number SC015263
>
>
>
>
> -----Original Message-----
> From: dftb-plus-user-bounces at dftb-plus.info on behalf of Marcelo Zimmer
> Sent: Tue 07/10/2008 15:10
> To: User list for DFTB+ related questions
> Subject: Re: [DFTB-Plus-User] Max Number of Atoms
>
> Dr. B. Hourahine,
>
> Thank you for prompt answer. Actually, by "unexpected error" I meant: the
> job stoped without any error message. For this, I was wondering if the
> problem was caused by lack of memory or any limitation related to vector
> size allocation, which would limit the maximum number of atoms independent
> on hardware availability. Anyway, my calculations rely on the first case
> you've mentioned and I'm convinced the problem is lack of memory, although
> my expectations were that my calculations would require ~10Gb.
>
> Best regards,
>
> Marcelo Zimmer
>
> --
> ===================================================
> Marcelo Zimmer S. Flores
> (PhD Physics Student)
>
> Address:
> Universidade Estadual de Campinas - UNICAMP
> Instituto de Fisica Gleb Wataghin - IFGW
> Grupo de Solidos Organicos e Novos Materiais - GSONM
> Departamento de Fisica Aplicada (DFA) - Room 61
> Campinas - Sao Paulo - Brazil
> 13083-970
> Caixa Postal: 6165
> zimmer at ifi.unicamp.br
> ===================================================
>
> On Mon, Oct 6, 2008 at 19:26, Benjamin Hourahine <
> benjamin.hourahine at strath.ac.uk> wrote:
>
> > Hello Marcelo,
> >
> > I'll need some more information to give an accurate answer, but
> > if I assume all of the atoms have an SP basis, you have SCC
> > turned on, are using molecular boundary conditions (not periodic),
> > no spin polarization, and the 'standard' or 'relatively robust'
> > choice of eigensolvers then I'd estimate your memory requirements
> > at about 17 Gb for that system.
> >
> > If you use the default eigensolver (divide and conqer) this would rise
> > to a peak of about 28 Gb when it requires workspace. Similarly, changing
> > any of the above calculational conditions could also modify the memory
> use.
> >
> > This suggests you are running out of memory. What exactly is the
> > 'unexpected error'?
> >
> > Cheers
> >
> > Ben
> >
> > Dr. B. Hourahine, Department of Physics, SUPA, University of
> > Strathclyde,
> > John Anderson Building, 107 Rottenrow, Glasgow G4 0NG, United
> Kingdom
> > Ph +44 141 548 2325 FAX +44 141 552 2891
> > benjamin.hourahine at strath.ac.uk
> >
> > The University of Strathclyde is a Scottish registered charitable body,
> > number SC015263
> >
> >
> >
> >
> > -----Original Message-----
> > From: dftb-plus-user-bounces at dftb-plus.info on behalf of Marcelo Zimmer
> > Sent: Mon 06/10/2008 19:47
> > To: User list for DFTB+ related questions
> > Subject: [DFTB-Plus-User] Max Number of Atoms
> >
> > Hello all DFTB+ users,
> >
> > I`d like to know what is the maximum number of atoms that DFTB+ can carry
> > on
> > calculation for a molecule. I`m trying to compute approx. 8,000 atoms and
> > I`m having unexpected error during the first SCC cycle. I`m trying to run
> > this job under 8 Intel(R) Xeon(R) CPU E5420 @ 2.50GHz with 16Gb of RAM.
> >
> > Thank you in advance,
> >
> > Marcelo Zimmer
> >
> > --
> > ===================================================
> > Marcelo Zimmer S. Flores
> > (PhD Physics Student)
> >
> > Address:
> > Universidade Estadual de Campinas - UNICAMP
> > Instituto de Fisica Gleb Wataghin - IFGW
> > Grupo de Solidos Organicos e Novos Materiais - GSONM
> > Departamento de Fisica Aplicada (DFA) - Room 61
> > Campinas - Sao Paulo - Brazil
> > 13083-970
> > Caixa Postal: 6165
> > zimmer at ifi.unicamp.br
> > ===================================================
> >
> >
> > _______________________________________________
> > DFTB-Plus-User mailing list
> > DFTB-Plus-User at dftb-plus.info
> > http://www.dftb-plus.info/mailman/listinfo/dftb-plus-user
> >
>
>
>
> --
> ===================================================
> Marcelo Zimmer S. Flores
> (PhD Physics Student)
>
> Address:
> Universidade Estadual de Campinas - UNICAMP
> Instituto de Fisica Gleb Wataghin - IFGW
> Grupo de Solidos Organicos e Novos Materiais - GSONM
> Departamento de Fisica Aplicada (DFA) - Room 61
> Campinas - Sao Paulo - Brazil
> 13083-970
> Caixa Postal: 6165
> zimmer at ifi.unicamp.br
> ===================================================
>
>
> _______________________________________________
> DFTB-Plus-User mailing list
> DFTB-Plus-User at dftb-plus.info
> http://www.dftb-plus.info/mailman/listinfo/dftb-plus-user
>
--
===================================================
Marcelo Zimmer S. Flores
(PhD Physics Student)
Address:
Universidade Estadual de Campinas - UNICAMP
Instituto de Fisica Gleb Wataghin - IFGW
Grupo de Solidos Organicos e Novos Materiais - GSONM
Departamento de Fisica Aplicada (DFA) - Room 61
Campinas - Sao Paulo - Brazil
13083-970
Caixa Postal: 6165
zimmer at ifi.unicamp.br
===================================================
More information about the DFTB-Plus-User
mailing list