[DFTB-Plus-User] GPU-accelerated MPI/OpenMP jobs using GFN2-xtb

Jerry Tanoury jerry_tanoury at vrtx.com
Mon May 8 16:53:03 CEST 2023 

Thank you for the details Balint.  I've finally traced my problem to my MAGMA build.  I'll update once I work through that.

Jerry

Gerald J. Tanoury, Ph.D.
Principal Scientific Fellow
Vertex Pharmaceuticals
50 Northern Avenue
Boston, MA 02210
Office: 617-341-6219
Cell: 508-740-4986
jerry_tanoury at vrtx.com
www.vrtx.com

-----Original Message-----
From: DFTB-Plus-User <dftb-plus-user-bounces at mailman.zfn.uni-bremen.de> On Behalf Of Bálint Aradi
Sent: Monday, May 8, 2023 10:48 AM
To: dftb-plus-user at mailman.zfn.uni-bremen.de
Subject: Re: [DFTB-Plus-User] GPU-accelerated MPI/OpenMP jobs using GFN2-xtb

CAUTION: This email originated from outside of Vertex. Do not open links or attachments unless you know the sender and believe it is safe. If you suspect that this email is a potentially malicious email, please click on the Report Phish button.



Dear Jerry,

Strange. I've just tested it now: I took DFTB+ main (commit 783c8b1d) without any modifications of config.cmake or of anything else, configured it via

cmake -DWITH_TBLITE=1 -DWITH_PLUMED=1 -DWITH_GPU=1 ~/sync/dftb_devel/dftbplus/

then built it. With the resulting binary I could run the following input without any problems:

Geometry = GenFormat {
2 C
  H
     1    1    0.00000000000000E+00    0.00000000000000E+00
-3.71762583041627E-01
     2    1    0.00000000000000E+00    0.00000000000000E+00
3.71762583041627E-01
}

Hamiltonian = XTB {
   Method = "GFN2-xTB"
   Solver = Magma {}
}

I've used ifort and icc from oneAPI 2022.1 as compilers.

So, I am somewhat unsure, why this does not work for you.


Best regards,

Bálint





On 04.05.23 14:47, Jerry Tanoury wrote:
> Hi Ben,
> Thank you for the information and the help.  I'm still getting the
> error, however:
>
> ERROR!
> -> Unknown electronic solver
> Path: dftbplusinput/Hamiltonian/xTB/Solver/MAGMA
> Line: 13-13 (File: dftb_in.hsd)
>
> I'm not sure if it's my build or my input file.  I've attached my
> input file.
>
> My build proceeds as follows:
> 1. source dftb_build_setup (file attached, along with config.cmake).
>
> 2. add plumed to PKG_CONFIG_PATH:
>
> export
> PKG_CONFIG_PATH=/cluster/home/tanoury/DFTB+/22.1_dependencies/built/pl
> umed-2.8.1/lib/pkgconfig:/cluster/home/tanoury/DFTB+/22.1_dependencies
> /built/magma-2.7.0/lib/pkgconfig:$PKG_CONFIG_PATH
>
> 3. FC=ifort CC=icc CXX=icpc
> CMAKE_PREFIX_PATH=/cluster/home/tanoury/DFTB+/22.1_dependencies/built/elsi_interface cmake -DCMAKE_INSTALL_PREFIX=/cluster/home/tanoury/DFTB+/dftbplus-22.2 -DWITH_GPU=1 -B _build .
>
> Then, 'cmake --build _build -- -j 8'
>
> Then, 'cmake --install _build'
>
> Best regards,
> Jerry
>
> Gerald J. Tanoury, Ph.D.
> Principal Scientific Fellow
> *Vertex Pharmaceuticals*
> 50 Northern Avenue
> Boston, MA 02210
> Office: 617-341-6219
> Cell: 508-740-4986
> jerry_tanoury at vrtx.com
> http://www.vrtx.com/
> ----------------------------------------------------------------------
> --
> *From:* DFTB-Plus-User
> <dftb-plus-user-bounces at mailman.zfn.uni-bremen.de> on behalf of
> Benjamin Hourahine <benjamin.hourahine at strath.ac.uk>
> *Sent:* Thursday, May 4, 2023 6:48 AM
> *To:* User list for DFTB+ related questions
> <dftb-plus-user at mailman.zfn.uni-bremen.de>
> *Subject:* Re: [DFTB-Plus-User] GPU-accelerated MPI/OpenMP jobs using
> GFN2-xtb
>       *CAUTION:*This email originated from outside of Vertex. Do not
> open links or attachments unless you know the sender and believe it is safe.
> If you suspect that this email is a potentially malicious email,
> *please click on the Report Phish button*.
>
>
>
> Hello Jerry,
>
> We've updated the install information in the source tree (this will be
> in the next release, but for the moment see
> https://github.com/dftbplus/dftbplus/blob/main/INSTALL.rst
> <https://urldefense.com/v3/__https://github.com/dftbplus/dftbplus/blob/main/INSTALL.rst__;!!FRfS_D4!aFGz9DLWwOKgVQmuuzMMaQJTICksj_TyqHCN9O7VQwc45xRT38TiKAg8qDOhA_QKYKkDEZEg1Lf-NFZeG6MURn4d_Y819PjYyoo$> for the extra flags).
>
> Regards
>
> Ben
>
> Dr. B. Hourahine
> Senior Lecturer and Director of Student Support Department of Physics,
> SUPA University of Strathclyde, John Anderson Building,
> 107 Rottenrow, Glasgow G4 0NG, UK.
> +44 141 548 2325, benjamin.hourahine at strath.ac.uk
>
> University of Strathclyde
> *THE QUEEN’S ANNIVERSARY PRIZES*
> *2019 & 2021*
> For Higher and Further Education
> *UNIVERSITY OF THE YEAR*
> *2012 & 2019*
> Times Higher Education
> *SCOTTISH UNIVERSITY OF THE YEAR*
> *2020*
> The Times & The Sunday Times
> The University of Strathclyde is a charitable body, registered in
> Scotland, number SC015263.
> Please consider the environment before printing this e-mail.
>
>
> ----------------------------------------------------------------------
> --
> *From:* DFTB-Plus-User
> <dftb-plus-user-bounces at mailman.zfn.uni-bremen.de> on behalf of Jerry
> Tanoury <jerry_tanoury at vrtx.com>
> *Sent:* 04 May 2023 11:38
> *To:* User list for DFTB+ related questions
> <dftb-plus-user at mailman.zfn.uni-bremen.de>
> *Subject:* Re: [DFTB-Plus-User] GPU-accelerated MPI/OpenMP jobs using
> GFN2-xtb
> CAUTION: This email originated outside the University. Check before
> clicking links or attachments.
>
> Hi Balint,
> Thank you.  I was not aware of needing the flag.  Was there something
> I missed in INSTALL?
>
> Best regards,
>
> Jerry
>
> Gerald J. Tanoury, Ph.D.
> Principal Scientific Fellow
> Vertex Pharmaceuticals
> 50 Northern Avenue
> Boston, MA 02210
> Office: 617-341-6219
> Cell: 508-740-4986
> jerry_tanoury at vrtx.com
> http://www.vrtx.com/ <http://www.vrtx.com/>
>
> -----Original Message-----
> From: DFTB-Plus-User
> <dftb-plus-user-bounces at mailman.zfn.uni-bremen.de>
> On Behalf Of Bálint Aradi
> Sent: Thursday, May 4, 2023 3:43 AM
> To: User list for DFTB+ related questions
> <dftb-plus-user at mailman.zfn.uni-bremen.de>
> Subject: Re: [DFTB-Plus-User] GPU-accelerated MPI/OpenMP jobs using
> GFN2-xtb
>
> CAUTION: This email originated from outside of Vertex. Do not open
> links or attachments unless you know the sender and believe it is
> safe. If you suspect that this email is a potentially malicious email,
> please click on the Report Phish button.
>
>
>
> Dear Jerry,
>
> the same solvers should be supported as with the DFTB Hamiltonian. You
> need a binary built with the -DWITH_GPU=1 flag in order to use the
> Magma solver. The release binaries distributed via Conda or GitHub are
> all built without GPU support.
>
> Best regards,
>
> Bálint
>
>
>
> On 03.05.23 18:44, Jerry Tanoury wrote:
>> Thank you Balint.  For GFN2, what solvers are supported.  I tried 'Solver = MAGMA{}', but I got an error saying 'Unknown electronic solver'.  I was hoping to use GFN2 with GPU acceleration.
>>
>> Best regards,
>> Jerry
>>
>> Gerald J. Tanoury, Ph.D.
>> Principal Scientific Fellow
>> Vertex Pharmaceuticals
>> 50 Northern Avenue
>> Boston, MA 02210
>> Office: 617-341-6219
>> Cell: 508-740-4986
>> jerry_tanoury at vrtx.com
>> http://www.vrtx.com/ <http://www.vrtx.com/>
>>
>> -----Original Message-----
>> From: DFTB-Plus-User
>> <dftb-plus-user-bounces at mailman.zfn.uni-bremen.de> On Behalf Of
>> Bálint Aradi
>> Sent: Wednesday, May 3, 2023 10:46 AM
>> To: dftb-plus-user at mailman.zfn.uni-bremen.de
>> Subject: Re: [DFTB-Plus-User] GPU-accelerated MPI/OpenMP jobs using
>> GFN2-xtb
>>
>> CAUTION: This email originated from outside of Vertex. Do not open links or attachments unless you know the sender and believe it is safe. If you suspect that this email is a potentially malicious email, please click on the Report Phish button.
>>
>>
>>
>> Dear Jerry,
>>
>> the number of processes you use must be divisible by the number of processor groups. So, if you want to use 4 groups, you need 4n parallel processes.
>>
>> Best regards,
>>
>> Bálint
>>
>>
>> Ps. Please also note, that periodic structures were not taken into account, when GFN2 was parameterized. Make sure to check compare your results with reference ab initio calculation before making production runs for big systems.
>>
>>
>> On 24.04.23 23:08, Jerry Tanoury wrote:
>>> Hi,
>>>
>>> I'm trying to run a dftb+ v22.1 job using GFN2-xtb Hamiltonian, with
>>> GPU acceleration and MPI/OpenMP parallelization.  I am using a
>>> poscar structure file.  However, I am getting the following error message:
>>>
>>> ERROR!
>>>
>>> -> Number of groups (4) not compatible with number of processes (1)
>>>
>>> I have attached my input file, my output file, the cluster submit
>>> script, and a sourcing file that sets paths.
>>>
>>> What am I doing wrong?
>>>
>>> Thank you so much for your help,
>>>
>>> Jerry
>>>
>>> Gerald J. Tanoury, Ph.D.
>>>
>>> Principal Scientific Fellow
>>>
>>> *Vertex Pharmaceuticals*
>>>
>>> 50 Northern Avenue
>>>
>>> Boston, MA 02210
>>>
>>> Office: 617-341-6219
>>>
>>> Cell: 508-740-4986
>>>
>>> jerry_tanoury at vrtx.com
>>>
>>> http://www.vrtx.com/ <http://www.vrtx.com/>
>>>
>>>
>>>
>>> This email message and any attachments are confidential and intended
>>> for use by the addressee(s) only. If you are not the intended
>>> recipient, please notify me immediately by replying to this message,
>>> and destroy all copies of this message and any attachments. Thank you.
>>>
>>> _______________________________________________
>>> DFTB-Plus-User mailing list
>>> DFTB-Plus-User at mailman.zfn.uni-bremen.de
>>> https://mailman.zfn.uni-bremen.de/cgi-bin/mailman/listinfo/dftb-plus
>>> -
>>> <https://urldefense.com/v3/__https://mailman.zfn.uni-bremen.de/cgi-b
>>> in/mailman/listinfo/dftb-plus-__;!!FRfS_D4!aFGz9DLWwOKgVQmuuzMMaQJTI
>>> Cksj_TyqHCN9O7VQwc45xRT38TiKAg8qDOhA_QKYKkDEZEg1Lf-NFZeG6MURn4d_Y81h
>>> f7hyQg$>
>>> u
>>> ser
>>
>> --
>> Dr. Bálint Aradi
>> Bremen Center for Computational Materials Science, University of
>> Bremen http://www.bccms.uni-bremen.de/cms/people/b-aradi/
>> <https://urldefense.com/v3/__http://www.bccms.uni-bremen.de/cms/peopl
>> e/b-aradi/__;!!FRfS_D4!aFGz9DLWwOKgVQmuuzMMaQJTICksj_TyqHCN9O7VQwc45x
>> RT38TiKAg8qDOhA_QKYKkDEZEg1Lf-NFZeG6MURn4d_Y81rMgLNl8$>
>>
>>
>>
>>
>> This email message and any attachments are confidential and intended for use by the addressee(s) only. If you are not the intended recipient, please notify me immediately by replying to this message, and destroy all copies of this message and any attachments.  Thank you.
>> _______________________________________________
>> DFTB-Plus-User mailing list
>> DFTB-Plus-User at mailman.zfn.uni-bremen.de
>> https://mailman.zfn.uni-bremen.de/cgi-bin/mailman/listinfo/dftb-plus-
>> u
>> <https://urldefense.com/v3/__https://mailman.zfn.uni-bremen.de/cgi-bi
>> n/mailman/listinfo/dftb-plus-u__;!!FRfS_D4!aFGz9DLWwOKgVQmuuzMMaQJTIC
>> ksj_TyqHCN9O7VQwc45xRT38TiKAg8qDOhA_QKYKkDEZEg1Lf-NFZeG6MURn4d_Y81eEW
>> UMes$>
>> ser
>
> --
> Dr. Bálint Aradi
> Bremen Center for Computational Materials Science, University of
> Bremen http://www.bccms.uni-bremen.de/cms/people/b-aradi/
> <https://urldefense.com/v3/__http://www.bccms.uni-bremen.de/cms/people
> /b-aradi/__;!!FRfS_D4!aFGz9DLWwOKgVQmuuzMMaQJTICksj_TyqHCN9O7VQwc45xRT
> 38TiKAg8qDOhA_QKYKkDEZEg1Lf-NFZeG6MURn4d_Y81rMgLNl8$>
>
>
>
>
> This email message and any attachments are confidential and intended
> for use by the addressee(s) only. If you are not the intended
> recipient, please notify me immediately by replying to this message,
> and destroy all copies of this message and any attachments. Thank you.
> _______________________________________________
> DFTB-Plus-User mailing list
> DFTB-Plus-User at mailman.zfn.uni-bremen.de
> https://mailman.zfn.uni-bremen.de/cgi-bin/mailman/listinfo/dftb-plus-u
> ser
> <https://urldefense.com/v3/__https://mailman.zfn.uni-bremen.de/cgi-bin
> /mailman/listinfo/dftb-plus-user__;!!FRfS_D4!aFGz9DLWwOKgVQmuuzMMaQJTI
> Cksj_TyqHCN9O7VQwc45xRT38TiKAg8qDOhA_QKYKkDEZEg1Lf-NFZeG6MURn4d_Y81a7L
> Tf8o$>
>
>
> This email message and any attachments are confidential and intended
> for use by the addressee(s) only. If you are not the intended
> recipient, please notify me immediately by replying to this message,
> and destroy all copies of this message and any attachments. Thank you.
>
> _______________________________________________
> DFTB-Plus-User mailing list
> DFTB-Plus-User at mailman.zfn.uni-bremen.de
> https://mailman.zfn.uni-bremen.de/cgi-bin/mailman/listinfo/dftb-plus-u
> ser

--
Dr. Bálint Aradi
Bremen Center for Computational Materials Science, University of Bremen http://www.bccms.uni-bremen.de/cms/people/b-aradi/




This email message and any attachments are confidential and intended for use by the addressee(s) only. If you are not the intended recipient, please notify me immediately by replying to this message, and destroy all copies of this message and any attachments. Thank you.

More information about the DFTB-Plus-User mailing list