Nexus: Quantum Package interface #1093

jtkrogel · 2018-10-03T22:26:59Z

The purpose of this PR is to create a Nexus interface to Quantum Package. This interface will enable routine selected CI calculations with QMCPACK.

Saved for later PR when hdf5 route fully available

HF workflow (QP -> QMCPACK)
FCI workflow (QP -> QMCPACK)

qmc-robot · 2018-10-03T22:27:01Z

Can one of the maintainers verify this patch?

qmc-robot · 2018-10-03T22:27:03Z

Can one of the maintainers verify this patch?

…r/slave

jtkrogel · 2018-10-11T20:02:18Z

Ready for review and merge.

markdewing · 2018-10-11T20:05:28Z

okay to test

markdewing · 2018-10-11T20:39:27Z

It would have been made the review smaller and easier to understand if the changes to machines.py (and possibly related changes in ntest) were in a separate PR.
Do you have an example nexus file for running quantum package?

jtkrogel · 2018-10-11T20:55:50Z

I do, but it is a work in progress. These developments are for a project with Anouar, so I would like to see if there are adjustments needed in that context before providing a user facing example.

jtkrogel · 2018-10-12T13:00:57Z

Do you want to try running an example to verify the interface works (if so I can provide you one), or is this more in the interest of having a documented example for general use (see prior comment)?

markdewing · 2018-10-12T13:49:50Z

Both reasons.
I lack a general mental model on how to map between the description of input files in the Nexus code, and an actual Nexus input file. That's why I'm often asking for example files - to try understand the Nexus library code.

jtkrogel · 2018-10-12T14:11:34Z

The following is working example for an H2O molecule:

#! /usr/bin/env python

from nexus import settings,job,run_project
from nexus import generate_physical_system
from nexus import generate_quantum_package

settings(
    results       = '',
    status_only   = 0,
    generate_only = 0,
    sleep         = 3,
    machine       = 'cooley',
    account       = 'PSFMat',
    qprc          = '/soft/applications/quantum_package/quantum_package.rc',
    )

scf_job = job(nodes=4,threads=12,hours=1,queue='debug')

system = generate_physical_system(
    structure = 'H2O.xyz',
    )

qp = generate_quantum_package(
    path       = 'qp_test',
    identifier = 'h2o_scf',
    job        = scf_job,
    system     = system,
    prefix     = 'h2o',
    run_type   = 'SCF',
    ao_basis   = 'cc-pvtz',
    )

run_project()

Contents of H2O.xyz:

3

O  0.000000  0.000000  0.000000 
H  0.000000  0.757160  0.586260
H  0.000000  0.757160 -0.586260

I can run this on Cooley or Theta at ALCF as these are the two machines I have access to with a QM Package install.

The Quantum Package configuration file needs to be sourced prior to running the script:

source /soft/applications/quantum_package/quantum_package.rc

jtkrogel · 2018-10-12T14:30:44Z

The Nexus script performs steps equivalent to the following to create the input for Quantum Package and run the HF in a split node configuration:

# create basic ezfio input with a cc-pvtz basis
qp_create_ezfio_from_xyz -b cc-pvtz h2o.xyz

# force all QP variables to be created and initialized
qp_edit -c h2o.ezfio

# request one and two body integrals be saved
echo "Write" > h2o.ezfio/integrals_monoelec/disk_access_ao_one_integrals
echo "Write" > h2o.ezfio/integrals_monoelec/disk_access_mo_one_integrals
echo "Write" > h2o.ezfio/integrals_bielec/disk_access_ao_integrals
echo "Write" > h2o.ezfio/integrals_bielec/disk_access_mo_integrals

# run HF job
export OMP_NUM_THREADS=12
mpirun -np 1 qp_run SCF h2o.ezfio >h2o_scf.output 2>h2o_scf.error&
sleep 30
mpirun -np 3 qp_run -slave qp_ao_ints h2o.ezfio >h2o_scf_slave.output 2>h2o_scf_slave.error

jtkrogel · 2018-10-12T14:48:43Z

Quantum Package's approach to input is somewhat unique. The "input file" is actually a directory tree with each input variable cached in a separate ASCII file.

This directory/file structure also represents the internal state of the program, and so subsequent runs, e.g. to perform CI singles or selected CI, require updating input variables stored in these files. Following a run, some of these files/variables change value.

Below is the result of running "tree" on an ezfio input directory:

h2o.ezfio/
├── ao_basis
│   ├── ao_basis
│   ├── ao_cartesian
│   ├── ao_md5
│   └── ao_num
├── davidson
│   ├── davidson_sze_max
│   ├── disk_based_davidson
│   ├── distributed_davidson
│   ├── n_states_diag
│   ├── state_following
│   └── threshold_davidson
├── determinants
│   ├── n_det
│   ├── n_det_max
│   ├── n_det_max_jacobi
│   ├── n_det_max_property
│   ├── n_det_max_stored
│   ├── n_states
│   ├── only_single_double_dm
│   ├── read_wf
│   ├── s2_eig
│   ├── store_full_h_mat
│   ├── target_energy
│   ├── threshold_generators
│   └── threshold_selectors
├── electrons
│   ├── elec_alpha_num
│   └── elec_beta_num
├── ezfio
│   ├── creation
│   ├── library
│   └── user
├── full_ci_zmq
│   └── iterative_save
├── hartree_fock
│   ├── energy
│   ├── level_shift
│   ├── max_dim_diis
│   ├── mo_guess_type
│   ├── n_it_scf_max
│   ├── no_oa_or_av_opt
│   ├── scf_algorithm
│   ├── threshold_diis
│   └── thresh_scf
├── integrals_bielec
│   ├── direct
│   ├── disk_access_ao_integrals
│   ├── disk_access_mo_integrals
│   ├── no_ivvv_integrals
│   ├── no_vvv_integrals
│   ├── no_vvvv_integrals
│   ├── threshold_ao
│   └── threshold_mo
├── integrals_monoelec
│   ├── disk_access_ao_one_integrals
│   └── disk_access_mo_one_integrals
├── mo_basis
│   ├── ao_md5
│   ├── mo_label
│   ├── mo_occ.gz
│   └── mo_tot_num
├── mrpt_utils
│   └── do_third_order_1h1p
├── nuclei
│   ├── disk_access_nuclear_repulsion
│   └── nucl_num
├── perturbation
│   ├── correlation_energy_ratio_max
│   ├── do_pt2
│   ├── pt2_absolute_error
│   ├── pt2_max
│   ├── pt2_relative_error
│   ├── threshold_generators_pt2
│   └── threshold_selectors_pt2
├── properties
│   ├── threshld_two_bod_dm
│   └── z_one_point
├── pseudo
│   ├── do_pseudo
│   ├── pseudo_grid_rmax
│   ├── pseudo_grid_size
│   ├── pseudo_klocmax
│   ├── pseudo_kmax
│   └── pseudo_lmax
└── qmc
    └── ci_threshold

The nesting corresponds roughly to what Fortran namelists looks like, and so in practice the fundamental input structure is not more complicated than e.g. Quantum Espresso's (but working with it is).

The Nexus spec corresponding to this allowed input structure, including types, is found in quantum_package_input.py assigned to the variable "input_specification".

The actual variables (the terminal files in the tree above) are actually unique names, so in Nexus one is allowed to generate the input by specifying nothing more than a single list of input variable names and values supplied to "generate_quantum_package". Nexus' internal represenation of the input follows the directory tree however, with directories being labeled as "Section"'s and each section containing the keyword/value pairs.

Input writing is done by first calling "qp_create_ezfio_from_xyz" and "qp_edit -c" if the ezfio directory does not exist, then if/once it exists all variables are written to the corresponding files.

In practice, the Nexus inputs contain the delta's for the input required at each step, so for HF, for example, the Nexus input file is (print qp.input):

  ao_basis
    ao_basis        = cc-pvtz
  end ao_basis
  determinants
    n_det_max       = 5000
  end determinants
  electrons
    elec_alpha_num  = 5
    elec_beta_num   = 5
  end electrons
  integrals_bielec
    disk_access_ao_integrals = Write
    disk_access_mo_integrals = Write
  end integrals_bielec
  integrals_monoelec
    disk_access_ao_one_integrals = Write
    disk_access_mo_one_integrals = Write
  end integrals_monoelec
  run_control
    postprocess     = []
    prefix          = h2o
    run_type        = SCF
    sleep           = 30
  end run_control
  structure
    axes            = []
    background_charge = 0
    bconds          = []
    center          = [ 0.  0.  0.]
    dim             = 3
    elem            = ['O' 'H' 'H']
    folded_structure = None
    frozen          = None
    kaxes           = []
    kpoints         = []
    kweights        = []
    mag             = []
    pos             = [[ 0.       0.       0.     ]
                       [ 0.       0.75716  0.58626]
                       [ 0.       0.75716 -0.58626]]
    scale           = 1.0
    units           = A
  end structure

The named sections "ao_basis", "determinants", "electrons", "integrals_bielec", and "integrals_monoelec" correspond directly to ezfio directories and variables. Additionally, "structure" is present to represent the internal state of the ezfio file regarding the atomic structure and "run_control" is added to represent the action-style inputs required to run the code.

jtkrogel · 2018-10-12T14:54:58Z

Input generation and input representation and read/write are handled by generate_quantum_package_input() and QuantumPackageInput, repectively, in quantum_package_input.py. Any action required to update the input state (calling qp_create_ezfio/qp_edit/qp_set_frozen_core.py and writing variables to files) is handled by QuantumPackageInput.

Actions relating to major simulation functions, like SCF/CIS/FCI (running "qp_run ...") are handled by the QuantumPackage class in quantum_package.py.

jtkrogel · 2018-10-12T15:04:10Z

The need to run commands in a split fashion (master/slave) when using multiple nodes

mpirun -np 1 qp_run SCF h2o.ezfio >h2o_scf.output 2>h2o_scf.error&
sleep 30
mpirun -np 3 qp_run -slave qp_ao_ints h2o.ezfio >h2o_scf_slave.output 2>h2o_scf_slave.error

or more traditionally when run on a single node

mpirun -np 1 qp_run SCF h2o.ezfio >h2o_scf.output 2>h2o_scf.error

required extensions to Nexus' job framework represented by the Job class in machines.py. This is handled primarily by the "split_nodes" function in the Job class. This function is used to create unified or split application run commands in the "app_command" function of the QuantumPackage class. Additional extensions are also present there to run CIS in a loop to iteratively generate natural orbitals to form a better starting point for selected CI.

No tests existed for Machine and Job classes prior to this point and the extensions required for Job splitting presented the possibility that some functionality would not work as expected, so I added tests for Machine and Job classes (including an extensive check performed by the "check_job_idempotency" function previously resident in machines.py) to the ntest script. Tests there now include an explicit check that simple job launch commands (including for various node and thread counts) are created correctly for all currently supported machines. Tests were then added for the new job splitting facility.

markdewing · 2018-10-12T20:36:06Z

nexus/library/quantum_package_input.py

+#end def write_qp_value
+
+
+


This looks like it comes from extract_input_specification? If so, there should a comment about where this piece of code came from.

I mean the code following this comment. The email message sent a bit of code from before this comment.

Comment added.

markdewing · 2018-10-12T22:10:59Z

The descriptions were helpful - they should make it into the documentation.

…nto nx_qm_package

jtkrogel · 2018-10-15T13:48:45Z

Ready to merge now?

ye-luo · 2018-10-15T13:57:50Z

Okay to test

nexus: initial scaffolding for quantum package implementation

5799061

ghost assigned jtkrogel Oct 3, 2018

ghost added the in progress label Oct 3, 2018

jtkrogel added 10 commits October 4, 2018 00:03

nexus: add ezfio read and input spec extraction

faff900

nexus: add qp input generation

194e3b8

nexus: complete qp input generation and add write ability

37a4aa0

nexus: prepare qp input for hf workflow

d2acad4

nexus: fix xyz file read

867b26c

nexus: prepare for job splitting

71d931b

nexus: add machine tests

1849b15

nexus: enable job splitting by nodes to support quantum package maste…

b58e12c

…r/slave

nexus: enable hartree-fock workflow for quantum package

a1854a8

nexus: enable cis and fci workflows for quantum package

9c4dd09

jtkrogel changed the title ~~[WIP] Nexus: Quantum Package interface~~ Nexus: Quantum Package interface Oct 11, 2018

Merge branch 'develop' into nx_qm_package

8b00f58

ghost assigned markdewing Oct 11, 2018

markdewing reviewed Oct 12, 2018

View reviewed changes

jtkrogel added 2 commits October 15, 2018 13:46

nexus: add input spec comment

9257596

Merge branch 'nx_qm_package' of https://github.com/jtkrogel/qmcpack i…

11a1c55

…nto nx_qm_package

markdewing approved these changes Oct 15, 2018

View reviewed changes

Merge branch 'develop' into nx_qm_package

7bc6d52

markdewing merged commit a37e1a5 into QMCPACK:develop Oct 15, 2018

ghost removed the in progress label Oct 15, 2018

jtkrogel deleted the nx_qm_package branch January 8, 2019 16:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nexus: Quantum Package interface #1093

Nexus: Quantum Package interface #1093

jtkrogel commented Oct 3, 2018 •

edited

Loading

qmc-robot commented Oct 3, 2018

qmc-robot commented Oct 3, 2018

jtkrogel commented Oct 11, 2018

markdewing commented Oct 11, 2018

markdewing commented Oct 11, 2018

jtkrogel commented Oct 11, 2018

jtkrogel commented Oct 12, 2018

markdewing commented Oct 12, 2018

jtkrogel commented Oct 12, 2018 •

edited

Loading

jtkrogel commented Oct 12, 2018

jtkrogel commented Oct 12, 2018 •

edited

Loading

jtkrogel commented Oct 12, 2018 •

edited

Loading

jtkrogel commented Oct 12, 2018 •

edited

Loading

markdewing Oct 12, 2018

markdewing Oct 12, 2018

jtkrogel Oct 15, 2018

markdewing commented Oct 12, 2018

jtkrogel commented Oct 15, 2018

ye-luo commented Oct 15, 2018

Nexus: Quantum Package interface #1093

Nexus: Quantum Package interface #1093

Conversation

jtkrogel commented Oct 3, 2018 • edited Loading

qmc-robot commented Oct 3, 2018

qmc-robot commented Oct 3, 2018

jtkrogel commented Oct 11, 2018

markdewing commented Oct 11, 2018

markdewing commented Oct 11, 2018

jtkrogel commented Oct 11, 2018

jtkrogel commented Oct 12, 2018

markdewing commented Oct 12, 2018

jtkrogel commented Oct 12, 2018 • edited Loading

jtkrogel commented Oct 12, 2018

jtkrogel commented Oct 12, 2018 • edited Loading

jtkrogel commented Oct 12, 2018 • edited Loading

jtkrogel commented Oct 12, 2018 • edited Loading

markdewing Oct 12, 2018

Choose a reason for hiding this comment

markdewing Oct 12, 2018

Choose a reason for hiding this comment

jtkrogel Oct 15, 2018

Choose a reason for hiding this comment

markdewing commented Oct 12, 2018

jtkrogel commented Oct 15, 2018

ye-luo commented Oct 15, 2018

jtkrogel commented Oct 3, 2018 •

edited

Loading

jtkrogel commented Oct 12, 2018 •

edited

Loading

jtkrogel commented Oct 12, 2018 •

edited

Loading

jtkrogel commented Oct 12, 2018 •

edited

Loading

jtkrogel commented Oct 12, 2018 •

edited

Loading