Selected Configuration Interaction

These methods rely on the same principle as the usual CI approaches, except that determinants aren’t chosen a priori based on an occupation or excitation criterion, but selected on the fly among the entire set of determinants based on their estimated contribution to the FCI wave function. It has been noticed long ago that, even inside a predefined subspace of determinants, only a small number significantly contributes to the wave function. [A1][A2] Therefore, an on the fly selection of determinants is a rather natural idea that has been proposed in the late 60’s by Bender and Davidson [A3] as well as Whitten and Hackmeyer. [A4]

The approach we are using in the Quantum Package is based on CIPSI developed by Huron, Rancurel and Malrieu, [A5] that iteratively selects external determinants (determinants which are not present in the variational space) using a perturbative criterion.

There is however a computational downside. In a priori selected methods, the rule by which determinants are selected is known a priori, and therefore, one can map a particular determinant to some row or column index. [A6] As a consequence, it can be systematically determined to which matrix element of \(\hat H\) a two-electron integral contributes. This allows for the implementation of so-called integral-driven methods, that work essentially by iterating over integrals and are very fast.

On the contrary, in selected methods an explicit list of determinants has to be kept, and there is no immediate way to know whether a determinant has been selected, or what its index is in the list. Consequently, a determinant-driven approach will be used, in which the loops run over determinants rather than integrals. This can be a lot more computationally expensive since the number of determinants is typically much larger than the number of integrals.

What makes determinant-driven approaches possible here is:

  • the fact that selected CI methods will keep the number of determinants small enough, orders of magnitude smaller than in a priori selected methods for wave functions with equal energies,
  • an efficient way to compare determinants in order to extract the corresponding excitation operators [A7],
  • an intense filtering of the internal space to avoid as much as possible determinant comparisons of disconnected determinants,
  • a fast retrieval of the corresponding two-electron integrals in memory.

Simple Algorithm

The variational wave function \(|\Psi^{(n)}\rangle\) is defined over a set of determinants \(\{|D_I\rangle\}^{(n)}\) in which we diagonalize \(\hat H\).

\[|\Psi^{(n)}\rangle = \sum_{I} c_I^{(n)} |D_I\rangle\]

The determinants in \(\{|D_I\rangle\}^{(n)}\) will be characterized as internal.

  1. For all external determinants \(|\alpha \rangle\) \(\notin\) \(\{|D_I\rangle\}^{(n)}\), compute the Epstein-Nesbet second-order perturbative contribution to the energy

    \[e_\alpha = \frac{ \langle \Psi^{(n)}| {\hat H} | \alpha \rangle^2 }{E^{(n)} - \langle \alpha | {\hat H} | \alpha \rangle }.\]

    \(E^{(n)}\) is the variational energy of the wave function at the current iteration. Note that another perturbation theory could be used to estimate \(e_\alpha\).

  2. An estimate of the total missing correlation energy can be computed by summing all the \(e_\alpha\) contributions

    \[\begin{split}E_\text{PT2} & = \sum_{\alpha} e_\alpha \\ E_\text{FCI} & \approx E + E_\text{PT2}\end{split}\]

  3. \(\{ |\alpha \rangle \}_\star ^{(n)}\), the subset of determinants \(|\alpha \rangle\) with the largest contributions \(e_\alpha\), is added to the variational space

    \[\{ |D_I \rangle \}^{(n+1)} = \{|D_I\rangle\}^{(n)} \cup \{ |\alpha\rangle \}_\star^{(n)}\]

  4. Go to iteration n+1, or exit on some criterion (number of determinants in the wave function, low \(E_\text{PT2}\), …).

Of course, such a procedure can be applied on any state and therefore can allow to treat both ground and excited states.

Stochastic approximations for the selection and the computation of \(E_\text{PT2}\)

The simple algorithm would be too slow to make calculations possible. Instead, the Quantum Package uses a stochastic algorithm [13] in order to compute efficiently the \(E_\text{PT2}\) and to select on-the-fly the best Slater determinants.

In such a way, the selection step introduces no extra cost with respect to the \(E_\text{PT2}\) calculation and the \(E_\text{PT2}\) itself is unbiased but associated with a statistical error bar rapidly converging.

Deterministic approximations for the selection

The following description was used in a previous version of the CIPSI algorithm which was less efficient. Nonetheless, it introduces the notions of generator and selector determinants which are much more general than the CIPSI algorithm that targets the FCI and can be used to realize virtually any kind of CI in a selected way.

We define generator determinants, as determinants of the internal space from which the \(|\alpha \rangle\) are generated. We then define selector determinants, a truncated wave function used in the computation of \(e_\alpha\).

For calculations in the FCI space, the determinants are sorted by decreasing \(|c_I|^2\), and thresholds are used on the squared norm of the wave function. The default is to use determinants threshold_generators = 0.99 for the generators, and determinants threshold_selectors = 0.999 for the selectors.

This is nothing but the 3-class CIPSI approximation to accelerate the selection, [A8] where instead of generating all possible \(|\alpha \rangle\), we only generate a subset which are likely to be selected.

The computation of \(E_\text{PT2}\) using a truncated wave function is biased, so if an accurate estimate of the FCI energy is desired, it is preferable to recompute \(E_\text{PT2}\) with the hybrid deterministic/stochastic algorithm [A9] which is unbiased (this is the default).

Modifying the selection space

By changing the definition of generators, and the rules for the generation of the \(|\alpha \rangle\), it is easy to define selected variants of traditional CI methods.

For example, if one defines the HF determinant as the only generator, one will produce a selected CISD. If one also changes the rules for the generation to generate only the double excitations, one will have a selected CID.

The generators can also be chosen as determinants belonging to a CAS. If the rules allow only for excitations inside the CAS, we obtain a selected CAS CI. If the rules allow for excitations in the FCI space, we obtain a selected CAS-SD. And if one add the rule to prevent for doing double excitations with two holes and two particles outside of the active space, one obtains a selected DDCI method.

All such things can be done very easily when programming the Quantum Package.

[A1]Laimutis Bytautas and Klaus Ruedenberg. A priori identification of configurational deadwood. Chemical Physics, 356(1-3):64–75, feb 2009. URL: https://doi.org/10.1016%2Fj.chemphys.2008.11.021, doi:10.1016/j.chemphys.2008.11.021.
[A2]James S.M. Anderson, Farnaz Heidar-Zadeh, and Paul W. Ayers. Breaking the curse of dimension for the electronic schrodinger equation with functional analysis. Computational and Theoretical Chemistry, 1142:66–77, oct 2018. URL: https://doi.org/10.1016%2Fj.comptc.2018.08.017, doi:10.1016/j.comptc.2018.08.017.
[A3]Charles F. Bender and Ernest R. Davidson. Studies in configuration interaction: the first-row diatomic hydrides. Phys. Rev., 183(1):23–30, jul 1969. URL: http://dx.doi.org/10.1103/PhysRev.183.23, doi:10.1103/physrev.183.23.
[A4]J. L. Whitten and Melvyn Hackmeyer. Configuration interaction studies of ground and excited states of polyatomic molecules. i. the CI formulation and studies of formaldehyde. The Journal of Chemical Physics, 51(12):5584–5596, dec 1969. URL: https://doi.org/10.1063%2F1.1671985, doi:10.1063/1.1671985.
[A5]B. Huron, J. P. Malrieu, and P. Rancurel. Iterative perturbation calculations of ground and excited state energies from multiconfigurational zeroth-order wavefunctions. The Journal of Chemical Physics, 58(12):5745–5759, jun 1973. URL: https://doi.org/10.1063%2F1.1679199, doi:10.1063/1.1679199.
[A6]Peter J. Knowles and Nicholas C Handy. A new determinant-based full configuration interaction method. Chem. Phys. Letters, 111:315–321, 1984.
[A7]A. Scemama and E. Giner. An efficient implementation of Slater-Condon rules. ArXiv [physics.comp-ph], pages 1311.6244, November 2013. URL: https://arxiv.org/abs/1311.6244.
[A8]Stefano Evangelisti, Jean-Pierre Daudey, and Jean-Paul Malrieu. Convergence of an improved CIPSI algorithm. Chemical Physics, 75(1):91–102, feb 1983. URL: https://doi.org/10.1016%2F0301-0104%2883%2985011-3, doi:10.1016/0301-0104(83)85011-3.
[A9]Yann Garniron, Anthony Scemama, Pierre-François Loos, and Michel Caffarel. Hybrid stochastic-deterministic calculation of the second-order perturbative contribution of multireference perturbation theory. The Journal of Chemical Physics, 147(3):034101, jul 2017. URL: https://doi.org/10.1063%2F1.4992127, doi:10.1063/1.4992127.