## Topological flat bands in a kagome lattice multiorbital system

### Theoretical model

To begin with, we set up a multi-orbital tight-binding model on a kagome lattice

$${H}_{{{{{{{{\rm{t}}}}}}}}}=-\mathop{\sum}\limits_{\left\langle {{{{{{{\bf{r}}}}}}}}\,{{{{{{{{\bf{r}}}}}}}}}^{\prime}\right\rangle }\mathop{\sum}\limits_{\alpha \beta \sigma }\left({t}_{{{{{{{{\bf{r}}}}}}}}\,{{{{{{{{\bf{r}}}}}}}}}^{\prime}}^{\alpha \beta }{c}_{{{{{{{{\bf{r}}}}}}}}\alpha \sigma }^{{{{\dagger}}} }{c}_{{{{{{{{{\bf{r}}}}}}}}}^{\prime}\beta \sigma }+{{{{{{{\rm{H.c.}}}}}}}}\right),$$

(1)

as schematically shown in Fig. 1. Here, \({c}_{{{{{{{{\bf{r}}}}}}}}\alpha \sigma }^{({{{\dagger}}} )}\) is the annihilation (creation) operator of an electron at site **r**, orbital *α*, and with spin *σ* = *↑* or *↓*. As discussed by Meier et al.^{25}, CoSn-type kagome systems have several flat bands with {*y**z*, *x**z*}, {*x**y*, *x*^{2} − *y*^{2}}, or 3*z*^{3} − *r*^{2} character. We focus on a {*y**z*, *x**z*} subset for simplicity and use *α* = *a* for the *y**z* orbital and *b* for the *x**z* orbital. With this basis, nearest-neighbor hopping intensities \({t}_{{{{{{{{\bf{r}}}}}}}}\,{{{{{{{\bf{r}}}}}}}}^{\prime} }^{\alpha \beta }\) can be parameterized using Slater integrals^{56}. Between site **1** and site **2**, \({\hat{t}}_{{{{{{{{\bf{1}}}}}}}}{{{{{{{\bf{2}}}}}}}}}\) is diagonal in orbital indices as \({t}_{{{{{{{{\bf{1}}}}}}}}\,{{{{{{{\bf{2}}}}}}}}}^{aa}={t}_{\delta }\) and \({t}_{{{{{{{{\bf{1}}}}}}}}\,{{{{{{{\bf{2}}}}}}}}}^{bb}={t}_{\pi }\), corresponding to (*d**d**δ*) and (*d**d**π*), respectively, by Slater and Koster^{56}. Other components are obtained by rotating the basis *a* and *b* as shown in the Methods section. From now on, *t*_{π} is used as the unit of energy.

Because *y**z* and *x**z* are written using the eigenfunctions of angular momentum *l*_{z} = ±1 for *l* = 2 as \(\left|yz\right\rangle =\frac{{{{{{{{\rm{i}}}}}}}}}{\sqrt{2}}(\left|1\right\rangle +\left|-1\right\rangle )\) and \(\left|xz\right\rangle =-\frac{1}{\sqrt{2}}(\left|1\right\rangle -\left|-1\right\rangle )\), respectively, the SOC \(\lambda \overrightarrow{l}\cdot \overrightarrow{s}\) in the {*y**z*, *x**z*} subset is written as

$${H}_{{{{{{{{\rm{soc}}}}}}}}}=\frac{\lambda }{2}\mathop{\sum}\limits_{{{{{{{{\bf{r}}}}}}}}\,\sigma }\left({{{{{{{\rm{i}}}}}}}}{\sigma }_{\sigma \sigma }^{z}{c}_{{{{{{{{\bf{r}}}}}}}}a\sigma }^{{{{\dagger}}} }{c}_{{{{{{{{\bf{r}}}}}}}}b\sigma }+{{{{{{{\rm{H.c.}}}}}}}}\right),$$

(2)

where \({\hat{\sigma }}^{z}\) is the *z* component of the Pauli matrices.

As shown in Supplementary Note 1, an effective model for the {*x**y*, *x*^{2} − *y*^{2}} doublet has the same form as the above *H*_{t} + *H*_{soc}. By symmetry, there is no hopping matrix between the {*y**z*, *x**z*} doublet and the other orbitals *x**y*, *x*^{2} − *y*^{2}, and 3*z*^{2} − *r*^{2}, but the {*x**y*, *x*^{2} − *y*^{2}} doublet and the 3*z*^{2} − *r*^{2} singlet could be hybridized. As discussed briefly later, the degeneracy in the {*y**z*, *x**z*} doublet and in the {*x**y*, *x*^{2} − *y*^{2}} doublet could be lifted by a crystal field. Such band splitting is also induced by the difference between *t*_{δ} and *t*_{π}. Furthermore, all *d* orbitals could in principle be mixed by the SOC. Including these complexities is possible but depends on the material and they usually induce smaller perturbations, therefore, here they are left for future analyses.

### Non-interacting band topology

By diagonalizing the single-particle Hamiltonian *H*_{t} + *H*_{soc}, one obtains dispersion relations as shown in Fig. 2. In the simplest case, where the hopping matrix \({t}_{{{{{{{{\bf{r}}}}}}}}\,{{{{{{{\bf{r}}}}}}}}^{\prime} }^{\alpha \beta }\) does not distinguish *t*_{δ} and *t*_{π} and the SOC is absent, the dispersion relation is identical to the one for the single-band tight-binding model, consisting of flat bands and graphene-like bands as shown by gray lines in Fig. 2a. Note that each band is fourfold degenerate because of two orbitals and two spins per site. Including SOC does not change the dispersion curve but simply shifts \(\overrightarrow{l}\cdot \overrightarrow{s}=\pm 1/2\) bands (see Supplementary Note 1).

Including orbital dependence as *t*_{δ} ≠ *t*_{π} without SOC instead splits the fourfold degeneracy except for two points at the Γ point and two points at the K point. Quite intriguingly, Dirac dispersions emerge from the topmost flat bands as shown as blue lines in Fig. 2a (see Supplementary Note 1 for more discussion). Turning on the SOC further splits such fourfold degeneracy, leading to nontrivial band topology. In this particular example, the spin component along the *z* axis is conserved giving unique characteristics to this case. As shown in Fig. 2b, the spin up component of each band is characterized by a nonzero Chern number \({{{{{{{{\mathcal{C}}}}}}}}}_{n}\). Because of the time-reversal symmetry, spin down bands have opposite Chern numbers. The topological property is also confirmed by gapless modes in the dispersion relation with the ribbon geometry, as shown in Fig. 2c. Here, there appear one (two) pair of gapless modes between the highest and the second highest (between the second lowest and the third lowest) bands, shown as red (blue) curves, corresponding to the sum of Chern numbers below the gap, −1(−2). The other edge states are invisible because of the overlap with the bulk continuum.

A multi-orbital kagome model thus naturally shows quasi flat bands with nontrivial topology. However, close inspection revealed that, with *t*_{δ} = 0.5 and *λ* = 0.2, the minimum of the highest band at the K point is slightly lower than the maximum of the second highest band at the Γ point. Thus, instead of a TI, a topological semimetal is realized when the Fermi level is located between the highest band and the second highest band. In fact, there are ways to make the gap positive. Here, we consider second-neighbor hopping matrices \({\hat{t}}_{{{{{{{{\bf{r}}}}}}}}\,{{{{{{{\bf{r}}}}}}}}^{\prime} }^{(2)}\). As explained in Supplementary Note 1, these are also parametrized by *π*-bonding (*d**d**π*) and *δ*-bonding (*d**d**δ*), \({t}_{\pi }^{(2)}\) and \({t}_{\delta }^{(2)}\), respectively. For simplicity, we fix the ratio between *t*_{π} and \({t}_{\pi }^{(2)}\) and between *t*_{δ} and \({t}_{\delta }^{(2)}\) as \({t}_{\pi }^{(2)}/{t}_{\pi }={t}_{\delta }^{(2)}/{t}_{\delta }={r}_{2}\), and analyze the sign and magnitude of the band gap Δ_{gap} between the highest band and the second highest band, as well as the flatness of the highest band defined by \({{\Delta }}\varepsilon \equiv {\varepsilon }_{1,\max }-{\varepsilon }_{1,\min }\).

Figure 3a plots Δ*ε* as a function of *t*_{δ} and *r*_{2} with *λ* = 0.2. As mentioned previously, the perfectly flat band with Δ*ε* = 0 is realized at *t*_{δ} = 1 and *r*_{2} = 0, but band gap Δ_{gap} is zero. The flatness is immediately modified by reducing *t*_{δ} from 1. As indicated by an open square in the plot, *t*_{δ} = 0.5 and *r*_{2} = 0 gives Δ*ε* ~ 0.88 and negative band gap Δ_{gap} ~ −0.027. Nonzero *r*_{2} controls the relative energy between the zone center and the zone boundary. In particular, negative *r*_{2} pushes up the energy at the K point, hereby the flatness is recovered. Naturally, the flatness and the positive gap are correlated as indicated by red loops in the second and forth quadrants because the separation between the highest band and the second highest band is fixed by the SOC strength. As indicated by a filled circle, *t*_{δ} = 0.5 and *r*_{2} = −0.2 gives Δ*ε* ~ 0.22 and positive band gap Δ_{gap} ~ 0.17. Corresponding dispersion relation is shown in Fig. 3b. The Chern numbers remain unchanged by this *r*_{2}.

### Many-body effects

Having established the topological properties at the single-particle level, we turn our attention to many-body effects focusing on the highest-energy flat band. A unique property of the current model is that the topmost quasi flat band has Chern number \(| {{{{{{{\mathcal{C}}}}}}}}| =1\). Thus, a large spin polarization can be induced by many-body interactions^{57} or by a small magnetic field. Further intriguing possibilities are FCI states when a topological flat band has a fractional filling and the insulating gap is induced by correlation effects^{44,46,47,48,49,50,51,52}. We examine such a possibility in our kagome model. Assuming the spin polarization in the highest band, we introduce local and nearest-neighbor Coulomb repulsive interactions as \({H}_{{{{{{{{\rm{U}}}}}}}}}=U{\sum }_{{{{{{{{\bf{r}}}}}}}}}{n}_{{{{{{{{\bf{r}}}}}}}}a\uparrow }{n}_{{{{{{{{\bf{r}}}}}}}}b\uparrow }+V{\sum }_{\left\langle {{{{{{{\bf{r}}}}}}}}{{{{{{{\bf{r}}}}}}}}^{\prime} \right\rangle }{\sum }_{\alpha \beta }{n}_{{{{{{{{\bf{r}}}}}}}}\alpha \uparrow }{n}_{{{{{{{{\bf{r}}}}}}}}^{\prime} \beta \uparrow }\), where \({n}_{{{{{{{{\bf{r}}}}}}}}\alpha \sigma }={c}_{{{{{{{{\bf{r}}}}}}}}\alpha \sigma }^{{{{\dagger}}} }{c}_{{{{{{{{\bf{r}}}}}}}}\alpha \sigma }\). Here *U* is the effective Coulomb interaction given by \(U=U^{\prime} -J\) with the interorbital Coulomb repulsion \(U^{\prime}\) and the interorbital exchange interaction *J*. These interactions are then projected onto the highest band, leading to the effective Hamiltonian *H*_{eff} = *H*_{t} + *H*_{soc} + *H*_{U}.

Note that the *S*_{z} conservation is not essential to realize FCI. For our case and most of others, including complexities which break *S*_{z} conservation does not destroy FCI as long as the flat band has the nontrivial topology and is well separated from other bands, justifying projecting interaction terms onto the flat band and allowing for an accurate Lanczos calculation. While the computational cost would be expensive, direct calculations of multiband models with *S*_{z}-non-conserving terms would show FCI if the appropriate condition is fulfilled, but this possibility has not been fully explored yet.

The effective Hamiltonian *H*_{eff} is diagonalized in momentum space. For this purpose, we discretize the momentum space into *N*_{1} × *N*_{2} patches and express the Hamiltonian in the occupation basis, i.e., the Hilbert space is built up by \(\left|{\varphi }_{l}\right\rangle ={\prod }_{{{{{{{{\bf{k}}}}}}}}\in l}{\psi }_{1{{{{{{{\bf{k}}}}}}}}}^{{{{\dagger}}} }\left|0\right\rangle\), where *ψ*_{1k} is the single-particle wave function for the highest flat band at momentum **k**, and the combination of **k** is specified by *l*. Due to the translational symmetry and the momentum conservation of many-body interaction terms, *H*_{eff} is subdiagonalized according to the total momentum **k**_{tot} = ∑_{k∈l}**k** modulo **b**_{1} and **b**_{2}, with **b**_{1,2} being two reciprocal lattice vectors. In this study, we take *N*_{1} = 4 and *N*_{2} = 6 and consider *ν* = 1/3 filling, that is, the number electrons in the highest flat band is *N*_{e} = 8. Momentum sector will be specified using integer index (*k*_{1}, *k*_{2}) corresponding to the total momentum **k**_{tot} = **b**_{1}*k*_{1}/*N*_{1} + **b**_{2}*k*_{2}/*N*_{2}.

Figure 4a, b show the low-energy spectra of the interacting model with *t*_{δ} = 0.5 and *r*_{2} = 0 and *t*_{δ} = 0.5 and *r*_{2} = −0.2, respectively, with *U* = 2 and *V* = 1 as a function of total momentum. In (a), the energy spectrum has a unique ground state at total momentum (*k*_{1}, *k*_{2}) = (0, 0) (note that this is to show the competition between the wide band width and the correlation effects on the highest band). When *r*_{2} is introduced as −0.2, the highest band becomes flatter, leading to a drastic change in the energy spectrum. There appear three energy minima at (*k*_{1}, *k*_{2}) = (0, 0), (0, 2), and (0, 4), forming a threefold degenerate ground state manifold (GSM), which is separated from the other states by an energy gap ~0.03. As shown in Fig. 4c, the three sectors evolve with each other by inserting magnetic fluxes without having overlap with higher energy states (energy separation is slightly reduced to ~ 0.02). These results strongly suggest a *ν* = 1/3 FCI state.

To confirm that this threefold degenerate ground state really represents a FCI state instead of trivial states such as charge density waves, we compute Chern numbers \({{{{{{{{\mathcal{C}}}}}}}}}_{({k}_{1},{k}_{2})}\) by introducing twisted boundary conditions^{5,58}. Here, we discretize the boundary phase unit cell into 20 × 20 meshes, and numerically evaluate the Berry curvature \({F}_{({k}_{1},{k}_{2})}({\theta }_{1},{\theta }_{2})\) as detailed in the Methods section as well as in Supplementary Note 2. Figure 5 shows \({F}_{({k}_{1},{k}_{2})}({\theta }_{1},{\theta }_{2})\) in a discretized grid (*n*_{1}, *n*_{2}) for the GSM with *t*_{δ} = 0.5, *r*_{2} = −0.2, *λ* = 0.2 with *U* = 2 and *V* = 1. Along the *n*_{1} direction, these plots are periodic. Along the *n*_{2} direction, plot (a) is continuously connected to plot (b), plot (b) is connected to plot (c), and plot (c) is connected back to plot (a). This also confirms the threefold GSM, where inserting one flux quantum along the **b**_{2} direction shifts the sector (*k*_{1}, *k*_{2}) = (0, 0) to (0, 2), (0, 2) to (0, 4), and (0, 4) to (0, 0). By adding up the discretized values of \({F}_{({k}_{1},{k}_{2})}({\theta }_{1},{\theta }_{2})\), we obtain \({{{{{{{{\mathcal{C}}}}}}}}}_{(0,0)}=0.331489\), \({{{{{{{{\mathcal{C}}}}}}}}}_{(0,2)}=0.330318\), \({{{{{{{{\mathcal{C}}}}}}}}}_{(0,4)}=0.338193\), and the sum of the three Chern numbers is exactly 1 within the numerical accuracy. The slight deviation from the ideal value \({{{{{{{\mathcal{C}}}}}}}}=1/3\) is ascribed to finite-size effects. This proves the existence of a *ν* = 1/3 FCI phase with a quantized fractional Hall response \({\sigma }_{{{{{{{{\rm{H}}}}}}}}}=\frac{1}{3}{e}^{2}/h\), where *e* is the electron charge and *h* is the Planck constant. In our numerical analyses, we did not find a ground state with the threefold degeneracy and Chern number zero, thus excluding the charge density wave states. This is probably because the quantum effects make such states unstable, as discussed in ref. ^{42}.