Interaction effects in topological insulators

In this thesis we employ various mean-field approaches to study the shortrange interaction effects in topological insulators. We start with the Kane-Mele model on the decorated honeycomb lattice and study the stability of topological insulator phase against different perturbations. We establish an adiabatic connection between a noninteracting topological insulator and a strongly interacting spin liquid in its Majorana fermion representation. We use the Hartree-Fock mean-field approach, slave-rotor approach and slave-boson approach to study correlation effects related to topological insulators. With the spontaneous symmetry breaking mechanism, we can have an interaction driven topological insulator with extended Hubbard models on the kagome lattice and decorated honeycomb lattice. For the interplay among spin-orbit coupling, distortion and correlation effect in transition metal oxides, we use the slave-rotor mean-field approach to study its phase transition. We identify regimes where a strong topological Mott insulator and a weak topological insulator reside due to the strong Coulomb interaction and distortion. This is relevant to experiments with the transition metal oxides as they hold promise to realize topological insulators. To study the doping effects and a possible spin liquid in Kane-Mele-Hubbard model on the honeycomb lattice, we employ the slave-boson mean-field approach which is appropriate for the intermediate interaction strength. We compare our results with those obtained from other methods.