Broken selection rule in the quantum Rabi model