Teeth are made up of outer enamel followed by softer dentin and then the inner nerve. Nutrients (and stains) pass into the tooth via pores.
Teeth have pores just like skin. Over time, stains from food, drink, tobacco and medication cause natural yellowing. Simply scrubbing teeth will not remove deep set stains.