Why do the Sun and the Moon appear redder on the horizon than overhead?

Blue light (left) is short wavelength, high frequency (high energy) radiation. Red light (right) is long wavelength, low frequency (low energy) radiation.

Sketch of blue light, showing a short wavelength sine curve. Sketch of red light, showing a long wavelength sine curve.
[NMSU, N. Vogt]

Our atmosphere is made up of small particles. These particles preferentially interact with and scatter short wavelength (blue) light. The scattered light bounces off in all directions, while the long wavelength (red) light is unaffected and passes through the atmosphere along a relatively straight path. When the Sun or the Moon is directly overhead, at noon or at midnight, the light passes through a relatively short amount of the atmosphere before reaching us. When the Sun or the Moon is on the horizon (at sunrise or sunset), however, the light passes through a longer section of the atmosphere before reaching us. The blue light is preferentially scattered away from the direct path, and so less of it survives for us to see coming directly from the source.

Sketch of the Earth viewed from far above the North Pole, with an arrow indicating the Earth rotates counterclockwise from this viewpoint. Four observers are spaced evenly around the Earth's equator. At midnight the sky is dark; six hours later at sunrise the observer sees a red Sun as parallel light rays from the Sun must traverse a long path through the Earth's atmosphere; six hours later at noon the Sun appears yellower (less red) as the light must traverse a shorter path straight through the atmosphere and so more blue light survives; six hours later at sunset the Sun again appears red on the horizon.
[NMSU, N. Vogt]