The rays must bend!  Each color must bend a different amount!

 

How the rays bend as they leave the prism

A picture of the actual experiment