When you lose velocity off of your fastball, one thing that suffers is your off-speed pitches - the action on your off-speed pitches. Once you lose arm speed, it takes away rotation from your off-speed pitches. The rotation is what gives them the sharpness and the nastiness.
