Why Does Memorization Reign Supreme in Traditional Learning?