why is my credit score going down when I pay on time Payment Perfect, Score Plunging: Why Your Credit Score Might Be Dropping Payment Perfect, Score Plunging: Why Your Credit Score Might Be Dropping. Understand the factors beyond timely payments that can negatively affect your credit score.