CSA mistake in 2003 means I owe more money

I have been paying CSA since 1994. In 2003 I was made unemployed for the first time in 19 years and my assessment was placed on hold and they re-credited my CSA account by £1696.58 so all outstanding balances were zero’d.

About 4 months later I started working again and the CSA were on my tail instantly, they sent a new bill through which I started to pay, about 9 months later I became unemployed again but an outstanding balance remained £910.00 which I didn’t pay as I was contesting it, however in 2010 they took the full amount from my wages.

About 2 weeks ago I received a letter out of the blue stating that I am now to pay back the £1696.58 from a mistake they made back in 2003.

Surely this can’t happen it is so unfair……