Historical and traditional rivalry:
http://www.historyofparliamentonline.org/volume/1604-1629/constituencies/great-yarmouth
Actually I agree that Lowestoft has gone downhill, for various reasons, but then I'm a native of nearby Oulton Broad.
Reality has not been kind to the East Coast ports & towns, whether they be Grimsby, Gt Yarmouth, Skegness or Lowestoft.