I live here. It's a lovely town with the gloss taken off by the traffic. Some good cafes. The Buttercross tea rooms in the centre have space for bikes in the garden. It's cheap and cheerful. Classier meals can be had at The Fleece and the Castle Inns or Earsham Street or Jesters cafes by the castle. The campsite to the North isn't bad. There's no cycle shop nearer than Beccles (7m). NCN route 30 runs through the centre. There's good canoe hire at the campsite.
The local cycling club is the Godric. It's a good base for cycling to any point of the compass.