A short distance along Stour Street is the entry to the County Hotel's garage which on the left. Opposite is a guest house, Greyfriars House, in a building that has undergone significant restoration. The building was once the gatehouse to Greyfriar's Monastery established by Franciscan monks in 1267. You can see the wonderful little church that straddles the river by entering by the gate on the left of the Guest House. It's open only in the afternoons. You can also attend a service that's held there on 12.30 on Wednesdays.
Greyfriar's Guest House is a bound to be a good place to stay, it will be quiet at night and is very handy for the city centre.