This is now the Bristol School of Dancing. Their website says:
Built in 1893, it stands in the garden of 20 Vyvyan Terrace and it is rumoured that it was once a Swedish diplomatic building that contained a gymnasium. This is probably why on either side of the main entrance the words “Swedish Gymnasium” are carved.
GPS Coordinates:
51.45737, -2.61795
Location: Clifton College