There's some fascinating stuff on Victoria Square here, including a lot of jumping-off points for deeper research. The description of the bombings during the war is amazing.
This is the path that used to be known as Birdcage Walk—up until the war it was lined with tall railings—but most people now use that name for the corridor made by the pleached lime trees in St Andrew's Churchyard, just a stone's throw away. I suppose that's the one that most resembles a bird cage today. Here's a Vaughan Collection photo of what it looked like before the war.
Adjacent on the left (Merchant's Road) side would have been the church of St James the Apostle, which survived the war but was demolished for flats in the seventies.