But I don't think it's the original end -- if you check this picture from a history pamphlet I posted on Twitter you'll see that it looks like the Colonnade used to curve around rather more, and there's an even better view on this drawing from the British Library collection of Hotwell Parade. Looking at the historic basemap layers there on KYP it certainly seems like sometime between 1855 and 1874 (both Ashmead maps) the first couple of houses in the terrace were lopped off, leaving only numbers 3, 4, 5 and 6. I imagine they were shaved off at the same time as the second Hot Well House was demolished, which was in 1867. It used to stand on Hotwell Point, sticking out into the river, and the whole lot was removed to ease navigation.
I've never actually looked at the Colonnade door numbers to see if they're still like that—I'll try to remember the next time I pass.