“To find something so diminished, still existing but made irrelevant — as the Travers house now seems to be, with its added dormer windows, its startling blue paint — might be less hurtful in the long run.”