Financial problems forced Samuel and Olivia to close up their Hartford home and to move the family to Europe in 1891. The family would never live in Hartford again. Susy’s death in 1896 made it too hard for Olivia to return to their Hartford home, and the Clemenses sold the property in 1903 to the Richard Bissell family.

The Bissells lived in the house until 1917. For the next four years they rented the building to the Kingswood School for boys. In 1922 the house was sold to a developer, who immediately made plans to turn the house into an apartment building. According to a 1923 Hartford Courant article, “The apartments will be ready for occupancy on July 1. There will be eleven apartments and in the basement there is to be a large dining hall to be used for commercial purposes or for functions arranged by the tenants. Each apartment will have a large room with a fireplace and there will be three small bedrooms with folding beds ingeniously worked into the panels of the doors.”