The Morris-Jumel Mansion, designed in the neo-Palladian style, is the oldest surviving house in Manhattan, an irreplaceable artifact described by Duke Ellington as “the crown jewel of Sugar Hill,” one of the places where Lin-Manuel Miranda composed songs for “Hamilton.” In its current life, it supplies a strange and unwelcome souvenir — pieces of itself. The building has been so badly maintained that it is possible to touch it and walk away with a moist, splintered clump of wood siding in the palm of your hand.

The mansion was built in 1765 as a summer place for a British loyalist, and its age alone might have indemnified it against all too obvious neglect. But the house also serves as a monument to the highest ideals of the American Enlightenment era and some of its most exquisite gossip.

Once a base of operations for George Washington during the Revolutionary War, it eventually became the home of Eliza Jumel, who was born in a Rhode Island bordello run by a Black madam and rose to become one of the richest and most liberated women in the country, tripling a fortune based largely on her own shrewd real-estate investments. Her cook was Anne Northup, the wife of the abolitionist Solomon Northup, who wrote “Twelve Years a Slave.”

For a while, Aaron Burr lived in the mansion as well. Jumel married him 14 months after her first husband, a rich wine merchant, died after landing on a pitchfork during a mysterious fall from a hay wagon. When she divorced Burr, who was tearing through her money as if he were trying out for the Real Husbands of the Continental Army, she hired Alexander Hamilton Jr. to represent her and presumably enjoy some karmic retribution.