Some years ago, as my mother and I were walking on New York City’s Upper West Side, she pointed out a redbrick townhouse in the West 70s where, she said, my great-grandfather had lived as a child. It was an awkward building, the door set back under a large arch, the roof sharply peaked, and I wondered that it had survived as the city rose around it.

I have since discovered that many of the places in New York where my ancestors lived are still standing: tenements on the Lower East Side, brownstones in Brooklyn Heights, a squat apartment building in Astoria, a two-family building in Canarsie.

Look around most neighborhoods in the city and you’ll find that the stage on which New Yorkers live and play, the physical city, hasn’t changed much in a very long time. More than half the city’s housing is in buildings constructed before 1947.

I take pleasure in wandering around this museum of family history, but it also makes me sad. The buildings survive because New York is preserving the corporeal city of bricks and steel at the expense of its residents and of those who might live here.

Like other American cities, New York has erected layers of laws to protect existing buildings and to impede the construction of new ones. The result is a shortage of housing. That is the reason the rent is too damn high, the reason so many people who grow up here cannot stay, the reason the city is struggling to accommodate an influx of immigrants that once would have seemed like a drop in the melting pot.

The city could add a significant amount of housing simply by allowing new buildings to rise to the same heights as existing buildings in the same neighborhoods. An analysis by the architectural firm PAU concluded that New York could add more than 500,000 homes around transit stations by replacing vacant lots, parking lots and single-story retail with new housing, subject to a neighborhood height limit.

Applying the same rule to existing housing would make room for even more new homes.

New York also needs to clear away some of the interlocked regulatory barriers that have long impeded housing construction, including a tax code that bizarrely penalizes large apartment buildings, rules that effectively give neighborhood residents the power to veto development plans and a byzantine permitting process.

Mayor Adams has spoken of a “moonshot” goal of building 500,000 homes in the next decade. The PAU analysis shows that such a goal is within reach. But it can only be achieved if the city makes the changes necessary to unleash a building boom.

New York is not a great city because of its buildings. It is a great city because it provides people with the opportunity to build better lives.

To preserve that, the buildings must change.

David Moffat, among the earliest of my ancestors to arrive in New York, wrote in a memoir that he landed in Manhattan in June of 1829 with one dollar in his pocket, “gazing at the crowds of people not one of whom I knew, with a feeling of utter loneliness.” He found a boardinghouse, then a job in a drugstore, and in time, he built a business in leather goods.

Over the next century, my forebears came to New York from New England, Nova Scotia and Scotland and, later, from Belarus, Greece and Ukraine — and the city made room for them, and millions of others, by erasing and replacing older versions of itself.

Moffat lived in at least nine places between 1829 and 1851, mostly in a neighborhood known as the Swamp, a leather-making district on the northeast edge of the modern financial district. None of the houses there have survived. Even some of the streets are gone. Decades later, in 1910, one of my great-grandfathers, Hyman Appelbaum, newly arrived from Belarus, moved into a tenement a few blocks from one of Moffat’s former homes. That building is gone, too.

But Moffat’s final New York home has survived. By 1852, his leather business prospering, he had moved across the East River to a brownstone that still stands on Willow Street in Brooklyn Heights. Moffat’s son married a woman who had grown up a few blocks away, on Hicks Street, and her childhood home also survives. So does the house on Willow Street where the newlyweds — my great-great-grandparents — made their home.

Much of Brooklyn Heights, built in the 19th century, has survived into the 21st. Sometimes described as New York’s first suburb, it was also one of the first parts of the city to fossilize.

For a time, nobody could be bothered to knock down the old townhouses. In the late 1950s, Truman Capote, living in a basement on Willow Street across from the old Moffat house, opened an essay about the faded neighborhood: “I live in Brooklyn. By choice.”

Today, however, the old houses survive because it is against the law to replace them with apartment buildings. New York has declared much of the neighborhood a historic district. Brooklyn Heights no longer sits on the outskirts of the city; it now sits pretty much in the middle. But it retains its original scale. It’s a New York version of Colonial Williamsburg, except that it sits on some of the most valuable real estate in the world.

Historic districts only cover roughly 4 percent of the city’s land, but it’s not easy to build in the other 96 percent of New York, either. New York’s zoning laws, which define the limits of potential development, provide little room for new housing.

Roughly 15 percent of the land in America’s largest city is reserved for single-family homes. Even in central neighborhoods, it is often illegal to build new buildings on the same scale as existing buildings: Forty percent of the buildings in Manhattan could not be built today.

In 1961, Hyman Appelbaum’s son, a postal worker, and his wife, a public-school teacher, bought a new duplex in Canarsie, a neighborhood in southeastern Brooklyn where developers in the decades after World War II built thousands of low-cost homes.

In that same year, New York enacted a plan that limited future development throughout the city. As those rules hardened, construction declined and neighborhoods calcified, none more so than Canarsie, which holds the dubious distinction of being the neighborhood with the second-lowest number of new housing units in the entire city from 2010 to 2022.

The building where my grandparents lived for three decades is less than an hour by subway from the middle of Manhattan. It’s a good location for a small apartment building. But under city law, it is illegal to build anything larger than their old home.

In a country where most cities on both coasts are not building enough housing, New York still stands apart. Between 2014 and 2021, the city added about half as many homes per capita as Boston; about a third as many as Washington, D.C.; and a quarter as many as Miami.

The result is an increasingly frantic competition for the available housing. In recent decades, rents have climbed much faster than incomes. In 1991, the median rent in New York City was $900. By 2021, the median renter was paying $1,500 a month for housing.

In the 1960s, much of the area once known as the Swamp, where David Moffat and Hyman Appelbaum had lived, was redeveloped under a state program called Mitchell-Lama, to create more housing for middle-income families. New York subsidized the construction of Southbridge Towers, four high-rises containing 1,600 units.

In 2014, the residents voted to allow themselves to sell units on the open market. The prices are now well beyond the reach of middle-income families.

Developers can still make money building new homes for the rich, mostly in tall buildings in a few central neighborhoods. The luxury high-rises that have redefined the midtown skyline are a fitting emblem of the modern city, and they have sustained the appearance, now mostly an illusion, that New York remains a dynamic and growing city.

New York also subsidizes the construction of some new housing for lower-income families.

What is missing — what the city sorely needs — is mid-rise housing for the middle class.

I hope someday I’ll be walking with my children on the Lower East Side or the Upper West Side or Brooklyn Heights. We’ll pass one of the places where my ancestors lived, and the building will be gone. In its place will stand an apartment building, housing a new generation of New Yorkers.

The Times is committed to publishing a diversity of letters to the editor. We’d like to hear what you think about this or any of our articles. Here are some tips. And here’s our email: letters@nytimes.com.

Follow The New York Times Opinion section on Facebook, Twitter (@NYTopinion) and Instagram.