This week’s price data was the latest confirmation that the pandemic continues to reshape how and where many Americans want to live. Remote work prompted buyers to seek out more space, while pandemic-related restrictions made vacation houses more desirable, real-estate agents said.

The housing boom has been unusually widespread, with low mortgage rates fueling strong buyer interest across the U.S., especially for high-end properties.

But first-time buyers in particular have struggled to keep up with soaring prices, and those with limited budgets are often losing out to cash buyers. The length of time U.S. homeowners stay put, meanwhile, has been steadily rising. Some have decided not to move, or they postponed selling because of concerns about letting buyers into their homes during a pandemic.

The shortage of homes on the market is affecting home shoppers around the country. The number of active listings on Realtor.com was down 52% from a year earlier in the week ended May 1. (

News Corp,

parent of The Wall Street Journal, operates Realtor.com.)

“The record-high home prices are happening across nearly all markets, big and small, even in those metros that have long been considered off the radar in prior years for many home seekers,” said

Lawrence Yun,

NAR’s chief economist.

Home sales rose in 2020 to the highest level since 2006 and remained strong in the first quarter. Economists have said the current housing boom is less risky than the one in the mid-2000s that ended with a housing-market crash. Borrowers this time are more financially qualified, and mortgage lenders are maintaining tighter standards.

Still, brokers describe the current market as frenzied. Many homes receive multiple offers within days. The typical home that sold in March spent 18 days on the market, the fastest pace on record, NAR said.

Economists have said the pace of price increases is likely to slow later in the year and next as more people are priced out of the market, especially if mortgage rates tick higher. Mortgage-finance company Fannie Mae is forecasting median existing-home prices to rise 11.5% in 2021, then slow to a 4% increase in 2022.

“With low inventory already impacting the market, added skyrocketing costs have left many families facing the reality of being priced out entirely,” Mr. Yun said.

Many of the metro areas that posted the strongest price increases in the first quarter were vacation destinations, as second-home demand surged during the pandemic and continues to remain robust. The biggest gainer was Kingston, N.Y., with a 35.5% median-price increase from a year earlier. Kingston is in New York’s Hudson Valley, where many city dwellers temporarily or permanently relocated in the past year.

The only metro area to post a decline in the first quarter from a year earlier was Springfield, Ill., where median prices fell 2.4%, NAR said.

In the Boise, Idaho, metro area, where median home prices surged 32.8%,

Julie Cook

struggled to find a house within her budget. She and her mother moved to Boise from Florida in January. Ms. Cook had looked at house listings before she moved and planned to buy a house in Boise for under $300,000. But by the time she arrived, there was little that amount could buy.

Ms. Cook ended up purchasing a townhouse for $330,000 in March. “It’s really not my dream or anything,” she said. “But I felt like I needed to, for mine and my mom’s sake, find a place that we could afford.”

Nationwide, the median existing-home sales price rose 16.2% in the first quarter to $319,200, a record high in data going back to 1989, NAR said.

Prices are rising so rapidly that they are outweighing the benefit of rock-bottom borrowing rates. In the first quarter, the typical monthly mortgage payment rose to $1,067, from $995 a year earlier, NAR said, even as mortgage rates declined.

In many metro areas, demand would still exceed the available supply even if some buyers dropped out, real-estate executives said.

“Despite an 11% increase in our average sales price over the last year, demand for our homes has never been higher,” said

Eric Lipar,

chief executive of home builder LGI Homes Inc., in an earnings call this month.

Write to Nicole Friedman at nicole.friedman@wsj.com