NEW YORK (AP) – When Ryan David bought three rental properties back in 2017, he expected the $1,000-a-month he was pocketing after expenses would be regular sources of income well into his retirement years.

He also was counting on the rent money from the properties in Dupont, Pennsylvania, to help with the cash flow of his business buying and selling distressed properties, launched early last year.

But then the pandemic hit and federal and state authorities imposed moratoriums on evictions. The unpaid rent began to mount. Then, just when he thought the worst was over, the Centers for Disease Control and Prevention announced a new moratorium, lasting until October 3. A federal judge dismissed a legal challenge to the order last week.

Mr. David, the father of a 2 1/2-year-old who is expecting another child, fears the $2,000 he has owed in back rent will quickly climb to thousands more.

The latest moratorium “was the final gut punch,” said the 39-year-old, adding that he now plans to sell the apartments. “I have had this internal struggle going back and forth. I have lost sleep at night, and I have now come up with a decision to sell and walk away.”

Most evictions for unpaid rent have been halted since the early days of the pandemic and there are now more than 15 million people living in households that owe as much as $20 billion in back rent, according to the Aspen Institute.

A majority of single-family rental homeowners have been impacted, according to a survey from the National Rental Home Council, and 50 percent say they have tenants who have missed rent during the pandemic.

Smaller landlords with fewer than four units, who often do not have the financing of larger property owners, were hit especially hard, with as many as 58 percent having tenants behind on rent, according to the National Association of Realtors. More than half of back rent is owed to smaller landlords.

Landlords, big and small, are most angry about the moratoriums, which they consider illegal. Many believe some tenants could have paid rent, if not for the moratorium. And the $47 billion in federal rental assistance that was supposed to make landlords whole has been slow to materialize. By July, only $3 billion of the first tranche of $25 billion had been distributed.

Mr. David points to two tenants who received paychecks throughout the pandemic but did not pay rent or bother to file for rental assistance. Others singled out delinquent tenants who they claimed still managed to drive a luxury car, get food deliveries or go on vacation.

“Without rent, we’re out of business,” said Gary Zaremba, who sold 40 of his properties in Ohio due to the moratorium and still has a quarter of his tenants in the remaining 100 buildings struggling to pay rent. He has helped some apply for rental assistance, he said.

“It’s like a restaurant that doesn’t have patrons,” he said. “I don’t get the rent. I can’t pay my maintenance staff. I have to lay them off. I can’t fix the buildings and keep them in good repair. So, that means they are going to get even worse off. I can’t pay my taxes.”

Mr. Zaremba, who also owns a handful of properties in New York City, sold some of his single-family homes to home buyers and some multi-family commercial apartment buildings to small investors.

Many landlords are saddled with tens of thousands of dollars in lost rent—money that was meant for retirement, a college fund or for their investors, who themselves had sought a safe investment. They are maxing out credit cards or dipping into savings to pay property taxes, staff salaries, insurance, water bills and maintenance.

“I keep thinking to myself, when does my family get paid?” said Matthew Haines, who owns 253 units with his wife in the Dallas/Fort Worth area and is owed more than $300,000 in back rent. He has referred $250,000 of that to collections.

The couple put in $50,000 of their own money to avoid laying off their seven full-time and three part-time employees. Mr. Haines is also doing repairs like fixing an air conditioning unit or changing a pool light himself to save money. Their investors, retirees who typically get an annual return of 7 to 9 percent, got nothing last year on two multifamily apartments and 3 percent on a third one because of unpaid rent.

“We jumped through hoops to help our residents who were struggling. We have not evicted a single person trying to work with us, even though we have people who owe us seven, eight, nine months of rent,” he said. “We are trying to do the right thing but it’s becoming impossible.”

In upstate New York, Michael Reid sold three of his houses to stem losses—after paying some delinquent tenants thousands of dollars to leave. Already out more than $100,000 in back rent on 13 of his 31 units and more than $20,000 in unpaid water bills, Mr. Reid took out a $90,000 home equity loan on his house so he could pay property taxes and other bills. On Tuesday, he finally received $9,000 in federal rental assistance, a fraction of what he is owed.

“I’ve lost an incredible amount of money on top of the rent owed,” said Mr. Reid, who also works as a mortgage loan officer, referring to his delinquent tenants in Binghamton and Endicott, New York.

Some owners are taking advantage of a red-hot housing market to sell their units to deep-pocketed investors willing to wait out the moratorium or to families who plan to live in them. Buyers are increasingly out-of-town investors or equity funds, whom critics fear will renovate the properties and market them at much higher prices.

“A lot of landlords are disgusted. They are selling at losses. They are getting out, period,” Mr. Reid said of the dozens of investors he talks with.