For investors whose retirement is decades away, the stock market’s pullback should be of little concern — there’s plenty of time for your portfolio to recover before you need the money.

Yet if you are a new retiree or on the verge of retiring, it’s worth considering what a prolonged dip would mean for your portfolio over the long-term.

Basically, down markets can pose significant “sequence of returns” risk in the early years of retirement. That risk basically is about how the order, or sequence, of stock returns over time — combined with your portfolio withdrawals — can impact your balance down the road.

More from Personal Finance:
Here’s what to know about your 2022 Medicare costs
How rising inflation may affect your 2021 tax bill
Retirees need to keep this much cash, advisors say

“If there’s a downturn early on, it can derail a whole retirement plan,” said Wade Pfau, a professor of retirement income at the American College of Financial Services.

The major indexes have had a rough week. Through Thursday’s close, the S&P 500 index has shed 3.9%, the Dow Jones industrial average is off roughly 3.4% and the Nasdaq composite index has slid 4.9%. Year to date, the S&P has lost 5.9%, and the Dow and Nasdaq have dropped 4.4% and 9.5%, respectively.

Generally, down markets can present a buying opportunity for investors because they’re purchasing stock at a lower price than they would have otherwise.

However, it also means that if you sell, you’re doing so at depressed prices. And for retirees especially, that can be problematic.

“If there’s a big loss in the market and you’re taking withdrawals, you could be taking more from your portfolio than what it can make up for,” said certified financial planner Avani Ramnani, managing director at Francis Financial in New York.

“If that happens early in retirement … the recovery may be very weak and put you in danger of not recovering at all or being lower than where you would have been and therefore jeopardizing your retirement lifestyle,” Ramnani said.

Here’s how a sequence of returns risk can impact your savings: Say a person had retired at the turn of the century with $1 million invested in the S&P and withdrew $40,000 each year, with withdrawals after the first year adjusted 2% for inflation.

In 2020, the remaining balance would have been about $470,000, according to Ben Carlson, director of institutional asset management for Ritholtz Wealth Management, who crunched the numbers for a blog post.

In the above scenario, the portfolio would have been subject to a bear market at the outset of the person’s retirement, when the S&P lost 37% over three years during 2000-2002, but enjoyed a long-running bull market that began in 2009.

However, if the order of yearly returns were flipped — the gains posted by the S&P at the end of the 20 years happened first and that early bear market happened last — that same person would have more than $2.3 million after withdrawing the $40,000 or inflation-adjusted amount each year.

“It’s not the specific returns over time but the order of those returns that matter,” Pfau said.

The good news is that there are options for mitigating the risk.

The first is to simply plan to spend more conservatively, Pfau said. In other words, the less you spend consistently, the less you have to withdraw overall.

Another strategy is to adjust your spending when your portfolio performance is suffering. 

“You look at your expenses and see if there are any you can stop,” Ramnani said. “So maybe you don’t take a trip, or you delay doing a large renovation that would require a big distribution.”

You also can actively reduce risk in your portfolio, Pfau said. For instance, you could have a low stock allocation early in retirement but increase it over time, or use bonds for short-term expenses and stocks for long-term ones.

“You’re strategically reducing volatility,” Pfau said.

The last option is to have assets outside your investment portfolio that can support your spending needs when stocks are underperforming.

“You would use that as a temporary resource while you wait for your portfolio to recover,” Pfau said. 

He said that buffer could be cash, a reverse mortgage line of credit or permanent life insurance with a cash value, assuming it’s protected from market losses.

Additionally, given how well the market has generally performed over the last decade, you may simply be able to meet your goals without taking on the risk that comes with stocks.

“You could take some of that volatility off the table,” Pfau said.