I'm going to give a very much anecdotal answer here because I don't think you'll find any research at all on this, though it'd be great if we had some. (If someone does have some, send it over!)
The distinction between short-term and longterm is critical here. However, what constitutes what? I don't know! But I'll give some scenarios.
I'm a personal trainer who works with everyday people. Average age has probably been around the mid 40s. (We often forget the average adult age in the U.S. is 38!)
I've gotten a lot of clients who say something to the tune of "I used to be very active in my 20s." Or "I played sports growing up."
In my experience, when it's been something like 10-20 years (or more) since the person was active, you ignore it. It's true some of them and you'll be able to tell "Oh, they're picking this up quickly." Or "This is definitely easier for them than most."
But it's not like the common muscle memory scenario we're all familiar with. The 20 year old who has been lifting for a long time, takes a few months off, and then in a few weeks is basically back at full strength.
In fact, with the older crowd who tells me they were once very active, I'm more inclined to hold them back. I've had some who say, start doing push-ups, and then all of a sudden get mentally transformed to thinking they're 25 years old again, only to be greatly disappointed when they can't pump out 20 in a row anymore. That is, I worry about them trying to do too much, too soon, and getting hurt.
A skill these people often never attained was knowing their body at their older age. They don't know how much longer it will take them to recover, how much more sore they'll be, how much less their joints will like something.
Of course, I've given you two extremes. A few months off vs decades off. What about a year off? Or two years? Well, you can certainly be more aggressive than 20 years off, but it's hard to quantify it.
No matter what, I always make people's first one or two workouts back a 50% day. Where after the workout, I want them to leave thinking "Yeah, I could go do all that again." Then we pretty much go by the eyeball and how-do-you-feel test from there.