Oooh... I read your post too fast.Of course I was not the applicable user. It's the HD from a fried laptop. I Pluged it into the new laptop via USB. I couldn't access the folder before granting permissions to the user on the new laptop. And that sure took longer than I expected...
In that case, I've experienced much the same thing when copying "special" folders to another PC.
Haven't found or really looked for a solution, but I also haven't experienced a wait of more than a couple minutes to gain permission.
There must be some sort of metadata to designate a directory as being special (like Program Files). I'm very interested in whether that metadata can be thrown away somehow.
Just a few weeks ago, I got a new PC and simply threw away most of a backup of an old Google Drive folder since random files throughout refused to be moved, even after assigning myself Ownership.