Why hands are the hardest region
A hand is a small region of the image with an enormous amount of structure: five articulated chains, a palm with its own geometry, and usually another hand or an object interacting with it.
It is also almost always occluded. Fingers overlap each other, the palm hides the lower joints, and the hand disappears into a sleeve or behind an object, which means the parts that would explain the pose are not visible.
Training data makes this worse rather than better. The web is full of hands in the same few poses, and far less of the unusual angles that photographs and illustration actually need.
So the failure is not mysterious or arbitrary. It is a small, complex, self-occluding object being reconstructed from ambiguous evidence, which is also why it survives longer than faces do.