Yes, your suspicion can be very much true. You can try simple experiment and ask your hubby to prepare another sample but using disposable gloves. When I used to work in forensic field we would refer to it as a "touch DNA", where sample could be obtained from fingerprint. There is no magic but very small fragments of dried skin left behind. If you handle sticky objects such as masking tape, there will be many of those tiny skin fragments attached to it. That's how you can actually lift DNA sample from objects for analysis.
For more details, please check the link: