Remove workaround for nice serial number of Yubikeys

GnuPG-bug-id: 5100
2 jobs for master in 17 minutes and 9 seconds (queued for 12 minutes and 56 seconds)
Status Name Job ID Coverage
  Build
passed build_clazy_clang_tidy #79977

00:11:02

passed linux kf5-qt5 qt5.14 #79976

00:06:06

0.0%