backends/drm: some fixes for hardware rotation

commit 1: don't test hardware rotation if not supported by the driver. @vladz this should fix the long waiting time you saw with the NVidia driver

commit 2: fix bitmask properties. The properties were effectively assumed to have fixed values despite the enum mapping, and the enum mapping was done wrong. Now it should be correct