vbltest: Use util_open()

Use the new util_open() helper instead of open-coding the method for
finding a usable device. While at it, this adds -D and -M command-line
options to vbltest to make its usage more consistent with its siblings
modetest and proptest.

Signed-off-by: Thierry Reding <treding@nvidia.com>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>