xf86drm: Be sure to closedir before return

removed in commit bb45ce4e

Adding it back as it is still needed in the case where we don't find a
match.

Signed-off-by: Kevin Strasser <kevin.strasser@intel.com>
Fixes: bb45ce4e "libdrm: Use readdir instead of readdir_r to
                             avoid build warnings"$
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>