Prevent inadvertently writing config when destroying SensorFaceController
In certain cases, we may end up with a dirty config which we do not want to store. So explicitly cleanup the config objects before destruction to prevent them from being written.