Also avoid trying to evict more tiles if the tiles manager is empty (all tiles were previously removed)