Make sure we don't reset the paginated models when items are refreshed
outside of the viewed scope.
Should speed up boot, reduce some allocations and remove some clunkinesses