mm: workingset: don't drop refault information prematurely