Discussion:
13-CURRENT: several GB swap being used despite plenty of free RAM
Add Reply
Rebecca Cran via freebsd-hackers
2018-11-17 21:44:41 UTC
Reply
Permalink
I'm running 13-CURRENT from a few days ago. I noticed the system using several
GB of swap despite having 90GB RAM still free. I know FreeBSD will use *some*,
but 3450MB seems excessive when there's still 90GB RAM free.

CPU: 0.1% user, 0.0% nice, 0.0% system, 0.0% interrupt, 99.8% idle
Mem: 3392M Active, 5489M Inact, 390M Laundry, 25G Wired, 90G Free
ARC: 19G Total, 8080M MFU, 9242M MRU, 64K Anon, 197M Header, 1570M Other
15G Compressed, 26G Uncompressed, 1.73:1 Ratio
Swap: 8192M Total, 3450M Used, 4742M Free, 42% Inuse, 36K In

Quitting firefox caused the swap usage to drop to just 460MB. I don't
understand why it would decide to swap out so much, especially since I have
vm.swap_idle_enabled set to 0.
--
Rebecca
Rebecca Cran via freebsd-hackers
2018-11-17 22:21:33 UTC
Reply
Permalink
On Saturday, 17 November 2018 14:44:41 MST Rebecca Cran via freebsd-hackers
Post by Rebecca Cran via freebsd-hackers
Quitting firefox caused the swap usage to drop to just 460MB. I don't
understand why it would decide to swap out so much, especially since I have
vm.swap_idle_enabled set to 0.
Of course it was because just before looking at top, I'd been running
something that used up all the memory. I'd run "grep -rHn" and it had wanted
to use over 120GB memory, which caused all the swapouts.
--
Rebecca
Wojciech Puchar
2018-11-17 21:52:22 UTC
Reply
Permalink
freebsd will not swap with that lots of free ram.
but it's 90GB free NOW, how about before?

If something got swapped it isn't read back just because there is free
memory but when it will be needed.
Post by Rebecca Cran via freebsd-hackers
I'm running 13-CURRENT from a few days ago. I noticed the system using several
GB of swap despite having 90GB RAM still free. I know FreeBSD will use *some*,
but 3450MB seems excessive when there's still 90GB RAM free.
CPU: 0.1% user, 0.0% nice, 0.0% system, 0.0% interrupt, 99.8% idle
Mem: 3392M Active, 5489M Inact, 390M Laundry, 25G Wired, 90G Free
ARC: 19G Total, 8080M MFU, 9242M MRU, 64K Anon, 197M Header, 1570M Other
15G Compressed, 26G Uncompressed, 1.73:1 Ratio
Swap: 8192M Total, 3450M Used, 4742M Free, 42% Inuse, 36K In
Quitting firefox caused the swap usage to drop to just 460MB. I don't
understand why it would decide to swap out so much, especially since I have
vm.swap_idle_enabled set to 0.
--
Rebecca
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
Rebecca Cran via freebsd-hackers
2018-11-17 22:22:51 UTC
Reply
Permalink
Post by Wojciech Puchar
freebsd will not swap with that lots of free ram.
but it's 90GB free NOW, how about before?
If something got swapped it isn't read back just because there is free
memory but when it will be needed.
That was it: just before, I'd run grep and canceled it after it reached about
120GB memory usage!
--
Rebecca
Ian Lepore
2018-11-17 23:59:48 UTC
Reply
Permalink
Post by Wojciech Puchar
freebsd will not swap with that lots of free ram.
but it's 90GB free NOW, how about before?
Your information is outdated. For at least a couple years now (since
approximately the 10.1 - 10.2 timeframe is my vague estimate), freebsd
will page out application memory that hasn't been referenced for some
time, even when the system has no shortage of free memory at all.

The advice I was recently given to revert to the old behavior is:

  sysctl vm.pageout_update_period=0

I've been using it on a couple systems here for a few days now, and so
far results are promising, I am no longer seeing gratuitous swapfile
usage on systems that have so much free physical ram that they should
never need to page anything out. I haven't yet pushed one of those
systems hard enough to check what happens when they do need to start
proactively paging out inactive memory due to shortages -- it could be
that turning off the new behavior has downsides for some workloads.

-- Ian
Post by Wojciech Puchar
 
If something got swapped it isn't read back just because there is
free 
memory but when it will be needed.
Post by Rebecca Cran via freebsd-hackers
I'm running 13-CURRENT from a few days ago. I noticed the system using several
GB of swap despite having 90GB RAM still free. I know FreeBSD will use *some*,
but 3450MB seems excessive when there's still 90GB RAM free.
CPU:  0.1% user,  0.0% nice,  0.0% system,  0.0% interrupt, 99.8%
idle
Mem: 3392M Active, 5489M Inact, 390M Laundry, 25G Wired, 90G Free
ARC: 19G Total, 8080M MFU, 9242M MRU, 64K Anon, 197M Header, 1570M Other
    15G Compressed, 26G Uncompressed, 1.73:1 Ratio
Swap: 8192M Total, 3450M Used, 4742M Free, 42% Inuse, 36K In
Quitting firefox caused the swap usage to drop to just 460MB. I don't
understand why it would decide to swap out so much, especially since I have
vm.swap_idle_enabled set to 0.
Mark Johnston
2018-11-18 00:13:18 UTC
Reply
Permalink
Post by Ian Lepore
Post by Wojciech Puchar
freebsd will not swap with that lots of free ram.
but it's 90GB free NOW, how about before?
Your information is outdated. For at least a couple years now (since
approximately the 10.1 - 10.2 timeframe is my vague estimate), freebsd
will page out application memory that hasn't been referenced for some
time, even when the system has no shortage of free memory at all.
No, FreeBSD will only ever swap when there is a free page shortage. The
difference is that we now slowly age unreferenced pages into the
inactive queue, which makes them candidates for pageout and subsequent
eviction. With pageout_update_period=0, anonymous memory won't get
paged out unless there's a shortage of inactive pages, or an application
calls madvise(MADV_DONTNEED) on a range of memory (which moves any
backing pages to the inactive queue).
Post by Ian Lepore
  sysctl vm.pageout_update_period=0
I've been using it on a couple systems here for a few days now, and so
far results are promising, I am no longer seeing gratuitous swapfile
usage on systems that have so much free physical ram that they should
never need to page anything out. I haven't yet pushed one of those
systems hard enough to check what happens when they do need to start
proactively paging out inactive memory due to shortages -- it could be
that turning off the new behavior has downsides for some workloads.
Mark Millard via freebsd-hackers
2018-11-18 00:41:55 UTC
Reply
Permalink
Post by Mark Johnston
Post by Ian Lepore
Post by Wojciech Puchar
freebsd will not swap with that lots of free ram.
but it's 90GB free NOW, how about before?
Your information is outdated. For at least a couple years now (since
approximately the 10.1 - 10.2 timeframe is my vague estimate), freebsd
will page out application memory that hasn't been referenced for some
time, even when the system has no shortage of free memory at all.
No, FreeBSD will only ever swap when there is a free page shortage. The
difference is that we now slowly age unreferenced pages into the
inactive queue, which makes them candidates for pageout and subsequent
eviction. With pageout_update_period=0, anonymous memory won't get
paged out unless there's a shortage of inactive pages, or an application
calls madvise(MADV_DONTNEED) on a range of memory (which moves any
backing pages to the inactive queue).
Swapping is built on top of paging as I understand. The system
can page without swapping but can not swap without (effectively)
paging to implement the swapping, if I understand right. If I
understand right, swapped-out means that kernel stacks have
been written out and have to be loaded back in RAM before the
process/threads can even run. (I might not understand.)

If I've got that right, are there distinctions here for
paging that is not part of swapping vs. actual swapping
(and its use of paging)? Saying that something does not
swap does not necessarily imply that it does not page:
it still could have paging activity that does not include
moving the kernel stacks for the process to backing media?

At times I have trouble interpreting when wording goes back
and forth between swapping and paging, both for the intended
meaning and for the technical implications.
Post by Mark Johnston
Post by Ian Lepore
sysctl vm.pageout_update_period=0
I've been using it on a couple systems here for a few days now, and so
far results are promising, I am no longer seeing gratuitous swapfile
usage on systems that have so much free physical ram that they should
never need to page anything out. I haven't yet pushed one of those
systems hard enough to check what happens when they do need to start
proactively paging out inactive memory due to shortages -- it could be
that turning off the new behavior has downsides for some workloads.
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Mark Johnston
2018-11-18 01:51:43 UTC
Reply
Permalink
Post by Mark Millard via freebsd-hackers
Post by Mark Johnston
Post by Ian Lepore
Post by Wojciech Puchar
freebsd will not swap with that lots of free ram.
but it's 90GB free NOW, how about before?
Your information is outdated. For at least a couple years now (since
approximately the 10.1 - 10.2 timeframe is my vague estimate), freebsd
will page out application memory that hasn't been referenced for some
time, even when the system has no shortage of free memory at all.
No, FreeBSD will only ever swap when there is a free page shortage. The
difference is that we now slowly age unreferenced pages into the
inactive queue, which makes them candidates for pageout and subsequent
eviction. With pageout_update_period=0, anonymous memory won't get
paged out unless there's a shortage of inactive pages, or an application
calls madvise(MADV_DONTNEED) on a range of memory (which moves any
backing pages to the inactive queue).
Swapping is built on top of paging as I understand. The system
can page without swapping but can not swap without (effectively)
paging to implement the swapping, if I understand right.
Right.
Post by Mark Millard via freebsd-hackers
If I
understand right, swapped-out means that kernel stacks have
been written out and have to be loaded back in RAM before the
process/threads can even run. (I might not understand.)
When free pages are scarce, one measure that the kernel may take to
address the shortage is to swap out the kernel stacks of the threads in
a process, thus allowing the pages backing the stacks to be reused for
some other purpose, but preventing that process from running on a CPU
until the stacks are swapped back in and "locked" (wired) into memory.

Most of the pages consumed by an application like firefox are not used
for kernel stacks. Most of them are used for the application's heap
memory, and are thus private to that process. In general, pieces of
such memory are subject to being paged out to the swap device,
particularly when they are not frequently referenced (read or written
to) by the application, in order to replenish the pool of free pages.
Such memory is often said to be swapped out.

As a side note, there are some system calls that modify this behaviour.
mlock(2) effectively prevents the kernel from swapping out pages backing
the specified virtual addresses; this guarantees that an access of the
virtual memory range will never incur the cost of an expensive page-in.
madvise(MADV_FREE) tells the kernel that the specified pages may be
freed without first being written to the swap device. Thus, a
subsequent read of an affected page may return the page's previous
contents (if the page had not yet been reclaimed to make up for a
shortage), or all zeroes (if the page had been freed without saving its
contents to swap).
Post by Mark Millard via freebsd-hackers
If I've got that right, are there distinctions here for
paging that is not part of swapping vs. actual swapping
(and its use of paging)? Saying that something does not
it still could have paging activity that does not include
moving the kernel stacks for the process to backing media?
Indeed, as I tried to describe above, kernel stack swapouts usually
represent only a small portion of many applications' total swap
usage, to the point where I at least usually don't think much about
them.
Post by Mark Millard via freebsd-hackers
At times I have trouble interpreting when wording goes back
and forth between swapping and paging, both for the intended
meaning and for the technical implications.
When discussing paging activity related to the swap pager, I
typically use those terms interchangeably. Sorry for the confusion.
Loading...