Alan Somers
2018-10-22 16:02:59 UTC
The SNIA Storage Developers' Conference was held in Santa Clara during the
last week of September. Jim Harris, John Hixon, Nick Principe, Michael
Dexter, and myself attended. As far as FreeBSD goes, here's a summary of
the juiciest bits:
NVDIMM/PMEM: A lot of companies are still pushing persistent memory
products. They're getting better, but still quite vendor-specific.
Fortunately, there are standardization efforts in place. JEDEC is
standardizing the hardware (NVDIMM-N, NVDIMM-P, NVDIMM-F). Every major
memory company (but not CPU company) is on-board. SNIA is also trying to
standardize a programming model (but not the precise API). Windows and
Linux currently support it, with differences. There will probably be some
additional changes to the model.
https://static.ptbl.co/static/attachments/187585/1537988510.pdf?1537988510
. iX Systems reported some impressive benchmarks using an NVDIMM as a ZFS
slog device. Several databases are adding pmem support. A few filesystems
have some level of NVDIMM support, and the NOVA filesystem is being written
from the ground up to take full advantage of NVDIMM. For example,
directories are stored as in-memory data structures that never get
serialized. The lesson here is that FreeBSD needs to support the standard
NVDIMM programming model too.
OpenChannel SSDs: These are SSDs that expose more of their internal
implementation details to the host. Specifically, they rely on the host
for at least part of garbage collection. They also expose their multiple
internal busses to the host, so it can choose how to stripe data across
them. Overall, the programming model is surprisingly similar to that of
SMR hard drives. Unfortunately, the standard is a bit murky. Different
speakers could not even agree on whether there is a standard. This is the
best presentation on the topic:
https://static.ptbl.co/static/attachments/187321/1537829929.pdf?1537829929
and this is the closest thing there is to a standard ATM:
https://openchannelssd.readthedocs.io/en/latest/ . The lesson here is that
FreeBSD needs to plumb these devices' properties up to userland and perhaps
expose them in zonectl(8) (easy) and add filesystem support (very hard).
NVMe: If there were an award for most popular buzzword, it would've gone to
"NVMe". Everybody and their mother had something to say about it. But I
personally paid little attention (except as regards OpenChannel).
Seagate dual-actuator hard drives: Seagate is coming out with hard drives
that pack two servos into a single case. Each servo can access half of the
platters. The drive reports each servo as a separate LUN to the host.
There is little FreeBSD needs to do here. To make zfsd(8) work correctly,
we should add lun info to the drives' physical path strings. And it might
be nice if zpool(8) prevented the user from adding both LUNs of the same
physical drive to the same RAID group. But that's arguably out of our
domain.
SPDK: The storage-plane developer's kit is like Intel's version of Netmap,
but for storage. It's a say for userland programs to access storage
devices directly, bypassing the kernel. The benefits are negligible for
spinning media, but can be significant for fast NVMe drives. SPDK has
multiple backends for different I/O controllers, including some that are
kernel-based. Notably lacking is a POSIX AIO backend. That's probably the
biggest gap in its FreeBSD support.
iX Systems wrote a blog post about the conference, too. It covers
Swordfish and Samba, two topics I ignored.
https://www.ixsystems.com/blog/snia-sdc-2018/
-Alan
last week of September. Jim Harris, John Hixon, Nick Principe, Michael
Dexter, and myself attended. As far as FreeBSD goes, here's a summary of
the juiciest bits:
NVDIMM/PMEM: A lot of companies are still pushing persistent memory
products. They're getting better, but still quite vendor-specific.
Fortunately, there are standardization efforts in place. JEDEC is
standardizing the hardware (NVDIMM-N, NVDIMM-P, NVDIMM-F). Every major
memory company (but not CPU company) is on-board. SNIA is also trying to
standardize a programming model (but not the precise API). Windows and
Linux currently support it, with differences. There will probably be some
additional changes to the model.
https://static.ptbl.co/static/attachments/187585/1537988510.pdf?1537988510
. iX Systems reported some impressive benchmarks using an NVDIMM as a ZFS
slog device. Several databases are adding pmem support. A few filesystems
have some level of NVDIMM support, and the NOVA filesystem is being written
from the ground up to take full advantage of NVDIMM. For example,
directories are stored as in-memory data structures that never get
serialized. The lesson here is that FreeBSD needs to support the standard
NVDIMM programming model too.
OpenChannel SSDs: These are SSDs that expose more of their internal
implementation details to the host. Specifically, they rely on the host
for at least part of garbage collection. They also expose their multiple
internal busses to the host, so it can choose how to stripe data across
them. Overall, the programming model is surprisingly similar to that of
SMR hard drives. Unfortunately, the standard is a bit murky. Different
speakers could not even agree on whether there is a standard. This is the
best presentation on the topic:
https://static.ptbl.co/static/attachments/187321/1537829929.pdf?1537829929
and this is the closest thing there is to a standard ATM:
https://openchannelssd.readthedocs.io/en/latest/ . The lesson here is that
FreeBSD needs to plumb these devices' properties up to userland and perhaps
expose them in zonectl(8) (easy) and add filesystem support (very hard).
NVMe: If there were an award for most popular buzzword, it would've gone to
"NVMe". Everybody and their mother had something to say about it. But I
personally paid little attention (except as regards OpenChannel).
Seagate dual-actuator hard drives: Seagate is coming out with hard drives
that pack two servos into a single case. Each servo can access half of the
platters. The drive reports each servo as a separate LUN to the host.
There is little FreeBSD needs to do here. To make zfsd(8) work correctly,
we should add lun info to the drives' physical path strings. And it might
be nice if zpool(8) prevented the user from adding both LUNs of the same
physical drive to the same RAID group. But that's arguably out of our
domain.
SPDK: The storage-plane developer's kit is like Intel's version of Netmap,
but for storage. It's a say for userland programs to access storage
devices directly, bypassing the kernel. The benefits are negligible for
spinning media, but can be significant for fast NVMe drives. SPDK has
multiple backends for different I/O controllers, including some that are
kernel-based. Notably lacking is a POSIX AIO backend. That's probably the
biggest gap in its FreeBSD support.
iX Systems wrote a blog post about the conference, too. It covers
Swordfish and Samba, two topics I ignored.
https://www.ixsystems.com/blog/snia-sdc-2018/
-Alan