Discussion:
cap_sysctlbyname for hw.vmm.destroy
Ionuț Mihalache
2021-04-26 14:16:14 UTC
Permalink
Hello,

I am working on adding capsicum support for the bhyve snapshot feature. At
the end of the suspend process, the guest should be destroyed and the code
handles this part with a sysctlbyname call which is not working in
capability mode. I don't know what is the problem but even when using
cap_sysctlbyname I still get the same error code (EPERM). I tried the
example from the documentation aswell [1] and still the same error code.
What could be the problem? I have a FreeBSD13 host and a FreeBSD13 guest.

[1] -
https://www.freebsd.org/cgi/man.cgi?query=cap_sysctl&apropos=0&sektion=0&manpath=FreeBSD+13-current&arch=default&format=html
Mark Johnston
2021-04-26 17:39:52 UTC
Permalink
Post by Ionuț Mihalache
Hello,
I am working on adding capsicum support for the bhyve snapshot feature. At
the end of the suspend process, the guest should be destroyed and the code
handles this part with a sysctlbyname call which is not working in
capability mode. I don't know what is the problem but even when using
cap_sysctlbyname I still get the same error code (EPERM). I tried the
example from the documentation aswell [1] and still the same error code.
What could be the problem? I have a FreeBSD13 host and a FreeBSD13 guest.
I'm not sure why it would happen unless the casper process is somehow
running as a non-root user. Can you share the code you're testing
somewhere?
Ionuț Mihalache
2021-04-28 11:30:26 UTC
Permalink
I tried to test the example from the documentation between here [1] and
here [2]. The code stops here [3].

[1] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1538
[2] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1585
[3] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1581
Post by Mark Johnston
Post by Ionuț Mihalache
Hello,
I am working on adding capsicum support for the bhyve snapshot feature.
At
Post by Ionuț Mihalache
the end of the suspend process, the guest should be destroyed and the
code
Post by Ionuț Mihalache
handles this part with a sysctlbyname call which is not working in
capability mode. I don't know what is the problem but even when using
cap_sysctlbyname I still get the same error code (EPERM). I tried the
example from the documentation aswell [1] and still the same error code.
What could be the problem? I have a FreeBSD13 host and a FreeBSD13 guest.
I'm not sure why it would happen unless the casper process is somehow
running as a non-root user. Can you share the code you're testing
somewhere?
Mark Johnston
2021-04-28 13:52:55 UTC
Permalink
Post by Ionuț Mihalache
I tried to test the example from the documentation between here [1] and
here [2]. The code stops here [3].
I think you're referencing an old version of the cap_sysctl man page?
See the example from the copy in your repo:
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/lib/libcasper/services/cap_sysctl/cap_sysctl.3#L122
In particular, when setting limits consumers should not be building
nvlists directly.
Post by Ionuț Mihalache
[1] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1538
[2] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1585
[3] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1581
Post by Mark Johnston
Post by Ionuț Mihalache
Hello,
I am working on adding capsicum support for the bhyve snapshot feature.
At
Post by Ionuț Mihalache
the end of the suspend process, the guest should be destroyed and the
code
Post by Ionuț Mihalache
handles this part with a sysctlbyname call which is not working in
capability mode. I don't know what is the problem but even when using
cap_sysctlbyname I still get the same error code (EPERM). I tried the
example from the documentation aswell [1] and still the same error code.
What could be the problem? I have a FreeBSD13 host and a FreeBSD13 guest.
I'm not sure why it would happen unless the casper process is somehow
running as a non-root user. Can you share the code you're testing
somewhere?
Ionuț Mihalache
2021-04-28 14:22:22 UTC
Permalink
I update the code now [1] but still the same error. Even without any limits
the cap_sysctlbyname fails after using cap_enter.

[1] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/c54dce7590b065a757dff0f68fd921aca380670f/usr.sbin/bhyve/bhyverun.c#L1567
Post by Mark Johnston
Post by Ionuț Mihalache
I tried to test the example from the documentation between here [1] and
here [2]. The code stops here [3].
I think you're referencing an old version of the cap_sysctl man page?
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/lib/libcasper/services/cap_sysctl/cap_sysctl.3#L122
In particular, when setting limits consumers should not be building
nvlists directly.
Post by Ionuț Mihalache
[1] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1538
Post by Ionuț Mihalache
[2] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1585
Post by Ionuț Mihalache
[3] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1581
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
Hello,
I am working on adding capsicum support for the bhyve snapshot
feature.
Post by Ionuț Mihalache
Post by Mark Johnston
At
Post by Ionuț Mihalache
the end of the suspend process, the guest should be destroyed and the
code
Post by Ionuț Mihalache
handles this part with a sysctlbyname call which is not working in
capability mode. I don't know what is the problem but even when using
cap_sysctlbyname I still get the same error code (EPERM). I tried the
example from the documentation aswell [1] and still the same error
code.
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
What could be the problem? I have a FreeBSD13 host and a FreeBSD13
guest.
Post by Ionuț Mihalache
Post by Mark Johnston
I'm not sure why it would happen unless the casper process is somehow
running as a non-root user. Can you share the code you're testing
somewhere?
Mark Johnston
2021-04-28 14:41:51 UTC
Permalink
There are two bugs in the example, also present in your WIP. I fixed
them here:
https://cgit.freebsd.org/src/commit/?id=44bbda649dc6c1cdc5a99641e14c77157967e140
Post by Ionuț Mihalache
I update the code now [1] but still the same error. Even without any limits
the cap_sysctlbyname fails after using cap_enter.
[1] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/c54dce7590b065a757dff0f68fd921aca380670f/usr.sbin/bhyve/bhyverun.c#L1567
Post by Mark Johnston
Post by Ionuț Mihalache
I tried to test the example from the documentation between here [1] and
here [2]. The code stops here [3].
I think you're referencing an old version of the cap_sysctl man page?
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/lib/libcasper/services/cap_sysctl/cap_sysctl.3#L122
In particular, when setting limits consumers should not be building
nvlists directly.
Post by Ionuț Mihalache
[1] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1538
Post by Ionuț Mihalache
[2] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1585
Post by Ionuț Mihalache
[3] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1581
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
Hello,
I am working on adding capsicum support for the bhyve snapshot
feature.
Post by Ionuț Mihalache
Post by Mark Johnston
At
Post by Ionuț Mihalache
the end of the suspend process, the guest should be destroyed and the
code
Post by Ionuț Mihalache
handles this part with a sysctlbyname call which is not working in
capability mode. I don't know what is the problem but even when using
cap_sysctlbyname I still get the same error code (EPERM). I tried the
example from the documentation aswell [1] and still the same error
code.
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
What could be the problem? I have a FreeBSD13 host and a FreeBSD13
guest.
Post by Ionuț Mihalache
Post by Mark Johnston
I'm not sure why it would happen unless the casper process is somehow
running as a non-root user. Can you share the code you're testing
somewhere?
Ionuț Mihalache
2021-04-28 15:19:09 UTC
Permalink
Even with the fixes cap_sysctl still returns EPERM for that simple example.
Post by Mark Johnston
There are two bugs in the example, also present in your WIP. I fixed
https://cgit.freebsd.org/src/commit/?id=44bbda649dc6c1cdc5a99641e14c77157967e140
Post by Ionuț Mihalache
I update the code now [1] but still the same error. Even without any
limits
Post by Ionuț Mihalache
the cap_sysctlbyname fails after using cap_enter.
[1] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/c54dce7590b065a757dff0f68fd921aca380670f/usr.sbin/bhyve/bhyverun.c#L1567
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
I tried to test the example from the documentation between here [1]
and
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
here [2]. The code stops here [3].
I think you're referencing an old version of the cap_sysctl man page?
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/lib/libcasper/services/cap_sysctl/cap_sysctl.3#L122
Post by Ionuț Mihalache
Post by Mark Johnston
In particular, when setting limits consumers should not be building
nvlists directly.
Post by Ionuț Mihalache
[1] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1538
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
[2] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1585
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
[3] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/3a08ffe4839de9b8396b1760f1dc42b066428807/usr.sbin/bhyve/bhyverun.c#L1581
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
Hello,
I am working on adding capsicum support for the bhyve snapshot
feature.
Post by Ionuț Mihalache
Post by Mark Johnston
At
Post by Ionuț Mihalache
the end of the suspend process, the guest should be destroyed
and the
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
Post by Mark Johnston
code
Post by Ionuț Mihalache
handles this part with a sysctlbyname call which is not working
in
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
capability mode. I don't know what is the problem but even when
using
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
cap_sysctlbyname I still get the same error code (EPERM). I
tried the
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
example from the documentation aswell [1] and still the same
error
Post by Ionuț Mihalache
Post by Mark Johnston
code.
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
What could be the problem? I have a FreeBSD13 host and a
FreeBSD13
Post by Ionuț Mihalache
Post by Mark Johnston
guest.
Post by Ionuț Mihalache
Post by Mark Johnston
I'm not sure why it would happen unless the casper process is
somehow
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
Post by Mark Johnston
running as a non-root user. Can you share the code you're testing
somewhere?
Mark Johnston
2021-04-28 15:41:18 UTC
Permalink
Post by Ionuț Mihalache
Even with the fixes cap_sysctl still returns EPERM for that simple example.
Are you sure that libcap_sysctl is actually being used? In other words,
are you compiling with -DWITH_CASPER? If not, the program will call
cap_enter() and then try to read the sysctl directly, since
cap_sysctlbyname() will be a simple wrapper for sysctlbyname().
Post by Ionuț Mihalache
Post by Mark Johnston
There are two bugs in the example, also present in your WIP. I fixed
https://cgit.freebsd.org/src/commit/?id=44bbda649dc6c1cdc5a99641e14c77157967e140
Post by Ionuț Mihalache
I update the code now [1] but still the same error. Even without any
limits
Post by Ionuț Mihalache
the cap_sysctlbyname fails after using cap_enter.
Ionuț Mihalache
2021-05-01 09:29:26 UTC
Permalink
I did a clean buildworld with -DWITH_CASPER and -DHAVE_CASPER and now there
is a linking problem. Here [1] I have the includes to test the example and
here [2] I have the code. When trying to link there is a problem with
cap_sysctl_limit_init,
cap_sysctl_limit_name, cap_sysctl_limit, cap_sysctlbyname; the symbols are
not found. If I comment those lines though, cap_service_open fails(this
happens in bhyverun.c). When using the API here [3] there are no linking
problems and cap_service_open works, but cap_sysctlbyname returns EPERM(I
have here [4] the includes that I used).


[1] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/2373ecaa6dea4f0491a3ccc679017c025cbaf243/usr.sbin/bhyve/bhyverun.c#L36
[2] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/2373ecaa6dea4f0491a3ccc679017c025cbaf243/usr.sbin/bhyve/bhyverun.c#L1569
[3] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/2373ecaa6dea4f0491a3ccc679017c025cbaf243/lib/libvmmapi/vmmapi.c#L141
[4] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/2373ecaa6dea4f0491a3ccc679017c025cbaf243/lib/libvmmapi/vmmapi.h#L38
Post by Ionuț Mihalache
Post by Ionuț Mihalache
Even with the fixes cap_sysctl still returns EPERM for that simple
example.
Are you sure that libcap_sysctl is actually being used? In other words,
are you compiling with -DWITH_CASPER? If not, the program will call
cap_enter() and then try to read the sysctl directly, since
cap_sysctlbyname() will be a simple wrapper for sysctlbyname().
Post by Ionuț Mihalache
Post by Mark Johnston
There are two bugs in the example, also present in your WIP. I fixed
https://cgit.freebsd.org/src/commit/?id=44bbda649dc6c1cdc5a99641e14c77157967e140
Post by Ionuț Mihalache
Post by Mark Johnston
Post by Ionuț Mihalache
I update the code now [1] but still the same error. Even without any
limits
Post by Ionuț Mihalache
the cap_sysctlbyname fails after using cap_enter.
Mark Johnston
2021-05-01 13:29:30 UTC
Permalink
Post by Ionuț Mihalache
I did a clean buildworld with -DWITH_CASPER and -DHAVE_CASPER and now there
is a linking problem. Here [1] I have the includes to test the example and
here [2] I have the code. When trying to link there is a problem with
cap_sysctl_limit_init,
cap_sysctl_limit_name, cap_sysctl_limit, cap_sysctlbyname; the symbols are
not found. If I comment those lines though, cap_service_open fails(this
happens in bhyverun.c). When using the API here [3] there are no linking
problems and cap_service_open works, but cap_sysctlbyname returns EPERM(I
have here [4] the includes that I used).
- You do not need to do a full buildworld. It is sufficient to rebuild
bhyve itself with -DWITH_CASPER. Note that the bhyve Makefile does
not define WITH_CASPER when snapshot support is enabled.
- You need to add cap_sysctl to the list of libraries linked into bhyve,
same as it currently links cap_pwd and cap_grp.
Post by Ionuț Mihalache
[1] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/2373ecaa6dea4f0491a3ccc679017c025cbaf243/usr.sbin/bhyve/bhyverun.c#L36
[2] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/2373ecaa6dea4f0491a3ccc679017c025cbaf243/usr.sbin/bhyve/bhyverun.c#L1569
[3] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/2373ecaa6dea4f0491a3ccc679017c025cbaf243/lib/libvmmapi/vmmapi.c#L141
[4] -
https://github.com/FreeBSD-UPB/freebsd-src/blob/2373ecaa6dea4f0491a3ccc679017c025cbaf243/lib/libvmmapi/vmmapi.h#L38
Loading...