A few build system questions

[Eric McCorkle]

Hi,

I have a few questions about how to accomplish some things with the
build system.

First, I want to create some libraries that exist only as static
archives, meaning no shared object (ex. libsomething.a, but no
libsomething.so)

Second, how do I arrange to have sources be generated prior to the
build? I can probably get away with having a shell script that does it,
but I may require a tool to be built.

Third, how would I go about creating a kind of derived library- that is,
one created by invoking a whole bunch of linker/objcopy/etc commands on
some other library? That is to say, "I want to get libsomething.a by
running this script that processes libsomethingelse.a".

Thanks,
Eric

[Brooks Davis]

Post by Eric McCorkle
Hi,
I have a few questions about how to accomplish some things with the
build system.
First, I want to create some libraries that exist only as static
archives, meaning no shared object (ex. libsomething.a, but no
libsomething.so)

If it's something for use by only the base system PRIVATELIB= will
do it (and change the name to libprivatesomethi.a). If you want to be
broadly available outside the base system, NO_SHARED= should do it IIRC.

Post by Eric McCorkle
Second, how do I arrange to have sources be generated prior to the
build? I can probably get away with having a shell script that does it,
but I may require a tool to be built.

If it's a shell script add a target for the sources. If there's a
compiled tool then you need a build-tools target to build the tool in
the right stage of the bootstrap process.

Post by Eric McCorkle
Third, how would I go about creating a kind of derived library- that is,
one created by invoking a whole bunch of linker/objcopy/etc commands on
some other library? That is to say, "I want to get libsomething.a by
running this script that processes libsomethingelse.a".

You'll probably want to add a libsomething.a target that depends on
libsomethingelse.a and the script and add libsomething.a to _LIBS in the
libsomethingelse Makefile. You will also need to make certain the
script has the right failure modes (e.g. doesn't create libsomething.a
unless it succeeds even if it or it's children are killed with kill -9)
or life will get very confusing. You'll want to get review from at
least bdrewery@ and brd@ for anything non-standard.

-- Brooks

[Eric McCorkle]

Post by Brooks Davis

Post by Eric McCorkle
Hi,
I have a few questions about how to accomplish some things with the
build system.
First, I want to create some libraries that exist only as static
archives, meaning no shared object (ex. libsomething.a, but no
libsomething.so)

If it's something for use by only the base system PRIVATELIB= will
do it (and change the name to libprivatesomethi.a). If you want to be
broadly available outside the base system, NO_SHARED= should do it IIRC.

If loader and kernel are able to use private libraries, then that is
probably better, actually.

[Warner Losh]

Post by Eric McCorkle

Post by Brooks Davis

Post by Eric McCorkle
Hi,
I have a few questions about how to accomplish some things with the
build system.
First, I want to create some libraries that exist only as static
archives, meaning no shared object (ex. libsomething.a, but no
libsomething.so)

If it's something for use by only the base system PRIVATELIB= will
do it (and change the name to libprivatesomethi.a). If you want to be
broadly available outside the base system, NO_SHARED= should do it IIRC.

If loader and kernel are able to use private libraries, then that is
probably better, actually.

They aren't. Src/stand depends on no objects in the system other than what
is built in src/stand. What you are proposing is a non-starter. The loader

Warner

[Eric McCorkle]

Post by Eric McCorkle

Post by Brooks Davis

Post by Eric McCorkle
Hi,
I have a few questions about how to accomplish some things with the
build system.
First, I want to create some libraries that exist only as static
archives, meaning no shared object (ex. libsomething.a, but no
libsomething.so)

If it's something for use by only the base system PRIVATELIB= will
do it (and change the name to libprivatesomethi.a).  If you want to be
broadly available outside the base system, NO_SHARED= should do it

IIRC.
If loader and kernel are able to use private libraries, then that is
probably better, actually.
They aren't.  Src/stand depends on no objects in the system other than
what is built in src/stand. What you are proposing is a non-starter. The
loader

I think you hit "send" too soon...

Basically what I'm trying to do at this point is two things.

First, I want to convert some certs into C declarations and embed them
into a static library, which can in turn be used to embed them into
applications. This is essentially the same thing that some drivers do,
where they embed the firmware binaries directly into the driver.

Second, and a (somewhat) separate thing, I'm trying to see if I can get
a PoC of extracting the specific primitives out of OpenSSL and using
them in place of the current software crypto implementations in kernel
and loader. (objcopy ought to be able to do this to a static library in
theory)

[Simon J. Gerraty]

Post by Eric McCorkle
First, I want to convert some certs into C declarations and embed them
into a static library, which can in turn be used to embed them into

FYI a mechanism to do this is in D16335

Post by Eric McCorkle
Second, and a (somewhat) separate thing, I'm trying to see if I can get
a PoC of extracting the specific primitives out of OpenSSL and using
them in place of the current software crypto implementations in kernel
and loader. (objcopy ought to be able to do this to a static library in
theory)

Um... ick ;-)

[Warner Losh]

Post by Eric McCorkle

Post by Eric McCorkle

Post by Brooks Davis

Post by Eric McCorkle
Hi,
I have a few questions about how to accomplish some things with

the

Post by Eric McCorkle

Post by Brooks Davis

Post by Eric McCorkle
build system.
First, I want to create some libraries that exist only as static
archives, meaning no shared object (ex. libsomething.a, but no
libsomething.so)

If it's something for use by only the base system PRIVATELIB= will
do it (and change the name to libprivatesomethi.a). If you want

to be

Post by Eric McCorkle

Post by Brooks Davis
broadly available outside the base system, NO_SHARED= should do it

IIRC.
If loader and kernel are able to use private libraries, then that is
probably better, actually.
They aren't. Src/stand depends on no objects in the system other than
what is built in src/stand. What you are proposing is a non-starter. The
loader

I think you hit "send" too soon...
Basically what I'm trying to do at this point is two things.
First, I want to convert some certs into C declarations and embed them
into a static library, which can in turn be used to embed them into
applications. This is essentially the same thing that some drivers do,
where they embed the firmware binaries directly into the driver.

This is easy.

Post by Eric McCorkle
Second, and a (somewhat) separate thing, I'm trying to see if I can get
a PoC of extracting the specific primitives out of OpenSSL and using
them in place of the current software crypto implementations in kernel
and loader. (objcopy ought to be able to do this to a static library in
theory)

Such a strategy won't fly for various reasons. Userland, the loader and the
kernel are all compiled with different options. You'll need to recompile
for each and not do objcopy tricks because the current build model doesn't
allow for that.

Warner

[Eric McCorkle]

Post by Eric McCorkle

     >> Hi,
     >>
     >> I have a few questions about how to accomplish some things with the
     >> build system.
     >>
     >> First, I want to create some libraries that exist only as static
     >> archives, meaning no shared object (ex. libsomething.a, but no
     >> libsomething.so)
     >
     > If it's something for use by only the base system PRIVATELIB= will
     > do it (and change the name to libprivatesomethi.a).  If you want to be
     > broadly available outside the base system, NO_SHARED= should do it
     IIRC.
     If loader and kernel are able to use private libraries, then that is
     probably better, actually.
They aren't.  Src/stand depends on no objects in the system other than
what is built in src/stand. What you are proposing is a non-starter. The
loader

I think you hit "send" too soon...
Basically what I'm trying to do at this point is two things.
First, I want to convert some certs into C declarations and embed them
into a static library, which can in turn be used to embed them into
applications.  This is essentially the same thing that some drivers do,
where they embed the firmware binaries directly into the driver.
This is easy.

For future reference, this seems to be more or less what I'm after:

for f in /etc/trust/root/certs/*.pub.pem; do echo -n "static const char
`basename ${f%%.*}`_data[] = {"; openssl x509 -outform DER -in
/etc/trust/root/certs/local.pub.pem | hexdump -v -e '1/1 "0x%02x,"';
echo "};"; done

Post by Eric McCorkle
Second, and a (somewhat) separate thing, I'm trying to see if I can get
a PoC of extracting the specific primitives out of OpenSSL and using
them in place of the current software crypto implementations in kernel
and loader.  (objcopy ought to be able to do this to a static library in
theory)
Such a strategy won't fly for various reasons. Userland, the loader and
the kernel are all compiled with different options. You'll need to
recompile for each and not do objcopy tricks because the current build
model doesn't allow for that.

Yeah, I'd say that shipwrecks any hopes I'd had there :(

[Conrad Meyer]

Post by Eric McCorkle
for f in /etc/trust/root/certs/*.pub.pem; do echo -n "static const char
`basename ${f%%.*}`_data[] = {"; openssl x509 -outform DER -in
/etc/trust/root/certs/local.pub.pem | hexdump -v -e '1/1 "0x%02x,"';
echo "};"; done

Another option instead of the hexdump format is just "xxd -i" (xxd is
part of vim), although that works best if the intermediary output is
written to a file (it populates the struct name automatically).
Something like:

for f in .../*pub.pem; do openssl x509 -outform DER -in ... >
"$(basename "...")_data" && xxd -i "$(basename "...")_data" ; done

Its output is nicely formatted with spaces and line wrapping.

Best,
Conrad

[John-Mark Gurney]

Post by Eric McCorkle
for f in /etc/trust/root/certs/*.pub.pem; do echo -n "static const char
`basename ${f%%.*}`_data[] = {"; openssl x509 -outform DER -in
/etc/trust/root/certs/local.pub.pem | hexdump -v -e '1/1 "0x%02x,"';
echo "};"; done

There's also file2c that does the conversion for you.

[Simon J. Gerraty]

Post by Eric McCorkle
Second, how do I arrange to have sources be generated prior to the
build? I can probably get away with having a shell script that does it,
but I may require a tool to be built.

[I think brooks answered the others.]

You can use the 'beforebuild' target to trigger that.

If however the src generation is non-trvial and especially if you need
to build them for multiple arch, it is better to do the generation in a
separate dir (eg subdir like gen/) so that it is easy to do the
generation once only.

Separate dirs make build orchestration simpler.