openpkg/README@c10fb90893b9 (annotated)
openpkg/README
Thu, 04 Oct 2012 20:30:05 +0200
- author
- Michael Schloh von Bennewitz <michael@schloh.com>
- date
- Thu, 04 Oct 2012 20:30:05 +0200
- changeset 715
- c10fb90893b9
- parent 428
- f880f219c566
- permissions
- -rw-r--r--
Correct out of date build configuration, porting to Solaris 11 network
link infrastructure and new libpcap logic. This additionally allows for
device drivers in subdirectories of /dev. Correct packaged nmap
personalities and signatures to work out of the box. Finally, hack
arpd logic to properly close sockets and quit on TERM by repeating
signaling in the run command script. Sadly, all this fails to correct
the run time behaviour of honeyd which fails to bind to the IP layer.
| michael@13 | 1 | |
| michael@428 | 2 | OpenPKG Framework Bootstrap Package, Source Tree |
| michael@428 | 3 | ================================================ |
| michael@13 | 4 | |
| michael@428 | 5 | This is the source tree for the OpenPKG Framework "bootstrap" package. |
| michael@428 | 6 | What you find here is rather complex and tricky stuff, so don't be |
| michael@428 | 7 | confused if you don't understand everything immediately. |
| michael@13 | 8 | |
| michael@13 | 9 | The Files |
| michael@13 | 10 | --------- |
| michael@13 | 11 | |
| michael@13 | 12 | README .................. this file ;-) |
| michael@13 | 13 | |
| michael@13 | 14 | openpkg.spec ............ the regular build procedure |
| michael@13 | 15 | openpkg.boot ............ the bootstrapping procedure |
| michael@13 | 16 | |
| michael@13 | 17 | patch-*.tar.gz .......... untouched distribution tarball of the GNU patch tool |
| michael@13 | 18 | bash-*.tar.gz ........... untouched distribution tarball of the GNU bash tool |
| michael@428 | 19 | make-*.tar.gz ........... untouched distribution tarball of the GNU make tool |
| michael@13 | 20 | tar-*.tar.gz ............ untouched distribution tarball of the GNU tar tool |
| michael@428 | 21 | config-*.tar.gz ......... untouched distribution tarball of the GNU config scripts |
| michael@445 | 22 | gzip-*.tar .............. preunpacked distribution tarball of the GNU gzip tool |
| michael@445 | 23 | gzip-*-openpkg-r*.tar ... prepatched distribution files of the GNU gzip tool |
| michael@428 | 24 | perl-*-mini.tar.gz ...... stripped down distribution tarball of the Perl tool |
| michael@428 | 25 | rpm-*.tar.gz ............ untouched distribution tarball of the RPM tool |
| michael@428 | 26 | openssl-*.tar.gz ........ untouched distribution tarball of the OpenSSL toolkit |
| michael@13 | 27 | uuid-*.tar.gz ........... untouched distribution tarball of the OSSP uuid tool |
| michael@13 | 28 | curl-*.tar.gz ........... untouched distribution tarball of the cURL tool |
| michael@428 | 29 | beecrypt-*.tar.gz ....... untouched distribution tarball of the BeeCrypt library |
| michael@13 | 30 | bzip2-*.tar.gz .......... untouched distribution tarball of the BZIP2 library |
| michael@13 | 31 | zlib-*.tar.gz ........... untouched distribution tarball of the ZLIB library |
| michael@428 | 32 | popt-*.tar.gz ........... untouched distribution tarball of the POPT library |
| michael@428 | 33 | sqlite-*.tar.gz ......... untouched distribution tarball of the SQLite library |
| michael@428 | 34 | pcre-*.tar.gz ........... untouched distribution tarball of the SQLite library |
| michael@428 | 35 | diffutils-*.tar.gz ...... untouched distribution tarball of the GNU diffutils toolkit |
| michael@428 | 36 | svs-*.tar.gz ............ untouched distribution tarball of the OSSP svs tool |
| michael@428 | 37 | xz-*.tar.gz ............. untouched distribution tarball of the XZ tool |
| michael@13 | 38 | |
| michael@428 | 39 | bash.patch .............. patch for GNU bash tool |
| michael@428 | 40 | bash.patch.vendor ....... patch for GNU bash tool (upstream vendor only) |
| michael@428 | 41 | bash.patch.vendor.sh .... patch for GNU bash tool (generation utility, NOT BUNDLED) |
| michael@428 | 42 | beecrypt.patch .......... patch for BeeCrypt library |
| michael@428 | 43 | make.patch .............. patch for GNU make tool |
| michael@428 | 44 | openssl.patch ........... patch for OpenSSL toolkit |
| michael@428 | 45 | perl.patch .............. patch for Perl tool |
| michael@428 | 46 | tar.patch ............... patch for GNU tar tool |
| michael@428 | 47 | rpm.patch ............... patch for RPM tool |
| michael@428 | 48 | popt.patch .............. patch for POPT library |
| michael@428 | 49 | sqlite.patch ............ patch for SQLite library |
| michael@428 | 50 | curl.patch .............. patch for cURL library |
| michael@428 | 51 | diffutils.patch ......... patch for GNU diffutils tool |
| michael@428 | 52 | libarchive.patch ........ patch for BSD libarchive tool |
| michael@428 | 53 | pcre.patch .............. patch for PCRE library |
| michael@428 | 54 | xz.patch ................ patch for XZ tool |
| michael@13 | 55 | |
| michael@13 | 56 | rpmpopt ................. replacements/extensions for RPM's POPT configuration |
| michael@13 | 57 | rpmmacros ............... replacements/extensions for RPM's macros |
| michael@13 | 58 | |
| michael@13 | 59 | root.README ............. the source for installed <prefix>/README |
| michael@13 | 60 | local.README ............ the source for installed <prefix>/local/README |
| michael@13 | 61 | dot.bashrc .............. the source for installed <prefix>/.bashrc |
| michael@13 | 62 | dot.bash_login .......... the source for installed <prefix>/.bash_login |
| michael@13 | 63 | dot.lsyncrc ............. the source for installed <prefix>/local/.lsyncrc |
| michael@13 | 64 | |
| michael@445 | 65 | openpkg.c ............... the OpenPKG frontend (setuid wrapper) |
| michael@428 | 66 | openpkg.mk .............. the OpenPKG frontend (build procedure) |
| michael@13 | 67 | openpkg.sh .............. the OpenPKG frontend (main script) |
| michael@13 | 68 | openpkg.pod ............. the OpenPKG frontend manual page (source) |
| michael@445 | 69 | openpkg.1 ............... the OpenPKG frontend manual page (pregenerated output) |
| michael@13 | 70 | |
| michael@445 | 71 | rc ...................... the OpenPKG runcommand handling script |
| michael@445 | 72 | rc.func ................. the OpenPKG runcommand function definitions |
| michael@445 | 73 | rc.conf ................. the OpenPKG runcommand configuration template |
| michael@445 | 74 | rc.openpkg .............. the OpenPKG runcommand script for the bootstrap package |
| michael@445 | 75 | rc.pod .................. the OpenPKG runcommand script manual page (source) |
| michael@445 | 76 | rc.8 .................... the OpenPKG runcommand script manual page (pregenerated output) |
| michael@13 | 77 | |
| michael@13 | 78 | release.sh .............. the OpenPKG release information utility script |
| michael@13 | 79 | release.pod ............. the OpenPKG release information utility manual page (source) |
| michael@445 | 80 | release.8 ............... the OpenPKG release information utility manual page (pregenerated output) |
| michael@13 | 81 | |
| michael@13 | 82 | uuid.sh ................. the OpenPKG UUID management utility script |
| michael@13 | 83 | uuid.pod ................ the OpenPKG UUID management utility manual page (source) |
| michael@445 | 84 | uuid.8 .................. the OpenPKG UUID management utility manual page (pregenerated output) |
| michael@13 | 85 | |
| michael@13 | 86 | rpmdb ................... the OpenPKG RPM database management utility |
| michael@13 | 87 | |
| michael@13 | 88 | rpmtool ................. the rpmtool tool (program source) |
| michael@13 | 89 | rpmtool.8 ............... the rpmtool tool (manual page) |
| michael@13 | 90 | rpmtool.pod ............. the rpmtool tool (manual page source) |
| michael@13 | 91 | shtool .................. the portable shell tool (GNU shtool) |
| michael@13 | 92 | |
| michael@13 | 93 | rpm-config.sh ........... the RPM C API helper script |
| michael@13 | 94 | rpm-config.pod .......... the RPM C API helper manual page (source) |
| michael@445 | 95 | rpm-config.8 ............ the RPM C API helper manual page (pregenerated output) |
| michael@13 | 96 | |
| michael@13 | 97 | lsync ................... the lsync tool (program source) |
| michael@13 | 98 | lsync.8 ................. the lsync tool (manual page) |
| michael@13 | 99 | lsync.pod ............... the lsync tool (manual page source) |
| michael@13 | 100 | |
| michael@428 | 101 | etc.usrgrp.sh ........... user/group name/id determination script |
| michael@428 | 102 | etc.prereq.sh ........... prerequisite checking script |
| michael@428 | 103 | etc.wrapsrc.sh .......... wrapper script for generating openpkg-V-R.src.sh |
| michael@428 | 104 | etc.wrapbin.sh .......... wrapper script for generating openpkg-V-R.P-L.sh |
| michael@13 | 105 | |
| michael@13 | 106 | man.sh .................. helper script for "openpkg man" command |
| michael@428 | 107 | |
| michael@428 | 108 | rpm.sh .................. wrapper for RPM |
| michael@428 | 109 | curl.sh ................. wrapper for cURL |
| michael@428 | 110 | |
| michael@428 | 111 | dev.pl .................. the OpenPKG package development utility script |
| michael@428 | 112 | dev.pod ................. the OpenPKG package development utility manual page (source) |
| michael@445 | 113 | dev.8 ................... the OpenPKG package development utility manual page (pregenerated output) |
| michael@428 | 114 | |
| michael@428 | 115 | index.pl ................ the OpenPKG package indexing utility script |
| michael@428 | 116 | index.pod ............... the OpenPKG package indexing utility manual page (source) |
| michael@445 | 117 | index.8 ................. the OpenPKG package indexing utility manual page (pregenerated output) |
| michael@428 | 118 | |
| michael@428 | 119 | search.pl ............... the OpenPKG package searching utility script |
| michael@428 | 120 | search.pod .............. the OpenPKG package searching utility manual page (source) |
| michael@445 | 121 | search.8 ................ the OpenPKG package searching utility manual page (pregenerated output) |
| michael@428 | 122 | |
| michael@428 | 123 | sea.sh .................. the OpenPKG shell execution archive utility script |
| michael@428 | 124 | sea.pod ................. the OpenPKG shell execution archive utility manual page (source) |
| michael@445 | 125 | sea.8 ................... the OpenPKG shell execution archive utility manual page (pregenerated output) |
| michael@428 | 126 | |
| michael@428 | 127 | mirror.pl ............... the OpenPKG package mirroing utility script |
| michael@428 | 128 | mirror.pod .............. the OpenPKG package mirroing utility manual page (source) |
| michael@445 | 129 | mirror.8 ................ the OpenPKG package mirroing utility manual page (pregenerated output) |
| michael@428 | 130 | |
| michael@428 | 131 | build.pl ................ the OpenPKG package building utility script |
| michael@428 | 132 | build.pod ............... the OpenPKG package building utility manual page (source) |
| michael@445 | 133 | build.8 ................. the OpenPKG package building utility manual page (pregenerated output) |
| michael@428 | 134 | |
| michael@428 | 135 | makeproxy.pl ............ the OpenPKG proxy package building utility script |
| michael@428 | 136 | makeproxy.pod ........... the OpenPKG proxy package building utility manual page (source) |
| michael@445 | 137 | makeproxy.8 ............. the OpenPKG proxy package building utility manual page (pregenerated output) |
| michael@13 | 138 | |
| michael@13 | 139 | openpkg.org.pgp ......... the OpenPGP public key "OpenPKG <openpkg@openpkg.org>" |
| michael@13 | 140 | openpkg.com.pgp ......... the OpenPGP public key "OpenPKG GmbH <openpkg@openpkg.com>" |
| michael@13 | 141 | openpkg.net.pgp ......... the OpenPGP public key "OpenPKG Foundation e.V. <openpkg@openpkg.net>" |
| michael@13 | 142 | |
| michael@428 | 143 | license.sh .............. the OpenPKG license management utility script |
| michael@428 | 144 | license.pod ............. the OpenPKG license management utility manual page (source) |
| michael@445 | 145 | license.8 ............... the OpenPKG license management utility manual page (pregenerated output) |
| michael@428 | 146 | license-BOOT.txt ........ the OpenPKG Framework Bootstrapping License |
| michael@428 | 147 | license-COMMUNITY.txt ... the OpenPKG Framework Community License |
| michael@428 | 148 | license-EVAL.txt ........ the OpenPKG Framework Evaluation License |
| michael@428 | 149 | license-EXAMPLE.txt ..... the OpenPKG Framework Example License |
| michael@428 | 150 | license-PROMO.txt ....... the OpenPKG Framework Promotion License |
| michael@428 | 151 | license-RECOVERY.txt .... the OpenPKG Framework Recovery License |
| michael@428 | 152 | license.lua ............. the OpenPKG Framework license processor script |
| michael@428 | 153 | |
| michael@428 | 154 | Makefile ................ development build procedures |
| michael@428 | 155 | |
| michael@13 | 156 | The Bootstrapping Procedure |
| michael@13 | 157 | --------------------------- |
| michael@13 | 158 | |
| michael@13 | 159 | The complexity of this OpenPKG RPM package results from the fact that |
| michael@13 | 160 | we force us to treat this bootstrapping package equal to every other |
| michael@13 | 161 | regular OpenPKG RPM package. First, this implies that the packaging |
| michael@428 | 162 | tool RPM is packaged with itself as an OpenPKG RPM package -- which |
| michael@428 | 163 | means that its build procedure is a real RPM .spec file and it can be |
| michael@428 | 164 | installed and upgraded through a binary or source RPM. Second, RPM is |
| michael@428 | 165 | installed into the same filesystem hierarchy as all other packages. |
| michael@428 | 166 | Third, RPM manages its own files. The reason for this approach should |
| michael@428 | 167 | be obvious: 100% consistency for the whole OpenPKG software packaging |
| michael@428 | 168 | facility! |
| michael@13 | 169 | |
| michael@13 | 170 | The drawback is that this package requires a very tricky bootstrapping |
| michael@13 | 171 | procedure which had cost a lot of time to figure out and establish. If |
| michael@13 | 172 | you ever wanted to know the gory details, here they are... |
| michael@13 | 173 | |
| michael@13 | 174 | The first step was that we wrote the regular openpkg.spec file for |
| michael@13 | 175 | building the bootstrap package with OpenPKG RPM under the assumption |
| michael@13 | 176 | that OpenPKG RPM is already available. This way we can provide OpenPKG |
| michael@13 | 177 | RPM as an RPM package. Just remains the problem how we actually |
| michael@13 | 178 | bootstrap in case where OpenPKG RPM is still not available, i.e., |
| michael@13 | 179 | when we reach a new platform and have to build the package from |
| michael@13 | 180 | scratch. Here the "openpkg.boot" script comes into play. It executes |
| michael@13 | 181 | the "openpkg.spec" build procedure very similar to the way the real |
| michael@13 | 182 | OpenPKG RPM would do ("openpkg rpm -bb"). That is, "openpkg.boot" |
| michael@13 | 183 | partly emulates OpenPKG RPM -- just enough that "openpkg.spec" works. |
| michael@13 | 184 | As a result, "openpkg.spec" cannot use any fancy OpenPKG RPM features |
| michael@13 | 185 | or other things before "openpkg.boot" is able to emulate it, of |
| michael@13 | 186 | course. |
| michael@13 | 187 | |
| michael@13 | 188 | After "openpkg.boot" executed the "%prep", "%build" and "%install" |
| michael@13 | 189 | scripts of "openpkg.spec", there is a fresh version of the target |
| michael@13 | 190 | filesystem hierarchy staying under a temporary "build root". The |
| michael@13 | 191 | "openpkg.boot" script then creates a very special temporary "openpkg |
| michael@428 | 192 | rpm" command which allows the installed "openpkg rpm" command |
| michael@428 | 193 | inside the "build root" to work (although it is built for the |
| michael@428 | 194 | final target filesystem path). Then the $OPENPKG_BOOT variable |
| michael@428 | 195 | is set and the package is _again_ build via "openpkg.spec" -- |
| michael@428 | 196 | but this time with the real OpenPKG RPM. To avoid unneccessary |
| michael@445 | 197 | recompilation, the "openpkg.spec" skips "%prep", "%build" and |
| michael@428 | 198 | "%install" sections if $OPENPKG_BOOT is defined. So, on this second |
| michael@428 | 199 | build phase, only the "%files" section is effectively executed, i.e., |
| michael@428 | 200 | a binary OpenPKG RPM package "openpkg-V-R.P-T.rpm" is rolled from the |
| michael@428 | 201 | files in the "build root". Additionally, a source OpenPKG RPM package |
| michael@428 | 202 | "openpkg-V-R.src.rpm" is rolled for consistency reasons, too. |
| michael@13 | 203 | |
| michael@13 | 204 | Finally, we override the installation in the "build root" |
| michael@13 | 205 | again by installing the now rolled binary OpenPKG RPM package |
| michael@13 | 206 | "openpkg-V-R.P-T.rpm" by using the real OpenPKG RPM. This way |
| michael@13 | 207 | we achieve that OpenPKG RPM is remembered as a real OpenPKG RPM |
| michael@13 | 208 | package in the RPM database. We just have to make sure the package |
| michael@13 | 209 | is still relocated to the "build root" while installing. For this |
| michael@13 | 210 | we could use "--prefix=$RPM_BUILD_ROOT%{l_prefix}", but this would |
| michael@13 | 211 | create an incorrect file list for the package "openpkg" in the RPM |
| michael@13 | 212 | database. Instead we use the tricky "--justdb" option for "openpkg |
| michael@13 | 213 | rpm" which means "openpkg rpm" behaves as it would install into the |
| michael@13 | 214 | real location, but does not actually install anything. But as a |
| michael@445 | 215 | side effect, the database inside the "build root" is now correct. |
| michael@13 | 216 | |
| michael@13 | 217 | After this procedure, the "build root" contains the target filesystem |
| michael@13 | 218 | hierarchy with OpenPKG RPM installed with itself. What is now just |
| michael@13 | 219 | remaining is to roll a bootstrap package "openpkg" with this stuff for |
| michael@13 | 220 | initial installation without OpenPKG RPM. For this the "build root" |
| michael@13 | 221 | is packed into a "tarball", compressed, again wrapped into another |
| michael@13 | 222 | tarball together with the uncompression tools ("bzip2" and "tar"), |
| michael@13 | 223 | and finally wrapped into a self-extracting shell script by appending |
| michael@428 | 224 | "etc.wrapbin.sh" (padded to 64KB for easier unpacking of the attached |
| michael@13 | 225 | tarball) to its front. |
| michael@13 | 226 | |
| michael@13 | 227 | The result is the binary bootstrap script "openpkg-V-R.P-T.sh" which |
| michael@428 | 228 | can be used to install the target hierarchy from scratch without |
| michael@445 | 229 | any preinstalled OpenPKG RPM. Nevertheless, the installed target |
| michael@428 | 230 | hierarchy looks _exactly_ as it would have been installed with OpenPKG |
| michael@428 | 231 | RPM. If one later wants to upgrade this hierarchy one can just use the |
| michael@13 | 232 | generated (or a newer) "openpkg-V-R.P-T.rpm". |
| michael@13 | 233 | |
| michael@445 | 234 | To allow one to easily repeat this from source bootstrapping procedure |
| michael@13 | 235 | on other machines, one can run "./openpkg.boot -s" which rolls |
| michael@13 | 236 | a "openpkg-V-R.src.sh" script which is a self-extracting script |
| michael@13 | 237 | containing an attached tarball of the sources of this directory. This |
| michael@13 | 238 | script contains the same contents like "openpkg-V-R.src.rpm", but |
| michael@13 | 239 | is intended for running the described bootstrapping procedure from |
| michael@13 | 240 | scratch without any OpenPKG RPM. |
| michael@13 | 241 |
