openpkg/README@a2c6460cfb16 (annotated)
openpkg/README
Mon, 28 Jan 2013 17:37:18 +0100
- author
- Michael Schloh von Bennewitz <michael@schloh.com>
- date
- Mon, 28 Jan 2013 17:37:18 +0100
- changeset 758
- a2c6460cfb16
- parent 428
- f880f219c566
- permissions
- -rw-r--r--
Correct socket error reporting improvement with IPv6 portable code,
after helpful recommendation by Saúl Ibarra Corretgé on OSips devlist.
| michael@13 | 1 | |
| michael@428 | 2 | OpenPKG Framework Bootstrap Package, Source Tree |
| michael@428 | 3 | ================================================ |
| michael@13 | 4 | |
| michael@428 | 5 | This is the source tree for the OpenPKG Framework "bootstrap" package. |
| michael@428 | 6 | What you find here is rather complex and tricky stuff, so don't be |
| michael@428 | 7 | confused if you don't understand everything immediately. |
| michael@13 | 8 | |
| michael@13 | 9 | The Files |
| michael@13 | 10 | --------- |
| michael@13 | 11 | |
| michael@13 | 12 | README .................. this file ;-) |
| michael@13 | 13 | |
| michael@13 | 14 | openpkg.spec ............ the regular build procedure |
| michael@13 | 15 | openpkg.boot ............ the bootstrapping procedure |
| michael@13 | 16 | |
| michael@13 | 17 | patch-*.tar.gz .......... untouched distribution tarball of the GNU patch tool |
| michael@13 | 18 | bash-*.tar.gz ........... untouched distribution tarball of the GNU bash tool |
| michael@428 | 19 | make-*.tar.gz ........... untouched distribution tarball of the GNU make tool |
| michael@13 | 20 | tar-*.tar.gz ............ untouched distribution tarball of the GNU tar tool |
| michael@428 | 21 | config-*.tar.gz ......... untouched distribution tarball of the GNU config scripts |
| michael@445 | 22 | gzip-*.tar .............. preunpacked distribution tarball of the GNU gzip tool |
| michael@445 | 23 | gzip-*-openpkg-r*.tar ... prepatched distribution files of the GNU gzip tool |
| michael@428 | 24 | perl-*-mini.tar.gz ...... stripped down distribution tarball of the Perl tool |
| michael@428 | 25 | rpm-*.tar.gz ............ untouched distribution tarball of the RPM tool |
| michael@428 | 26 | openssl-*.tar.gz ........ untouched distribution tarball of the OpenSSL toolkit |
| michael@13 | 27 | uuid-*.tar.gz ........... untouched distribution tarball of the OSSP uuid tool |
| michael@13 | 28 | curl-*.tar.gz ........... untouched distribution tarball of the cURL tool |
| michael@428 | 29 | beecrypt-*.tar.gz ....... untouched distribution tarball of the BeeCrypt library |
| michael@13 | 30 | bzip2-*.tar.gz .......... untouched distribution tarball of the BZIP2 library |
| michael@13 | 31 | zlib-*.tar.gz ........... untouched distribution tarball of the ZLIB library |
| michael@428 | 32 | popt-*.tar.gz ........... untouched distribution tarball of the POPT library |
| michael@428 | 33 | sqlite-*.tar.gz ......... untouched distribution tarball of the SQLite library |
| michael@428 | 34 | pcre-*.tar.gz ........... untouched distribution tarball of the SQLite library |
| michael@428 | 35 | diffutils-*.tar.gz ...... untouched distribution tarball of the GNU diffutils toolkit |
| michael@428 | 36 | svs-*.tar.gz ............ untouched distribution tarball of the OSSP svs tool |
| michael@428 | 37 | xz-*.tar.gz ............. untouched distribution tarball of the XZ tool |
| michael@13 | 38 | |
| michael@428 | 39 | bash.patch .............. patch for GNU bash tool |
| michael@428 | 40 | bash.patch.vendor ....... patch for GNU bash tool (upstream vendor only) |
| michael@428 | 41 | bash.patch.vendor.sh .... patch for GNU bash tool (generation utility, NOT BUNDLED) |
| michael@428 | 42 | beecrypt.patch .......... patch for BeeCrypt library |
| michael@428 | 43 | make.patch .............. patch for GNU make tool |
| michael@428 | 44 | openssl.patch ........... patch for OpenSSL toolkit |
| michael@428 | 45 | perl.patch .............. patch for Perl tool |
| michael@428 | 46 | tar.patch ............... patch for GNU tar tool |
| michael@428 | 47 | rpm.patch ............... patch for RPM tool |
| michael@428 | 48 | popt.patch .............. patch for POPT library |
| michael@428 | 49 | sqlite.patch ............ patch for SQLite library |
| michael@428 | 50 | curl.patch .............. patch for cURL library |
| michael@428 | 51 | diffutils.patch ......... patch for GNU diffutils tool |
| michael@428 | 52 | libarchive.patch ........ patch for BSD libarchive tool |
| michael@428 | 53 | pcre.patch .............. patch for PCRE library |
| michael@428 | 54 | xz.patch ................ patch for XZ tool |
| michael@13 | 55 | |
| michael@13 | 56 | rpmpopt ................. replacements/extensions for RPM's POPT configuration |
| michael@13 | 57 | rpmmacros ............... replacements/extensions for RPM's macros |
| michael@13 | 58 | |
| michael@13 | 59 | root.README ............. the source for installed <prefix>/README |
| michael@13 | 60 | local.README ............ the source for installed <prefix>/local/README |
| michael@13 | 61 | dot.bashrc .............. the source for installed <prefix>/.bashrc |
| michael@13 | 62 | dot.bash_login .......... the source for installed <prefix>/.bash_login |
| michael@13 | 63 | dot.lsyncrc ............. the source for installed <prefix>/local/.lsyncrc |
| michael@13 | 64 | |
| michael@445 | 65 | openpkg.c ............... the OpenPKG frontend (setuid wrapper) |
| michael@428 | 66 | openpkg.mk .............. the OpenPKG frontend (build procedure) |
| michael@13 | 67 | openpkg.sh .............. the OpenPKG frontend (main script) |
| michael@13 | 68 | openpkg.pod ............. the OpenPKG frontend manual page (source) |
| michael@445 | 69 | openpkg.1 ............... the OpenPKG frontend manual page (pregenerated output) |
| michael@13 | 70 | |
| michael@445 | 71 | rc ...................... the OpenPKG runcommand handling script |
| michael@445 | 72 | rc.func ................. the OpenPKG runcommand function definitions |
| michael@445 | 73 | rc.conf ................. the OpenPKG runcommand configuration template |
| michael@445 | 74 | rc.openpkg .............. the OpenPKG runcommand script for the bootstrap package |
| michael@445 | 75 | rc.pod .................. the OpenPKG runcommand script manual page (source) |
| michael@445 | 76 | rc.8 .................... the OpenPKG runcommand script manual page (pregenerated output) |
| michael@13 | 77 | |
| michael@13 | 78 | release.sh .............. the OpenPKG release information utility script |
| michael@13 | 79 | release.pod ............. the OpenPKG release information utility manual page (source) |
| michael@445 | 80 | release.8 ............... the OpenPKG release information utility manual page (pregenerated output) |
| michael@13 | 81 | |
| michael@13 | 82 | uuid.sh ................. the OpenPKG UUID management utility script |
| michael@13 | 83 | uuid.pod ................ the OpenPKG UUID management utility manual page (source) |
| michael@445 | 84 | uuid.8 .................. the OpenPKG UUID management utility manual page (pregenerated output) |
| michael@13 | 85 | |
| michael@13 | 86 | rpmdb ................... the OpenPKG RPM database management utility |
| michael@13 | 87 | |
| michael@13 | 88 | rpmtool ................. the rpmtool tool (program source) |
| michael@13 | 89 | rpmtool.8 ............... the rpmtool tool (manual page) |
| michael@13 | 90 | rpmtool.pod ............. the rpmtool tool (manual page source) |
| michael@13 | 91 | shtool .................. the portable shell tool (GNU shtool) |
| michael@13 | 92 | |
| michael@13 | 93 | rpm-config.sh ........... the RPM C API helper script |
| michael@13 | 94 | rpm-config.pod .......... the RPM C API helper manual page (source) |
| michael@445 | 95 | rpm-config.8 ............ the RPM C API helper manual page (pregenerated output) |
| michael@13 | 96 | |
| michael@13 | 97 | lsync ................... the lsync tool (program source) |
| michael@13 | 98 | lsync.8 ................. the lsync tool (manual page) |
| michael@13 | 99 | lsync.pod ............... the lsync tool (manual page source) |
| michael@13 | 100 | |
| michael@428 | 101 | etc.usrgrp.sh ........... user/group name/id determination script |
| michael@428 | 102 | etc.prereq.sh ........... prerequisite checking script |
| michael@428 | 103 | etc.wrapsrc.sh .......... wrapper script for generating openpkg-V-R.src.sh |
| michael@428 | 104 | etc.wrapbin.sh .......... wrapper script for generating openpkg-V-R.P-L.sh |
| michael@13 | 105 | |
| michael@13 | 106 | man.sh .................. helper script for "openpkg man" command |
| michael@428 | 107 | |
| michael@428 | 108 | rpm.sh .................. wrapper for RPM |
| michael@428 | 109 | curl.sh ................. wrapper for cURL |
| michael@428 | 110 | |
| michael@428 | 111 | dev.pl .................. the OpenPKG package development utility script |
| michael@428 | 112 | dev.pod ................. the OpenPKG package development utility manual page (source) |
| michael@445 | 113 | dev.8 ................... the OpenPKG package development utility manual page (pregenerated output) |
| michael@428 | 114 | |
| michael@428 | 115 | index.pl ................ the OpenPKG package indexing utility script |
| michael@428 | 116 | index.pod ............... the OpenPKG package indexing utility manual page (source) |
| michael@445 | 117 | index.8 ................. the OpenPKG package indexing utility manual page (pregenerated output) |
| michael@428 | 118 | |
| michael@428 | 119 | search.pl ............... the OpenPKG package searching utility script |
| michael@428 | 120 | search.pod .............. the OpenPKG package searching utility manual page (source) |
| michael@445 | 121 | search.8 ................ the OpenPKG package searching utility manual page (pregenerated output) |
| michael@428 | 122 | |
| michael@428 | 123 | sea.sh .................. the OpenPKG shell execution archive utility script |
| michael@428 | 124 | sea.pod ................. the OpenPKG shell execution archive utility manual page (source) |
| michael@445 | 125 | sea.8 ................... the OpenPKG shell execution archive utility manual page (pregenerated output) |
| michael@428 | 126 | |
| michael@428 | 127 | mirror.pl ............... the OpenPKG package mirroing utility script |
| michael@428 | 128 | mirror.pod .............. the OpenPKG package mirroing utility manual page (source) |
| michael@445 | 129 | mirror.8 ................ the OpenPKG package mirroing utility manual page (pregenerated output) |
| michael@428 | 130 | |
| michael@428 | 131 | build.pl ................ the OpenPKG package building utility script |
| michael@428 | 132 | build.pod ............... the OpenPKG package building utility manual page (source) |
| michael@445 | 133 | build.8 ................. the OpenPKG package building utility manual page (pregenerated output) |
| michael@428 | 134 | |
| michael@428 | 135 | makeproxy.pl ............ the OpenPKG proxy package building utility script |
| michael@428 | 136 | makeproxy.pod ........... the OpenPKG proxy package building utility manual page (source) |
| michael@445 | 137 | makeproxy.8 ............. the OpenPKG proxy package building utility manual page (pregenerated output) |
| michael@13 | 138 | |
| michael@13 | 139 | openpkg.org.pgp ......... the OpenPGP public key "OpenPKG <openpkg@openpkg.org>" |
| michael@13 | 140 | openpkg.com.pgp ......... the OpenPGP public key "OpenPKG GmbH <openpkg@openpkg.com>" |
| michael@13 | 141 | openpkg.net.pgp ......... the OpenPGP public key "OpenPKG Foundation e.V. <openpkg@openpkg.net>" |
| michael@13 | 142 | |
| michael@428 | 143 | license.sh .............. the OpenPKG license management utility script |
| michael@428 | 144 | license.pod ............. the OpenPKG license management utility manual page (source) |
| michael@445 | 145 | license.8 ............... the OpenPKG license management utility manual page (pregenerated output) |
| michael@428 | 146 | license-BOOT.txt ........ the OpenPKG Framework Bootstrapping License |
| michael@428 | 147 | license-COMMUNITY.txt ... the OpenPKG Framework Community License |
| michael@428 | 148 | license-EVAL.txt ........ the OpenPKG Framework Evaluation License |
| michael@428 | 149 | license-EXAMPLE.txt ..... the OpenPKG Framework Example License |
| michael@428 | 150 | license-PROMO.txt ....... the OpenPKG Framework Promotion License |
| michael@428 | 151 | license-RECOVERY.txt .... the OpenPKG Framework Recovery License |
| michael@428 | 152 | license.lua ............. the OpenPKG Framework license processor script |
| michael@428 | 153 | |
| michael@428 | 154 | Makefile ................ development build procedures |
| michael@428 | 155 | |
| michael@13 | 156 | The Bootstrapping Procedure |
| michael@13 | 157 | --------------------------- |
| michael@13 | 158 | |
| michael@13 | 159 | The complexity of this OpenPKG RPM package results from the fact that |
| michael@13 | 160 | we force us to treat this bootstrapping package equal to every other |
| michael@13 | 161 | regular OpenPKG RPM package. First, this implies that the packaging |
| michael@428 | 162 | tool RPM is packaged with itself as an OpenPKG RPM package -- which |
| michael@428 | 163 | means that its build procedure is a real RPM .spec file and it can be |
| michael@428 | 164 | installed and upgraded through a binary or source RPM. Second, RPM is |
| michael@428 | 165 | installed into the same filesystem hierarchy as all other packages. |
| michael@428 | 166 | Third, RPM manages its own files. The reason for this approach should |
| michael@428 | 167 | be obvious: 100% consistency for the whole OpenPKG software packaging |
| michael@428 | 168 | facility! |
| michael@13 | 169 | |
| michael@13 | 170 | The drawback is that this package requires a very tricky bootstrapping |
| michael@13 | 171 | procedure which had cost a lot of time to figure out and establish. If |
| michael@13 | 172 | you ever wanted to know the gory details, here they are... |
| michael@13 | 173 | |
| michael@13 | 174 | The first step was that we wrote the regular openpkg.spec file for |
| michael@13 | 175 | building the bootstrap package with OpenPKG RPM under the assumption |
| michael@13 | 176 | that OpenPKG RPM is already available. This way we can provide OpenPKG |
| michael@13 | 177 | RPM as an RPM package. Just remains the problem how we actually |
| michael@13 | 178 | bootstrap in case where OpenPKG RPM is still not available, i.e., |
| michael@13 | 179 | when we reach a new platform and have to build the package from |
| michael@13 | 180 | scratch. Here the "openpkg.boot" script comes into play. It executes |
| michael@13 | 181 | the "openpkg.spec" build procedure very similar to the way the real |
| michael@13 | 182 | OpenPKG RPM would do ("openpkg rpm -bb"). That is, "openpkg.boot" |
| michael@13 | 183 | partly emulates OpenPKG RPM -- just enough that "openpkg.spec" works. |
| michael@13 | 184 | As a result, "openpkg.spec" cannot use any fancy OpenPKG RPM features |
| michael@13 | 185 | or other things before "openpkg.boot" is able to emulate it, of |
| michael@13 | 186 | course. |
| michael@13 | 187 | |
| michael@13 | 188 | After "openpkg.boot" executed the "%prep", "%build" and "%install" |
| michael@13 | 189 | scripts of "openpkg.spec", there is a fresh version of the target |
| michael@13 | 190 | filesystem hierarchy staying under a temporary "build root". The |
| michael@13 | 191 | "openpkg.boot" script then creates a very special temporary "openpkg |
| michael@428 | 192 | rpm" command which allows the installed "openpkg rpm" command |
| michael@428 | 193 | inside the "build root" to work (although it is built for the |
| michael@428 | 194 | final target filesystem path). Then the $OPENPKG_BOOT variable |
| michael@428 | 195 | is set and the package is _again_ build via "openpkg.spec" -- |
| michael@428 | 196 | but this time with the real OpenPKG RPM. To avoid unneccessary |
| michael@445 | 197 | recompilation, the "openpkg.spec" skips "%prep", "%build" and |
| michael@428 | 198 | "%install" sections if $OPENPKG_BOOT is defined. So, on this second |
| michael@428 | 199 | build phase, only the "%files" section is effectively executed, i.e., |
| michael@428 | 200 | a binary OpenPKG RPM package "openpkg-V-R.P-T.rpm" is rolled from the |
| michael@428 | 201 | files in the "build root". Additionally, a source OpenPKG RPM package |
| michael@428 | 202 | "openpkg-V-R.src.rpm" is rolled for consistency reasons, too. |
| michael@13 | 203 | |
| michael@13 | 204 | Finally, we override the installation in the "build root" |
| michael@13 | 205 | again by installing the now rolled binary OpenPKG RPM package |
| michael@13 | 206 | "openpkg-V-R.P-T.rpm" by using the real OpenPKG RPM. This way |
| michael@13 | 207 | we achieve that OpenPKG RPM is remembered as a real OpenPKG RPM |
| michael@13 | 208 | package in the RPM database. We just have to make sure the package |
| michael@13 | 209 | is still relocated to the "build root" while installing. For this |
| michael@13 | 210 | we could use "--prefix=$RPM_BUILD_ROOT%{l_prefix}", but this would |
| michael@13 | 211 | create an incorrect file list for the package "openpkg" in the RPM |
| michael@13 | 212 | database. Instead we use the tricky "--justdb" option for "openpkg |
| michael@13 | 213 | rpm" which means "openpkg rpm" behaves as it would install into the |
| michael@13 | 214 | real location, but does not actually install anything. But as a |
| michael@445 | 215 | side effect, the database inside the "build root" is now correct. |
| michael@13 | 216 | |
| michael@13 | 217 | After this procedure, the "build root" contains the target filesystem |
| michael@13 | 218 | hierarchy with OpenPKG RPM installed with itself. What is now just |
| michael@13 | 219 | remaining is to roll a bootstrap package "openpkg" with this stuff for |
| michael@13 | 220 | initial installation without OpenPKG RPM. For this the "build root" |
| michael@13 | 221 | is packed into a "tarball", compressed, again wrapped into another |
| michael@13 | 222 | tarball together with the uncompression tools ("bzip2" and "tar"), |
| michael@13 | 223 | and finally wrapped into a self-extracting shell script by appending |
| michael@428 | 224 | "etc.wrapbin.sh" (padded to 64KB for easier unpacking of the attached |
| michael@13 | 225 | tarball) to its front. |
| michael@13 | 226 | |
| michael@13 | 227 | The result is the binary bootstrap script "openpkg-V-R.P-T.sh" which |
| michael@428 | 228 | can be used to install the target hierarchy from scratch without |
| michael@445 | 229 | any preinstalled OpenPKG RPM. Nevertheless, the installed target |
| michael@428 | 230 | hierarchy looks _exactly_ as it would have been installed with OpenPKG |
| michael@428 | 231 | RPM. If one later wants to upgrade this hierarchy one can just use the |
| michael@13 | 232 | generated (or a newer) "openpkg-V-R.P-T.rpm". |
| michael@13 | 233 | |
| michael@445 | 234 | To allow one to easily repeat this from source bootstrapping procedure |
| michael@13 | 235 | on other machines, one can run "./openpkg.boot -s" which rolls |
| michael@13 | 236 | a "openpkg-V-R.src.sh" script which is a self-extracting script |
| michael@13 | 237 | containing an attached tarball of the sources of this directory. This |
| michael@13 | 238 | script contains the same contents like "openpkg-V-R.src.rpm", but |
| michael@13 | 239 | is intended for running the described bootstrapping procedure from |
| michael@13 | 240 | scratch without any OpenPKG RPM. |
| michael@13 | 241 |
