Easier patching with pkgmk
Hi everyone, I had an idea to make software patching easier, and lower the need to fork ports. Sometimes, I just fork some port from contrib because I want to add a patch (eg: thttpd), or I need to modify a file before actually compiling the software (eg: config.h on contrib/st). I figured out that by adding a single variable to `/etc/pkgmk.conf`, it would be easier to "tweak" sofwares by patching them beforehand: ``` /etc/pkgmk.conf [...] PKGMK_PATCH_DIR=/usr/patch And then, have ports include something like: ``` /usr/port/contrib/example/PKgfile [...] cd $name-$version cat $PKGMK_PATCH_DIR/$name/*.diff | patch -Np1 make make DESTDIR=$PKG PREFIX=/usr MANDIR=/usr/man install So if I need a patch for thttpd, I'll just add it to `/usr/patch/thttpd`. To have the `st` port configured with my `config.h`, I'll just create a patch for it in `/usr/patch/st`. All ports might not need this obviously, but this could result in a simple and efficient way to let user tweak packages from the "official" port trees without any need to actually fork them, and double the effort. What do you think about it? -- Willy Goiffon (z3bra on #crux)
On 18/10, Willy Goiffon wrote:
What do you think about it?
-- Willy Goiffon (z3bra on #crux)
Hi, I think it's a great idea. I'm going to have to try this locally. Thanks for sharing! Regards, -- - Brian Tomlinson GPG: 3694569D | https://www.brianctomlinson.com
On 18.10.2015 17:20, Willy Goiffon wrote:
Hi everyone,
I had an idea to make software patching easier, and lower the need to fork ports.
Sometimes, I just fork some port from contrib because I want to add a patch (eg: thttpd), or I need to modify a file before actually compiling the software (eg: config.h on contrib/st).
I figured out that by adding a single variable to `/etc/pkgmk.conf`, it would be easier to "tweak" sofwares by patching them beforehand:
``` /etc/pkgmk.conf [...] PKGMK_PATCH_DIR=/usr/patch
And then, have ports include something like:
``` /usr/port/contrib/example/PKgfile [...] cd $name-$version cat $PKGMK_PATCH_DIR/$name/*.diff | patch -Np1 make make DESTDIR=$PKG PREFIX=/usr MANDIR=/usr/man install
So if I need a patch for thttpd, I'll just add it to `/usr/patch/thttpd`. To have the `st` port configured with my `config.h`, I'll just create a patch for it in `/usr/patch/st`.
But you still would have to maintain an additional source (your patches) - whenever the original port changes, a patch may become obsolete, not applicable, or plainly wrong. I never include ports from contrib directly, but copy them into my own repository, update the version (most of them are outdated), adjust the Pkgfile to my needs, make patches etc. This has the advantage that everything that belongs to the port is concentrated in a single directory. Regards, Bernd
Bernd Eggink wrote:
On 18.10.2015 17:20, Willy Goiffon wrote:
Hi everyone,
I had an idea to make software patching easier, and lower the need to fork ports.
Sometimes, I just fork some port from contrib because I want to add a patch (eg: thttpd), or I need to modify a file before actually compiling the software (eg: config.h on contrib/st).
I figured out that by adding a single variable to `/etc/pkgmk.conf`, it would be easier to "tweak" sofwares by patching them beforehand:
``` /etc/pkgmk.conf [...] PKGMK_PATCH_DIR=/usr/patch
And then, have ports include something like:
``` /usr/port/contrib/example/PKgfile [...] cd $name-$version cat $PKGMK_PATCH_DIR/$name/*.diff | patch -Np1 make make DESTDIR=$PKG PREFIX=/usr MANDIR=/usr/man install
So if I need a patch for thttpd, I'll just add it to `/usr/patch/thttpd`. To have the `st` port configured with my `config.h`, I'll just create a patch for it in `/usr/patch/st`.
But you still would have to maintain an additional source (your patches) - whenever the original port changes, a patch may become obsolete, not applicable, or plainly wrong. I never include ports from contrib directly, but copy them into my own repository, update the version (most of them are outdated), adjust the Pkgfile to my needs, make patches etc. This has the advantage that everything that belongs to the port is concentrated in a single directory.
Sure you'd have to maintain the patches yourself, but you'd have to do it too in case of a fork. If you want to keep everything local to the port directory, you could just set PKGMK_PATCH_DIR=./patch In pkgmk.conf, and have everything stored along with the port itself. I think it's a non-intrusive way of allowing port customisation to every user without decreasing the quality or simplicity of existing ports. It is also easy to understand on first read, so new users would be able to get the hang of it really quickly.
On 19.10.2015 13:58, Willy Goiffon wrote:
Bernd Eggink wrote:
On 18.10.2015 17:20, Willy Goiffon wrote:
Hi everyone,
I had an idea to make software patching easier, and lower the need to fork ports.
Sometimes, I just fork some port from contrib because I want to add a patch (eg: thttpd), or I need to modify a file before actually compiling the software (eg: config.h on contrib/st).
I figured out that by adding a single variable to `/etc/pkgmk.conf`, it would be easier to "tweak" sofwares by patching them beforehand:
``` /etc/pkgmk.conf [...] PKGMK_PATCH_DIR=/usr/patch
And then, have ports include something like:
``` /usr/port/contrib/example/PKgfile [...] cd $name-$version cat $PKGMK_PATCH_DIR/$name/*.diff | patch -Np1 make make DESTDIR=$PKG PREFIX=/usr MANDIR=/usr/man install
So if I need a patch for thttpd, I'll just add it to `/usr/patch/thttpd`. To have the `st` port configured with my `config.h`, I'll just create a patch for it in `/usr/patch/st`.
But you still would have to maintain an additional source (your patches) - whenever the original port changes, a patch may become obsolete, not applicable, or plainly wrong. I never include ports from contrib directly, but copy them into my own repository, update the version (most of them are outdated), adjust the Pkgfile to my needs, make patches etc. This has the advantage that everything that belongs to the port is concentrated in a single directory.
Sure you'd have to maintain the patches yourself, but you'd have to do it too in case of a fork. If you want to keep everything local to the port directory, you could just set
PKGMK_PATCH_DIR=./patch
In pkgmk.conf, and have everything stored along with the port itself.
I think it's a non-intrusive way of allowing port customisation to every user without decreasing the quality or simplicity of existing ports. It is also easy to understand on first read, so new users would be able to get the hang of it really quickly. _______________________________________________ CRUX mailing list CRUX@lists.crux.nu https://lists.crux.nu/mailman/listinfo/crux
Hi all, sry to bring up the past, but I recently realized I have a few ports myself which are only cloned due to some simple patches, e.g.: - additional directive to ./configure in curl - very short patch for conky to display upto 20 instead of 10 processes - ... Thus I modified /usr/bin/ports itself to do something similar as proposed by Willy. Now after a "ports -u", it applies patches (named *.patch) laying in /usr/patches/$repo/$port/ on files in /usr/ports/$repo/$port/ and possibly adds files (named *.new) to /usr/ports/$repo/$port/ (minus the ending ".new") and their md5sums to /usr/ports/$repo/$port/.md5sum. The first gives the ability to add directives to Pkgfile (e.g. new sources in the source array), the second gives the posibility to add patch files or other new source files. Maybe someone can use this or has some comments on this approach? For illustration, I attached the content of /usr/patches/core/portutils itself. I prefer this method to Willys one (although his is much shorter), because of two things: 1. Pkgfiles should only contain stuff, that may differ from port to port - and not some generic "apply patch if available"-command. However, this could be shifted into pkgmk, but 2. I really like pkgmk to not involve any magic, because when I stumble over problems when compiling custom packages, I want to track them down separately from hidden magic. cheers, Erich
Erich Eckner wrote:
On 19.10.2015 13:58, Willy Goiffon wrote:
Bernd Eggink wrote:
On 18.10.2015 17:20, Willy Goiffon wrote:
Hi everyone,
I had an idea to make software patching easier, and lower the need to fork ports.
Sometimes, I just fork some port from contrib because I want to add a patch (eg: thttpd), or I need to modify a file before actually compiling the software (eg: config.h on contrib/st).
I figured out that by adding a single variable to `/etc/pkgmk.conf`, it would be easier to "tweak" sofwares by patching them beforehand:
``` /etc/pkgmk.conf [...] PKGMK_PATCH_DIR=/usr/patch
And then, have ports include something like:
``` /usr/port/contrib/example/PKgfile [...] cd $name-$version cat $PKGMK_PATCH_DIR/$name/*.diff | patch -Np1 make make DESTDIR=$PKG PREFIX=/usr MANDIR=/usr/man install
So if I need a patch for thttpd, I'll just add it to `/usr/patch/thttpd`. To have the `st` port configured with my `config.h`, I'll just create a patch for it in `/usr/patch/st`.
But you still would have to maintain an additional source (your patches) - whenever the original port changes, a patch may become obsolete, not applicable, or plainly wrong. I never include ports from contrib directly, but copy them into my own repository, update the version (most of them are outdated), adjust the Pkgfile to my needs, make patches etc. This has the advantage that everything that belongs to the port is concentrated in a single directory.
Sure you'd have to maintain the patches yourself, but you'd have to do it too in case of a fork. If you want to keep everything local to the port directory, you could just set
PKGMK_PATCH_DIR=./patch
In pkgmk.conf, and have everything stored along with the port itself.
I think it's a non-intrusive way of allowing port customisation to every user without decreasing the quality or simplicity of existing ports. It is also easy to understand on first read, so new users would be able to get the hang of it really quickly. _______________________________________________ CRUX mailing list CRUX@lists.crux.nu https://lists.crux.nu/mailman/listinfo/crux
Hi all,
sry to bring up the past, but I recently realized I have a few ports myself which are only cloned due to some simple patches, e.g.: - additional directive to ./configure in curl - very short patch for conky to display upto 20 instead of 10 processes - ...
Thus I modified /usr/bin/ports itself to do something similar as proposed by Willy. Now after a "ports -u", it applies patches (named *.patch) laying in /usr/patches/$repo/$port/ on files in /usr/ports/$repo/$port/ and possibly adds files (named *.new) to /usr/ports/$repo/$port/ (minus the ending ".new") and their md5sums to /usr/ports/$repo/$port/.md5sum. The first gives the ability to add directives to Pkgfile (e.g. new sources in the source array), the second gives the posibility to add patch files or other new source files.
Maybe someone can use this or has some comments on this approach? For illustration, I attached the content of /usr/patches/core/portutils itself.
This looks like a 'higher-level' approach to me. My idea was to provide a way to patch the software directly, while yours is provides a way to patch the whole port (Pkgfile/md5sum).
I prefer this method to Willys one (although his is much shorter), because of two things: 1. Pkgfiles should only contain stuff, that may differ from port to port - and not some generic "apply patch if available"-command. However, this could be shifted into pkgmk, but
While building myself an LFS system, I realized how important patches are, and that it would be great to have a well-defined mechanism for this task, and no leave it "Pkgfile's implementation defined". The typical process for building a software is 0. download 1. extract 2. patch 3. build 4. package/install Each task above has its corresponding function defined in the `pkgmk`, except for "patch". So I figured having a generic solution for it would be a good idea.
2. I really like pkgmk to not involve any magic, because when I stumble over problems when compiling custom packages, I want to track them down separately from hidden magic.
Magic is always a bad thing when it comes to debugging. But having a directory for patches isn't "hidden" magic (as long as it's clear in the documentation that after fetching/extracting the sources, the code is patched). It is, in my opinion, cleaner to only patch the software rather that the whole port. Because with my solution, you don't need to change anything to the port itself, simply to the software being built. Your solution "hot-patches" the Pkgfile/md5sum, resulting in a totally new port. Cheers. -- Willy
On 16May26:1221+0200, Willy Goiffon wrote:
Erich Eckner wrote:
On 19.10.2015 13:58, Willy Goiffon wrote:
Bernd Eggink wrote:
On 18.10.2015 17:20, Willy Goiffon wrote:
I had an idea to make software patching easier, and lower the need to fork ports.
But you still would have to maintain an additional source (your patches) - whenever the original port changes, a patch may become obsolete, not applicable, or plainly wrong. I never include ports from contrib directly, but copy them into my own repository, update the version (most of them are outdated), adjust the Pkgfile to my needs, make patches etc. This has the advantage that everything that belongs to the port is concentrated in a single directory.
Sure you'd have to maintain the patches yourself, but you'd have to do it too in case of a fork.
sry to bring up the past, but I recently realized I have a few ports myself which are only cloned due to some simple patches, e.g.: - additional directive to ./configure in curl - very short patch for conky to display upto 20 instead of 10 processes - ...
Thus I modified /usr/bin/ports itself to do something similar as proposed by Willy.
This looks like a 'higher-level' approach to me. My idea was to provide a way to patch the software directly, while yours is provides a way to patch the whole port (Pkgfile/md5sum).
I follow Mother's rule number 1 in "What Mother Never Told You About VM Service" (http://www.leeandmelindavarian.com/Melinda/tutorial.pdf), that still holds a lot of good basic sysadmin rules even though most of the specifics are different: "Never change anything IBM sends you." In context that means keeping IBM's file contents and organization unmodified and making copies elsewhere and editing the copies, then ensuring that maintenance processing uses your copies instead of IBM's. The VM/CMS disk search order facility makes that easy and I think inspired Plan 9's union directory facility. I apply this to CRUX by staying the heck out of the canonical repositories (I'm not certain what rsync is going to do to any changes I make in there, anyway). This also means I can just build anything exactly as the distribution intended simply by temporarily inserting an override /etc/prt-get.conf prtdir for the original directory:package as the first prtdir; e.g., prtdir /usr/ports/opt:grub2 The point is I know it's a canonical build if the port resides in an official port collection, I never have to answer the question, "Was this build tainted in some way?" (other than my environment's idiosyncracies, of course). If I need to modify a distributed port (who doesn't ever need to do this?), I cp -a copy that port into a non-distributed collection (I use /usr/ports/local) and that is normally at the head of the prtdir records. I then edit the Pkgfile and, as Mother said in rule number 4: Always leave tracks, I add a "Localized:" header documenting the raison-d'etre; e.g., tmpfix, the source collection, the target system(s), and the localizer (me). Below the headers I add commentary as needed to explain why this localization was made. A diff between the two Pkgfiles makes it very clear what is afoot. In regard to local patches, they are no different than official patches except they exist in a non-distributed collection. If a patch applies to multiple ports, you can symlink to one actual patch but need to put them into a fixed location (I just copy such patches as needed since I move retired local ports from local to local.inactive which appears not in /etc/prt-get.conf). -- <not cent from sell> May the LORD God bless you exceedingly abundantly! Dave_Craig______________________________________________ "So the universe is not quite as you thought it was. You'd better rearrange your beliefs, then. Because you certainly can't rearrange the universe." __--from_Nightfall_by_Asimov/Silverberg_________________
participants (5)
-
Bernd Eggink -
Brian Tomlinson -
David L. Craig -
Erich Eckner -
Willy Goiffon