Hi all, I'm using the subject line of a 15-year-old thread [1] to restart this discussion, even though the metadata field is more commonly called "Optional" these days. The feature in question is even more expansive than what Jose proposed in [1]. I've been developing for prt-get the additional logic to recognize optional dependencies when it sorts the install targets. A few months ago, Tim raised the issue that a sysup operation would sometimes update vulkan-loader AFTER some of the ports that optionally depend on it, unaware of the eager linking that the maintainers try to document using the "Optional" metadata field. In response to Tim's comment, I started adding to the prt-get source code the ability to respect optional dependencies. Here's Johannes acknowledging that this feature is not something we can accomplish using the existing 'depinst' command: On Thu May 1 12:17 UTC 2008, Johannes Winkelmann wrote:
prt-get depinst pinentry --optional=gtk,qt3 Note that this would be the same as running prt-get depinst pinentry gtk qt3 err, no wait. There's no guarantee that gtk an qt3 will be installed before, so this would indeed need additional code.
On Thu May 1 14:13 UTC 2008, Johannes Winkelmann wrote:
It'll calculate the dependecies of all ports specified on the command line, but since there's no dependency relationship between qt3 (or gtk) and pinentry qt3 may as well be installed after pinentry, thus not installing the optional pinentry stuff.
Interestingly, this 15-year-old example no longer gives us a good test case, because the current Pkgfile for pinentry explicitly disables the gtk2, qt, and qt5 interfaces. So instead I tested my forked prt-get using a more recent example, libxkbcommon which optionally enables x11 support. You can find my report under the doc/ directory of [2], i3-softdeps.test is the filename. Users of gtk or qt5 who want the corresponding pinentry interface might resort to creating their own port, calling it pinentry-gtk or pinentry-qt5, respectively. But in FS#1576 Tim expressed his preference for flexible Pkgfiles, rather than letting a large number of nearly-identical ports clutter up the portdb. On Mon Aug 29 07:35 UTC 2022, Tim Biermann wrote:
I am making up this example: having a port named rust-clang for example that overlays opt/rust because a user wanted a rust binary compiled by clang and linked by lld is not what I ideally want to see. We don't need to reproduce Archs AUR where people add things like that. It could be a concious decision by opt/rust to detect both clang and lld, then set the flags to make that happen. ... I am very much pro "fluid Pkgfiles" which can suit all use-cases. It minimises the need to bump a ton of dups.
Although most maintainers these days follow Tim's lead and attempt to suit all use-cases with the ports they care for, this practice was less widely accepted at the time thread [1] was active. Anton [3] speaks to the then-common practice of writing build() with no branching logic, performing identically no matter what system is used to run pkgmk. On Tue Aug 29 16:16 UTC 2006, Anton wrote:
What is CRUX? ISO images? No. Pkgutils? No - they are really not necessary, you can write simple scripts in shell using tar with similar functionality. Pkgfiles? Bingo! Indeed, Pkgfiles is really valuable thing in CRUX. That's it, Pkgfiles should be as KISS as possible. If they are not, then it's not CRUX anymore.
A few years later, Tilman expresses a subtler form of this preference for rigid Pkgfiles with his contribution to thread [1]. On Thu May 1 20:51 UTC 2008, Tilman Sauerbeck wrote:
Danny started using Optional as an alias for nice-to-have. So it's a different name for the same thing, and apparently it's not supposed to introduce pkginfo-i-grep-something games in build() per se.
But now the pendulum has swung in the other direction, and instead of a portdb cluttered with thousands of near-duplicate Pkgfiles (KISS because non-fluid), we have a trimmed-down portdb that hides its versatility inside fluid Pkgfiles. The prt-get that we inherited is better suited for a portdb shaped along the lines of Anton's or Tilman's remarks, and less suited for a portdb whose maintainers make full use of the "Optional" metadata field and "if-prt-get-isinst-something games in build()". Before incorporating into an official release of prt-get any code from my softdep fork [4] or its sister fork that merges "install" and "update" [2], we should address the question of a possible increase in maintenance burden. On Thu May 1 16:52 UTC 2008, Juergen Daubert wrote:
TBH I don't know why/who/when this field was introduced. And TBH I don't like it at all. In many cases it's not really maintainable, cause the maintainer isn't able to test all the possible options a program is providing, e.g php or apache.
Back in 2008 when "Optional" was still new, Juergen had a valid point that if prt-get acquired the ability to do something with the field, maintainers might feel obligated to populate it to the best of their ability. These days the field is in widespread use even with no official support in prt-get; maintainers just got in the habit of having a convenient place to record their "if-prt-get-isinst-something games in build()". And if build() is designed and tested only for a narrow range of configurations, what then? Most likely, the downstream users who want that particular port configured differently will create their own port, either keeping it in a private overlay or adding to the proliferation of near-dups on the portdb. Either way, the downstream user experience in this latter scenario is not ideal. But does it really relieve the CRUX team from having to "test all the possible options"? On the contrary! In the scenario where the portdb starts getting cluttered with near-dups that are on radically different version bump schedules, maintainers might feel pressured to restore order by incorporating the various compile-time options into the build() of the official port. And in the scenario where everyone's private overlay is cluttered with near-dups that reflect their particular use-case, the official maintainers might still find their inboxes flooded with support requests asking how to accomplish in build() something specific to that use-case. To my mind, inflexible Pkgfiles (or the softdep-ignorant prt-get which encourages them) only offer the illusion of distributing the maintenance burden among the broader CRUX community. The core CRUX team still ends up working just as hard, perhaps moreso when they help troubleshoot the problems of someone who tried making an unofficial port just because the official one is not flexible enough for the desired use-case. So why not let our official tools acknowledge the current reality of fluid Pkgfiles and widespread use of the "Optional" field? With prt-get able to respect those optional dependencies that the maintainer has bothered to test, the downstream user is better equipped to avoid build errors, like the `prt-get depinst i3` failure that Matt pointed out in [5], or to feel tempted to throw together yet another dup that hard-codes the desired dependencies for their use-case. -- John [1] https://lists.crux.nu/pipermail/crux-devel/2008-May/003366.html [2] https://git.crux.nu:82/farkuhar/prt-get/src/branch/mixed-upinst [3] https://lists.crux.nu/pipermail/crux-devel/2006-August/001912.html [4] https://git.crux.nu:82/farkuhar/prt-get/src/branch/softdeps [5] https://libera.irclog.whitequark.org/crux/2023-08-21 (16:36)
Hi all, jmcquah@disroot.org wrote:
I've been developing for prt-get the additional logic to recognize optional dependencies when it sorts the install targets.
Sorting the install targets is only one of the commands that my fork does differently, depending on whether it sees the --softdeps flag (or the corresponding directive in prt-get.conf). Other commands include depends, dependent, deptree, and listorphans, as explained in the revised man-page. To get an idea for how big a patch is needed to implement these changes, I ran cloc on three different prt-get source trees (because running `diff -r` will also pick up the cosmetic whitespace changes). Here are the results. master branch, commit 8bc31c07455, scanned by github.com/AlDanial/cloc ---------------------------------------------------------------------- Language files blank comment code ---------------------------------------------------------------------- C++ 18 869 708 4478 C/C++ Header 17 270 267 704 m4 1 99 15 870 make 4 14 1 35 Bourne Shell 5 720 953 5612 ---------------------------------------------------------------------- SUM: 45 1972 1944 11699 ---------------------------------------------------------------------- softdeps branch, scanned by github.com/AlDanial/cloc ---------------------------------------------------------------------- Language files blank comment code ---------------------------------------------------------------------- C++ 18 871 714 4642 C/C++ Header 17 269 267 711 Meson 4 4 0 45 ---------------------------------------------------------------------- SUM: 39 1144 981 5398 ---------------------------------------------------------------------- Key takeaways: * implementing the softdeps feature only increased the C++ codebase by about 200 lines. * switching from autotools to meson trims the SLOC count by half! mixed-upinst branch, scanned by github.com/AlDanial/cloc ---------------------------------------------------------------------- Language files blank comment code ---------------------------------------------------------------------- C++ 18 802 681 4419 C/C++ Header 17 265 266 711 Markdown 1 13 0 97 Meson 4 4 0 45 ---------------------------------------------------------------------- SUM: 40 1084 947 5272 ---------------------------------------------------------------------- Key takeaways: * modest refactoring brought the C++ SLOC count down to where it was at the 5.19.6 release ... * ... while also incorporating Alan's proposed --group flag, the caching of getPkgmkSetting() values (FS#595), and question 1 at the bottom of TODO ("to install and update, which ever method is needed") Some of the refactoring in my mixed-upinst branch might also benefit the softdeps branch. But in at least one key behaviour (dependency resolution by default) the two branches are different, so adapting the refactored functions is a more delicate task. In the thread "RFC: next generation prt-get" from 15 years ago, Tilman and Johannes reported their own habits regarding dependency resolution. At the time, manual dependency tracking was apparently still very common, albeit with the assistance of a visual aid like `prt-get deptree` [1]. If that's still the majority preference among CRUX users, then even without all the refactoring that went into mixed-upinst, the softdeps branch of my fork is probably the better way to address Tim's point about suboptimal ordering of targets during sysup. In any case, as Fun points out in FS#1313, "CRUX is incredibly libertarian" so everyone is free to use whichever package management implementation they want. [1] https://lists.crux.nu/pipermail/crux/2008-June/001799.html -- John
Off-topic, but graphicsmagick i tried out (since imagemagick GUI usage is seriously broken), but it does not work out at all since it tries to open MODULE.la files (aka libtool), yet only MODULE.so (aka ELF shared object) are installed (here). (I would not have been able to use it since it does not seem to have a GUI at all.) If only gimp would not have such a dependency chain, i mean gnutls, brotli, rust, that is just nothing but totally sick. A simple thing like Microsoft Paint in the 90s. Being able to optionally try out / use that Optional: thing would be cool. Thanks, --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Hallo Steffen, Steffen Nurpmeso <steffen@sdaoden.eu> wrote:
Off-topic, but graphicsmagick does not work out at all since it tries to open MODULE.la files (aka libtool), yet only MODULE.so (aka ELF shared object) are installed (here).
Try the latest version (graphicsmagick 1.3.41). I noticed the same problem a few weeks ago, and revised the Pkgfile to be less aggressive about deleting libtool garbage [1]. The commands I use most frequently (gm convert, gm resize) now seem to work as expected.
If only gimp would not have such a dependency chain, i mean gnutls, brotli, rust, that is just nothing but totally sick. A simple thing like Microsoft Paint in the 90s.
What good does it do to complain about the dependency chain on this list (unless some gimp developers happen to be subscribed)? The folks who determine gimp's dependencies are the upstream developers; we packagers are only responsible for ensuring that our repository's versions of those dependencies will satisfy gimp. If you want do do image editing with a simpler dependency chain, you can try mtpaint [2], an app that runs in the cloud [3], or bundle your own lua scripts into a custom mpv config [4]. If you want gimp to have a simpler dependency chain, then reach out to the gimp developers and ask if they can ship an equally featureful product without linking to gnutls, brotli, or whatever else you consider bloat.
Being able to optionally try out / use that Optional: thing would be cool.
Just to be clear, this feature only lets prt-get do something useful with the metadata that maintainers are already providing; it won't magically transform an existing port to make your unwanted dependencies optional. With that in mind, feel free to clone my repo [5] and take it for a test drive. Footnotes: [1] by adding -maxdepth 1 to the find command. Previously all *.la files under $PKG/usr/lib were being deleted, based on the claim that libtool "doesn't actually need them anyway and it will gladly continue without them" (https://github.com/baguette/crux-ports/blob/master/Util/bin/ libtool-garbage-day.sh) [2] https://github.com/baguette/crux-ports/tree/master/mtpaint [3] but then you might complain about the dependency chain of the browser; obviously something more complex than lynx is needed to edit images in the cloud. [4] gemini://rawtext.club/~jmq/recycled/mvi-jpg-crop.gmi [5] https://git.sdf.org/jmq/prt-get/src/branch/mixed-upinst -- John
John McQuah wrote in <2XUSZPAOPXUBS.2WEMHMK6A0UV2@disroot.org>: |Steffen Nurpmeso <steffen@sdaoden.eu> wrote: |> Off-topic, but graphicsmagick does not work out at all since |> it tries to open MODULE.la files (aka libtool), yet only MODULE.so |> (aka ELF shared object) are installed (here). | |Try the latest version (graphicsmagick 1.3.41). I noticed the same |problem a few weeks ago, and revised the Pkgfile to be less aggressive |about deleting libtool garbage [1]. The commands I use most frequently |(gm convert, gm resize) now seem to work as expected. It was contrib/ thing. |> If only gimp would not have such a dependency chain, i mean |> gnutls, brotli, rust, that is just nothing but totally sick. |> A simple thing like Microsoft Paint in the 90s. ... |those dependencies will satisfy gimp. If you want do do image editing |with a simpler dependency chain, you can try mtpaint [2], an app that That is overly cool (though without Pkgfile here, just works like this), thanks for the pointer! Unfortunately i seem to be too stupid to use it, no matter the manual. ... then again i tried again, and after four hours i am where i wanted to be. Thanks! ... |> Being able to optionally try out / use that Optional: thing would |> be cool. | |Just to be clear, this feature only lets prt-get do something useful Yes i guess an additional command line option to make use of it? --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
participants (3)
-
jmcquah＠disroot.org -
John McQuah
-
Steffen Nurpmeso