A patch for httpup to support whitelisting/blacklisting files
Hi, I made a patch for httpup to support whitelisting/blacklisting files via patterns (shell glob). I think there are several benefits to have this feature: 1. With whitelist, we fetch only the files we really wanted, thus it speeds up the process of updating httpup port trees, and saves some disk space (trivial). 2. We can blacklist some specific files, like */.footprint. It also makes httpup more useful to use it for other things other than sharing port repositories. 3. The most important thing is, it helps to reuse existing ports. As far as I know, most unofficial ports are shared via httpup. When we are looking for some ports made by others, now we have 2 choices, either to add that entire repository to our port tree, or to fork and maintain that single port by ourself. Many people choose the second option, because option one has many disadvantages, (see 1), while option 2 is even worse, it leads to more forks, I think that's one of the reasons why we have so many duplicated ports in portdb. Yes, we still have another option, we can instruct httpup to fetch only a "fragment" of a repository instead of the entire of it. (see /etc/ports/drivers/httpup), to use this feature, we just need to do something like this in repo.httpup: URL="http://repo-url/#port1 http://repo-url/#port2 ..." It works fine, but that means for every single port, we need to fetch the REPO file from scratch, if the URL list is long enough, you would get nowhere to speed up. The patch solved these problems by allowing whitelisting and or blacklisting files via shell globs. It add two options to httpup, -i (include) and -x (exclude), both of them accept one argument, that is the file to read patterns from, the argument can also be '-' to indicate stdin. The pattern works the way as fnmatch with FNM_PATHNAME enabled. To use httpup with a whitelist of ports, just: echo "porta portb portc" | tr ' ' '\n' | httpup sync -i - url . Note: It may have bugs. I've done some basic test, it should work fine most of the time, if you run into some bugs and you want to try debugging it, #define DEBUG_PATTERNS at the begining of httpup.cpp, and you'll see some printed logs related to this patch. Hope it helps - phi
participants (1)
-
phi