Hi there,
Nowadays most of us select a specific dl.sf.net mirror my means of
/etc/hosts; this works good in most cases, however it has a couple of
disadvantages:
- it's static; added/removed hosts lead to errors
- you potentially don't use the fastest server
- sometimes, even your stable server might not have a file.
As in our last IRC meeting the idea of addressing this in pkgutils
was frowned upon (:-)), I chose a pure "userspace" approach, where the
script is called from pkgmk.conf, and changes the entries in the
'source' array to select a mirror.
The tool initially reads in a file and checks whether these servers (in
order of appearance) have the source file to be downloaded by means of
an HTTP HEAD request; if no mirror in this file has it, it'll run 'dig
dl.sf.net', ping the hosts, sort them by ping time, and check whether
one of those hosts have the file (chances are they do ;-)).
Of course, the script can also generate an initial mirror file, which
contains just a list of hosts sorted by their ping time; chances are you
want to reorder it slightly to give preference to an "optimal" host,
even if their ping time is slightly worse.
When replacing a /etc/hosts setup, you gain a fallback when your main
mirror doesn't have the file, or if it's offline; you pay that with an
additional HTTP HEAD request. Or to put it slightly different, when
using this script, if a download fails then it's definitely not a
mirroring/DNS RR problem :-).
The script can be found at
http://jw.tks6.net/files/crux/sf-mirror-tool
All you need is the following line in your pkgmk.conf, preferably after
the last PKGMK_SOURCE_DIR setting (since it'll otherwise do an HTTP HEAD
even if the file has already been downloaded):
if [ "$PKGMK_DOWNLOAD" = "yes" ]; then
. /path/to/sf-mirror-tool
sf_replace_source verbose
fi
There are a number of things left to do, especially providing better
feedback to the user (i.e. to point out that certain mirrors always
fail, or even automatically disable them), but I'd still appreciate some
feedback along the way, hence this post :-).
Best regards, Johannes
--
Johannes Winkelmann mailto:jw@tks6.net
Zurich, Switzerland http://jw.tks6.net
Hi,
As you certainly have seen the new website is online, including most of
the original content of the wiki. However, you'll need to create new
accounts for the wiki and the bug tracking, therefore I'd appreciate if
you could take the time and go to
https://crux.nu/bugs/?do=register
and register an account there, and then contact me or Simone to
a) get your account enabled as developer account (in flyspray), and
b) get the access code to register a wiki account at
https://crux.nu/wikka.php?wakka=UserSettings
Best regards, Johannes
--
Johannes Winkelmann mailto:jw@tks6.net
Zurich, Switzerland http://jw.tks6.net
