Hi, I am maintaining the namenlos repostitory. The git repository is at repo.or.cz [1] and the httpup directory is at the webserver of my company [2]. Every time I update a port I have to update the httpup repository, too. So I was wondering, whether it would be possible to tell ports(8) to fetch the files from the git repository directly instead of doing all by myself. So I wrote a git driver for ports (attached) and some example configuration files for the namenlos and opt repositories (also attached). What do you think? Imho it would be neat to get this driver into crux by default. bye richi (namenlos) [1] http://repo.or.cz/w/namenlos-ports.git [2] http://www.sti2.at/~richardp/ports/ -- quoting guide: http://www.xs4all.nl/~hanb/documents/quotingguide.html
Hi Richard,
On Tue, May 06, 2008 at 09:55:10 +0200, Richard Pöttler wrote:
> Hi,
> [...] So I wrote a git driver for
> ports (attached) and some example configuration files for the namenlos and opt
> repositories (also attached).
>
> What do you think?
This has a couple of nice properties:
- I think when coupled with gitweb this is better than rsync,
since you can link to the actual files in the portdb.
- Also, if we could actually recommend our users to use the public hosting
on git.or.cz (not sure if that's okay with them) or any other git
hosting, we would never have to hear the "I have no webspace to host my
ports" argument anymore :-).
There are a couple of downsides though:
- no nice output; i.e. you don't see which ports are updated
- clone currently gets the whole repository; currently, I get the
following numbers (opt):
git: 16708 kB
rsync: 10458 kb
in other words, 50% more data is transferred and stored. This will get
worse over time with more history, although chances are that thanks to
compression, this won't be too bad
- users behind a restrictive firewall might have to use the http://
access, which is fairly slow, especially for the initial clone
If git gains a "partial clone" feature, that would definitely help.
Regards, Johannes
--
Johannes Winkelmann mailto:jw@smts.ch
Zurich, Switzerland http://jw.smts.ch
On Tue, May 06, 2008 at 13:00:54 +0200, Johannes Winkelmann wrote: [...]
- clone currently gets the whole repository; currently, I get the following numbers (opt): git: 16708 kB rsync: 10458 kb To be fair, the clone without checkout is smaller (6154 kB here) so the amount of data to be transfered during the initial clone is potentially smaller.
Johannes -- Johannes Winkelmann mailto:jw@smts.ch Zurich, Switzerland http://jw.smts.ch
Johannes Winkelmann [2008-05-06 13:00]:
Hi Richard,
On Tue, May 06, 2008 at 09:55:10 +0200, Richard Pöttler wrote:
Hi, [...] So I wrote a git driver for ports (attached) and some example configuration files for the namenlos and opt repositories (also attached).
What do you think? This has a couple of nice properties: [...] If git gains a "partial clone" feature, that would definitely help.
git clone URL --depth n seems to give you a clone that only contains the n last changesets. I never tried it though. Regards, Tilman -- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail?
Tilman Sauerbeck [2008-05-06 23:33]:
Johannes Winkelmann [2008-05-06 13:00]:
Hi Richard,
On Tue, May 06, 2008 at 09:55:10 +0200, Richard Pöttler wrote:
Hi, [...] So I wrote a git driver for ports (attached) and some example configuration files for the namenlos and opt repositories (also attached).
What do you think? This has a couple of nice properties: [...] If git gains a "partial clone" feature, that would definitely help.
git clone URL --depth n seems to give you a clone that only contains the n last changesets.
I never tried it though.
How about this for the git driver: git clone --depth 1 --bare /tmp/gitdriverclone.git git archive --format tar /tmp/gitdriverclone.git | tar -C -x /tmp/co Then we can sync /tmp/co with /usr/ports/whatever, which means we can print nice stats about which file was updated. It's horribly inefficient, but it should be okay for small(ish) repositories. Regards, Tilman -- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail?
Hi, On Wed, May 07, 2008 at 08:36:54 +0200, Tilman Sauerbeck wrote:
Tilman Sauerbeck [2008-05-06 23:33]:
Johannes Winkelmann [2008-05-06 13:00]: [...]
[...] If git gains a "partial clone" feature, that would definitely help.
git clone URL --depth n seems to give you a clone that only contains the n last changesets. Cool, I (obviously) didn't know :-). It would be nice to prune based on branches too.
How about this for the git driver:
git clone --depth 1 --bare /tmp/gitdriverclone.git git archive --format tar /tmp/gitdriverclone.git | tar -C -x /tmp/co
Another approach would be to revert the working copy, rerun fetch (or clone initially), and then run git-diff between origin/2.4 and the WC. The following command looks like a good start: git diff --name-status origin/2.4 -R Johannes -- Johannes Winkelmann mailto:jw@smts.ch Zurich, Switzerland http://jw.smts.ch
Hi, Another remark on the driver itself: On Tue, May 06, 2008 at 09:55:10 +0200, Richard Pöttler wrote: [...]
#!/bin/sh # # /etc/ports/drivers/git: git driver script for ports(8) #
OLDPWD=$PWD
if [ $# -ne 1 ] then echo "usage: $0 <file>" exit 1 fi
. $1
if [ ! -d $DESTINATION ] then # there was no initial clone git-clone $URL $DESTINATION cd $DESTINATION else # there was already a clone cd $DESTINATION git-fetch git-gc fi
git-checkout $BRANCH
There probably needs to be a revert before that, since otherwise locally modified files (Pkgfiles/md5sums/footprints/patches etc.) will be merged, which may not be desired. Regards, Johannes -- Johannes Winkelmann mailto:jw@smts.ch Zurich, Switzerland http://jw.smts.ch
On Wed, May 07, 2008 at 09:12:27 +0200, Johannes Winkelmann wrote: [...]
git-checkout $BRANCH
There probably needs to be a revert before that which in git lingo is not git-revert but git-reset; something like
git-diff origin/2.4 -R git-reset --hard -q Should show what we're planning to do, and then get us back in the pristine state. Regards, Johannes -- Johannes Winkelmann mailto:jw@smts.ch Zurich, Switzerland http://jw.smts.ch
I got one more: On Tue, May 06, 2008 at 09:55:10 +0200, Richard Pöttler wrote: [...]
#!/bin/sh # # /etc/ports/drivers/git: git driver script for ports(8) #
OLDPWD=$PWD
if [ $# -ne 1 ] then echo "usage: $0 <file>" exit 1 fi
. $1
if [ ! -d $DESTINATION ] ^ this should probably also check for $DESTINATION/.git/ , and whether $DESTINATION/.git/config (or `git config --get remote.origin.url`) matches $URL. I'm not quite sure what to if one of the two fail, we could just get rid of the current content in $DESTINATION and clone, or fail with an error. In any case this should probably be handled.
Regards, Johannes -- Johannes Winkelmann mailto:jw@smts.ch Zurich, Switzerland http://jw.smts.ch
Hi, thanks for your input. I have modified the driver according to your replies (I hope I understood them all). The actual version is attached. Are there any further suggestions/recommendations/comments? bye richi (namenlos) -- quoting guide: http://www.xs4all.nl/~hanb/documents/quotingguide.html
Hi Richard, Thanks for the new version, On Wed, May 07, 2008 at 15:52:04 +0200, Richard Pöttler wrote:
Hi,
thanks for your input. I have modified the driver according to your replies (I hope I understood them all). The actual version is attached. Are there any further suggestions/recommendations/comments?
I've attached to a patch which does the following: - make it harder for the user to make local commits, and thus make clear the this is not intended (via custom GIT_DIR) - avoid error if git config --get is empty - remove unused OLDPWD Cheers, Johannes -- Johannes Winkelmann mailto:jw@smts.ch Zurich, Switzerland http://jw.smts.ch
Hi, On Wed, May 07, 2008 at 15:52:04 +0200, Richard Pöttler wrote: > Hi, > > thanks for your input. I have modified the driver according to your replies (I > hope I understood them all). The actual version is attached. Are there any > further suggestions/recommendations/comments? I've reworked the script quite a bit, introduce the following changes: - move all DESTINATION verification and cleanup into one function - add check to make sure URL, BRANCH and DESTINATION are not empty - use special GIT_DIR to make local commits less easy - exit on the first error to avoid a series of failed git calls - mimick output of cvsup - add --no-checkout to clone to make sure the diff is also printed for BRANCH=master - silence clone I also did some formatting changes to match my preference. This seems now pretty stable, although it still suffers from some of the disadvantages discussed earlier in this thread. Cheers, Johannes -- Johannes Winkelmann mailto:jw@smts.ch Zurich, Switzerland http://jw.smts.ch
participants (3)
-
Johannes Winkelmann -
Richard Pöttler -
Tilman Sauerbeck