[clc-devel] Re: contrib preparation scripts

Jay Dolan jasonthomasdolan at yahoo.com
Fri Nov 19 04:19:49 UTC 2004

--- Johannes Winkelmann <jw at tks6.net> wrote:

> Hi Jay,
> I've been pretty busy lately and didn't manage to
> join IRC yesterday. So
> I'll just guess and send you my scripts to prepare a
> big repository of
> ports (PrepRep). I commented out certain lines from
> preprep.conf, since
> I guess they're not all the same right now (jue
> moved his repo), and one
> of them was a local one to simulate clashes between
> ports.
> I guess it might be a good idea to start from
> scratch, and I guess
> python or perl would be the weapon of choice as we
> already discussed at
> CRUXCon. Since bash scripts are usually rather
> verbose and not too clear
> to understand what's going on, here a summary of
> what I'd expect it do
> do:
> 1. sync a list of repositories to an input directory
>   - note if a repo can't be synced
> 2. Create a list of ports 'contribports'
> 2. Find duplicates in 'contribports', and where they
> appear
>   -> send notifications to the crux-contrib mailing
> list
>   -> remove those from 'contribports'
> 3. create a list of ports in base and opt
> ('coreports')
> 4. remove 'coreports' from 'contribports'
>   // 'contribports' contains now a list of ports
> which can be updated in
>   // the main collection
> 5. if a list of 'contribports' which have been
> synced the last time is
>   available, compare the two lists
>   - note which ports are new, and which one have
> gone
> 6. update the ports listed in 'contribports' from
> the input directory to
>    the output directory
>   - create a list of ports that were synced in this
> run (to be used the
>     next time (5.) is called
> 7. Mark the ports which are not maintained anymore
> (from 6.) as
>    '# Maintainer: UNMAINTAINED'
> 9. Create a new httpup REPO file
>    (maybe run portspage from vektori)
> A. Send a report to the contrib-admin or
> crux-contrib, containing:
>    a) new ports
>    b) ports newly unmaintained
>    c) repos that couldn't be synced (from 1.)
> B. Send a notification to maintainers who have
> conflicting ports
> C. Send a notification to maintainers of
> repositories which couldn't be
>    synced (from 1.)
> This is just from the top of my head, so consider
> this just a
> description what I was trying (or wanting) to do.
> Finally, it would be
> great if the script would run hourly, but it would
> really be a PITA if
> people would get notifications hourly; so there
> needs to be some kind of
> management when the people where notified.
> Reading through this, it seems that this is quite an
> interesting task,
> but definitely challenging. Actually I'd really like
> to do some coding
> on this as well, but probably not before mid
> december :-/. Maybe you
> could ask Matt to set up a CVS (or Subversion)
> repository for that.
> Feel free to ask if I don't make any sense!
> So much for now,
> Kind regards, Johannes

I miss you already.

I took my own stab at this (considering that you
forgot to actually attach your scripts ;)), and ended
up settling on bash :/  The more I thought about it,
the more it seemed appropriate.  Tools like find and
fgrep can be used for just about all of the
complicated parts of this task.

So take a look at prtsync in my httpup repository, if
you will.  It works well, is less than 80 lines, and
does almost all of what you mentioned in your last
mail.  I think it would be fine to run just nightly. 
The ugliest part is actually the call to sendmail.

Sweet dreams, I'm spent..

Jay Dolan
Software Engineer, Systems Analyst
Windmill Cycles, Inc.

Do you Yahoo!? 
The all-new My Yahoo! - Get yours free! 

More information about the crux-devel mailing list