[crux-devel] [RFC] new portdb

Lucas Hazel lucas at die.net.au
Wed Oct 8 12:55:50 UTC 2008

On Wed, 8 Oct 2008 14:14:29 +0200
Johannes Winkelmann <jw at smts.ch> wrote:

> Hi Lucas,
> On Wed, Oct 08, 2008 at 21:23:32 +1100, Lucas Hazel wrote:
> > 
> > So it appears I've been handed the "portdb maintainer" hat.
> > 
> > My first duty in this roll was to add a few repos to the portdb
> > cacher. But alas the pdbcacher seems to have locked up and I don't
> > have the privileges to kill the process.
> > 
> > So rather than find out what's wrong with it, I've decided to write
> > a new one (sans PHP) with a focus on me having to do as little data
> > entry as possible.
> Nice
> > 
> > A few features I have in mind are,
> > 
> > * Allow users to add repositories by themselves.
> Cool; if we have proper user management, that could also be used for a
> number of other things. It would be nice if users wouldn't need three
> accounts for crux.nu (wiki, flyspray, portdb); not sure which
> application could/should be the "master" though.

I'm writing it using the django web frameowrk, while it has it's own
user system, it is quite easy to plug in custom authentication systems.

A stand alone database containing user credentials would be a better
idea than assigning authentication to a specific app.

> > * Allow users to create custom collections from other repositories.
> Not sure if that should be part of crux.nu's portdb. I can see that
> users create meta repositories, but I'm not sure if we should host
> them. Or is this mean to be a private custom collection, or just a
> special ports configuration file for a yet-to-be written ports(8)
> driver that can merge repositories?

This idea was sort of inspired with the pull action I implemented in my
command line interface[1][2] to my current implementation of the

Yes, this could theoretically be a future ports driver (and is planned).
The advantage here is that users could have a single ports repo on their
local system containing only the ports they use.

Such a system already works using 'pdb pull' but the whole process is
quite slow as each port has to be individually downloaded, processed
and have it's dependencies found.

Initially I thought this would be mostly useful for private use. But so
called "meta repositories" could also be used to solve some of the
issues regarding DE specific repositories.

> > * Partial and full caching of repositories.
> Does this mean that one can browse the files on crux.nu? I'd really
> like to see that again, as right now (at least for rsync) you can't
> e.g. check versions and the like.

Yeah this was one of the features I missed most about the old portdb.

The plan here is for partial caching to only store the big 3
(pkgfile,footprint and md5sum) and full caching to essentially act as a
mirror for the repository.

> > * Make more information available, such as processing footprints,
> >   md5sums, meta data and verification results.
> Sounds good. It would also be nice if users could flag packages
> outdated, which would trigger a bug report to portdb, similar to what
> the Arch guys have.

Allowing users to flag ports for age or incorrectness is a good idea,
I'll add that to the list.

> Maybe portdb could also show if there's already an outdated (and maybe
> also other bugs?) reported.

A comments system would achieve this and is planned, hooking it into
flyspray is also a nice idea.
> > * XML output, RSS feeds, email alerts, etc.
> > * ???
> > * Profit
> > 
> > Does anyone else have any nifty ideas for what could go into a new
> > portdb?
> Well, one thing I'd like to animate is that repo maintainers must
> confirm on a regular basis that they're looking after their
> repositories; I would suggest that we send a mail with a generated
> URL, which confirms that a repository is active when visited. If a
> maintainer fails to confirm (maybe only the second time), his
> repositories are deactivated (hidden) from portdb. He can always log
> in later on to reactivate them.

Perhaps having the portdb keep track of outdated ports through
something like ck4up. Then emailing maintainers when a new version is

> Thanks for working on that!

No worries, the cheque's in the mail ;)

> Cheers, Johannes

Lucas Hazel <lucas at die.net.au>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.crux.nu/pipermail/crux-devel/attachments/20081008/4cfaf18b/attachment.asc>

More information about the crux-devel mailing list