So it appears I've been handed the "portdb maintainer" hat. My first duty in this roll was to add a few repos to the portdb cacher. But alas the pdbcacher seems to have locked up and I don't have the privileges to kill the process. So rather than find out what's wrong with it, I've decided to write a new one (sans PHP) with a focus on me having to do as little data entry as possible. A few features I have in mind are, * Allow users to add repositories by themselves. * Allow users to create custom collections from other repositories. * Partial and full caching of repositories. * Make more information available, such as processing footprints, md5sums, meta data and verification results. * XML output, RSS feeds, email alerts, etc. * ??? * Profit Does anyone else have any nifty ideas for what could go into a new portdb? -- Lucas Hazel <lucas@amacdigital.com.au> AMAC Digital Products Technician 209 Beardy Street, Armidale NSW [phone: +61267711287]
Hi Lucas, On Wed, Oct 08, 2008 at 21:23:32 +1100, Lucas Hazel wrote:
So it appears I've been handed the "portdb maintainer" hat.
My first duty in this roll was to add a few repos to the portdb cacher. But alas the pdbcacher seems to have locked up and I don't have the privileges to kill the process.
So rather than find out what's wrong with it, I've decided to write a new one (sans PHP) with a focus on me having to do as little data entry as possible.
Nice
A few features I have in mind are,
* Allow users to add repositories by themselves.
* Allow users to create custom collections from other repositories. Not sure if that should be part of crux.nu's portdb. I can see that users create meta repositories, but I'm not sure if we should host them. Or is this mean to be a private custom collection, or just a special
Cool; if we have proper user management, that could also be used for a number of other things. It would be nice if users wouldn't need three accounts for crux.nu (wiki, flyspray, portdb); not sure which application could/should be the "master" though. ports configuration file for a yet-to-be written ports(8) driver that can merge repositories?
* Partial and full caching of repositories. Does this mean that one can browse the files on crux.nu? I'd really like to see that again, as right now (at least for rsync) you can't e.g. check versions and the like.
* Make more information available, such as processing footprints, md5sums, meta data and verification results. Sounds good. It would also be nice if users could flag packages outdated, which would trigger a bug report to portdb, similar to what the Arch guys have.
Maybe portdb could also show if there's already an outdated (and maybe also other bugs?) reported.
* XML output, RSS feeds, email alerts, etc. * ??? * Profit
Does anyone else have any nifty ideas for what could go into a new portdb? Well, one thing I'd like to animate is that repo maintainers must confirm on a regular basis that they're looking after their repositories; I would suggest that we send a mail with a generated URL, which confirms that a repository is active when visited. If a maintainer fails to confirm (maybe only the second time), his repositories are deactivated (hidden) from portdb. He can always log in later on to reactivate them.
Thanks for working on that! Cheers, Johannes -- Johannes Winkelmann mailto:jw@smts.ch Zurich, Switzerland http://jw.smts.ch
Hi Lucas,
On Wed, Oct 08, 2008 at 21:23:32 +1100, Lucas Hazel wrote: [...]
* XML output, RSS feeds, email alerts, etc. You probably meant this with the RSS feed, but a long time ago we had a web site (not only RSS feed) which listed added/changed/removed ports;
On Wed, Oct 08, 2008 at 14:14:29 +0200, Johannes Winkelmann wrote: that was kinda neat. Also, for arch (x86/x86_64 etc) maintainers, it would be handy to track individual ports (or rather a custom list of ports). Also, here's two other random ideas: * Web based ports management This is not really related to portdb per se, but it would be nice if I could subscribe to an RSS feed which included ck4up, prtverify and check_urls output. * "Port of the week" Occasionally, I find a new tool/library/* which is very nice, and would like to inform the rest of the world about it ;-) Maybe we could add some kind of submission/vote mechanism where the users can vote a new port "port of the week", which is then listed more prominentely somewhere on our website. Of course, it's not the port (i.e. Pkgfile) that should get the award but the packaged software. Cheers, Johannes -- Johannes Winkelmann mailto:jw@smts.ch Zurich, Switzerland http://jw.smts.ch
On Wed, 8 Oct 2008 14:14:29 +0200 Johannes Winkelmann <jw@smts.ch> wrote:
Hi Lucas,
On Wed, Oct 08, 2008 at 21:23:32 +1100, Lucas Hazel wrote:
So it appears I've been handed the "portdb maintainer" hat.
My first duty in this roll was to add a few repos to the portdb cacher. But alas the pdbcacher seems to have locked up and I don't have the privileges to kill the process.
So rather than find out what's wrong with it, I've decided to write a new one (sans PHP) with a focus on me having to do as little data entry as possible.
Nice
A few features I have in mind are,
* Allow users to add repositories by themselves.
Cool; if we have proper user management, that could also be used for a number of other things. It would be nice if users wouldn't need three accounts for crux.nu (wiki, flyspray, portdb); not sure which application could/should be the "master" though.
I'm writing it using the django web frameowrk, while it has it's own user system, it is quite easy to plug in custom authentication systems. A stand alone database containing user credentials would be a better idea than assigning authentication to a specific app.
* Allow users to create custom collections from other repositories. Not sure if that should be part of crux.nu's portdb. I can see that users create meta repositories, but I'm not sure if we should host them. Or is this mean to be a private custom collection, or just a special ports configuration file for a yet-to-be written ports(8) driver that can merge repositories?
This idea was sort of inspired with the pull action I implemented in my command line interface[1][2] to my current implementation of the portdb[3]. Yes, this could theoretically be a future ports driver (and is planned). The advantage here is that users could have a single ports repo on their local system containing only the ports they use. Such a system already works using 'pdb pull' but the whole process is quite slow as each port has to be individually downloaded, processed and have it's dependencies found. Initially I thought this would be mostly useful for private use. But so called "meta repositories" could also be used to solve some of the issues regarding DE specific repositories.
* Partial and full caching of repositories. Does this mean that one can browse the files on crux.nu? I'd really like to see that again, as right now (at least for rsync) you can't e.g. check versions and the like.
Yeah this was one of the features I missed most about the old portdb. The plan here is for partial caching to only store the big 3 (pkgfile,footprint and md5sum) and full caching to essentially act as a mirror for the repository.
* Make more information available, such as processing footprints, md5sums, meta data and verification results. Sounds good. It would also be nice if users could flag packages outdated, which would trigger a bug report to portdb, similar to what the Arch guys have.
Allowing users to flag ports for age or incorrectness is a good idea, I'll add that to the list.
Maybe portdb could also show if there's already an outdated (and maybe also other bugs?) reported.
A comments system would achieve this and is planned, hooking it into flyspray is also a nice idea.
* XML output, RSS feeds, email alerts, etc. * ??? * Profit
Does anyone else have any nifty ideas for what could go into a new portdb? Well, one thing I'd like to animate is that repo maintainers must confirm on a regular basis that they're looking after their repositories; I would suggest that we send a mail with a generated URL, which confirms that a repository is active when visited. If a maintainer fails to confirm (maybe only the second time), his repositories are deactivated (hidden) from portdb. He can always log in later on to reactivate them.
Perhaps having the portdb keep track of outdated ports through something like ck4up. Then emailing maintainers when a new version is detected.
Thanks for working on that!
No worries, the cheque's in the mail ;)
Cheers, Johannes
-- Lucas Hazel <lucas@die.net.au>
Hi, First off all thanks a lot Lucas, I'm glad with your work.
* Allow users to create custom collections from other repositories. Not sure if that should be part of crux.nu's portdb. I can see that users create meta repositories, but I'm not sure if we should host them. Or is this mean to be a private custom collection, or just a special ports configuration file for a yet-to-be written ports(8) driver that can merge repositories?
This idea was sort of inspired with the pull action I implemented in my command line interface[1][2] to my current implementation of the portdb[3].
Yes, this could theoretically be a future ports driver (and is planned). The advantage here is that users could have a single ports repo on their local system containing only the ports they use. I write something similar some time ago and I'm using it ATM. I think it would be useful to take ideas: http://mikeux.dyndns.org/crux/files/ports-mult-driver.html
Best regards, Jose V Beneyto
On 10/08/08 10:23 Lucas Hazel wrote:
So it appears I've been handed the "portdb maintainer" hat.
My first duty in this roll was to add a few repos to the portdb cacher. But alas the pdbcacher seems to have locked up and I don't have the privileges to kill the process.
It ran fine right now, we may want to track down the problem anyway while we're using the old version. Btw it seems the following collections are not availabe at this time: chw lxde mrks roliveira singular
So rather than find out what's wrong with it, I've decided to write a new one (sans PHP) with a focus on me having to do as little data entry as possible.
nice!
A few features I have in mind are,
* Allow users to add repositories by themselves. * Allow users to create custom collections from other repositories. * Partial and full caching of repositories. * Make more information available, such as processing footprints, md5sums, meta data and verification results. * XML output, RSS feeds, email alerts, etc. * ??? * Profit
Does anyone else have any nifty ideas for what could go into a new portdb?
Sounds good, random notes from past experiences/discussions: - I'd avoid storing the files locally, I'm not that fond of getting random files from untrusted sources. - IIRC we disabled the form to add a repository due to spam, you may want to add some check (ie: check if it's a real repo, email confirmation,...) - We tried (and failed ;)) to stick to a common platform for web applications to simplify the setup. I guess using python is not a big deal since we already have mailman and I don't see it being replaced anytime soon. - Since you mentioned verification results, I think your job will be easier if you enable a auto-disable mechanism for unreachable repos (ie after 10 days), and make the cacher notify you only the first time and maybe when the repo is removed. Thank you for taking care of this, Regards, Simone -- Simone Rota Bergamo, Italy - http://www.varlock.com
participants (4)
-
Johannes Winkelmann -
Jose V Beneyto -
Lucas Hazel -
Simone Rota