[clc-devel] Multi-platform ports management
Hi there, I know that the PPC team as well as Peter Banks who took over the sparc port are reading this, so let's start discussion issues related to ports management. I think it's important to first agree on a goal for this, and cover the technical details later if there's a consensus. Also let me state that this is my personal opinion only, not the one of CLC or the CRUX project. My goal is to have a system with the following properties: 1. The user only gets ports tested on this platform 2. The user gets the same system on every system 3. No single master tree (as it is now) 4. The package management stays as easy as it is now (i.e. no special cases) 5. Allow to start ports to new architectures easily, without requiring too many developers Is this something we can agree on? Kind regards, Johannes -- Johannes Winkelmann mailto:jw@tks6.net Bern, Switzerland http://jw.tks6.net
Hi, On Mon, Dec 19, 2005 at 13:55:09 +0100, Johannes Winkelmann wrote:
Hi there, [...] My goal is to have a system with the following properties:
1. The user only gets ports tested on this platform 2. The user gets the same system on every system That should of course be: The user gets the same system on every architecture.
Johannes -- Johannes Winkelmann mailto:jw@tks6.net Bern, Switzerland http://jw.tks6.net
Johannes Winkelmann wrote:
Hi there,
I know that the PPC team as well as Peter Banks who took over the sparc port are reading this, so let's start discussion issues related to ports management. I think it's important to first agree on a goal for this, and cover the technical details later if there's a consensus. Also let me state that this is my personal opinion only, not the one of CLC or the CRUX project.
My goal is to have a system with the following properties:
1. The user only gets ports tested on this platform
i think this is very difficult to realize and not useful... infact $ARCH developers should perform a test for every known port and this is not acceptable specially for little communities if the $ARCH community can instead use the "master" ports tree (it should be ok in the 90% of cases), the $ARCH developers can focalize the attention in the porting and the rimanent 10% of buggy cases
2. The user gets the same system on every system
ok for me
3. No single master tree (as it is now)
see point 1
4. The package management stays as easy as it is now (i.e. no special cases)
ok for me
5. Allow to start ports to new architectures easily, without requiring too many developers
see point 1
Is this something we can agree on?
regards -- Giulivo Navigante http://cruxppc.sunsite.dk
Hi, On Mon, Dec 19, 2005 at 17:23:48 +0100, Giulivo Navigante wrote:
Johannes Winkelmann wrote: [...]
My goal is to have a system with the following properties:
1. The user only gets ports tested on this platform
i think this is very difficult to realize and not useful... infact $ARCH developers should perform a test for every known port and this is not acceptable specially for little communities I'm not quite sure here. Adapting the base and opt ports for CRUX/SPARC took me only a few days, and I did it alone. In real life, those changes don't happen all at the same time, so after the initial port, it's like 1 or 2 updates per day.
What makes you believe it's not acceptable? Are you currently using any tools to sync your trees with the x86 one for those ports which require changes?
if the $ARCH community can instead use the "master" ports tree (it should be ok in the 90% of cases) However, this only works if there's a single master tree. We from the x86 port lost a good number of important projects due to Daniel's move to x86_64, and chances are this will happen more often in the future. I don't think that a model with a single master tree is realistic for the future, considering the size of our team.
Regards, Johannes -- Johannes Winkelmann mailto:jw@tks6.net Bern, Switzerland http://jw.tks6.net
Johannes Winkelmann wrote:
Hi,
I'm not quite sure here. Adapting the base and opt ports for CRUX/SPARC took me only a few days, and I did it alone. In real life, those changes don't happen all at the same time, so after the initial port, it's like 1 or 2 updates per day.
What makes you believe it's not acceptable?
not acceptable for little communities because they remains 1 or 2 updates per day if the number of ports remains of little entity, as it is now ... in order consider that "tests" can be made only by $ARCH developers
Are you currently using any tools to sync your trees with the x86 one for those ports which require changes?
no, we've in mind to have /usr/ports/x86 and /usr/ports/ppc in the second tree will exists only the ports that needs to be modified on ppc
if the $ARCH community can instead use the "master" ports tree (it should be ok in the 90% of cases)
However, this only works if there's a single master tree. We from the x86 port lost a good number of important projects due to Daniel's move to x86_64, and chances are this will happen more often in the future. I don't think that a model with a single master tree is realistic for the future, considering the size of our team.
our idea instead is based on a "master" single tree where we should merge forces :P -- Giulivo Navigante http://cruxppc.sunsite.dk
Hey guys...i think i'd like to put my $0.02 in now.
I'm not quite sure here. Adapting the base and opt ports for CRUX/SPARC took me only a few days, and I did it alone. In real life, those changes don't happen all at the same time, so after the initial port, it's like 1 or 2 updates per day.
What makes you believe it's not acceptable?
not acceptable for little communities because they remains 1 or 2 updates per day if the number of ports remains of little entity, as it is now ... in order consider that "tests" can be made only by $ARCH developers
I have to agree with Johannes here, the testing should happen on the dev side and there should be some type of "QOS" control for a port that we produce.
Are you currently using any tools to sync your trees with the x86 one for those ports which require changes?
no, we've in mind to have /usr/ports/x86 and /usr/ports/ppc in the second tree will exists only the ports that needs to be modified on ppc
if the $ARCH community can instead use the "master" ports tree (it should be ok in the 90% of cases)
not to throw a wrench in this discussion, but i did the server migration off crux.nu for the hell of it on the ppc install that i did last night, and so far every port i've tried works fine. I even built firefox 1.5 with the deps from Matts gnome repo and it worked like a charm. I know i'm going to run into some problems eventually, but hell, i'm telling yall that maybe the best was would be to just submit patches to the tree for ppc/sparc? use the repos and create tools for the two other $ARCHS to patch them to make sure the work? -j -- Jonathan Asghar phone: 512.619.0722
Jonathan Asghar wrote:
I have to agree with Johannes here, the testing should happen on the dev side and there should be some type of "QOS" control for a port that we produce.
this implies that $ARCH developers should test every known port of core/opt (and contrib?) collections... and it is not acceptable for me, specially for little communities, infact it produces two negaive aspects for the 90% of ports: 1. $ARCH dev are really overloaded 2. $ARCH ports can remains outdated for a long period and the only 1 positive effective for the remanent 10% of buggy ports: 1. port usable without sending a ticket to the $ARCH developers consider also the "obviously tested" ports distributed on the $ARCH iso image regards -- Giulivo Navigante http://cruxppc.sunsite.dk
this implies that $ARCH developers should test every known port of core/opt (and contrib?) collections...
I dont truly believe that, first off, we have to "test" the majority if not all of core/opt because we install the OS right? yes contrib is huge, i concede that, but i dont think it's unrealistic for us to have a test plan for each release. plus thats where bug reports come into play, for the small amount of users right now for cruxppc we can set up a mailing list type bug report system so we can get ontop of the problem ports. Yes, i agree it seems like a hack as of right now, but i guess in my mind i'd prefer us to get a road layed down before moving towards the superhighway, if you catch my drift. my priorities in general would be to get a good uptodate ports system, and solid base to work with, then move towards almost "recruiting" users so the community grows and we can start worrying about stream lining the "problem ports." Also like i keep saying, i rsynced the ports for the x86 tree and so far so good...i can send out the pkginfo -i to see what compiled fine with no tweaking if yall like. regards, -j -- Jonathan Asghar phone: 512.619.0722
Jonathan Asghar wrote:
I have to agree with Johannes here, the testing should happen on the dev side and there should be some type of "QOS" control for a port that we produce.
this implies that $ARCH developers should test every known port of core/opt (and contrib?) collections... and it is not acceptable for me, specially for little communities You've stated this a couple of times now so I assume you're convinced
Hi Giulivo, On Mon, Dec 19, 2005 at 21:11:09 +0100, Giulivo Navigante wrote: that it doesn't make sense to even try it. I'm willing to believe that too, but since I'm currently confident that it is possible even for a small crowd to maintain a properly tested system using the right tools, I convinced Daniel to do a test run with a x86_64 port which syncs from (and hopefully also to) x86. We plan to start in january, and we'll certainly post our results here to let you know how it goes. I'm sure that the outcome of this test will give us a good idea about what's possible and what's not, and will hopefully allow to use more facts and less emotional arguments in this discussion. Best regards, Johannes -- Johannes Winkelmann mailto:jw@tks6.net Bern, Switzerland http://jw.tks6.net
Johannes Winkelmann wrote:
We plan to start in january, and we'll certainly post our results here to let you know how it goes. I'm sure that the outcome of this test will give us a good idea about what's possible and what's not, and will hopefully allow to use more facts and less emotional arguments in this discussion.
i'm perfectly in accord with you we will do the same for the ppc platform ... a propos, we've already do a similar work before distribute cruxppc 1.3 and 2.0, so consider my idea not only based on emotional arguments, but also on facts :) regards -- Giulivo Navigante http://cruxppc.sunsite.dk
Johannes Winkelmann wrote:
Hi,
I'm not quite sure here. Adapting the base and opt ports for CRUX/SPARC took me only a few days, and I did it alone. In real life, those changes don't happen all at the same time, so after the initial port, it's like 1 or 2 updates per day.
What makes you believe it's not acceptable?
not acceptable for little communities because they remains 1 or 2 updates per day if the number of ports remains of little entity, as it is now ... in order consider that "tests" can be made only by $ARCH developers Okay. With a community of three developers, that means that one will have to update ~5 ports every week with 2 updates per day. I definitely expect the x86 maintainers to be able to cope with this kind of load, and they're even required to follow the mailing lists etc. If we assume
On Mon, Dec 19, 2005 at 18:54:01 +0100, Giulivo Navigante wrote: the ports build fine, that's maybe an average of half an hour per port, which is 2.5 h per week. How much work/time are you guys willing to spend on this project?
Are you currently using any tools to sync your trees with the x86 one for those ports which require changes?
no, we've in mind to have /usr/ports/x86 and /usr/ports/ppc in the second tree will exists only the ports that needs to be modified on ppc No, I mean those ports which are modified; how do you sync those?
if the $ARCH community can instead use the "master" ports tree (it should be ok in the 90% of cases)
However, this only works if there's a single master tree. We from the x86 port lost a good number of important projects due to Daniel's move to x86_64, and chances are this will happen more often in the future. I don't think that a model with a single master tree is realistic for the future, considering the size of our team.
our idea instead is based on a "master" single tree where we should merge forces :P I'm not quite sure I understand you here. You want one tree which supports all architecures? Like gentoo, with if [ "$arch" = "ppc" ]; ... elif ... elif ... fi all over the Pkgfiles?
Regards, Johannes -- Johannes Winkelmann mailto:jw@tks6.net Bern, Switzerland http://jw.tks6.net
On Monday 19 December 2005 17:23, Giulivo Navigante wrote:
i think this is very difficult to realize and not useful... infact $ARCH developers should perform a test for every known port and this is not acceptable specially for little communities
I don't agree with you in this point. I had to test EVERY particular port for x86_64 because there were bugs sometimes even if the source compiled cleanly (e.g. 'bc' -> sporadic segmentation faults, 'irssi 0.8.9' -> connect (tcp) errors, [..]). It's not our job to provide as much ports as possible. A small, good running set of ports should be sufficient for the most users. bye, danm -- Daniel Mueller Berlin, Germany OpenPGP: 1024D/E4F4383A
On Mon, Dec 19, 2005 at 05:23:48PM +0100, Giulivo Navigante wrote:
Johannes Winkelmann wrote:
Hi there,
I know that the PPC team as well as Peter Banks who took over the sparc port are reading this, so let's start discussion issues related to ports management. I think it's important to first agree on a goal for this, and cover the technical details later if there's a consensus. Also let me state that this is my personal opinion only, not the one of CLC or the CRUX project.
My goal is to have a system with the following properties:
1. The user only gets ports tested on this platform
i think this is very difficult to realize and not useful... infact $ARCH developers should perform a test for every known port and this is not acceptable specially for little communities
I have to full agree with Johannes here. For me it's not acceptable to have untested ports in a 'official' ports tree. Quality should go over quantity. regards Jürgen -- Juergen Daubert | mailto:jue@jue.li Korb, Germany | http://jue.li/crux
On Monday 19 December 2005 13:55, Johannes Winkelmann wrote:
1. The user only gets ports tested on this platform
yep.
2. The user gets the same system on every system
This is not possible. Lilo doesn't work on all platforms and devfs was thrown out of the kernel source tree. Library paths differ from $arch to $arch and configuration files sometimes need special adjustments.
3. No single master tree (as it is now)
Mhh, a separated ports tree would be nice. A mechanism which prevents me from accidently touching x86 ports :-)
4. The package management stays as easy as it is now (i.e. no special cases)
:-( I'm not allowed to write a 'patch-helper' function in special cases? It's annoying to write fifteen 'patch -p1 < $SRC/no1.patch' lines.
5. Allow to start ports to new architectures easily, without requiring too many developers
What do you mean? bye, danm -- Daniel Mueller Berlin, Germany OpenPGP: 1024D/E4F4383A
Hi Daniel,
On Mon, Dec 19, 2005 at 19:14:04 +0100, Daniel Mueller wrote:
> On Monday 19 December 2005 13:55, Johannes Winkelmann wrote:
>
> > 1. The user only gets ports tested on this platform
>
> yep.
>
> > 2. The user gets the same system on every system
>
> This is not possible. Lilo doesn't work on all platforms and devfs was thrown
> out of the kernel source tree. Library paths differ from $arch to $arch and
> configuration files sometimes need special adjustments.
Oh, you took my words too literarily.
What I meant is that it's /usr/ports/{core,opt,contrib} on every arch
for example.
The PPC team seems to be in favour of a /usr/ports/{x86,ppc} special
solution on their system; it's more this kind of things I mean.
Of course, ports have to be chosen per plaform, changes have to be made
for pathes and so an. As for devfs, that's not really an issue of archs;
if we had sync'ed released, all archs would have udev.
I would however be in favour that all archs have more or less the same
features in their ports; as a silly example, let's say mplayer is build
without GUI on all archs. However, I think that's already the case now.
> > 3. No single master tree (as it is now)
>
> Mhh, a separated ports tree would be nice. A mechanism which prevents me from
> accidently touching x86 ports :-)
It will also allow us to distribute the primary maintenance (following
the mailing lists, checking for announcements and conflicts) to several
architectures, making it easier to maintain a port on a another
architecture (let's say I maintain libxml2 on sparc which you maintain
on x86_64; in this case, I can simply follow your port, but not the
libxml2 mailing list).
> > 4. The package management stays as easy as it is now (i.e. no special
> > cases)
>
> :-(
>
> I'm not allowed to write a 'patch-helper' function in special cases? It's
> annoying to write fifteen 'patch -p1 < $SRC/no1.patch' lines.
Peanuts :-). What I meant here was that it's not something like:
If you execute pkgmk, it will
1. check for Pkgfile.$arch, and build it if it's there
2. choose Pkgfile otherwise
and the same for md5sum and footprint
That's what I meant with special cases for the package management. I'm
mainly against this since the users who look for an error in a build
will have to go through the same process: "where's the error, is it in
Pkgfile.$arch, or in Pkgfile...".
> > 5. Allow to start ports to new architectures easily, without requiring
> > too many developers
>
> What do you mean?
Basically two things.
1. that in order to start an architecture which only reuses ports from
the other architectures and requires little work (i.e. provide a sync
tool which is adapted to our needs).
2. that the setup allows to start working on it without requiring
special accounts or rights. The typical situation would be to have a
central SVN repo, where everything's developed and users need write
permissions to start working on a new archtecture.
Well, I guess my initial mail wasn't too clear then :-). Sorry for that,
and thanks for your questions, I hope I could clear up my previous mail
a bit.
Kind regards, Johannes
--
Johannes Winkelmann mailto:jw@tks6.net
Bern, Switzerland http://jw.tks6.net
On Monday 19 December 2005 19:32, Johannes Winkelmann wrote:
3. No single master tree (as it is now)
Mhh, a separated ports tree would be nice. A mechanism which prevents me from accidently touching x86 ports :-)
It will also allow us to distribute the primary maintenance (following the mailing lists, checking for announcements and conflicts) [..]
I don't know much about SVN yet - is it possible to create a branch for this or would we get our 'own' repository (one per architecture). Giulivo: What do you think? My ports tree (AMD64) would look like this: /usr/ports/ base opt compat32 (32bit ports) contrib + /usr/ports-x86/ base opt contrib The /usr/ports-x86 directory is NOT activated in /etc/prt-get.conf (not even mentioned!). It's just there for the users cause we can't provide all x86 ports. bye, danm -- Daniel Mueller Berlin, Germany OpenPGP: 1024D/E4F4383A
Hi, On Mon, Dec 19, 2005 at 20:00:31 +0100, Daniel Mueller wrote:
On Monday 19 December 2005 19:32, Johannes Winkelmann wrote:
3. No single master tree (as it is now)
Mhh, a separated ports tree would be nice. A mechanism which prevents me from accidently touching x86 ports :-)
It will also allow us to distribute the primary maintenance (following the mailing lists, checking for announcements and conflicts) [..]
I don't know much about SVN yet - is it possible to create a branch for this or would we get our 'own' repository (one per architecture). We could host certain architectures in the same repository, but we shouldn't make that a requirement.
I'd like to see a setup which doesn't depend on a particular revision control system, since this will make it easier for new ports to get started. Instead, I'd just sync all trees with "primary ports" to the HDD, and track/sync/merge them there. Regards, Johannes -- Johannes Winkelmann mailto:jw@tks6.net Bern, Switzerland http://jw.tks6.net
participants (5)
-
Daniel Mueller -
Giulivo Navigante -
Johannes Winkelmann -
Jonathan Asghar -
Juergen Daubert