[clc-devel] Proposal for multi-architecture ports management
Hi, Multi architecture support was of some interest for some time already, and the momentum and demand for crux on x86_64 is growing; also, there are some maintainers here that have x86_64 cpu's already. I'm one of them, but ran the regular 32-bit version so far, mainly because I'm still maintaining a few ports for the 32-bit version and didn't want to get into a conflict there. I think though that the time has come to seriously consider solutions to improve the support for multiple architectures, and to enable ports (of CRUX) to non-x86 architectures to use and re-use the work we put into the ports tree. You might be aware that I've experimented a long time ago with a merge tool called mppm, which worked rather nice, and was used by Simone for uCRUX. http://jw.tks6.net/files/crux/mppm.html Also, I wrote a merge point tracking for subversion called xsvn, which I used for CRUX/SPARC. http://jw.tks6.net/files/crux/xsvn/xsvn-howto.html I think that using a pure revision control tool isn't really suitable to solve our problem, and that using a semi-smart tool for the task of syncing ports is much better. As a consequence of the experience gained from using mppm and xsvn, I've tried to come up with an improved concept for multi-architecutre support. The result is a rather long document, and can be found at http://jw.tks6.net/files/crux/multi-arch-proposal I understand that not all of you are interested in this at this point in time, which is perfectly fine. If you are though, I'd appreciate if you could read through this document, ask and/or comment. If wanted, I can add it to the CLC wiki, but figured that at this point in time, I'd rather keep it on my website to underline it's inofficial state. Thanks for your attention, Regards, Johannes -- Johannes Winkelmann mailto:jw@tks6.net Bern, Switzerland http://jw.tks6.net
Hello, Johannes, here is _my_ point of view. At the moment there are two different CRUX-x86_64 versions out: - Jeremy's CRUX-x86-64 which is (very) close to Per's CRUX i686, and - Daniel's CRUX64, which has experienced some heavy changes. I consider my CRUX64 as a fork of Per's CRUX. I'll give you an examples and the reason why I have made the change.
- arch maintainers are not supposed to change features, e.g. enable the gui in mplayer. OTOH the primary maintainer will have to make sure that his port suits the needs of the CRUX community (arch independent)
I have replaced devfs against Matt's hotplug/udev ports. If you download the newest kernel 2.6.13-rc5 you'll see that devfs is no longer available - they (Greg Kroah-Hartman?) removed the corresponding menu entries so that you cannot choose it during the kernel configuration. I personally think it was a mistake to ship CRUX 2.1 with devfs{d} again (Per knows about my opinion). Therefore CRUX64 comes with some more ports in its base collection. => CRUX64 behaves differently than CRUX (i686) especially files in /dev. Since the AMD64/EM64T technology offers both, 64bit and 32bit operation modes we need to separate those libraries from each other. In the majority of cases the configure scripts get some additional options but sometimes there are also patches needed.
Then there's the gentoo approach, using if clauses to enable/disable packaging instructions specifically per architectures
After deeply thinking about I agree with you. Different Pkgfiles / .footprint / .md5sums/.. for each architecture are the best (and only?) way to keep CRUX's main goal: "keep it simple". I would separate each arch in its own trunk and only merge if possible. For sure, that will result in a decreased number of ports (the price for K.I.S.S.). /usr/ports/ base-i686 base-x86_64 base-ppc (looks ugly..)
There have also been complaints from PPC users to x86 maintainers about their ports not building in the past, simply because the "maintainer" field obviously listed the x86 maintainer.
*cough* sometimes I also left the 'Maintainer' flag as it is. I don't want to earn all the credit for things I've not fully done myself...
4. A smart tool is used to get ports from other architectures, and merge them
Mhhh, I don't think there is a need for such a tool or is there? A tool which informs me (as AM [architecture maintainer :-]) about a newer/modified port in ARCH xyz would be nice. .oO(mppa?) I like to manually do the necessary port adjustments (note: by hand). okay, that's all so far. Hope you were able to fight through my DEnglish :) bye, danm -- Daniel Mueller http://www.danm.de Berlin, Germany OpenPGP: 1024D/E4F4383A
Hi, On Sat, Aug 06, 2005 at 20:34:12 +0200, Daniel Mueller wrote:
Hello,
Johannes, here is _my_ point of view. :-)
At the moment there are two different CRUX-x86_64 versions out: - Jeremy's CRUX-x86-64 which is (very) close to Per's CRUX i686, and - Daniel's CRUX64, which has experienced some heavy changes.
I consider my CRUX64 as a fork of Per's CRUX. I'll give you an examples and the reason why I have made the change.
- arch maintainers are not supposed to change features, e.g. enable the gui in mplayer. OTOH the primary maintainer will have to make sure that his port suits the needs of the CRUX community (arch independent)
I have replaced devfs against Matt's hotplug/udev ports. If you download the newest kernel 2.6.13-rc5 you'll see that devfs is no longer available [...] I guess I wasn't too clear, but the context for this statement is really important. I think it would be important that CRUX/x86 feels the same as CRUX/SPARC; using a system of one single primary maintainer would imply this anyway.
Also, there will always be differences between core ports for the architectures, but those ports will probably have primary maintainers on all platforms. I'll add this as a note to this section.
=> CRUX64 behaves differently than CRUX (i686) especially files in /dev. That's pretty much logical, considering that you want to support current kernels :-). That's really just an issue of timing; if CRUX 2.2 was to be released anytime soon, it would most certainly use udev as well, since - as you mention - devfsd has been thrown out of the configuration.
Then there's the gentoo approach, using if clauses to enable/disable packaging instructions specifically per architectures
After deeply thinking about I agree with you. Different Pkgfiles / .footprint / .md5sums/.. for each architecture are the best (and only?) way to keep CRUX's main goal: "keep it simple".
I would separate each arch in its own trunk and only merge if possible. For sure, that will result in a decreased number of ports (the price for K.I.S.S.). If we make it easy enough for architecture maintainers to keep track of changes, we can hopefully keep large parts of the trees.
/usr/ports/ base-i686 base-x86_64 base-ppc
(looks ugly..) Yeah, here as well it should look the same on every architecture; arch specifics should only affect developers, not users.
4. A smart tool is used to get ports from other architectures, and merge them
Mhhh, I don't think there is a need for such a tool or is there? A tool which informs me (as AM [architecture maintainer :-]) about a newer/modified port in ARCH xyz would be nice. .oO(mppa?) I like to manually do the necessary port adjustments (note: by hand). Well, some tools would certainly help to keep track of ports. I'd expect architecture maintainers to have a large number of ports, so some kind of tracking will certainly be needed. Whether you want to enjoy automated merging or not is certainly up to every maintainer. I know I would us it for certain updates (note that this is just for merging changes to your tree; you can still do a 'cvs diff' then to see what happend, and throw away those changes again (assuming your primary tree is in CVS)).
okay, that's all so far. Hope you were able to fight through my DEnglish :) Obviously, I was ;-). Thanks for the comments.
Johannes -- Johannes Winkelmann mailto:jw@tks6.net Bern, Switzerland http://jw.tks6.net
On 08/06/05 15:53 Johannes Winkelmann wrote: Hi,
As a consequence of the experience gained from using mppm and xsvn, I've tried to come up with an improved concept for multi-architecutre support. The result is a rather long document, and can be found at http://jw.tks6.net/files/crux/multi-arch-proposal
We briefly discussed abou multi-arch support and I confirm my latest reflections on the topic: - Good to have separate branches for different archs - A dedicated client to help maintainers would be very handy and will certainly cause less headaches than traditional merging techniques (btw, the stm name is nice!) The only think that scares me a bit is not to have a arch-based "main" source but rather a maintainer-based one: (quoting from the link above)
1. within the pool of official ports, there exists only one "primary maintainer" (PM). He/She maintains this port on his/her preferred architecture.
3. If there's a maintainer for port X (which is already maintained by a PM), he/she becomes the "arch maintainer" (AM) for this port.
In the past I suggested of having an "official" main repository (today x86, tomorrow could be x86_64) because: - there are higher chances of ports playing well together. With the per-maintainer base pool you propose there could be a set of related ports not fully tested on the same arch. - extension of the previous point to user feedback: the current mainstream arch would almost certainly get more user feedback and test cases (I'd call it more tested), so to me it's a natural step to base other archs on that. On the other side I understand that this puts some maintainer in a bad position when he wants to maintain a port for $hippie_arch that it's not in the main pool. Regards, Simone -- Simone Rota WEB : http://www.varlock.com Bergamo, Italy MAIL: sip@varlock.com
On Sun, Aug 07, 2005 at 00:10:11 +0200, Simone Rota wrote:
On 08/06/05 15:53 Johannes Winkelmann wrote:
Hi, [...] We briefly discussed abou multi-arch support and I confirm my latest reflections on the topic:
- Good to have separate branches for different archs
- A dedicated client to help maintainers would be very handy and will certainly cause less headaches than traditional merging techniques (btw, the stm name is nice!) I hope we can keep the name :-)
The only think that scares me a bit is not to have a arch-based "main" source but rather a maintainer-based one: (quoting from the link above) [...] In the past I suggested of having an "official" main repository (today x86, tomorrow could be x86_64) because:
- there are higher chances of ports playing well together. With the per-maintainer base pool you propose there could be a set of related ports not fully tested on the same arch. Yeah, I agree the risk is there. Of course, if the arch maintainers work well together, broken sets of ports should be fixed in no time :-)
- extension of the previous point to user feedback: the current mainstream arch would almost certainly get more user feedback and test cases (I'd call it more tested), so to me it's a natural step to base other archs on that. In this case, I'd really hope that the maintainers work well together. This is one of the reasons why I think we'll need to have a close look at the bug tracking in the future; I'd really hope that if general bugs are assigned to the arch maintainer, that he/she reassigns it to the primary maintainer if there's a problem. Some more powerful system than cvstrac could help with that (that said, with CVSTrac's custom reports, we can probably do everything we need as well).
Considering that bugs reported on archs might find their way to the primary maintainer could even help to find certain issues which weren't detected on other platforms yet; this is something we don't have yet (AFAICT). Regards, Johannes -- Johannes Winkelmann mailto:jw@tks6.net Bern, Switzerland http://jw.tks6.net
Hi, On Sat, Aug 06, 2005 at 15:53:40 +0200, Johannes Winkelmann wrote:
Hi,
Multi architecture support was of some interest for some time already, [...] As a consequence of the experience gained from using mppm and xsvn, I've tried to come up with an improved concept for multi-architecutre support. The result is a rather long document, and can be found at http://jw.tks6.net/files/crux/multi-arch-proposal
I've put together an example document to illustrate the approach, and a poor drawing. Text here: http://jw.tks6.net/files/crux/multi-arch-example Drawing: http://jw.tks6.net/files/crux/multiarch.png Note that the example is of course imaginary, and probably not going to happen exactly like this in real world. I just wanted to keep the number of ports and maintainers low, in reality a lot more people would/could be involved. Maybe (hopefully) this allows to dive into this more quickly. Regards, Johannes -- Johannes Winkelmann mailto:jw@tks6.net Bern, Switzerland http://jw.tks6.net
On Mon, 15 Aug 2005 16:13:12 +0200 Johannes Winkelmann wrote:
ehm, Johannes you've way to much spare time. It's nice though. ;-) I like your idea of primary/arch maintainers. One problem might be the server. Team work requires shared svn/cvs access - we need a dedicated system. I'm sorry to say that I cannot offer such a service at crux.danm.de. bye, danm -- Daniel Mueller http://www.danm.de Berlin, Germany OpenPGP: 1024D/E4F4383A
participants (3)
-
Daniel Mueller -
Johannes Winkelmann -
Simone Rota