On Thu, 22 May 2008 00:15:26 +0200 Johannes Winkelmann <jw@smts.ch> wrote: Hey Johannes,

Hi Jose,

Thanks for your posting. I have a couple of comments and questions:

On Wed, May 21, 2008 at 21:41:31 +0200, Jose V Beneyto wrote:

...

Hi people,

As surely someone has known, pitillo and me are involved on developing a safe environment where test crux ports.

Apparently, ATM don't exist a unified way for testing/developing ports. Well, there's the chroot testing procedure which is described on http://crux.nu/Main/ChrootTesting

like we talk at irc channel, is this a general/unified/standard way for packagers/maintainers to build ports?. If its objective is this, I think some maintainers don't know about this procedure and like we talk yesterday, IMHO if this is intended to be the way to go, someone must check the process. I told this and I repeat here again, is it really needed to start from a core/opt installation?, doesn't has much sense to start only with core and then build ports with the minimal/real dependencies instead of using all opt ports?

I've quickly tested this for a 2.4 test ISO, and with an additional mount for /sys it works fairly well.

I started some time ago from this page/notes to build a little script with this steps to keep ports up to date with their minimal dependencies.

While this setup may be less powerful, it has served well in the past to test package builds, and I like the fact that it's very transparent, i.e. it's very clear to me what actually happens.

I think the process is more clear in that webpage, but I think safe-env isn't hard to read and it's quite easy to check what it does.

...

As we known, developers are using prtutils, chroot environments or unfortunately other are using their own crux system for building and testing their ports, and that sounds terrible in some cases. I guess it could be argued how terrible this is, but okay :-)

Well, may be terrible isn't the propper word, but looking sometimes some footprints you can make diffs and see missing files which aren't really needed by a port, because seems that port was built in a normal system, with more libs than the minimal needed by a port. May be these were commented in the optional/nice to have lines, which aren't really official too, and can be good to use only 1 name in all Pkgfiles too, but the point is that the maintainer what must provide? A real build with what he thinks it's fine or with the real/minimal dependencies?

...

For that, we are working on 'safe-env', a environment similar to a "freshinstall", where testing ports and try to discover bugs, and also the most issues that crux users can find when they are building ports for their systems. I think as far as packaging bugs go, using fakeroot should catch most of them already. Functional tests as well as missing dependencies however could be easier to find.

That's true.

What bugs did you think of here?

I think we can talk about missing dependencies or footprints mistmatchs with missing files which aren't really needed by a port or which were installed in the maintainer system.

Also, are there any automated checks to see if a package installation misbehaved, especially directory permission checks? Also, what about post-installs scripts? I think especially the later might be something interesting to look into!

This can be a start point. I had in mind to use safe-env and make it bigger with some set of tests to make more checks to a port. This is one of the reasons we show you this enviroment, to see if it can be interesting to the comunity and may be someone can add, bit by bit, more attibutes to it.

In any case, maybe we can define some criteria for ports testing, i.e. which ports should be there to start with, always rebuilding vs. keeping packages etc.

And if this can be good to point all maintainers to some ways to build ports, I think this can be good too. Doesn't really matter if he will use the confortable process of the webpage or the "big"? enviroment. But I think the objective is to keep all maintainers informed about ways to build clean ports for the comunity.

Thanks for the clarifications, Johannes Thank you too for your time spent reading this mails and the enviroment.

-- Learning bit by bit. -pitillo-

On Fri, 23 May 2008 17:03:25 +0200 Jose V Beneyto <sepen@users.sourceforge.net> wrote:

Heyo Jue,

First of all, I'm very glad of having your response. Thanks a lot.

Juergen Daubert wrote:

...

On Wed, May 21, 2008 at 09:41:31PM +0200, Jose V Beneyto wrote:

...

Hi people,

Hi Jose,

...

Apparently, ATM don't exist a unified way for testing/developing ports. As we known, developers are using prtutils, chroot environments or unfortunately other are using their own crux system for building and testing their ports, and that sounds terrible in some cases.

For that, we are working on 'safe-env', a environment similar to a "freshinstall", where testing ports and try to discover bugs, and also the most issues that crux users can find when they are building ports for their systems.

it would help much if you could clarify the following points:

- which problems do you try to fix with safe-env

Encourage lacy people to use a chroot environment, by saving a lot of his time. The main goal, as I exposed in my other mails and pitillo too, was keeping the idea of ChrootTesting, but being more automated for getting it running in a easy way, without spend time for build the environment.

...

- how does it work, e.g. some kind of description or pseudo-code

1 - first of all select what kind of environment you want to have (crux_version, source_iso, img_size, img_fstype, etc)

$ vim etc/safe-env.conf

2 - start the automation and create one environment with the options you selected previously - init does: * download selected iso * check md5 * mount iso with loop * create (with dd) a new fs image (with fstype/size you selected) * mount the fsimage * fakesetup all core ports from iso loop-mounted image to fsimage (fakesetup iso/mnt mnt/) * finish the fsimage (it will be named 'core' by default, and stored in img/core) * optional that can be added: execute you own extra post-installations tasks * umount all

$ sudo ./safe-env init

3 - use the environment * mount img/core * execute fakelogin and whatever you personalize on init

$ sudo ./safe-env use core

4 - test the building process of a port using prt-get depinst and snapshots (for more info doc/HOWTO-Working-with-prt-clean.txt)

First of all I like update the enviroment, prt-get sysup and then store the snapshot. Then you can start working in the enviroment with a clean/fresh up to date core.

* make the inital snapshot

# prt-clean store "Initial snaphot"

* install depinstallation of a port

# prt-get depinst foo

* if it success # prt-clean clean (or prt-clean restore 1)

* if not report the problem for building the port or his dependencies

[...] * exit from the safe-env

# exit

Also can be used in many different ways, just test it.

...

thanks and regards Juergen Thanks you too and best regards

Jose V Beneyto

Regards, pitillo. -- Learning bit by bit. -pitillo-

On Fri, 23 May 2008 22:06:35 +1000 Lucas Hazel <lucas@die.net.au> wrote: Hello Lucas, thank you for posting here your opinion.

I don't really see a present need to "unify" the testing/development process. CRUX has a set of guidelines that ports should adhere to and at the end of the day, this is all that is really important. How you get there is the least important thing.

Well, then we can take this like another way to test ports, instead of using it in a unified way.

I agree that a clean build environment is an important tool in the port development process. The only advantage I see to such projects is that they automate some of the tedious groundwork required in creating such an environment. Once that groundwork has been laid out, then what?

Chroot environments are great for testing the build process and eliminating soft dependencies that can show up as part of the library detection that some configure scripts employ, however, this is only one aspect of testing.

This is one reason we posted/showed safe-env. May be it can be extended with prt-utils to make more than look for missing deps only.

I think package database snapshots are a good idea. I've been using such a tool myself for a long time now. I'm not sure if I ever announced it on the ML, but many people here would already aware of it's existence.

I readed it first time some days ago. I didn't know about it, may be you announced when I wasn't here.

Anyway, I digress. My point is I don't think there is a need to unify these processes. I happen to like the freestyle nature of CRUX users, we all think differently, have different goals, approaches, methods and an almost complete lack of hierarchy (excluding a very small core team). Yet we still manage to produce one of the finest Linux distributions available.

Well, I think this was showed to talk about a unified way to build ports, no to tell that this must be the way to build ports. (I think there is a FS where mike told this). I'm glad to know your opinion about the unified way to build ports and you are right, people is different and have differents ways to work, but if we can give people another way to test and see if it's interesting for them (for test ports or whatever they want to use it) I think it's a good reason to show it here to the comunity.

While I think it's great we share ideas on port development and maintenance techniques, at the end of the day ports either meet the requirements of CRUX or they don't.

Yes, that is why we show this. There isn't nothing hidden in these posts.

It's my belief that if we all thought and did things the same way more bugs would appear than got fixed.

I am not sure about this last sentence... in this comunity there is lot of knowledge... may be a unified way in this sense can be good, because all people will be looking to the same things and it can grow in a good way (letting in a side about people's customs for do things like you said above). May be can be hard in its start. Btw, this is only my point of view and sure I need more knowledge and experiencie. Thank you a lot for posting here and tell us your opinion. Regards, pitillo. -- Learning bit by bit. -pitillo-