Package managers

[Jonathan R]

I've been contemplating making my own distribution. To do this means I need to pick a package manager. Now I know a fair amount about package managers. Source packages, in my opinion, are ideal. They come out the fastest, you don't have to wait for them to be made into various packages, and they can be customized for your box. The problem is, the installation is slow.

Binary packages, such as rpm or deb, offer faster installs, but as I mentioned above, you have to wait till they are packaged, and you are limited to the packaging constraints.

Ideally, I'd like to have a happy merger of the two. Basically, a source package that installs fast. If a binary package can do it, I'm sure it can be done with source as well. So the question would be how.

Now, in this poll I deliberately did not get into things like portage, urpmi, YaST, smart, yum, aptitude, apt-get, and the list goes on. Those all use one of the primary package managers. So, I want to hear what you like and why. References are helpful.

[mark]

Source packages, in my opinion, are ideal. They come out the fastest, you don't have to wait for them to be made into various packages...

While you don't have to wait for them to be made into various packages, there is still a need to create the build scripts and make sure once it's built that it will function properly (that is, there's still bug testing, and someone will have to build it anyways, before it goes out to the repository).  I don't really think there is a real advantage to source packages as far as reducing time from upstream release to release from the distro.

Ideally, I'd like to have a happy merger of the two. Basically, a source package that installs fast. If a binary package can do it, I'm sure it can be done with source as well. So the question would be how.

You can't really make a source install faster, the speed of a source install depends completely on the PC it is being built on and the size/complexity of the package being built.  You can, however, get binaries released quite fast, if you have a good group of package maintainers with enough time on their hands to build things as soon as they're out.

I like arch's system.  You have pacman, the actual package manager, which is binary-based.  Then you also have ABS, the Arch Build System, which contains the build spec files for the packages.  You can use these to build from source, even modify them to build the latest and greatest version.  Since this is not an option in the poll, and I'm not a fan of the 3 options you have, I won't be voting :P

Now let me ask you a question: Why do you want to make your own distribution?  How will yours be different from other distributions out there?  Does it have a special purpose?  Do you really have enough free time for a project like this?  Building your own distro is not simple, and takes a large amount of time.

[Jonathan R]

I have used ports before and pkgtools. I thought about including them in the poll, but I'm trying to keep it somewhat simple. I can add "other".

The average user does have to wait for the package to be made.

I disagree. As I mentioned before, if binary can be fast, there must be a way to make source fast.

You said "a" question. You asked four.
Cause I want to. Maybe I can tackle some issues such as package management. Perhaps this will answer some of your questions. http://thecompletecomputerresource.com/forum/index.php?topic=135.0

[mark]

The average user does have to wait for the package to be made.

I didn't disagree with that.  My point was, whether it's a source distribution or a binary distribution, the distro maintainers still have to build the package at least once before it goes into even testing (and quite a few more times before it goes to stable).  So theoretically, source is not quicker at getting things out than binary - it depends more on the maintainers than the type of package system.  Proof: I've seen Arch come out with new packages quicker than Gentoo put out ebuilds for the same package (some years ago when I was switching between the two).

I disagree. As I mentioned before, if binary can be fast, there must be a way to make source fast.

Source can not be fast (or rather, as fast as binary) because you have to compile the application.  There is really nothing you can do to speed up compiles except upgrade your hardware.  Unless you have a magic wand, there is no way to get around that.

You said "a" question. You asked four.

Oh no!  I didn't count all the questions!!!  The end of the world is nigh! 

Anyways, the point of the question(s) was more to get you thinking about the implications of starting your own distribution.

[phunni]

Another voice in favour of pacman - itis, imho, the best package manager I've used.  It's probably one of the main reasons why I stick rigidly to arch these days...

[Jonathan R]

The average user does have to wait for the package to be made.

I didn't disagree with that.  My point was, whether it's a source distribution or a binary distribution, the distro maintainers still have to build the package at least once before it goes into even testing (and quite a few more times before it goes to stable).  So theoretically, source is not quicker at getting things out than binary - it depends more on the maintainers than the type of package system.  Proof: I've seen Arch come out with new packages quicker than Gentoo put out ebuilds for the same package (some years ago when I was switching between the two).

You have things confused. When a programmer writes a program, whether it's alpha, beta, or final, it is always in source. This is why source is the fastest released. The others have to be packaged. Even with Arch, it still has to be packaged.

Your proof, while interesting, again, does not mean anything. All it means is that it takes time for a packager to build packages.

Look, if a binary package takes the source of the program, and then builds it into a package, and that's faster, then something is awry. I think you're forgetting a lot of what goes on in the back end. Look at some spec files. You will always see the source of the program there. To build the binary package, i.e. rpm, deb, abs, whatever, the package must be built, which means it must go through compiling. You can not escape that. The only difference is, that when the package is built, the compiling happens on the packagers computer.

I guess you didn't get my sense of humor. I do enjoy yours though. I have been thinking. I first started posting about this July 26th of 2007. I was thinking about it before that. Due to working with smart, I have been researching package managers for over 2 years solid. We are working to add pacman/abs functionality to smart. We have even received a couple of requests for smart on Gentoo. 

[ac_dispatcher]

I voted .deb

Mainly because I like a distro I install and then never have to reinstall again.  Just upgrade to next version.  I have had far more luck grading to the next version with apt-get than I ahve with any .rpm distro.

With one exception so far.  PCLinuxOS.

Tex built PClinuxOS with that in mind. So that is what I stick with.

[lavaeolus]

I voted other, because I use both .deb and .rpm-based distributions and do not see that much differences, at least not from the end-user perspective, so I really cannot say I prefer one of them. I think source is not a good solution if your planned distribution is targeted at novice users. If you compare .deb and .rpm, imho the wrappers around them (apt-for-rpm, urpmi, yum, apt-get, aptitude to name some) have more impact on the user-experience than the underlying package-manager itself, apt-for-rpm gives you in many ways the same user-experience as any .deb-based distro despite using the .rpm package format. I cannot comment on pacman, since I haven't tried it so far. Seems I should fill this gap in my knowledge  ;D.

[Jonathan R]

I voted other, because I use both .deb and .rpm-based distributions and do not see that much differences, at least not from the end-user perspective, so I really cannot say I prefer one of them. I think source is not a good solution if your planned distribution is targeted at novice users. If you compare .deb and .rpm, imho the wrappers around them (apt-for-rpm, urpmi, yum, apt-get, aptitude to name some) have more impact on the user-experience than the underlying package-manager itself, apt-for-rpm gives you in many ways the same user-experience as any .deb-based distro despite using the .rpm package format. I cannot comment on pacman, since I haven't tried it so far. Seems I should fill this gap in my knowledge  ;D.

Your view point is very close to mine. I just know rpm better. I find that most users barely know the package manager. Like with rpm, who knows or understands the rpm macros? Deb also has something similar. who can tell me what this is? How about this; can you use rpm itself, as a package manager like urpmi, yum, whatever? (I already know the answer to these).

I'll wait for some more votes.

[skipper]

I don't know what are the differences between RPM and DEB and I have no any opinion about which is better, that is why i voted "other". I mostly use RPM (with apt-get on PCLinuxOS), but i tried DEB based distros and it seems like DEBs are a little bit faster to work with. Other than this - i have no clue, but i would like to know better. If you could summarize in two words all that package thing - I'll be grateful.

[mark]

You have things confused.

No, Jon, you are the one who is confused.  The thing you are forgetting is that any distribution which has repositories and releases software for that distribution must test things before they release them to the repositories.  They still have an internal testing process.  And in that testing process, they still have to build the package, have to figure out it's dependencies, have to figure out if other packages on the system have to be upgraded or if some other hack is required to get the package working on their specific distribution.  They still have to write the build instructions for their distribution (i.e. ebuilds, which are essentially spec files) and then test those spec files by building the package to ensure it works.  They do not just skip all this testing and building and drop the spec file out to the repositories.  You are forgetting what goes on in the back end ;)

One could argue that source distributions get things out to their testing repositories faster, but in the end they all go through the same internal process before being release as stable.  This includes building the package several times.

[Jonathan R]

The functionality of deb and rpm are quite similar. The philosophy is even similar, what's different are some approaches.

dpkg

maximum rpm

You have things confused.

No, Jon, you are the one who is confused.  The thing you are forgetting is that any distribution which has repositories and releases software for that distribution must test things before they release them to the repositories.  They still have an internal testing process.  And in that testing process, they still have to build the package, have to figure out it's dependencies, have to figure out if other packages on the system have to be upgraded or if some other hack is required to get the package working on their specific distribution.  They still have to write the build instructions for their distribution (i.e. ebuilds, which are essentially spec files) and then test those spec files by building the package to ensure it works.  They do not just skip all this testing and building and drop the spec file out to the repositories.  You are forgetting what goes on in the back end ;)

One could argue that source distributions get things out to their testing repositories faster, but in the end they all go through the same internal process before being release as stable.  This includes building the package several times.

Yes, and those are called binary packages. Not source. A distribution packages software. So on one hand we have the programmer who makes the program. This is called source code. Then the various distributions pick this up and package it for their distribution. This is known as source. Source is not dependent on the distribution. It is the raw source code of the program.

No tyme, I am not forgetting. Have you ever noticed tyme, that when a package is built it has the version of the program and then the build of the package. For example; smart-0.52-1, this would be the first build of the 52 version of smart. http://labix.org/smart Or you can just install it from source,

Code:

python setup.py build

. I could go to freshmeat or sourceforge and point out thousands more source programs.

Please, let's keep this straight. Packages are built from the source, by the distribution. Source is the raw program code of the developer.

[mark]

My posts were all in reference to this comment:

Basically, a source package that installs fast. If a binary package can do it, I'm sure it can be done with source as well. So the question would be how.

In this comment you are not talking about the source released from the developer but, as I understood it, the idea of a source-based distribution like Gentoo.  You are basically saying you want source packages that build as fast as binary packages install.  I then went on to discuss the process these source-based packages go through within the distribution.  At no time was I discussing the source released from the developer themselves, and I did not see that as being part of the discussion, since we are talking about package managers and distributions. 

Perhaps that's causing some confusion...?

[Jonathan R]

My posts were all in reference to this comment:

Basically, a source package that installs fast. If a binary package can do it, I'm sure it can be done with source as well. So the question would be how.

In this comment you are not talking about the source released from the developer but, as I understood it, the idea of a source-based distribution like Gentoo.  You are basically saying you want source packages that build as fast as binary packages install.  I then went on to discuss the process these source-based packages go through within the distribution.  At no time was I discussing the source released from the developer themselves, and I did not see that as being part of the discussion, since we are talking about package managers and distributions. 

Perhaps that's causing some confusion...?

Yes, that would vcause it. What I am talking about here, is a way to do a package so that it installs like binary packages, but has the power of source.

See, generally, when talking about source packages, it is referring to the developers package.

I did start talking with a Mandriva developer about some of this. Since it was in the rpm5 channel (#rpm), I limited the conversation to rpm.

<Jonathan_R> i have some questions
* spoleeba has quit ("Leaving")
* [siftin-com] has quit (Connection timed out)
<jbj_> here now
* pjones has quit ("Crustacean drifted by pier")
* mxcarron (n=mxcarron@fedora/Pingoomax) has joined #rpm
* spoleeba (n=one@fedora/Jef) has joined #rpm
* WildPikachu has quit (Read error: 113 (No route to host))
<Jonathan_R> ah there you are jbj_
<Jonathan_R> i want to write a wrapper in python (cause its the only language i know at this point) to enhance rpm
<Jonathan_R> maybe this is covered in rpm macros
<Jonathan_R> if so, tell me
<proyvind> what exactly do you want to enhance?
<Jonathan_R> lol
<Jonathan_R> when installing an rpm, i want there to be options for ./configure
<Jonathan_R> and use flags and mask options
<proyvind> uhm
<proyvind> something like gentoo portage..?
<Jonathan_R> yep
<proyvind> you do know about source rpms..?
<proyvind> you can't really do ./configure on binary rpms
<Jonathan_R> yes
<Jonathan_R> i know
<Jonathan_R> hense the problem
<proyvind> what you maybe could do
<proyvind> is to fetch sourcerpm tag from header
<Jonathan_R> but making a distro off of src.rpm's would take forever in installing
<proyvind> then fetch the source rpm from this, and then work against it, but that would require you to add a bunch of stuff to the rpm spec
<Jonathan_R> yeah
<proyvind> I think you're missing some fundamental aspects of rpm packaging here, if you want to build everything, you have to build all the source rpms
<Jonathan_R> i'm sure
<Jonathan_R> i've only built 1 rpm
<Jonathan_R> well i know i have to build src.rpms
<Jonathan_R> i know that
<proyvind> one approach could maybe to do something like a portage thingie wrapper/replacement for spec files
<Jonathan_R> but see, what i want is the speed of rpm
<proyvind> what speed?
<Jonathan_R> but the power and customability of source
<Jonathan_R> as far as installation
<proyvind> then you would have to deal with src.rpms
<Jonathan_R> the time it takes to install a binary package
<Jonathan_R> versus the time it takes to install source
<proyvind> Ithink you miss some crucial concepts here, but I have to get back to my coding..
<Jonathan_R> ok
<proyvind> jbj_ might be able to understand you better and if so provide better answers
<Jonathan_R> ok
<Jonathan_R> i must admit proyvind that i am a novice at programming
<Jonathan_R> and i dont get my ideas in programming accross very well yet
<proyvind> hm
<proyvind> I can give it another try
<proyvind> what you basically want is portage for rpms, right?
<Jonathan_R> basically
<proyvind> okay
<Jonathan_R> but with the speed of rpm
<proyvind> I've had some thoughts on that one myself
* mxcarron has quit (Read error: 110 (Connection timed out))
<Jonathan_R> cool
<proyvind> well
<proyvind> with the speed
<Jonathan_R> i know
<Jonathan_R> that part is tricky
<proyvind> you refer to the binary rpms?
<Jonathan_R> yep
<proyvind> you can't get both at once, you need first to build a package once, then you have a binary rpm which can be installed many times
<proyvind> but one idea I've had
<Jonathan_R> unless the repo contains only src rpms with a script to detect the users computer and compile an rpm for the user right there
<Jonathan_R> but i dunno
<Jonathan_R> ok
<proyvind> is to create some portage like interface for rpm, for the same task as spec files
* Jonathan_R listens now
<Jonathan_R> ok, how?
<Jonathan_R> through spec files, or rpm macros or what
<proyvind> that could be interesting for gentoo people in general as AFAIK the gentoo binary packages aren't kung fu material
<proyvind> well
<proyvind> through a lot of coding against librpm API I'd guess
<proyvind> not an easy task, not at all
<Jonathan_R> well, i guess thats why i was thinking of a wrapper
<proyvind> or might be easy, but still demanding and not something that is doable in short time
<proyvind> well
<proyvind> then I can stop you shourt of saying that's just not possible ;)
<Jonathan_R> deffinately not in short time
<proyvind> short*
<Jonathan_R> ah ok
<Jonathan_R> so it has to be at the core of rpm
<proyvind> if you want to do it properly at least..
<Jonathan_R> yeah
<proyvind> maybe you can hack something incredibly horrible, obscurr and crappy in some other ways, but probably not something you'll be proud of, most likely haunt your dreams ;)
<Jonathan_R> lol then i'd rather do it right
<Jonathan_R> i'm a perfectionist
<proyvind> but what I suggested is a recurring idea of mine, not something I've been thinking of doing nor having much interest in myself, but that others might do and would be very beneficial and helping share efforts between gentoo and rpm distros
<Jonathan_R> i'm willing to try
<proyvind> a gentoo friend of mine has the same idea, I hope to persuade him into looking into it

[mystified]

I thought you didn't like portage?