Siduction Forum
Siduction Forum => Ideas & Improvements => Topic started by: michaa7 on 2014/07/20, 15:01:43
-
I absolutely have no clue how much work it would be to fire up your build machinery and configure it appropriately for a 32 bit paintitblack.iso, but I'd like the idea of a not so outdated iso for my architekture ...
-
You can make one yourself
git clone git://git.siduction.org/.../pyfll.git
-
seriously? No need to dive into the abyss of whateverconfig or scripting?
Simply execute command and wait some hours (here on my sempron lowend machinery)?
-
seriously? No need to dive into the abyss of whateverconfig or scripting?
Simply execute command and wait some hours (here on my sempron lowend machinery)?
seriously ! so basically you want someone else to do it for you (waiting hours), I don't even own a 32 bit machine, the best I can do is to make a 32/64 bit iso, so then you have a choice, again, It would take time to build it.
Back in time before I had a 64bit, I use to take the time to build on my 32 bit machine, it's not as bad as you are making it out to be, and in the process you get to learn a little/lot.
It's a good feeling installing something you built, whether it is stock or customized to your liking :)
-
seriously? No need to dive into the abyss of whateverconfig or scripting?
Simply execute command and wait some hours (here on my sempron lowend machinery)?
seriously ! so basically you want someone else to do it for you (waiting hours),...
No, that was not my intention, and yeah, I would learn a lot. I think I'll give it a try. But one last question: How much space do I need for git data and iso all together approximately. I assume I'd do it as normal user in my home? And re time, how many hours should I expect to wait (sempron 2900, 1 GB RAM) 2h? 5h? 10h? overnight?
-
There is no need to do that on a 32bit machine. A 32bit iso can be build on a 64bit machine.
-
Thanks towo.
That is very usefull info as this means I could use my laptop (much more RAM and CPU power). But the HD-space is limited. So how much disk space should I reserve for git plus ISO?
And I certainly would need to read at least how to configure the built of a 32 bit ISO on a 64 bit computer. What do I first need to read?
-
3-5 gigs should be plenty for 1 iso & git (stock)
git folder (pyfll) = 2.0 MiB (272 files, 32 sub-folders)
Even my custom is only 3.4 gigs
-
And I certainly would need to read at least how to configure the built of a 32 bit ISO on a 64 bit computer. What do I first need to read?
You should first have a look in fll.conf, then it should be clear.
-
thanks to both of you, I'll give it a try soon.
-
Hmm, I got problems trying that:
root@HP615RS:/usr/src# git clone git://git.siduction.org/.../pyfll.git
Klone nach 'pyfll'...
fatal: remote error: access denied or repository not exported: /.../pyfll.git
Any idea what I did wrong ?
regards
Reiner
-
1. Never checkout sourcecode as root!
2. git clone git://git.siduction.org/code/pyfll.git
-
@towo: Ok, thank you for the info.
regards
Reiner
-
I now installed git on my laptop and will checkout soon.
should I install one/some/all of the suggested files? If so, possibly provide why I should do so.
git-daemon-run git-daemon-sysvinit git-doc git-el git-email git-gui gitk gitweb git-arch git-cvs git-mediawiki git-svn
Up to now I have installed git without any suggested or recommended package.
???
I have created a new partition labeled "git" which I mount with defaults into /home/<user>/git
I assume I go to this directory when I checkout and execute the command which you posted above. Or is it advisable to do it somehow differently?
Some more questions:
Is
http://wiki.siduction.de/index.php?title=Erstellen_eines_siduction-%C3%A4hnlichen_ISO
still valid?
What happens in build_dir = /media/fll/build_dir/ ?
The wiki says I don't need pbuilder, but if I use it, how do I have to configure build_dir ? And is the build process aware of my local /var/cache/apt/archives ? I don't want to redownload all those packages again.
What does fll and pyfll mean? Over the years I saw these expressions again and agian without having a clue what they mean. What those letters stand for? Could someone enlighten me?
It is not that simple to understand what to do, but step by step I think I understand what I have to configure, but it's not always obvious.
Thanks so far, infos to my questions would be very appreciated.
n8
-
I wonder what prevents developers from releasing x86 binaries? Considering that it will be, as some have said earlier in this thread "only a couple of hours of waiting" it seems to be a huge downside to have support for all but one of their x86 images only due to the lack of the last needed step.
This dissapoints me, as I have specifically looked for a distribution that supports LxQt natively and I almost found it in Siduction, yet still I am limited by my hardware.
-
First: The developer - right, we have limited manpower, so only one developer care about the lxqt stuff
Second: The binariese are there, apt will tell you more.
Third: From today the binaries are available {i386, amd64} {qt4, qt5} - all untested, bleeding edge - but they compile. And that was a time consuming process with a lot of work, swear words, upstream patches.
if you need a iso with your architecture - sorry, not available at the moment, but you can use pyfll to create one - and no, it isn't created and tested yet, we have only our release iso for LinuxTag Berlin. I'm sure that would change in the next few weeks.
If it is to slow for you: Get involved or provide the money so the dev can have a living without a regular day job.
-
How long will it take until Siduction LxQt release on x86 release, approximately? Can I expect it until the end of summer?
-
maybe - maybe not - there is no eta
Please keep in mind that paintitblack is not a regular release - its a dev release. So if things stabilize and it make sense, we will make a new dev-iso - or not. I'm sure we will release a new dev with LXQt 0.8 - when ever it will be released. It will be done, when it is done.
-
...
Second: The binariese are there, apt will tell you more.
...
Could you please more specific?
@ all
It is cristal clear, that limited manpower prevents siduction devs from releasing whatever or from support for pyffl. There shouldn't be any complaints about this fact.
It would be helpfull, though, if people (not devs) who did built an ISO themselves jumped in and give some infos. It would be very helpfull if someone with deeper understanding could comment *his* fll.conf a bit more. Maybe or probably a well commented fll.conf would enable more people to build there own ISO which as I see it would be a benefit to the distribution.
I myself am a guy who gets discouraged easily from a project if the number of my (may be stupid) questions superates a certain amount, no matter whether they are relevant or not. As an *example*, not knowing what *pyfll* realy means doesn't technically prevent me from building an ISO but nontheless creates uncertainty. The less uncerainty I feel, the more I am able to care about mistakes I make during a new lesson.
[EDIT] reeding /git/pyfll/pyfll/doc/fll.8.txt helps: fll=fullstory, the old sidux repo, pyfll=python fullstory scrips. So this part is solved[/EDIT]
So here is my proposal: I post here the original ffl.conf with some additional comments or questions. I'd appreciate it very much if users with experience in building an ISO could make comments to an in it (please, for readability and better orientation choose an other color):
########################################
# I assume this works like any **python**-script, so I comment out
# what I don't need
# adapt paths to my local specifics
# and write down questions as a comment
# Hopefully I get some answers as new comments in this script
########################################
[ 'sourcedistro' ]
# this section is experimental, modify at your own risk
# name = debian
# codename = unstable
[ 'packages' ]
# I might choose the DE between those defined in /git/pyfll/pyfll
profile = lxde
# may I comment within the following list or do I need to delete lines with unwanted i18n_s?
i18n = """
de_DE
en_GB
en_US
it_IT
pl_PL
pt_BR
"""
# same here
lang = """
de_DE
en_GB
en_US
it_IT
pl_PL
pt_BR
"""
# ??? do I need to make a list of all wanted packages here?
#packages = """
#"""
# ???
#deps = """
#"""
# ???
#debconf = """
#"""
[ 'archs' ]
# ok, seems clear, here i choose the architekture
[[ 'i386' ]]
linux = siduction-686
#[ 'amd64' ]]
#linux = siduction-amd64
[ 'repos' ]
[[ 'debian' ]]
label = debian
uri = http://ftp.de.debian.org/debian/
#cached = http://176.9.46.77:3142/ftp.de.debian.org/debian/
# would my local archive /var/cache/apt/archives suit here
# although it is not an URL but a path to a directory?
suite = unstable
components = main
[[ 'base' ]]
label = base
uri = http://packages.siduction.org/base
#cached = http://176.9.46.77:3142/packages.siduction.org/base
suite = unstable
components = main
# what do I do here? Use your key?
#gpgkey = 151E2489
#gpgkey = /usr/share/aptosid-archive-keyring/0xE3BD538B.asc
keyring = siduction-archive-keyring
[[ 'extra' ]]
label = extra
uri = http://packages.siduction.org/extra
# cached = http://176.9.46.77:3142/packages.siduction.org/extra
suite = unstable
components = main
# gpgkey = 151E2489
# gpgkey = /usr/share/aptosid-archive-keyring/0xE3BD538B.asc
keyring = siduction-archive-keyring
[[ 'fix' ]]
label = fixes
uri = http://packages.siduction.org/fixes
#cached = http://176.9.46.77:3142/packages.siduction.org/fixes
suite = unstable
components = main contrib non-free
#gpgkey = 45C45076
#gpgkey = /usr/share/siduction-archive-keyring/45C45076.asc
keyring = siduction-archive-keyring
# [[ 'kdenext' ]]
# label = kdenext
# uri = http://packages.siduction.org/kdenext
# suite = unstable
# components = main
# gpgkey = /usr/share/siduction-archive-keyring/45C45076.asc
# [[ 'experimental' ]]
# label = experimental
# uri = http://packages.siduction.org/experimental
# cached = http://176.9.46.77:3142/packages.siduction.org/experimental
# suite = unstable
# components = main contrib non-free
# #gpgkey = 45C45076
# #gpgkey = /usr/share/siduction-archive-keyring/45C45076.asc
# keyring = siduction-archive-keyring
[[ 'release' ]]
label = release
uri = http://packages.siduction.org/release
#cached = http://176.9.46.77:3142/packages.siduction.org/release
suite = unstable
components = main contrib non-free
#gpgkey = 45C45076
#gpgkey = /usr/share/siduction-archive-keyring/45C45076.asc
keyring = siduction-archive-keyring
[ 'options' ]
#build_dir = /var/cache/pyfll
# would "build_dir = /home/<user>/git/<sidon>/pyfll" be a valid choice?
# here I am lost
output_dir = /home/debbuilder/iso-agaida
#build_log = #/path/to/build.log
#media_include = /path/to/release/notes
#apt_preferences = /etc/apt/preferences
#http_proxy =
#ftp_proxy =
#boot_cmdline = quiet
#boot_timeout = 30
#apt_recommends = yes
# since cdebootstrap is borked, overwrite default with
# debootstrap. Reverted towo's change (hardcoded debootstrap)
bootstrapper = debootstrap
#squashfs_comp = xz
[ 'distro' ]
# I assume this could serve to personalize the ISO.
# For now it may be ok as is.
FLL_DISTRO_NAME = "siduction"
FLL_DISTRO_URL = "http://siduction.org"
FLL_IMAGE_DIR = "siduction"
FLL_IMAGE_FILE = "siduction"
#FLL_IMAGE_FS = "ext4"
#FLL_IMAGE_MB = "16384"
FLL_UNION_MODULE = "aufs"
FLL_MOUNTPOINT = "/fll/siduction"
FLL_MEDIA_NAME = "siduction.iso"
FLL_LIVE_USER = "siducer"
FLL_LIVE_USER_GROUPS = "dialout dip fuse cdrom audio video plugdev users floppy netdev powerdev scanner vboxusers kvm lp"
FLL_WALLPAPER = "/usr/share/wallpapers/siduction-desperado"
FLL_GFXBOOT_THEME = "12.1-desperado"
FLL_IRC_SERVER = "irc.oftc.net"
FLL_IRC_PORT = "6667"
FLL_IRC_CHANNEL = "#siduction"
#FLL_CDROM_INDEX = "siduction release notes"
#FLL_CDROM_INDEX_ICON = "release/release-notes"
# here it may be a good idea to adapt the naming appropriately
FLL_DISTRO_VERSION = 12.1
FLL_DISTRO_CODENAME_SAFE = desperadobeta
FLL_DISTRO_CODENAME = Desperado-Beta
FLL_DISTRO_CODENAME_REV_SAFE =
FLL_DISTRO_CODENAME_REV =
-
@michaa7: pyfll is doumented enough - there are the configuration files for every iso in git. so one can checkout the repo and is nearly done with the configuration. Maybe some pathes or special memory settings, but it should work out of the box.
I would be glad if someone would write a Wiki entry - but wait - its done: http://wiki.siduction.de/index.php?title=Erstellen_eines_siduction-%C3%A4hnlichen_ISO
Sure, there is room for improvement, but hey - its a wiki. Some additions: I don't see the need for a own partition - but why not. I've setup a debian mirror for our isos to speed up the build process a little bit - this may be a little bit of overkill for a home user, but hey - its really cool :) - One need only approx 350 G for the repo in booth architectures and a internet connection that allows approx 5-15G daily download to maintain the mirror. I also setup the build dir in tmpfs - 5-8 G sould be sufficient, i take 20, because we share the buildplace with pbuilder.
There are three time consuming things within the build: 1. Downloading the needed packages (approx 1..3G, it depends), not that bad with a good connection, 2. unpack and configure the packages (nice, if one have enough Ram, one should use it), 3. Creating the squash filesystem (more and faster Processors are better, doing that in Ram is cool too).
On a normal PC with a good Internet connection a build should take approx 15-30 mins, we need 4-7 min on the buildserver, so one can add the time for downloading the finished iso. But the really timeconsuming thing is create/modify/debug a configuration, that can take some days.
-
Instead of this romance it would have been more helpfull to answer my questions.
@michaa7: pyfll is doumented enough
Sure, for you.
....so one can checkout the repo and is nearly done with the configuration. Maybe some pathes or special memory settings, but it should work out of the box.
I am sure it does, exept for my questions.
I would be glad if someone would write a Wiki entry - but wait - its done: http://wiki.siduction.de/index.php?title=Erstellen_eines_siduction-%C3%A4hnlichen_ISO
I did read it as stated above.
There are three time consuming things within the build: 1. Downloading the needed packages (approx 1..3G, it depends)
See? After "apt-get autoclean" I have # df -h /var/cache/apt/archives
Dateisystem Größe Benutzt Verf. Verw% Eingehängt auf
/dev/sdb3 11G 7,5G 2,5G 76% /var/cache/apt/archives
. So It would be a shame to redownload what I already have avaliable. Do I really need a mirror software or is there a way to point pyfll to this directory, and if not, where does pyfll store the packages. I should be able to mount my partition containing the apt archive to this folder, isn't it?
3. Creating the squash filesystem ....
This is to short for me ...
If it is too time consuming, don't answer. But I you do, please do it to my questions.
-
you may ask piper or dibl for help
-
Little late for me tonight, perhaps tomorrow, I can help him with just a stock build, mind you, if debian is broke, so will building :)
-
I'm following along too. If I can build the iso's here, I can check to see how they install with my UEFI motherboard. That is, if it's ready for UEFI.
-
GoinEasy9 & all: it makes no sense to try a build now - pyfll is partly broken, some packages are gone - i will take care today (and weekend) - i think, some changes are ongoing.
-
Little late for me tonight, perhaps tomorrow, I can help him with just a stock build, mind you, if debian is broke, so will building :)
Thank's piper.
But there is no need to build for me. Now I have checked out and I began to read. I want to built an ISO myself. I'd appreciate someone answering my questions and posting his fll.conf.
And if it were advisable to install a debian mirror software (to avoid the new d/l of packages which sure as hell I have in my 7,5GB /var/cache/apt/archive), whichone should I use? apt-mirror? debpartial-mirror? debmirror? apt-move? reprepro?
And then it seems I have to wait until pyfll is fixed ...
-
Re. Mirror Software: I use apt-cacher-ng for providing/caching packages for my boxes.
regards
Reiner
-
forget about apt caches - it will not work reliable. But hey - one has to make his own experiences. devil can eventually talk about approx or apt-cacher. and reprepro is not a cache - after over a half year of testing and trying we came to the conclusion, that a mirror owned by us is a clean and working solution. but ymmv.
-
hmm, using apt-cacher-ng as cache now since long time as I have very limited bandwidth over here. So far it has been quite reliable for that. I'll see how it will work with pyfll (If its working again).
regards
Reiner
-
ReinerS: We tested apt-cacher for iso building - long story short - we don't recommend it 8)
mmh - Maybe, if one like not reproduceble build errors or like a new adventure - in that case i would recommend approx or app-cacher. But to be true - pyfll is enough of an adventure for me.
-
ReinerS: We tested apt-cacher for iso building - long story short - we don't recommend it (http://forum.siduction.org/Smileys/default/cool.gif)
I will keep that in mind and will try to be extremly carefull ;)
regards
Reiner
-
@ReinerS: its not about being careful - the cache mechanisms don't fit for iso building
hmm - some ideas:
- one can try reprepro and convert a manifest into a reprepro list file - so one can build a partial debian mirror.
- debian and the siduction files coud be included into pyfll to serve packages not in reprepro
The problems we encounter was mostly old packages in the cache which break our builds - and there was no reliable way to update the cache. with this lists on can do a upgrade in reprepro - problem solved.
if hosted on the build machine or a little server this repo can be used to upgrade several machines - ok, creating a proper list - or some lists could be a little bit tricky, but should worth a try.
The more i though about it reprepro seems the way to go without a full debian mirror
-
I will have a look into reprepro then. I have a bandwitdth still of 384 K here. Others around here are still limited to ISDN ( >:( ).
So working without some cache or mirror is not really an option if you experiment with such tools ;)
regards
Reiner
-
i talked with piper yesterday about this - the reprepro way seems a good way to go - but it need a lot of testing. Imho the new created Repo should be the primary repo for the Build, a debian repo should be in the pyfll configuration as fallback - beside the fixes and other siduction repositories. Doing things that way should minimize the needed network traffic dramatically - all one has to to is create such a repo, make it available via a webserver and throw that into his configuration.
So basic tasks will be:
* setup a local webserver (i would suggest nginx for that)
* install, understand and configure reprepro right (its poorly documented, but we have sample configs that work for the siduction repos)
* create a local repository with the needed packages (may be the hardest part because of the several Source-Repositories)
* create some package lists and update rules
* modify the pyfll config(s) the way the local repo is used.
-
But there is no need to build for me. Now I have checked out and I began to read. I want to built an ISO myself. I'd appreciate someone answering my questions and posting his fll.conf.
Hi michaa7,
All the necessary *.conf files are in place, no need to edit something for the first try!
Last week I built two i386 ISOs (lxde and nox as recovery cd) on my amd64 system without any configuration change:
1. as user:git clone git://git.siduction.org/code/pyfll.git
2. as root:cd pyfll; ./lxde_next_i386
after about 10..120 minutes depending on your machine you'll find the ISO in /srv/iso/lxde/i386
If you have kvm installed you can test it on your machine (found this hint in the wiki (http://wiki.siduction.de/index.php?title=Erstellen_eines_siduction-%C3%A4hnlichen_ISO)):
kvm -m 1024 siduction-14.1.0-paintitblack-lxde-i386-201407201841.iso
EDIT: easier call works fine (and faster). -m 1024 allocates 1024 MB instead of the default value of 128 MB
HTH
Ciao, Martin
-
...
2. as root:cd pyfll; ./lxde_next_i386
...
Thanks Martin,
you are sure it really is neccessary to run this script *as root*?
@ melma(r)ker
thanks for the info about mirror software. I though it could be sufficient to have a *partial* mirror of only the needed packages, so reprepo seems a good idea. But there are still other packages for *partial* mirrors: debmirror, debpartial-mirror or apt-move.
And, BTW, apt-cacher-ng != apt-cacher .
-
Yes, you have to be root because it uses /var/cache/pyfll and puts the ISO into /srv/iso/...
I didn't want to read a lot to change the configuration as it was late at the evening ;)
Maybe someone can clarify if its possible to work as user without a lot of changes?
Ciao, Martin
-
micha7: we thested both apt-cacher and apt-cacher-ng. Conclusion: not reliable - reprepro seems to be the much cleaner and sophisticated way to do this. reprepro configured the right way can have some other advances like a central repro for updates, updates out of the build process etc
imho the biggest advantage is the ability fo fill the repro initial without caching and keep the repo up to date with a few simple commands.
-
horo: working as user is possible, if sudo is configured the right way. towo introduced sudo su in pyfll a long time ago.
i would suggest a own user for iso builds - the current user can be added to the group without pain - work with jenkins without flaws.
i can post some very restrictive sudo rules later.
-
Well, root is still needed the way I build, but, I don't use sudo (can edit fll for that), I do however stress to use sudo especially for new people that are tinkering until you get a better understanding
* However, if using jenkins, sudo is a * MUST * I can't stress this enough (same with melmarker ;) )
I also don't use /srv/iso/
Everything is in my /home
[ 'options' ]
build_dir = /home/piper/pyfll/pyfll/debbuilder/pbuilder/build
output_dir = /home/piper/pyfll/pyfll/debbuilder/iso-piper
build_log = /home/piper/pyfll/pyfll/build_log/build.log
I am not the greatest with building and it has changed dramatically over the years when kelmo, (founder of pyfll), sidux team, devil, had to put up with me, especially devil who *gasp* spoon-fed me ;) learnin to build.
When we can build again, I can *try and help, but, like I had to learn, it is trial & error & a lot of swearing.
This is all about stock siduction, please don't pm me or discuss here about *custom* distro or remaster, your on your own with that.
-
...
Everything is in my /home
[ 'options' ]
build_dir = /home/piper/pyfll/pyfll/debbuilder/pbuilder/build
output_dir = /home/piper/pyfll/pyfll/debbuilder/iso-piper
build_log = /home/piper/pyfll/pyfll/build_log/build.log
Thanks for that.
... especially devil who *gasp* spoon-fed me ;) learnin to build.
You can't imagine how much I'd loved to see this video documented ;-)
-
You can't imagine how much I'd loved to see this video documented ;-)
That's cutting into my drinking time ;)
-
After building this machine close to 2 years ago, I have never seen all 8 cores working full blast. It looked brilliant on gkrellm
2014-07-28 12:20:21,553 INFO - creating squashfs filesystem of amd64 chroot...
Parallel mksquashfs: Using 8 processors
Creating 4.0 filesystem on siduction.amd64, block size 131072.
[==========================================================================================-] 127495/127495 100%
After a few problems with dependencies, I seem to have it working. Geez, I wanted to pyfll for years.
real 52m58.226s
user 14m22.825s
sys 3m58.518s