Siduction Forum

Siduction Forum => Experimental => Topic started by: ralul on 2012/07/03, 14:57:17

Title: if using BTRFileSystem
Post by: ralul on 2012/07/03, 14:57:17
This is not a general Debian issue as Wheezy is stuck with Linux-3.2. But with siduction use of btrfs might become problematic!
I just saw this news while upgrading my Gentoo installation:
Quote
btrfs-progs-0.19-r3::gentoo
# Joe Peterson <lavajoe@gentoo.org> (02 Jul 2012)
# This old version has become out of sync with kernel btrfs, so is dangerous
# (except for those who have an older kernel and know what they are doing).
# Use of the live build (btrfs-progs-9999) is recommended, since no upstream
# versioning is available, and the live build keeps pace with new kernels.
# See bug #420477
Gentoo "live build" is a notion for compiling directly from upstream git source ...
Title: RE: if using BTRFileSystem
Post by: dibl on 2012/07/03, 18:52:13
Which kernel version appears with this warning?

Thank ralul.
Title: RE: if using BTRFileSystem
Post by: DeepDayze on 2012/07/03, 19:42:26
For Debian, could there be a script thrown together for building this package?
Title: RE: if using BTRFileSystem
Post by: ralul on 2012/07/03, 19:55:43
The bug link https://bugs.gentoo.org/show_bug.cgi?id=420477
tells something about using Linux-3.4
This is why reading it, I thought about you guys!

But myself beware of such an unstable filesystem yet ...
Title: RE: if using BTRFileSystem
Post by: dibl on 2012/07/04, 23:18:29
I have been running a btrfs filesystem since December, 2010. This is installed on a pair of WD1002FAEX "SATA III" drives, connected to the Marvell 9128 6Gb/s controller on my Asus P6X58D-E system board.

Code: [Select]
root@imerabox:/# blkid -c /dev/null -o list                                                            
device               fs_type   label      mount point              UUID                                
-------------------------------------------------------------------------------------------------------
/dev/sda1            ext4                 /                        bea3a748-3411-4024-acd0-39f3882ddaf9
/dev/sda2            ext4      SDA2       /mnt/SDA2                8cfe2acc-7572-4b45-b25f-ed021bb1d78b
/dev/sdb1            ext4      revodata   /mnt/REVODATA            ec21f5b3-7fd4-4f4b-af8d-cf787b147ae8
/dev/sdc1            ext2                 /boot                    ac7da829-aebb-46f0-806c-04a4d81a945a
/dev/sdc2            swap                 <swap>                   0d939b7d-48f1-47dd-aebe-77e7bd8c3503
/dev/sdd             btrfs                (in use)                 c112ed57-0e33-4d4b-82c9-5c55932c529d
/dev/sde             btrfs                (in use)                 c112ed57-0e33-4d4b-82c9-5c55932c529d


Today I checked it:
Code: [Select]
root@imerabox:/# btrfs filesystem show
failed to read /dev/sr0
Label: none  uuid: c112ed57-0e33-4d4b-82c9-5c55932c529d
        Total devices 2 FS bytes used 771.37GB
        devid    2 size 931.51GB used 519.63GB path /dev/sde
        devid    1 size 931.51GB used 519.65GB path /dev/sdd                                            
                                                                                                       
Btrfs Btrfs v0.19                                                                                      
root@imerabox:/# btrfs filesystem df /mnt/DATA                                                        
Data, RAID0: total=964.00GB, used=769.82GB                                                              
Data: total=8.00MB, used=0.00                                                                          
System, RAID1: total=8.00MB, used=84.00KB                                                              
System: total=4.00MB, used=0.00                                                                        
Metadata, RAID1: total=37.62GB, used=1.56GB                                                            
Metadata: total=8.00MB, used=0.00


I ran
Code: [Select]
root@imerabox:/# btrfs scrub start /mnt/DATA
scrub started on /mnt/DATA, fsid c112ed57-0e33-4d4b-82c9-5c55932c529d (pid=4572)


(couple of hours later after it finished):
Code: [Select]
root@imerabox:/# btrfs scrub status /mnt/DATA
scrub status for c112ed57-0e33-4d4b-82c9-5c55932c529d                                                  
        scrub started at Wed Jul  4 13:47:02 2012 and finished after 3577 seconds                      
        total bytes scrubbed: 772.93GB with 0 errors


So, it seems OK at this point.  Probably a good time to back up ....
Title: RE: if using BTRFileSystem
Post by: DeepDayze on 2012/07/05, 15:07:14
How does btrfs stack up against the existing filesystems as well as ZFS?
Title: RE: if using BTRFileSystem
Post by: dibl on 2012/07/05, 17:15:30
BTRFS is clearly designed for one target application -- massive, flexible storage.  Think of server farms and cluster computing.  I don't think it offers any advantage for running an OS -- it is not faster than ext4.  It offers multi-device performance very similar to RAID, with no special controllers or drivers, so that makes life simpler.  For my 2-drive BTRFS filesystem, I used the default options, which sets metadata to be mirrored (similar to RAID 1) and data to be striped (similar to RAID 0).  I have not done any benchmark testing -- you can read in Phoronix and elsewhere how the performance compares to ext4.  I have about 750GB of music, videos, images, and docs in my /mnt/DATA filesystem, and it has shown no problems in 18 months.  When I first installed it, there was no defrag and no fsck available.  The first time I ran scrub (the btrfs version of fsck), it found correctible errors on 13 inodes, and fixed them, and there were zero non-correctible errors.  It has never found any more errors, as you see above.  I worked through a full defrag one time, which is a 2-step process, and that went fine.  But I kinda think it was not necessary, similar to ext4.

I have never tried ZFS -- I can't comment on that one.
Title: RE: if using BTRFileSystem
Post by: ralul on 2012/07/05, 17:35:36
all what dibl said
plus snapshot feature - this would be interesting for use with Debian sid: All of the troubles gone ...
Title: RE: if using BTRFileSystem
Post by: DeepDayze on 2012/07/05, 18:46:03
Btrfs does sound promising, and looking forward to trying this on my home NAS once it is rock solid.

Filesystems are quite complex underneath the hood so it must take a lot of time to debug and tune

Quote from: "ralul"
all what dibl said
plus snapshot feature - this would be interesting for use with Debian sid: All of the troubles gone ...


Would be cool to easily roll back should you mess up a DU or a major bug creeps in.

Testing software would also be a breeze so should something break or that you don't want the app, just roll back!
Title: Re: RE: if using BTRFileSystem
Post by: dibl on 2012/07/05, 19:41:07
Quote from: "ralul"

plus snapshot feature - this would be interesting for use with Debian sid: All of the troubles gone ...


Huh -- I had not thought of that, but that is one good reason to run this OS on btrfs.

Hmmm, I think maybe Windows could use btrfs!   :twisted:
Title: Re: RE: if using BTRFileSystem
Post by: agaida on 2012/07/05, 20:53:11
dibl,
Windows dont need this feature, since they have a similar thing in windows 8. And for the records: they have partial backups of systemsettings before updates ages ago. ;)

EDIT: I don't think the so called "Wiederherstellungspunkte" are usable - but i think at the same development level like btrfs. They simply dont work in 8/10 Times.
Title: Re: RE: if using BTRFileSystem
Post by: ralul on 2012/07/05, 22:15:55
That Ms "Wiederherstellungspunkt" (nobody knows the english word) is something artificial on higher OS level. A Btrfs-snapshot is the original filesystem before the snapshot without any copying - very low level.
Title: Re: RE: if using BTRFileSystem
Post by: dibl on 2012/07/05, 22:27:10
Quote from: "ralul"
That Ms "Wiederherstellungspunkt" (nobody knows the english word)


Is that what MS called "System Restore" on Win XP, under the System Tools?

I never knew anyone who saved their Windows system with that junk -- I don't think the viruses respected it.    :lol:
Title: RE: Re: RE: if using BTRFileSystem
Post by: vilde on 2012/07/06, 00:12:07
Wiederherstellungspunkt = Återställningspunkt in Swedish, I don't either know the English word and it never worked for me either ;)
Title: Re: RE: if using BTRFileSystem
Post by: DeepDayze on 2012/07/06, 03:10:08
Quote from: "dibl"
Quote from: "ralul"

plus snapshot feature - this would be interesting for use with Debian sid: All of the troubles gone ...


Huh -- I had not thought of that, but that is one good reason to run this OS on btrfs.

Hmmm, I think maybe Windows could use btrfs!   :twisted:


Just what we need..a "system restore" for Linux :lol:
Title: Re: RE: if using BTRFileSystem
Post by: agaida on 2012/07/06, 10:40:20
System Backup:

Code: [Select]

cp -a source backup


System Restore:

Code: [Select]

cp -a backup source


:twisted:
Title: BTRFS Snapshot - The Perfect companion to Siduction?
Post by: bad_aptitude on 2013/05/01, 04:56:11
Is the time almost upon us that siduction could use something like "apt-btrfs-snapshot" (unfortunately its only a Ubuntu hack at the moment) to manage the rolling back of gnarly dist-upgrades. As valuable as the siduction utlility for managing kernels is, it only solves half of the potential problems associated with dist upgrades.
I'm not a programmer - but I would be happy to test such a feature.

Regards
bad-aptitude
Title: BTRFS Snapshot - The Perfect companion to Siduction?
Post by: agaida on 2013/05/01, 10:49:35
if you found one who develop and test this and it work reliable we would be happy to distribute such a package.

EDIT: Reliability is the point. Such a tool should not base on a experimental filesystem.
Title: Re: RE: if using BTRFileSystem
Post by: michaa7 on 2013/05/01, 11:01:34
Quote from: "agaida"
System Backup:

Code: [Select]

cp -a source backup



I'd suggest a slightly enhanced version of this command:
Code: [Select]

cp -ax /path/to/<source_partition>/. /path/to/<backup>/


The "-x" option prevents "cp" to dive into linked partitions, and the "/." at the end of the source path makes sure only the content is copied (not the containing folder).
Title: Re: RE: if using BTRFileSystem
Post by: agaida on 2013/05/01, 11:59:27
good point, i'm a litte bit lazy some times
Title: RE: Re: RE: if using BTRFileSystem
Post by: bad_aptitude on 2013/05/02, 00:46:43
One advantage, using the snapshot feature of BTRFS would have over
Code: [Select]
cp -a source backup
is the snapshot of a BTRFS directory is effectively a set of hardlinks to the original files. So when taking a snapshot prior to a dist-upgrade on BTRFS, only the files changed by the dist-upgrade are "duplicated".  At least that is my understanding of the process.
It would seem that this approach could save a lot of time and space.
Title: RE: Re: RE: if using BTRFileSystem
Post by: agaida on 2013/05/02, 01:06:43
when btrfs will be stable - in 5 or 10 years - i would eventually agree
Title: if using BTRFileSystem
Post by: bad_aptitude on 2013/05/02, 08:18:36
But isn't stable just "dead"?
Title: if using BTRFileSystem
Post by: agaida on 2013/05/02, 17:20:10
let me cite from the unix-hater-handbook:
"Sure - the new filesystem will corrupt your data. But look, how fast it is!"

The Clue with btrfs is: I tested it a while ago, it was fucking slow at writes and i decided that i don't like it on my main machines. Imho btrfs is a wonderful thing with netbooks and small notebook - machines, where you mostly need high reading speed. Given that it is reliable and have reliable tools for crash recovery.