[EN] [Solved] BTRFS tree first key mismatch detected

[Zombiefleischer]

Hello fellow linux users,

I got (I think) a bitflip today as my main drive sprung into read-only mode with the message " tree first key mismatch detected"
After rebooting I landed in initramfs as /root could not be mounted. In my tries to recover/repair this I accidentally deleted the message with

Code: [Select]

btrfs rescue zero-log /dev/nvme0n1p2I'm now at a point, where I don't know, what I should do now.
I tried:

Code: [Select]

siducer@siduction:~$ sudo btrfsck -p /dev/nvme0n1p2
Opening filesystem to check...
Checking filesystem on /dev/nvme0n1p2
UUID: 678ea3a2-d45e-4198-a28a-3a7d93329e7f
[1/7] checking root items                      (0:00:08 elapsed, 5484866 items checked)
ref mismatch on [7601295360 8192] extent item 9, found 1elapsed, 399549 items checked)
[2/7] checking extents                         (0:00:24 elapsed, 405437 items checked)
ERROR: errors found in extent allocation tree or chunk allocation
[3/7] checking free space tree                 (0:00:01 elapsed, 1076 items checked)
[4/7] checking fs roots                        (0:01:47 elapsed, 297815 items checked)
[5/7] checking csums (without verifying data)  (0:00:00 elapsed, 876800 items checked)
[6/7] checking root refs                       (0:00:00 elapsed, 59 items checked)
[7/7] checking quota groups skipped (not enabled on this FS)
found 1122057031680 bytes used, error(s) found
total csum bytes: 1088396968
total tree bytes: 6641614848
total fs tree bytes: 4897996800
total extent tree bytes: 541425664
btree space waste bytes: 1040216153
file data blocks allocated: 5041668685824
 referenced 1470731497472

siducer@siduction:~$ sudo btrfs rescue chunk-recover /dev/nvme0n1p2
Scanning: 852492288 in dev0scan chunk headers error

siducer@siduction:~$ sudo btrfs check -p --check-data-csum /dev/nvme0n1p2
Opening filesystem to check...
Checking filesystem on /dev/nvme0n1p2
UUID: 678ea3a2-d45e-4198-a28a-3a7d93329e7f
[1/7] checking root items                      (0:00:08 elapsed, 5484866 items checked)
ref mismatch on [7601295360 8192] extent item 9, found 1elapsed, 399765 items checked)
[2/7] checking extents                         (0:00:23 elapsed, 405437 items checked)
ERROR: errors found in extent allocation tree or chunk allocation
[3/7] checking free space tree                 (0:00:01 elapsed, 1076 items checked)
[4/7] checking fs roots                        (0:01:47 elapsed, 297815 items checked)
[5/7] checking csums against data              (0:41:49 elapsed, 876800 items checked))
[6/7] checking root refs                       (0:00:00 elapsed, 59 items checked)
[7/7] checking quota groups skipped (not enabled on this FS)
found 1122057031680 bytes used, error(s) found
total csum bytes: 1088396968
total tree bytes: 6641614848
total fs tree bytes: 4897996800
total extent tree bytes: 541425664
btree space waste bytes: 1040216153
file data blocks allocated: 5041668685824
 referenced 1470731497472

siducer@siduction:~$ sudo btrfs check -p -b /dev/nvme0n1p2
Opening filesystem to check...
Checking filesystem on /dev/nvme0n1p2
UUID: 678ea3a2-d45e-4198-a28a-3a7d93329e7f
[1/7] checking root items                      (0:00:07 elapsed, 5484866 items checked)
ref mismatch on [7601295360 8192] extent item 9, found 1elapsed, 401033 items checked)
[2/7] checking extents                         (0:00:24 elapsed, 405437 items checked)
ERROR: errors found in extent allocation tree or chunk allocation
[3/7] checking free space tree                 (0:00:01 elapsed, 1076 items checked)
[4/7] checking fs roots                        (0:01:46 elapsed, 297815 items checked)
[5/7] checking csums (without verifying data)  (0:00:01 elapsed, 876800 items checked)
[6/7] checking root refs                       (0:00:00 elapsed, 59 items checked)
[7/7] checking quota groups skipped (not enabled on this FS)
found 1122057031680 bytes used, error(s) found
total csum bytes: 1088396968
total tree bytes: 6641614848
total fs tree bytes: 4897996800
total extent tree bytes: 541425664
btree space waste bytes: 1040216153
file data blocks allocated: 5041668685824
 referenced 1470731497472

I have not tried --repair and --init-csum-tree/--init-extent-tree as they come with a lot of warnings I don't have that much experience with btrfs yet.
Can someone please help me out of this misery?

o/ Zombiefleischer

[Zombiefleischer]

Code: [Select]

[   36.175771] BTRFS error (device nvme0n1p2): tree first key mismatch detected, bytenr=641661665280 parent_transid=83112 key expected=(16040497152,168,134221824) has=(16040497152,168,4096)
[   36.175937] BTRFS error (device nvme0n1p2): tree first key mismatch detected, bytenr=641661665280 parent_transid=83112 key expected=(16040497152,168,134221824) has=(16040497152,168,4096)
[   36.175948] BTRFS error (device nvme0n1p2): failed to run delayed ref for logical 16040960000 num_bytes 4096 type 178 action 1 ref_mod 1: -5
[   36.175955] BTRFS error (device nvme0n1p2: state A): Transaction aborted (error -5)
[   36.175958] BTRFS: error (device nvme0n1p2: state A) in btrfs_run_delayed_refs:2167: errno=-5 IO failure
[   36.175960] BTRFS info (device nvme0n1p2: state EA): forced readonly

I got the original error again, so that is not lost 😅

[Zombiefleischer]

Well seems like it's bad RAM, memtest is spitting out a lot of errors 🥲

[Zombiefleischer]

To sum this post up:
Every btrfs repair/rescue command I tried (including --repair and --init-csum-tree and some btrfs-rescue commands) yielded nothing, forcing me to reinstall the OS on another drive. Luckily, the original drive is still mount- and readable, so nothing important is lost.

[DeepDayze]

Memtest errors might mean RAM issues. Try reseating the RAM and reboot. If error recurs then you should look into replacing the faulty RAM. It's possible RAM issues might be the underlying cause of the btrfs errors you posted and perhaps more knowledgeable members will chime in.

[Zombiefleischer]

Memtest errors might mean RAM issues. Try reseating the RAM and reboot. If error recurs then you should look into replacing the faulty RAM. It's possible RAM issues might be the underlying cause of the btrfs errors you posted and perhaps more knowledgeable members will chime in.

Has already been resolved. Found the faulty RAM Stick, replaced it and needed to reinstall the OS, the btrfs drive was not bootable. I could however mount it and copy my data until the drive completely corrupted. Formatted → and now a clean drive again.