btrfs 执行 scrub 后出现了一些 Uncorrectable 的错误，怎么知道是哪些文件的数据出错了？

[wniming]

执行 btrfs scrub 的内核日志如下：

[930620.240546] BTRFS info (device dm-2): scrub: started on devid 1
[930620.861664] BTRFS error (device dm-2): unable to fixup (regular) error at logical 2290286592 on dev /dev/mapper/develop-btrfs physical 3372417024
[930620.861851] BTRFS error (device dm-2): unable to fixup (regular) error at logical 2290352128 on dev /dev/mapper/develop-btrfs physical 3372482560
[930620.862502] BTRFS error (device dm-2): unable to fixup (regular) error at logical 2290417664 on dev /dev/mapper/develop-btrfs physical 3372548096
[930620.862542] BTRFS error (device dm-2): unable to fixup (regular) error at logical 2290417664 on dev /dev/mapper/develop-btrfs physical 3372548096
[930621.487688] BTRFS error (device dm-2): unable to fixup (regular) error at logical 2290352128 on dev /dev/mapper/develop-btrfs physical 3372482560
[930621.644231] BTRFS error (device dm-2): unable to fixup (regular) error at logical 2290352128 on dev /dev/mapper/develop-btrfs physical 3372482560
[930623.614946] BTRFS error (device dm-2): unable to fixup (regular) error at logical 8891203584 on dev /dev/mapper/develop-btrfs physical 9973334016
[930623.771505] BTRFS warning (device dm-2): checksum error at logical 8891203584 on dev /dev/mapper/develop-btrfs, physical 9973334016, root 257, inode 265, offset 2267738112, length 4096, links 1 (path: fedora.raw.vhost)
[930624.251037] BTRFS error (device dm-2): unable to fixup (regular) error at logical 9398255616 on dev /dev/mapper/develop-btrfs physical 10480386048
[930624.251886] BTRFS error (device dm-2): unable to fixup (regular) error at logical 9404088320 on dev /dev/mapper/develop-btrfs physical 10486218752
[930624.566158] BTRFS warning (device dm-2): checksum error at logical 9404088320 on dev /dev/mapper/develop-btrfs, physical 10486218752, root 257, inode 265, offset 2056241152, length 4096, links 1 (path: fedora.raw.vhost)
[930624.799369] BTRFS error (device dm-2): unable to fixup (regular) error at logical 9404088320 on dev /dev/mapper/develop-btrfs physical 10486218752
[930624.956925] BTRFS warning (device dm-2): checksum error at logical 9404088320 on dev /dev/mapper/develop-btrfs, physical 10486218752, root 257, inode 265, offset 2056241152, length 4096, links 1 (path: fedora.raw.vhost)
[930636.356820] scrub_stripe_report_errors: 8 callbacks suppressed
[930636.356827] BTRFS error (device dm-2): unable to fixup (regular) error at logical 70791266304 on dev /dev/mapper/develop-btrfs physical 71873396736
[930636.585935] scrub_stripe_report_errors: 8 callbacks suppressed
[930636.656702] BTRFS error (device dm-2): unable to fixup (regular) error at logical 70878560256 on dev /dev/mapper/develop-btrfs physical 71960690688
[930636.815440] BTRFS warning (device dm-2): checksum error at logical 70878560256 on dev /dev/mapper/develop-btrfs, physical 71960690688, root 257, inode 265, offset 466690048, length 4096, links 1 (path: fedora.raw.vhost)
[930637.537597] BTRFS error (device dm-2): unable to fixup (regular) error at logical 72395653120 on dev /dev/mapper/develop-btrfs physical 73477783552
[930637.537674] BTRFS error (device dm-2): unable to fixup (regular) error at logical 72395587584 on dev /dev/mapper/develop-btrfs physical 73477718016
[930637.537727] BTRFS error (device dm-2): unable to fixup (regular) error at logical 72395718656 on dev /dev/mapper/develop-btrfs physical 73477849088
[930638.013344] BTRFS error (device dm-2): unable to fixup (regular) error at logical 72395587584 on dev /dev/mapper/develop-btrfs physical 73477718016
[930641.642974] BTRFS info (device dm-2): scrub: finished on devid 1 with status: 0

[newborn]

感觉可以加入一块新盘，把 dm-3 这块盘踢出看看。

[wniming]

@newborn 感谢回复，dm-3 是个 lvm 的逻辑卷，我现在只想知道是哪个文件的数据有损坏，加入一个新盘起不了什么作用吧？

[wniming]

@newborn 应该是 dm-2 吧，我贴出的日志里没有 dm-3 呀

[newborn]

@wniming 我打错了。加新盘主要是保障你整个卷组健康，老盘踢出来之后可以上 hexdump 之类的工具看看损坏的内容有没有修复的可能性，或者整个踢出执行很顺利老盘也不用看了。

[wniming]

@newborn 可能是我需求描述的不够让你误解了，我这台机是专门做开发用的，有时候会直接用机箱上的 reset 按钮重启，这个 btrfs 的报错很可能是我这么操作导致的，我硬盘用的是 sn770 ，写入量才几个 TB ，硬件应该没有问题，我只是想找出有问题的文件，想试试把有问题的文件删掉这个报错会不会消失。

[yanqiyu]

[930624.956925] BTRFS warning (device dm-2): checksum error at logical 9404088320 on dev /dev/mapper/develop-btrfs, physical 10486218752, root 257, inode 265, offset 2056241152, length 4096, links 1 (path: fedora.raw.vhost)

这不是写了出问题的 path 是 fedora.raw.vhost 吗，是不是虚拟机开了 directio 但是没关 btrfs 的 cow ？

[wniming]

@yanqiyu #6 感谢回复，之前看得不仔细没发现这个 path ，刚才把这个文件删除后再次 scrub ，dmesg 报错比之前少了很多，但还是有一些同样的报错，而且没有 path 告诉我是哪个文件有问题：

[936404.398809] BTRFS info (device dm-2): scrub: started on devid 1
[936404.980677] BTRFS error (device dm-2): unable to fixup (regular) error at logical 2290286592 on dev /dev/mapper/develop-btrfs physical 3372417024
[936404.981100] BTRFS error (device dm-2): unable to fixup (regular) error at logical 2290352128 on dev /dev/mapper/develop-btrfs physical 3372482560
[936404.981180] BTRFS error (device dm-2): unable to fixup (regular) error at logical 2290417664 on dev /dev/mapper/develop-btrfs physical 3372548096
[936404.981236] BTRFS error (device dm-2): unable to fixup (regular) error at logical 2290417664 on dev /dev/mapper/develop-btrfs physical 3372548096
[936405.606635] BTRFS error (device dm-2): unable to fixup (regular) error at logical 2290352128 on dev /dev/mapper/develop-btrfs physical 3372482560
[936405.763104] BTRFS error (device dm-2): unable to fixup (regular) error at logical 2290352128 on dev /dev/mapper/develop-btrfs physical 3372482560
[936407.657754] BTRFS error (device dm-2): unable to fixup (regular) error at logical 8891203584 on dev /dev/mapper/develop-btrfs physical 9973334016
[936408.051643] BTRFS error (device dm-2): unable to fixup (regular) error at logical 9398255616 on dev /dev/mapper/develop-btrfs physical 10480386048
[936408.052556] BTRFS error (device dm-2): unable to fixup (regular) error at logical 9404088320 on dev /dev/mapper/develop-btrfs physical 10486218752
[936408.366738] BTRFS error (device dm-2): unable to fixup (regular) error at logical 9404088320 on dev /dev/mapper/develop-btrfs physical 10486218752
[936423.189046] BTRFS info (device dm-2): scrub: finished on devid 1 with status: 0


我虚拟机没有特别指定是否启用 directio ，btrfs 也是用默认的设置，应该是开了 cow 的，不过我刚才一查发现 btrfs 的 cow 和 qcow2 的 cow 最好不要同时使用，所以我又尝试把一个 qcow2 格式的磁盘文件删掉，再次 scrub 就没有报错了。

[yanqiyu]

@wniming #7 > 我虚拟机没有特别指定是否启用 directio

就是虚拟磁盘的 cache mode ，我记得除了 writethrough 都会有 O_DIRECT 的文件操作，然后现在 btrfs 的 O_DIRECT 真的是 DIRECT 不会 cow 的更新 checksum （ 6.14 修了，被 checksum 保护的文件始终会 cow 了）

[lurenjiaMAX]

我记得可以自己算的

[kuanat]

dmesg 里面找 checksum error ，大概是这么个字符串，因为我这好久没出过错记得不是很清楚了。