nfs-blkmap AND pNFS

pNFS (Parallel NFS) 是 NFS v4.1 标准中引入的一项革命性特性。NFSv4.1/4.2： 增加了对 pNFS (并行 NFS) 的支持，适用于大数据和高性能计算环境。

pNFS 解决了传统 NFS 的“单点瓶颈”问题，实现了客户端与存储设备之间的直接、并行数据传输。

从架构设计哲学上讲，pNFS 与 HDFS（Hadoop Distributed File System）有着异曲同工之妙。它们都采用了“控制平面与数据平面分离”的核心思想。我们可以把这两套系统的组件放在一起对比，发现它们的逻辑是高度重合的：

pNFS与HDFS的不同点：底层实现与应用场景

虽然架构相似，但它们在实现细节上有本质区别：

A. 实现层次 (The “Where”)
* pNFS：实现在 Linux 内核级。对于应用程序来说，它就是一个普通的文件夹（POSIX 标准）。你不需要修改代码，直接 open() 和 read() 就能享受加速。
* HDFS：实现在 应用级（用户态）。它不是标准的挂载点（虽然可以用 FUSE 挂载但性能一般），应用程序通常需要调用特殊的 API（如 hadoop fs -get）来操作。

B. 颗粒度 (The “How”)
* HDFS：通常处理超大文件，块大小（Block Size）默认为 128MB。它更强调流式读写。
* pNFS (Block Layout)：可以处理更细粒度的随机读写。配合 blkmapd，它可以直接操作底层的磁盘块，这让它非常适合数据库或虚拟化镜像这种对延迟敏感的场景。

C. 数据冗余机制
* HDFS：通常靠软件实现多副本（Replication）或纠删码（EC）。
* pNFS：数据冗余通常交给底层的 DS (Data Server) 处理。比如 DS 本身就是一个由硬件 RAID 保护的 SAN 存储，或者是一个分布式存储集群。

systemctl status -l nfs-blkmap
● nfs-blkmap.service - pNFS block layout mapping daemon
     Loaded: loaded (/usr/lib/systemd/system/nfs-blkmap.service; enabled; preset: enabled)
     Active: active (running) since Mon 2026-04-13 18:30:46 CST; 3 weeks 3 days ago
 Invocation: 8c48c63faf8248d5b442983c7c6aa7e0
       Docs: man:blkmapd(8)
   Main PID: 934 (blkmapd)
      Tasks: 1 (limit: 154286)
     Memory: 408K (peak: 1.8M)
        CPU: 8ms
     CGroup: /system.slice/nfs-blkmap.service
             └─934 /usr/sbin/blkmapd

Apr 13 18:30:46 debian systemd[1]: Starting nfs-blkmap.service - pNFS block layout mapping daemon...
Apr 13 18:30:46 debian blkmapd[934]: open pipe file /run/rpc_pipefs/nfs/blocklayout failed: No such file or directory
Apr 13 18:30:46 debian systemd[1]: Started nfs-blkmap.service - pNFS block layout mapping daemon.

[root@debian ~]$  which blkmapd
/usr/sbin/blkmapd

[root@debian ~]$  dpkg -S /usr/sbin/blkmapd
nfs-common: /usr/sbin/blkmapd

[root@debian ~]$  apt-file show nfs-common
nfs-common: /etc/default/nfs-common
nfs-common: /etc/init.d/nfs-common
nfs-common: /etc/request-key.d/id_resolver.conf
nfs-common: /usr/lib/systemd/system-generators/rpc-pipefs-generator
nfs-common: /usr/lib/systemd/system/auth-rpcgss-module.service
nfs-common: /usr/lib/systemd/system/nfs-blkmap.service
nfs-common: /usr/lib/systemd/system/nfs-client.target
nfs-common: /usr/lib/systemd/system/nfs-common.service
nfs-common: /usr/lib/systemd/system/nfs-idmapd.service
nfs-common: /usr/lib/systemd/system/nfs-utils.service
nfs-common: /usr/lib/systemd/system/proc-fs-nfsd.mount
nfs-common: /usr/lib/systemd/system/rpc-gssd.service
nfs-common: /usr/lib/systemd/system/rpc-statd-notify.service
nfs-common: /usr/lib/systemd/system/rpc-statd.service
nfs-common: /usr/lib/systemd/system/rpc-svcgssd.service
nfs-common: /usr/lib/udev/rules.d/99-nfs.rules
nfs-common: /usr/libexec/nfsrahead
nfs-common: /usr/sbin/blkmapd
nfs-common: /usr/sbin/mount.nfs
nfs-common: /usr/sbin/mount.nfs4
nfs-common: /usr/sbin/mountstats
nfs-common: /usr/sbin/nfsconf
nfs-common: /usr/sbin/nfsidmap
nfs-common: /usr/sbin/nfsiostat
nfs-common: /usr/sbin/nfsstat
nfs-common: /usr/sbin/rpc.gssd
nfs-common: /usr/sbin/rpc.idmapd
nfs-common: /usr/sbin/rpc.statd
nfs-common: /usr/sbin/rpc.svcgssd
nfs-common: /usr/sbin/rpcctl
nfs-common: /usr/sbin/rpcdebug
nfs-common: /usr/sbin/showmount
nfs-common: /usr/sbin/sm-notify
nfs-common: /usr/sbin/start-statd
nfs-common: /usr/sbin/umount.nfs
nfs-common: /usr/sbin/umount.nfs4
nfs-common: /usr/share/bug/nfs-common/control
nfs-common: /usr/share/bug/nfs-common/script
nfs-common: /usr/share/bug/nfs-utils/control
nfs-common: /usr/share/bug/nfs-utils/presubj
nfs-common: /usr/share/doc/nfs-common/NEWS.Debian.gz
nfs-common: /usr/share/doc/nfs-common/changelog.Debian.gz
nfs-common: /usr/share/doc/nfs-common/copyright
nfs-common: /usr/share/lintian/overrides/nfs-common
nfs-common: /usr/share/man/man3/nfs4_uid_to_name.3.gz
nfs-common: /usr/share/man/man5/nfs.5.gz
nfs-common: /usr/share/man/man5/nfs.conf.5.gz
nfs-common: /usr/share/man/man5/nfsmount.conf.5.gz
nfs-common: /usr/share/man/man5/nfsrahead.5.gz
nfs-common: /usr/share/man/man7/nfs.systemd.7.gz
nfs-common: /usr/share/man/man8/blkmapd.8.gz
nfs-common: /usr/share/man/man8/gssd.8.gz
nfs-common: /usr/share/man/man8/idmapd.8.gz
nfs-common: /usr/share/man/man8/mount.nfs.8.gz
nfs-common: /usr/share/man/man8/mountstats.8.gz
nfs-common: /usr/share/man/man8/nfsconf.8.gz
nfs-common: /usr/share/man/man8/nfsidmap.8.gz
nfs-common: /usr/share/man/man8/nfsiostat.8.gz
nfs-common: /usr/share/man/man8/nfsstat.8.gz
nfs-common: /usr/share/man/man8/rpc.gssd.8.gz
nfs-common: /usr/share/man/man8/rpc.idmapd.8.gz
nfs-common: /usr/share/man/man8/rpc.sm-notify.8.gz
nfs-common: /usr/share/man/man8/rpc.statd.8.gz
nfs-common: /usr/share/man/man8/rpc.svcgssd.8.gz
nfs-common: /usr/share/man/man8/rpcctl.8.gz
nfs-common: /usr/share/man/man8/rpcdebug.8.gz
nfs-common: /usr/share/man/man8/showmount.8.gz
nfs-common: /usr/share/man/man8/sm-notify.8.gz
nfs-common: /usr/share/man/man8/statd.8.gz
nfs-common: /usr/share/man/man8/svcgssd.8.gz
nfs-common: /usr/share/man/man8/umount.nfs.8.gz
nfs-common: /usr/share/nfs-common/conffiles/idmapd.conf
nfs-common: /usr/share/nfs-common/conffiles/nfs-common.default
nfs-common: /usr/share/nfs-common/conffiles/nfs.conf

[root@debian ~]$  apt show nfs-common |cat

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Package: nfs-common
Version: 1:2.8.3-1
Priority: optional
Section: net
Source: nfs-utils
Maintainer: Debian kernel team <[email protected]>
Installed-Size: 1,172 kB
Provides: nfs-client
Pre-Depends: init-system-helpers (>= 1.54~)
Depends: libc6 (>= 2.38), libcap2 (>= 1:2.10), libcom-err2 (>= 1.43.9), libdevmapper1.02.1 (>= 2:1.02.197), libevent-core-2.1-7t64 (>= 2.1.8-stable), libgssapi-krb5-2 (>= 1.17), libk5crypto3 (>= 1.6.dfsg.2), libkeyutils1 (>= 1.5.9), libkrb5-3 (>= 1.10+dfsg~alpha1), libmount1 (>= 2.24.2), libnfsidmap1 (= 1:2.8.3-1), libtirpc3t64 (>= 1.0.2), libwrap0 (>= 7.6-4~), rpcbind, adduser, ucf, keyutils
Recommends: python3
Suggests: open-iscsi, watchdog
Conflicts: nfs-client, nfs-kernel-server (<< 1:2.8.2-2~exp1~)
Replaces: nfs-client
Homepage: https://linux-nfs.org/
Tag: admin::filesystem, implemented-in::c, interface::commandline,
 interface::daemon, network::client, network::server, protocol::nfs,
 role::program
Download-Size: 269 kB
APT-Manual-Installed: yes
APT-Sources: http://mirrors.ustc.edu.cn/debian trixie/main amd64 Packages
Description: NFS support files common to client and server
 Use this package on any machine that uses NFS, either as client or
 server.  Programs included: lockd, statd, showmount, nfsstat, gssd,
 idmapd and mount.nfs.

[root@debian ~]$  mount | grep rpc_pipefs
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)

[root@debian ~]$  tree /run/rpc_pipefs
/run/rpc_pipefs
├── cache
├── gssd
│   └── clntXX
│       ├── gssd
│       └── info
├── lockd
├── mount
├── nfs
├── nfsd
├── nfsd4_cb
├── portmap
└── statd

11 directories, 2 files

# 加载块布局驱动模块
[root@debian ~]$  modprobe blocklayoutdriver

# 验证模块是否加载成功
[root@debian ~]$  lsmod | grep blocklayoutdriver

# 检查 pipefs 路径下是否出现了 blocklayout 管道
# Debian 13 默认路径通常在 /run/rpc_pipefs/ 或 /var/lib/nfs/rpc_pipefs/
[root@debian ~]$  tree /run/rpc_pipefs/nfs

[root@debian ~]$  systemctl restart nfs-blkmap.service

[root@debian ~]$  systemctl status -l nfs-blkmap.service

● nfs-blkmap.service - pNFS block layout mapping daemon
     Loaded: loaded (/usr/lib/systemd/system/nfs-blkmap.service; enabled; preset: enabled)
     Active: active (running) since Fri 2026-05-08 14:25:16 CST; 6s ago
 Invocation: 3e3adadadf1a451c980e9fdf3c679917
       Docs: man:blkmapd(8)
    Process: 1775644 ExecStart=/usr/sbin/blkmapd (code=exited, status=0/SUCCESS)
   Main PID: 1775646 (blkmapd)
      Tasks: 1 (limit: 154286)
     Memory: 348K (peak: 1.7M)
        CPU: 7ms
     CGroup: /system.slice/nfs-blkmap.service
             └─1775646 /usr/sbin/blkmapd

May 08 14:25:16 debian systemd[1]: Starting nfs-blkmap.service - pNFS block layout mapping daemon...
May 08 14:25:16 debian systemd[1]: Started nfs-blkmap.service - pNFS block layout mapping daemon.

blocklayoutdriver

blocklayoutdriver 是 Linux 内核 NFS 客户端子系统 的一部分。

ls -l /lib/modules/$(uname -r)/kernel/fs/nfs/blocklayout/
.rw-r--r-- 28k root  9 Mar 03:54 blocklayoutdriver.ko.xz

modinfo blocklayoutdriver
filename:       /lib/modules/6.12.74+deb13+1-amd64/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko.xz
alias:          nfs-layouttype4-5
alias:          nfs-layouttype4-3
description:    The NFSv4.1 pNFS Block layout driver
author:         Andy Adamson <[email protected]>
license:        GPL
depends:        nfsv4,sunrpc,nfs
intree:         Y
name:           blocklayoutdriver
retpoline:      Y
vermagic:       6.12.74+deb13+1-amd64 SMP preempt mod_unload modversions 
sig_id:         PKCS#7
signer:         Build time autogenerated kernel key
sig_key:        0A:88:BC:3D:A6:49:3C:97:00:8F:74:3B:38:D8:53:7A:59:A2:31:6F
sig_hashalgo:   sha256
signature:      30:66:02:31:00:E3:49:6F:A6:E1:6A:27:FE:33:8A:6B:99:94:AE:33:
                EF:95:62:CE:12:03:23:BD:89:73:3F:C4:19:95:32:F7:F5:C6:2F:95:
                93:53:B6:67:24:26:15:9C:DB:B6:56:85:08:02:31:00:AB:B4:6C:77:
                77:37:B1:81:A6:4B:29:B0:7E:42:9B:EE:38:B6:BA:6E:7A:AB:4D:02:
                76:85:40:08:C7:A4:05:3E:41:BE:F8:FB:41:1C:F6:B1:4E:35:54:D4:
                58:A9:21:04

如果你在挂载选项里指定了使用 pNFS，但没有启动 nfs-blkmap

nfs-blkmap 的作用不是“配置挂载”，而是“支持挂载后的并行数据传输”。 nfs-blkmap 是这条路上的“高速公路扩容包”，专门负责处理 pNFS Block Layout 这种特殊的高级路况。

• 挂载可能成功：初始的 mount 动作通常能完成。
• 性能退化：由于客户端无法完成“块映射”，自动退回到传统的 NFS 模式。
• 报错刷屏：内核会不断尝试寻找 blocklayout 管道并报错，影响系统稳定性。

nfs-blkmap 是一个可选但关键的加速组件。

• 如果你只是挂载普通的群晖 NAS 或简单的 Linux NFS Server，你不需要它。
• 如果你正在使用企业级分布式存储（如华为 OceanStor、NetApp 或高性能 Ceph/pNFS 集群），且通过 iSCSI/FC 暴露数据，那么 nfs-blkmap 是发挥存储性能的唯一钥匙。