git commit on NFS4.2 fails with "error when closing loose object file: Permission denied" — but only some of the time

Filed 2026-08-12, updated 2026-08-12 with fix-attempt results. Looking
for root-cause ideas, config fixes, or kernel bug references. Everything
below is reproducible and tested — no guesses.

Update: fix attempts tested (2026-08-12)

A detailed diagnosis (NFSv4.2 close-to-open race over mergerfs-FUSE) was
tested with the following results:

Validated root-cause refinement: the failing syscall is fsync()
(mapped by git to "error when closing loose object file") on the
tmp_obj_* temporary object files. Writing succeeds; the post-write
fsync over NFS4.2→mergerfs(FUSE) intermittently returns EACCES.

Conclusion after testing: neither client-side (noac, git fsync
tuning) nor server-side (sync export) NFS tuning fixes it. The failure
is in the mergerfs (FUSE) layer itself — it intermittently rejects
writes/fsyncs to newly created files (EACCES), and NFSd faithfully
propagates that. This is consistent with the "Hard Truth" in the
original diagnosis: git repositories do not belong on NFS backed by
mergerfs. All experimental changes were reverted to the original
state (fstab, exports, test dirs removed).

TL;DR

git commit inside a repository that lives on an NFSv4.2 mount
intermittently dies with:

fatal: error when closing loose object file: Permission denied

• It's not actually a permission problem: the file is writable, touch
  in the same directory works, the owner/mode are right, and the same
  commit succeeds on local ext4.
• It's intermittent — sometimes the first attempt works, sometimes
  only after retries, sometimes git commit prints success but writes
  nothing and the ref is unchanged.
• It happens on freshly-created git repos on NFS, and also on long-lived
  repos after certain operations (git add of a big file, cp -r of a
  directory into the working tree, rm -rf .git && git init).
• The workaround that reliably works is to redirect object writes to a
  local disk (GIT_OBJECT_DIRECTORY=/local/path + GIT_ALTERNATE_OBJECT_DIRECTORIES=<repo>/.git/objects) then cp -rn the objects back.
  But that's a band-aid and leaves the refs/index problem unsolved.

Environment

Client (where git runs):

• Host: gamingpc, Ubuntu (kernel 6.14.0-37-generic)
• git 2.43.0
• NFS mount:

  192.168.1.13:/personal /personal nfs4 rw,noatime,vers=4.2,rsize=1048576,
  wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,
  clientaddr=192.168.1.19,local_lock=none,addr=192.168.1.13
  

Server:

• Host: ThinkCentre, Ubuntu (kernel 7.0.0-28-generic), NFSv4.2 server
  (nfsd /proc/fs/nfsd reports +3 +4 +4.1 +4.2)
• Export (note async):

  /personal  192.168.1.0/24(rw,async,no_subtree_check,no_root_squash,insecure,fsid=101,crossmnt,nohide)
  

• Backing store: the export root is on mergerfs (fuse) over several
  disks:

  ...:/personal ... fuse.mergerfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other
  

Symptom details

The failure

$ git commit -m "test"
fatal: error when closing loose object file: Permission denied

Rust's cargo on the same mount fails similarly:

error: can't find crate for `fichub`
error: could not compile `fichub` (lib) due to 1 previous error

(often fixed by rm -rf target/debug/incremental, i.e. stale .rlib
close failures leave partial state).

What does NOT explain it

• Ownership/mode: stat -c "%a %U:%G" .git/objects → 775 alvaro:alvaro
  (owner-writable).
• Plain writes work: echo x > .git/objects/writetest succeeds.
• Same command on local ext4: git commit in /tmp/git-nfs-test
  succeeds instantly.
• Not a quota/disk-full: df shows 24% used.

What makes it worse

• Fresh git init on the NFS path, then a first commit (objects dir just
  created).
• After rm -rf .git && git init (re-created repo on NFS).
• After cp -r <bigdir> . into the working tree (lots of new loose objects
  in one commit).
• Sometimes the commit appears to succeed:

  $ git commit -m "x"   # prints nothing, exit 0
  $ git log --oneline -1
  fatal: your current branch 'main' does not have any commits yet
  

  → the ref update / HEAD write also silently failed.

Workaround we use (band-aid)

mkdir -p /home/alvaro/.cache/git-objects-xyz
export GIT_OBJECT_DIRECTORY=/home/alvaro/.cache/git-objects-xyz
export GIT_ALTERNATE_OBJECT_DIRECTORIES=/personal/documents/code/rust/fichub/.git/objects
git commit -m "msg"
cp -rn /home/alvaro/.cache/git-objects-xyz/* .git/objects/
unset GIT_OBJECT_DIRECTORY GIT_ALTERNATE_OBJECT_DIRECTORIES

• GIT_OBJECT_DIRECTORY alone (without GIT_ALTERNATE_OBJECT_DIRECTORIES)
  makes it worse: git can no longer read the existing HEAD from the NFS
  objects and fails with fatal: could not parse HEAD. Both must be set.
• This only fixes object writes. Index/ref writes to .git/index and
  .git/refs/heads/main still occasionally fail, so the workaround is
  partial.
• git config core.fsync none on the repo reduces frequency but does not
  eliminate it.

What we need help with

1. Root cause. The failure is at close() (or fsync/rename) of a
   loose object file on NFS4.2. The file is writable, local writes work,
   but git's close fails with EACCES. Is this a known NFSv4.2 client bug?
   Kernel 6.14 client + kernel 7.0 server? Something about
   async exports + close-to-open consistency? Or is the mergerfs
   backing store (fuse) underneath the export the real culprit — does the
   NFS server do something at close time that the fuse fs rejects?

2. Is the local_lock=none mount option relevant? We use
   local_lock=none (default for nfs4). Should it be local_lock=all?
   Would that affect close-time locking of newly created files?

3. Why do fresh repos fail more? Newly created .git/objects dir on
   the NFS export vs long-established ones. NFS attribute caching
   (negative dentry / missing post-create attribute refresh)?

4. Server-side: we export async (we know it's the safe default on
   this box). Does async + close-to-open on the client explain the
   intermittent EACCES at close? Would sync fix it (we can accept the
   perf hit)?

5. Is the mergerfs-under-NFS export a known-bad stack? The export is a
   mergerfs fuse mount of 8 disks. NFSd → fuse (mergerfs) → disks. Any
   known issues with NFS close/fsync/rename across that chain?

6. Has anyone seen git commit print success but write nothing (no
   objects, no ref update) on NFS? That one is the scariest — silent
   success is worse than an explicit error.

Things we already tried (with results)

Environment recap (for kernel/git folks)

• Client: Ubuntu kernel 6.14.0-37-generic, git 2.43.0, NFSv4.2
• Server: Ubuntu kernel 7.0.0-28-generic, export rw,async,no_subtree_check,no_root_squash,insecure,crossmnt
• Backing store under export: mergerfs (fuse) over 8 disks
• Mount: vers=4.2,hard,proto=tcp,timeo=600,retrans=2,sec=sys,local_lock=none

Any pointer to a kernel bug, a git issue, an NFSv4.2 close/fsync thread,
or a known mergerfs-via-NFS gotcha is appreciated.

I absolutely detest LLM- generated bug reports. Huge walls of text which waste maintainers' time.