Thursday, August 9, 2012

Still Difference Between Du and DF output?

Coming back to my blogs after a long time.

This is a continuation of my earlier post Difference in the output du and df?

I still have a filesystem with big difference between du and df output.

df -h /opt
Filesystem            Size  Used Avail Use% Mounted on
                      5.0G  4.2G  546M  89% /opt

Obviously did all the tricks with deleted files via lsof

# lsof | grep dele

Just gave me no output.

# du -sch /opt
24K    /opt
24K    total

# ls -a /opt/
.  ..  lost+found  spp.txt

Obviously nothing inside the directory, but still 4.2G is used.

Since I had exhausted all the options, had to spent some time to find out the culprit.

Couple of months back, I had an ISO file inside the /opt which was 4GB in size and had loop mounted it.

# losetup /dev/loop1 /opt/4gb.iso
# mount /dev/loop1 /mnt

Then obviously unmounted it after use.

# umount /mnt

But didnt' free the loop device /dev/loop1 after unmouting the iso which kept an fd open to the loop device which in turn points to the deleted iso and lsof failed to show to me. Bug in lsof?

I leave it to you the task to fix this by freeing the loop device.

I could have fixed this by rebooting the system like 99% of system administrators do without running behind the root cause and the cause of this should have remained unknown for me forever. Please don't do it :-)