Saturday, July 12, 2014

Re: Problem with vimdiff and file name resolution

Hello,

It looks like you are using a netapp filer to store the files.

The netapp uses filesystem snapshots to make backups of the files, then mounts these snapshots under the root of the mount point. On the netapp side, they are two distinct file systems. However, on the client side, it is a single NFS-mount, so the Linux client considers it a single filesystem (as it should, according to the NFS protocol.)

Unfortunately, the only ways that I know of to differentiate are:

* With realpath(), which would then misinterpret hard links.
* To walk up the filesystem looking for directories with the same filesystem and inode number, which is similar to how the find command detects filesystem loops, but would break in the case of a real file system loop.

Both are tradeoffs, but the second option seems the more natural method.

Hope this helps.

Best Regards,
Jeff

On Jul 11, 2014 1:55 AM, "Gary Johnson" <garyjohn@spocom.com> wrote:
On 2014-07-10, Gary Johnson wrote:
> On 2014-07-10, Gary Johnson wrote:
> > On 2014-07-10, Bram Moolenaar wrote:
>
> > > Did you check the inode numbers on the files?  ls --inode.
> > > Vim checks if files are the same, ignoring links, by comparing the inode
> > > and devide number.
> >
> > I was going to try that before, but thought it unnecessary since the
> > sizes and modification times were different.  I tried it just now,
> > changing nightly.0 to nightly.1 because the backups have rotated.
> >
> >     $ ls -il .snapshot/nightly.1/.configfiles/profile.common .configfiles/profile.common
> >     14988947 -rwx------ 1 gary gary 2269 Jul  9 15:00 .configfiles/profile.common
> >     14988947 -rwx------ 1 gary gary 2992 Jul  7 10:07 .snapshot/nightly.1/.configfiles/profile.common
> >
> > Sure enough, the inode numbers are the same!  I then tried to find
> > out something about the file systems to which those directories
> > belong, but I don't understand the results.
> >
> >     $ df -h .configfiles
> >     Filesystem                      Size  Used Avail Use% Mounted on
> >     fserver1b:/vol/users/gary/      3.0T  1.6T  1.5T  53% /home/gary
> >
> >     $ df -h .snapshot/nightly.1/.configfiles
> >     Filesystem                      Size  Used Avail Use% Mounted on
> >     fserver1b:/vol/users/gary/      538G  270G  269G  51% /home/gary
> >
> > The file systems have the same names, but different stats.
> >
> > I've asked one of our IT guys to explain that to me.
> >
> > It remains, though, that that "feature" of vimdiff is still causing
> > me difficulty.  I haven't looked yet at the code that Christian
> > pointed me to, but it seems that if vimdiff is going to try for some
> > sort of optimization by checking inode numbers, it ought to check
> > file systems as well.
>
> I haven't figured out how to distinguish the two file systems.  The
> stat command indicates that the device numbers (%d) are the same,
>
>     $ stat --format="%d %i %m %n" .snapshot/nightly.1/.configfiles/profile.common .configfiles/profile.common
>     37 14988947 /home/gary .snapshot/nightly.1/.configfiles/profile.common
>     37 14988947 /home/gary .configfiles/profile.common
>
> and that the file system IDs (%i) are the same,
>
>     $ stat -f --format="%i %b %n" .snapshot/nightly.1/.configfiles/profile.common .configfiles/profile.common
>     0 17616077 .snapshot/nightly.1/.configfiles/profile.common
>     0 99824436 .configfiles/profile.common
>
> I included the total data blocks in the file systems (%b) in the
> second result to help confirm that the file systems are indeed
> different.

I had a chat with our IT guy.  It turns out that our short-term
backup system, the one that periodically saves files to the
~/.snapshot directory, uses its own file system (perhaps a database
made to look like a file system) that preserves the inode numbers of
backed up files, as you can see in this output of ls.

    $ ls -li .configfiles/profile.common .snapshot/*/.configfiles/profile.common
    14988947 -rwx------ 1 gary gary 2269 Jul  9 15:00 .configfiles/profile.common
    14988947 -rwx------ 1 gary gary 2269 Jul  9 15:00 .snapshot/hourly.0/.configfiles/profile.common
    14988947 -rwx------ 1 gary gary 2269 Jul  9 15:00 .snapshot/hourly.1/.configfiles/profile.common
    14988947 -rwx------ 1 gary gary 2269 Jul  9 15:00 .snapshot/hourly.2/.configfiles/profile.common
    14988947 -rwx------ 1 gary gary 2269 Jul  9 15:00 .snapshot/hourly.3/.configfiles/profile.common
    14988947 -rwx------ 1 gary gary 3676 Jul  9 11:57 .snapshot/hourly.4/.configfiles/profile.common
    14988947 -rwx------ 1 gary gary 2992 Jul  7 10:07 .snapshot/hourly.5/.configfiles/profile.common
    14988947 -rwx------ 1 gary gary 2269 Jul  9 15:00 .snapshot/nightly.0/.configfiles/profile.common
    14988947 -rwx------ 1 gary gary 2992 Jul  7 10:07 .snapshot/nightly.1/.configfiles/profile.common

I also browsed through the Vim code for "inode".  It appears that
Vim uses the inode number and the device number whenever it can to
determine whether two files are the same.

It would agree that if the inode numbers of two files are different,
it is safe to conclude that the files are different.  However, as
observed here and as noted at lines 4639 and 4670 of memline.c, it
is not safe to assume that if the inode numbers of two files are
equal, the files are equal.  It might be better if another test was
used to verify whether two files having the same inode numbers are
actually the same.  This is done anyway on file systems without
inodes.

On the other hand, a file system such as we're using for backups
seems pretty unusual, and equal inode numbers over network file
systems is also pretty rare.  Is this problem worth fixing?  Would a
patch even be considered?  Or should I just resort to workarounds
for the few times I have to recover part of a file?

Regards,
Gary

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_use+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

No comments: