Tar Wild Card Interpretation

I had this email earlier this week:

I am trying to restore a file "\GL050".
I can see it on the tape listing, but I
can't get edge to find it.  I have tried
listing it the following ways:



./usr1/file/\\GL050

./usr1/file/\GL050



And I've tried it in quotes.  I also put the file path in a filelist

and tried to use edge that way.  It just will not find it.

Note that she is using Microlite Edge (http://aplawrence.com/Reviews/supertars.html), but that's really unimportant: you can observe the problem with tar.

$ cd /tmp

$ mkdir foo

$ touch "foo/\\filewithbackslash"

$ ls foo

\filewithbackslash

$ tar cvf test.tar foo

foo/

foo/\\filewithbackslash

We know have a tar file (test.tar) containing the problem file. Let's try restoring it as she did:

$ tar xvf test.tar foo/\\filewithbackslash 

tar: foo/\filewithbackslash: Not found in archive

tar: Error exit delayed from previous errors

$ tar xvf test.tar "foo/\\filewithbackslash"

tar: foo/\filewithbackslash: Not found in archive

tar: Error exit delayed from previous errors

Our wildcards look like they should work:

$ echo  "foo/\\filewithbackslash"

foo/\filewithbackslash

$ echo  foo/\\filewithbackslash 

foo/\filewithbackslash

At this point you may be saying "Are you crazy? Both those tar's work - with or without the quotes!"

And someone else would retort "Are YOU nuts?? No they don't!"

Who is nuts or not nuts has to be decided by some other method. Whether or not your tar happily works or complains as shown above simply depends upon how it handles wildcards and whether or not it gets to see them.

The complaining tar above identifies itself as "(GNU tar) 1.14" and was executed on Mac OS X 10.4.5. A non-complaining tar on a RedHat Linux systems says that it is "(GNU tar) 1.13.25".

Isn't that odd: the newer version seems to work "incorrectly". There's an interesting section of the "info tar" for the 1.13.25 version:

   There are some discussions floating in the air and asking for

modifications in the way GNU `tar' accomplishes wildcard matches.  We

perceive any change of semantics in this area as a delicate thing to

impose on GNU `tar' users.  On the other hand, the GNU project should be

progressive enough to correct any ill design: compatibility at all price

is not always a good attitude.  In conclusion, it is _possible_ that

slight amendments be later brought to the previous description.  Your

opinions on the matter are welcome.

Info on the Mac version lacks that paragraph - though it still strongly implies that our syntax should have worked:

"Globbing" is the operation by which "wildcard" characters, `*' or `?'

for example, are replaced and expanded into all existing files matching

the given pattern.  However, `tar' often uses wildcard patterns for

matching (or globbing) archive members instead of actual files in the

filesystem.  Wildcard patterns are also used for verifying volume

labels of `tar' archives.  This section has the purpose of explaining

wildcard syntax for `tar'.



   A PATTERN should be written according to shell syntax, using wildcard

characters to effect globbing.  Most characters in the pattern stand

for themselves in the matched string, and case is significant: `a' will

match only `a', and not `A'.  The character `?' in the pattern matches

any single character in the matched string.  The character `*' in the

pattern matches zero, one, or more single characters in the matched

string.  The character `\' says to take the following character of the

pattern _literally_; it is useful when one needs to match the `?', `*',

`[' or `\' characters, themselves.

That seems pretty plain, doesn't it? But it sure doesn't work as advertised.

Hold on, someone in the back is waving their arm frantically. They have a question. What's that? A little louder, please. Oh, yes.. the shell *does* expand wildcards.

If it can.

When it cannot, or when we prevent it, it's tar's responsibility entirely. Both man pages take note of that:

   The distinction between file names and archive member names is

especially important when shell globbing is used, and sometimes a

source of confusion for newcomers.  *Note Wildcards::, for more

information about globbing.  The problem is that shells may only glob

using existing files in the file system.  Only `tar' itself may glob on

archive members, so when needed, you must ensure that wildcard

characters reach `tar' without being interpreted by the shell first.

Using a backslash before `*' or `?', or putting the whole argument

between quotes, is usually sufficient for this.

Not sufficient on the Mac, though.

Another interesting anomaly: although the Mac man page doesn't mention it, the changelog of the .14 release mentions some new flags, including:

--wildcards

--no-wildcards

    When using wildcards (the default), *, ?, and [...] are the

    usual shell wildcards, and \ escapes wildcards. Otherwise, none

    of these characters are special, and patterns must match names

    literally.

Turns out that the older version groks those flags too, and adding them doesn't help the Mac test at all.

So how do you solve this if your tar doesn't do complete wildcards? One way is to do an interactive restore where you have to affirm each file before it is restored. In this particular case (using Microlite Edge) the pattern

"foo/?filewithbackslash"

cuts down on the number of matches and if there is no other "?filewithbackslash" will quickly restore the desired file.

GNU tar is now at 1.15, by the way: I haven't tested to see how it reacts to cases like this. I also haven't compiled 1.14 from scratch on Linux; this behaviour may be unique to the Mac instance for some reason I'm just not aware of.

The usual lessons apply: the same command works differently on different platforms, things change, read the man and info pages but don't trust them, and always be prepared to experiment.