@c Maintenance notes:
@c 1. Pay attention to @FIXME{}s and @UNREVISED{}s
@c 2. Before creating final variant:
-@c 1.1. Run `make check-options' to make sure all options are properly
+@c 2.1. Run `make check-options' to make sure all options are properly
@c documented;
-@c 2.1. Run `make master-menu' (see comment before the master menu).
+@c 2.2. Run `make master-menu' (see comment before the master menu).
@include rendition.texi
@include value.texi
-@defcodeindex op
+@defcodeindex op
@c Put everything in one index (arbitrarily chosen to be the concept index).
@syncodeindex fn cp
from archives.
Copyright @copyright{} 1992, 1994, 1995, 1996, 1997, 1999, 2000, 2001,
-2003, 2004, 2005, 2006 Free Software Foundation, Inc.
+2003, 2004, 2005, 2006, 2007, 2008 Free Software Foundation, Inc.
@quotation
Permission is granted to copy, distribute and/or modify this document
and with the Back-Cover Texts as in (a) below. A copy of the license
is included in the section entitled "GNU Free Documentation License".
-(a) The FSF's Back-Cover Text is: ``You are free to copy and modify
-this GNU Manual. Buying copies from GNU Press supports the FSF in
-developing GNU and promoting software freedom.''
+(a) The FSF's Back-Cover Text is: ``You have the freedom to
+copy and modify this GNU manual. Buying copies from the FSF
+supports it in developing GNU and promoting software freedom.''
@end quotation
@end copying
* Changes::
* Configuring Help Summary::
+* Fixing Snapshot Files::
* Tar Internals::
* Genfile::
* Free Software Needs Free Documentation::
* help::
* defaults::
* verbose::
+* checkpoints::
* interactive::
The Three Option Styles
Controlling the Archive Format
-* Portability:: Making @command{tar} Archives More Portable
* Compression:: Using Less Space through Compression
* Attributes:: Handling File Attributes
+* Portability:: Making @command{tar} Archives More Portable
* cpio:: Comparison of @command{tar} and @command{cpio}
+Using Less Space through Compression
+
+* gzip:: Creating and Reading Compressed Archives
+* sparse:: Archiving Sparse Files
+
Making @command{tar} Archives More Portable
* Portable Names:: Portable Names
* dereference:: Symbolic Links
+* hard links:: Hard Links
* old:: Old V7 Archives
* ustar:: Ustar Archives
* gnu:: GNU and old GNU format archives.
* Split Recovery:: Members Split Between Volumes
* Sparse Recovery:: Sparse Members
-Using Less Space through Compression
-
-* gzip:: Creating and Reading Compressed Archives
-* sparse:: Archiving Sparse Files
-
Tapes and Other Archive Media
* Device:: Device selection and switching
The third chapter presents the remaining five operations, and
information about using @command{tar} options and option syntax.
-@FIXME{this sounds more like a @acronym{GNU} Project Manuals Concept [tm] more
-than the reality. should think about whether this makes sense to say
-here, or not.} The other chapters are meant to be used as a
-reference. Each chapter presents everything that needs to be said
-about a specific topic.
+The other chapters are meant to be used as a reference. Each chapter
+presents everything that needs to be said about a specific topic.
One of the chapters (@pxref{Date input formats}) exists in its
entirety in other @acronym{GNU} manuals, and is mostly self-contained.
You can use @command{tar} archives in many ways. We want to stress a few
of them: storage, backup, and transportation.
-@FIXME{the following table entries need a bit of work..}
+@FIXME{the following table entries need a bit of work.}
@table @asis
@item Storage
Often, @command{tar} archives are used to store related files for
file system. You should have some basic understanding of directory
structure and how files are named according to which directory they are
in. You should understand concepts such as standard output and standard
-input, what various definitions of the term ``argument'' mean, and the
-differences between relative and absolute path names. @FIXME{and what
+input, what various definitions of the term @samp{argument} mean, and the
+differences between relative and absolute file names. @FIXME{and what
else?}
@item
This manual assumes that you are working from your own home directory
(unless we state otherwise). In this tutorial, you will create a
-directory to practice @command{tar} commands in. When we show path names,
-we will assume that those paths are relative to your home directory.
-For example, my home directory path is @file{/home/fsf/melissa}. All of
-my examples are in a subdirectory of the directory named by that path
+directory to practice @command{tar} commands in. When we show file names,
+we will assume that those names are relative to your home directory.
+For example, my home directory is @file{/home/fsf/melissa}. All of
+my examples are in a subdirectory of the directory named by that file
name; the subdirectory is called @file{practice}.
@item
of three forms: long (mnemonic) form, short form, and old style. Some
of the operations and options have no short or ``old'' forms; however,
the operations and options which we will cover in this tutorial have
-corresponding abbreviations. @FIXME{make sure this is still the case,
-at the end}We will indicate those abbreviations appropriately to get
-you used to seeing them. (Note that the ``old style'' option forms
-exist in @GNUTAR{} for compatibility with Unix
+corresponding abbreviations. We will indicate those abbreviations
+appropriately to get you used to seeing them. (Note that the ``old
+style'' option forms exist in @GNUTAR{} for compatibility with Unix
@command{tar}. In this book we present a full discussion of this way
of writing options and operations (@pxref{Old Options}), and we discuss
the other two styles of writing options (@xref{Long Options}, and
two different ways. People sometimes refer to @command{tar} ``commands''.
A @command{tar} @dfn{command} is the entire command line of user input
which tells @command{tar} what to do --- including the operation, options,
-and any arguments (file names, pipes, other commands, etc). However,
+and any arguments (file names, pipes, other commands, etc.). However,
you will also sometimes hear the term ``the @command{tar} command''. When
the word ``command'' is used specifically like this, a person is usually
referring to the @command{tar} @emph{operation}, not the whole line.
@option{--verbose} to show the differences.
Each instance of @option{--verbose} on the command line increases the
-verbosity level by one, so if you need more details on the output,
+verbosity level by one, so if you need more details on the output,
specify it twice.
When reading archives (@option{--list}, @option{--extract},
default. So, a single @option{--verbose} option shows the file names
being added to the archive, while two @option{--verbose} options
enable the full listing.
-
+
For example, to create an archive in verbose mode:
@smallexample
@item Owner name and group separated by a slash character.
If these data are not available (for example, when listing a @samp{v7} format
-archive), numeric ID values are printed instead.
+archive), numeric @acronym{ID} values are printed instead.
@item Size of the file, in bytes.
The archive member is a GNU @dfn{volume header} (@pxref{Tape Files}).
@item --Continued at byte @var{n}--
-Encountered only at the beginning of a multy-volume archive
+Encountered only at the beginning of a multi-volume archive
(@pxref{Using Multiple Tapes}). This archive member is a continuation
from the previous volume. The number @var{n} gives the offset where
-the original file was split.
-
-@item --Mangled file names--
-This archive member contains @dfn{mangled file names} declarations,
-a special member type that was used by early versions of @GNUTAR{}.
-You probably will never encounter this, unless you are reading a very
-old archive.
+the original file was split.
@item unknown file type @var{c}
An archive member of unknown type. @var{c} is the type character from
Now @command{cd} to the directory named @file{practice}; @file{practice}
is now your @dfn{working directory}. (@emph{Please note}: Although
-the full path name of this directory is
+the full file name of this directory is
@file{/@var{homedir}/practice}, in our examples we will refer to
this directory as @file{practice}; the @var{homedir} is presumed.
appear in the archive, as well as various attributes of the files at
the time they were archived. For example, you can examine the archive
@file{collection.tar} that you created in the last section with the
-command,
+command,
@smallexample
$ @kbd{tar --list --file=collection.tar}
names of members you identify. For example, @w{@kbd{tar --list
--file=afiles.tar apple}} would only print @file{apple}.
-Because @command{tar} preserves paths, file names must be specified as
+Because @command{tar} preserves file names, these must be specified as
they appear in the archive (i.e., relative to the directory from which
the archive was created). Therefore, it is essential when specifying
member names to @command{tar} that you give the exact member names.
@file{collection.tar} earlier (say, @file{blues}), you can extract it
from the archive without changing the archive's structure. Its
contents will be identical to the original file @file{blues} that you
-deleted.
+deleted.
First, make sure you are in the @file{practice} directory, and list the
files in the directory. Now, delete the file, @samp{blues}, and list
command line arguments as globbing patterns and @option{--no-anchored}
informs it that the patterns apply to member names after any @samp{/}
delimiter. The use of globbing patterns is discussed in detail in
-@xref{wildcards}.
+@xref{wildcards}.
You can extract a file to standard output by combining the above options
with the @option{--to-stdout} (@option{-O}) option (@pxref{Writing to Standard
@node going further
@section Going Further Ahead in this Manual
+@UNREVISED
@FIXME{need to write up a node here about the things that are going to
be in the rest of the manual.}
* help::
* defaults::
* verbose::
+* checkpoints::
* interactive::
@end menu
clearly diagnosed on @code{stderr}, after a line stating the nature of
the error.
-@GNUTAR{} returns only a few exit statuses. I'm really
-aiming simplicity in that area, for now. If you are not using the
-@option{--compare} @option{--diff}, @option{-d}) option, zero means
-that everything went well, besides maybe innocuous warnings. Nonzero
-means that something went wrong. Right now, as of today, ``nonzero''
-is almost always 2, except for remote operations, where it may be
-128.
+Possible exit codes of @GNUTAR{} are summarized in the following
+table:
+
+@table @asis
+@item 0
+@samp{Successful termination}.
+
+@item 1
+@samp{Some files differ}. If tar was invoked with @option{--compare}
+(@option{--diff}, @option{-d}) command line option, this means that
+some files in the archive differ from their disk counterparts
+(@pxref{compare}). If tar was given @option{--create},
+@option{--append} or @option{--update} option, this exit code means
+that some files were changed while being archived and so the resulting
+archive does not contain the exact copy of the file set.
+
+@item 2
+@samp{Fatal error}. This means that some fatal, unrecoverable error
+occurred.
+@end table
+
+If @command{tar} has invoked a subprocess and that subprocess exited with a
+nonzero exit code, @command{tar} exits with that code as well.
+This can happen, for example, if @command{tar} was given some
+compression option (@pxref{gzip}) and the external compressor program
+failed. Another example is @command{rmt} failure during backup to the
+remote device (@pxref{Remote Tape Server}).
@node using tar options
@section Using @command{tar} Options
most long and short forms, they do not have old style equivalent. The
rules for specifying an argument for such options are stricter than
those for specifying mandatory arguments. Please, pay special
-attention to them.
+attention to them.
@menu
* Long Options:: Long Option Style
Each option has at least one @dfn{long} (or @dfn{mnemonic}) name starting with two
dashes in a row, e.g., @option{--list}. The long names are more clear than
their corresponding short or old names. It sometimes happens that a
-single long option has many different different names which are
+single long option has many different names which are
synonymous, such as @option{--compare} and @option{--diff}. In addition,
long option names can be given unique abbreviations. For example,
@option{--cre} can be used in place of @option{--create} because there is no
available on some systems. However, mounting typically requires
superuser privileges and can be a pain to manage.
+@opsummary{auto-compress}
+@item --auto-compress
+@itemx -a
+
+During a @option{--create} operation, enables automatic compressed
+format recognition based on the archive suffix. The effect of this
+option is cancelled by @option{--no-auto-compress}. @xref{gzip}.
+
@opsummary{backup}
@item --backup=@var{backup-type}
This option tells @command{tar} to read or write archives through
@code{bzip2}. @xref{gzip}.
+@opsummary{check-device}
+@item --check-device
+Check device numbers when creating a list of modified files for
+incremental archiving. This is the default. @xref{device numbers},
+for a detailed description.
+
@opsummary{checkpoint}
@item --checkpoint[=@var{number}]
This option directs @command{tar} to print periodic checkpoint
messages as it reads through the archive. It is intended for when you
want a visual indication that @command{tar} is still running, but
-don't want to see @option{--verbose} output. For a detailed
-description, see @ref{Progress information}.
+don't want to see @option{--verbose} output. You can also instruct
+@command{tar} to execute a list of actions on each checkpoint, see
+@option{--checklist-action} below. For a detailed description, see
+@ref{checkpoints}.
+
+@opsummary{checkpoint-action}
+@item --checkpoint-action=@var{action}
+Instruct @command{tar} to execute an action upon hitting a
+breakpoint. Here we give only a brief outline. @xref{checkpoints},
+for a complete description.
+
+The @var{action} argument can be one of the following:
+
+@table @asis
+@item bell
+Produce an audible bell on the console.
+
+@item dot
+@itemx .
+Print a single dot on the standard listing stream.
+
+@item echo
+Display a textual message on the standard error, with the status and
+number of the checkpoint. This is the default.
+
+@item echo=@var{string}
+Display @var{string} on the standard error. Before output, the string
+is subject to meta-character expansion.
+
+@item exec=@var{command}
+Execute the given @var{command}.
+
+@item sleep=@var{time}
+Wait for @var{time} seconds.
+
+@item ttyout=@var{string}
+Output @var{string} on the current console (@file{/dev/tty}).
+@end table
+
+Several @option{--checkpoint-action} options can be specified. The
+supplied actions will be executed in order of their appearance in the
+command line.
+
+Using @option{--checkpoint-action} without @option{--checkpoint}
+assumes default checkpoint frequency of one checkpoint per 10 records.
@opsummary{check-links}
@item --check-links
output @footnote{Earlier versions of @GNUTAR{} understood @option{-l} as a
synonym for @option{--one-file-system}. The current semantics, which
complies to UNIX98, was introduced with version
-1.15.91. @xref{Changes}, for more information.}.
+1.15.91. @xref{Changes}, for more information.}.
+
+@xref{hard links}.
@opsummary{compress}
@opsummary{uncompress}
@opsummary{exclude-caches}
@item --exclude-caches
-Automatically excludes all directories
-containing a cache directory tag. @xref{exclude}.
+Exclude from dump any directory containing a valid cache directory
+tag file, but still dump the directory node and the tag file itself.
+
+@xref{exclude}.
+
+@opsummary{exclude-caches-under}
+@item --exclude-caches-under
+
+Exclude from dump any directory containing a valid cache directory
+tag file, but still dump the directory node itself.
+
+@xref{exclude}.
+
+@opsummary{exclude-caches-all}
+@item --exclude-caches-all
+
+Exclude from dump any directory containing a valid cache directory
+tag file. @xref{exclude}.
+
+@opsummary{exclude-tag}
+@item --exclude-tag=@var{file}
+
+Exclude from dump any directory containing file named @var{file}, but
+dump the directory node and @var{file} itself. @xref{exclude}.
+
+@opsummary{exclude-tag-under}
+@item --exclude-tag-under=@var{file}
+
+Exclude from dump the contents of any directory containing file
+named @var{file}, but dump the directory node itself. @xref{exclude}.
+
+@opsummary{exclude-tag-all}
+@item --exclude-tag-all=@var{file}
+
+Exclude from dump any directory containing file named @var{file}.
+@xref{exclude}.
+
+@opsummary{exclude-vcs}
+@item --exclude-vcs
+
+Exclude from dump directories and files, that are internal for some
+widely used version control systems.
+
+@xref{exclude}.
@opsummary{file}
@item --file=@var{archive}
@opsummary{force-local}
@item --force-local
-Forces @command{tar} to interpret the filename given to @option{--file}
+Forces @command{tar} to interpret the file name given to @option{--file}
as a local file, even if it looks like a remote tape drive name.
@xref{local and remote archives}.
@opsummary{group}
@item --group=@var{group}
-Files added to the @command{tar} archive will have a group id of @var{group},
+Files added to the @command{tar} archive will have a group @acronym{ID} of @var{group},
rather than the group from the source file. @var{group} is first decoded
as a group symbolic name, but if this interpretation fails, it has to be
-a decimal numeric group ID. @xref{override}.
+a decimal numeric group @acronym{ID}. @xref{override}.
Also see the comments for the @option{--owner=@var{user}} option.
@command{gzip}, allowing @command{tar} to directly operate on several
kinds of compressed archives transparently. @xref{gzip}.
+@opsummary{hard-dereference}
+@item --hard-dereference
+When creating an archive, dereference hard links and store the files
+they refer to, instead of creating usual hard link members.
+
+@xref{hard links}.
+
@opsummary{help}
@item --help
@itemx -?
@opsummary{ignore-case}
@item --ignore-case
Ignore case when matching member or file names with
-patterns. @xref{controlling pattern-matching}.
+patterns. @xref{controlling pattern-matching}.
@opsummary{ignore-command-error}
@item --ignore-command-error
@item --incremental
@itemx -G
-Used to inform @command{tar} that it is working with an old
+Informs @command{tar} that it is working with an old
@acronym{GNU}-format incremental backup archive. It is intended
primarily for backwards compatibility only. @xref{Incremental Dumps},
for a detailed discussion of incremental archives.
With other operations, informs @command{tar} that the archive is in
incremental format. @xref{Incremental Dumps}.
+@opsummary{lzma}
+@item --lzma
+@itemx -J
+
+This option tells @command{tar} to read or write archives through
+@command{lzma}. @xref{gzip}.
+
+@item --lzop
+
+This option tells @command{tar} to read or write archives through
+@command{lzop}. @xref{gzip}.
+
@opsummary{mode}
@item --mode=@var{permissions}
(see --info-script)
-@opsummary{seek}
-@item --seek
-@itemx -n
-
-Assume that the archive media supports seeks to arbitrary
-locations. Usually @command{tar} determines automatically whether
-the archive can be seeked or not. This option is intended for use
-in cases when such recognition fails.
-
@opsummary{newer}
@item --newer=@var{date}
@itemx --after-date=@var{date}
An exclude pattern can match any subsequence of the name's components.
@xref{controlling pattern-matching}.
+@opsummary{no-auto-compress}
+@item --no-auto-compress
+
+Disables automatic compressed format recognition based on the archive
+suffix. @xref{--auto-compress}. @xref{gzip}.
+
+@opsummary{no-check-device}
+@item --no-check-device
+Do not check device numbers when creating a list of modified files
+for incremental archiving. @xref{device numbers}, for
+a detailed description.
+
@opsummary{no-delay-directory-restore}
@item --no-delay-directory-restore
-Setting modification times and permissions of extracted
-directories when all files from this directory has been
-extracted. This is the default. @xref{Directory Modification Times and Permissions}.
+Modification times and permissions of extracted
+directories are set when all files from this directory have been
+extracted. This is the default.
+@xref{Directory Modification Times and Permissions}.
@opsummary{no-ignore-case}
@item --no-ignore-case
@opsummary{no-ignore-command-error}
@item --no-ignore-command-error
-Print warnings about subprocesses terminated with a non-zero exit
+Print warnings about subprocesses that terminated with a nonzero exit
code. @xref{Writing to an External Program}.
@opsummary{no-overwrite-dir}
@item --null
When @command{tar} is using the @option{--files-from} option, this option
-instructs @command{tar} to expect filenames terminated with @option{NUL}, so
+instructs @command{tar} to expect file names terminated with @acronym{NUL}, so
@command{tar} can correctly work with file names that contain newlines.
@xref{nul}.
@item -o
The function of this option depends on the action @command{tar} is
performing. When extracting files, @option{-o} is a synonym for
-@option{--no-same-owner}, i.e. it prevents @command{tar} from
+@option{--no-same-owner}, i.e., it prevents @command{tar} from
restoring ownership of files being extracted.
When creating an archive, it is a synonym for
@option{--old-archive}. This behavior is for compatibility
with previous versions of @GNUTAR{}, and will be
-removed in the future releases.
+removed in future releases.
@xref{Changes}, for more information.
@item --one-file-system
Used when creating an archive. Prevents @command{tar} from recursing into
directories that are on different file systems from the current
-directory @footnote{Earlier versions of @GNUTAR{} understood @option{-l} as a
-synonym for @option{--one-file-system}. This has changed in version
-1.15.91. @xref{Changes}, for more information.}.
+directory.
@opsummary{overwrite}
@item --overwrite
Specifies that @command{tar} should use @var{user} as the owner of members
when creating archives, instead of the user associated with the source
file. @var{user} is first decoded as a user symbolic name, but if
-this interpretation fails, it has to be a decimal numeric user ID.
+this interpretation fails, it has to be a decimal numeric user @acronym{ID}.
@xref{override}.
This option does not affect extraction from archives.
-@opsummary{transform}
-@item --transform=@var{sed-expr}
-
-Transform file or member names using @command{sed} replacement expression
-@var{sed-expr}. For example,
-
-@smallexample
-$ @kbd{tar cf archive.tar --transform 's,^\./,usr/,' .}
-@end smallexample
-
-@noindent
-will add to @file{archive} files from the current working directory,
-replacing initial @samp{./} prefix with @samp{usr/}. For the detailed
-discussion, @xref{transform}.
-
-To see transformed member names in verbose listings, use
-@option{--show-transformed-names} option
-(@pxref{show-transformed-names}).
-
-@opsummary{quote-chars}
-@item --quote-chars=@var{string}
-Always quote characters from @var{string}, even if the selected
-quoting style would not quote them (@pxref{quoting styles}).
-
-@opsummary{quoting-style}
-@item --quoting-style=@var{style}
-Set quoting style to use when printing member and file names
-(@pxref{quoting styles}). Valid @var{style} values are:
-@code{literal}, @code{shell}, @code{shell-always}, @code{c},
-@code{escape}, @code{locale}, and @code{clocale}. Default quoting
-style is @code{escape}, unless overridden while configuring the
-package.
-
@opsummary{pax-option}
@item --pax-option=@var{keyword-list}
This option is meaningful only with @acronym{POSIX.1-2001} archives
Specifying this option instructs @command{tar} that it should use the
permissions directly from the archive. @xref{Setting Access Permissions}.
+@opsummary{quote-chars}
+@item --quote-chars=@var{string}
+Always quote characters from @var{string}, even if the selected
+quoting style would not quote them (@pxref{quoting styles}).
+
+@opsummary{quoting-style}
+@item --quoting-style=@var{style}
+Set quoting style to use when printing member and file names
+(@pxref{quoting styles}). Valid @var{style} values are:
+@code{literal}, @code{shell}, @code{shell-always}, @code{c},
+@code{escape}, @code{locale}, and @code{clocale}. Default quoting
+style is @code{escape}, unless overridden while configuring the
+package.
+
@opsummary{read-full-records}
@item --read-full-records
@itemx -B
@opsummary{recursion}
@item --recursion
-With this option, @command{tar} recurses into directories.
+With this option, @command{tar} recurses into directories (default).
@xref{recurse}.
@opsummary{recursive-unlink}
@item --restrict
Disable use of some potentially harmful @command{tar} options.
-Currently this option disables shell invocaton from multi-volume menu
+Currently this option disables shell invocation from multi-volume menu
(@pxref{Using Multiple Tapes}).
@opsummary{rmt-command}
(See @option{--preserve-permissions}; @pxref{Setting Access Permissions}.)
+@opsummary{seek}
+@item --seek
+@itemx -n
+
+Assume that the archive media supports seeks to arbitrary
+locations. Usually @command{tar} determines automatically whether
+the archive can be seeked or not. This option is intended for use
+in cases when such recognition fails.
+
@opsummary{show-defaults}
@item --show-defaults
@opsummary{show-omitted-dirs}
@item --show-omitted-dirs
-Instructs @command{tar} to mention directories its skipping over when
+Instructs @command{tar} to mention the directories it is skipping when
operating on a @command{tar} archive. @xref{show-omitted-dirs}.
@opsummary{show-transformed-names}
Display file or member names after applying any transformations
(@pxref{transform}). In particular, when used in conjunction with one of
-archive creation operations it instructs tar to list the member names
-stored in the archive, as opposed to the actual file
+the archive creation operations it instructs @command{tar} to list the
+member names stored in the archive, as opposed to the actual file
names. @xref{listing member and file names}.
@opsummary{sparse}
Invokes a @acronym{GNU} extension when adding files to an archive that handles
sparse files efficiently. @xref{sparse}.
+@opsummary{sparse-version}
+@item --sparse-version=@var{version}
+
+Specifies the @dfn{format version} to use when archiving sparse
+files. Implies @option{--sparse}. @xref{sparse}. For the description
+of the supported sparse formats, @xref{Sparse Formats}.
+
@opsummary{starting-file}
@item --starting-file=@var{name}
@itemx -K @var{name}
@opsummary{strip-components}
@item --strip-components=@var{number}
Strip given @var{number} of leading components from file names before
-extraction.@footnote{This option was called @option{--strip-path} in
-version 1.14.} For example, if archive @file{archive.tar} contained
+extraction. For example, if archive @file{archive.tar} contained
@file{/some/file/name}, then running
@smallexample
rather than the data modification time stored in the archive.
@xref{Data Modification Times}.
+@opsummary{transform}
+@item --transform=@var{sed-expr}
+
+Transform file or member names using @command{sed} replacement expression
+@var{sed-expr}. For example,
+
+@smallexample
+$ @kbd{tar cf archive.tar --transform 's,^\./,usr/,' .}
+@end smallexample
+
+@noindent
+will add to @file{archive} files from the current working directory,
+replacing initial @samp{./} prefix with @samp{usr/}. For the detailed
+discussion, @xref{transform}.
+
+To see transformed member names in verbose listings, use
+@option{--show-transformed-names} option
+(@pxref{show-transformed-names}).
+
@opsummary{uncompress}
@item --uncompress
@item --verbose
@itemx -v
-Specifies that @command{tar} should be more verbose about the operations its
-performing. This option can be specified multiple times for some
-operations to increase the amount of information displayed.
+Specifies that @command{tar} should be more verbose about the
+operations it is performing. This option can be specified multiple
+times for some operations to increase the amount of information displayed.
@xref{verbose}.
@opsummary{verify}
@item --volno-file=@var{file}
Used in conjunction with @option{--multi-volume}. @command{tar} will
-keep track of which volume of a multi-volume archive its working in
+keep track of which volume of a multi-volume archive it is working in
@var{file}. @xref{volno-file}.
@opsummary{wildcards}
@item -G @tab @ref{--incremental}.
+@item -J @tab @ref{--lzma}.
+
@item -K @tab @ref{--starting-file}.
@item -L @tab @ref{--tape-length}.
@item -m @tab @ref{--touch}.
@item -o @tab When creating, @ref{--no-same-owner}, when extracting ---
-@ref{--portability}.
+@ref{--portability}.
-The later usage is deprecated. It is retained for compatibility with
-the earlier versions of @GNUTAR{}. In the future releases
+The latter usage is deprecated. It is retained for compatibility with
+the earlier versions of @GNUTAR{}. In future releases
@option{-o} will be equivalent to @option{--no-same-owner} only.
@item -p @tab @ref{--preserve-permissions}.
@smallexample
tar (GNU tar) @value{VERSION}
-Copyright (C) 2006 Free Software Foundation, Inc.
+Copyright (C) 2008 Free Software Foundation, Inc.
This is free software. You may redistribute copies of it under the terms
of the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.
@opindex show-defaults
@GNUTAR{} has some predefined defaults that are used when you do not
-explicitely specify another values. To obtain a list of such
+explicitly specify another values. To obtain a list of such
defaults, use @option{--show-defaults} option. This will output the
values in the form of @command{tar} command line options:
@smallexample
@group
@kbd{tar --show-defaults}
---format=gnu -f- -b20 --quoting-style=escape
+--format=gnu -f- -b20 --quoting-style=escape
--rmt-command=/etc/rmt --rsh-command=/usr/bin/rsh
@end group
@end smallexample
Print statistics upon delivery of signal @var{signo}. Valid arguments
are: @code{SIGHUP}, @code{SIGQUIT}, @code{SIGINT}, @code{SIGUSR1} and
@code{SIGUSR2}. Shortened names without @samp{SIG} prefix are also
-accepted.
+accepted.
@end table
Both forms of @option{--totals} option can be used simultaneously.
Thus, @kbd{tar -x --totals --totals=USR1} instructs @command{tar} to
extract all members from its default archive and print statistics
after finishing the extraction, as well as when receiving signal
-@code{SIGUSR1}.
+@code{SIGUSR1}.
@anchor{Progress information}
@cindex Progress information
-@opindex checkpoint
The @option{--checkpoint} option prints an occasional message
as @command{tar} reads or writes the archive. It is designed for
those who don't need the more detailed (and voluminous) output of
This example shows the default checkpoint message used by
@command{tar}. If you place a dot immediately after the equal
-sign, it will print a @samp{.} at each checkpoint. For example:
+sign, it will print a @samp{.} at each checkpoint@footnote{This is
+actually a shortcut for @option{--checkpoint=@var{n}
+--checkpoint-action=dot}. @xref{checkpoints, dot}.}. For example:
@smallexample
$ @kbd{tar -c --checkpoint=.1000} /var
...
@end smallexample
+The @option{--checkpoint} option provides a flexible mechanism for
+executing arbitrary actions upon hitting checkpoints, see the next
+section (@pxref{checkpoints}), for more information on it.
+
@opindex show-omitted-dirs
@anchor{show-omitted-dirs}
The @option{--show-omitted-dirs} option, when reading an archive---with
favor of the tape where the file appears earliest (closest to the
front of the tape). @xref{backup}.
+@node checkpoints
+@section Checkpoints
+@cindex checkpoints, defined
+@opindex checkpoint
+@opindex checkpoint-action
+
+A @dfn{checkpoint} is a moment of time before writing @var{n}th record to
+the archive (a @dfn{write checkpoint}), or before reading @var{n}th record
+from the archive (a @dfn{read checkpoint}). Checkpoints allow to
+periodically execute arbitrary actions.
+
+The checkpoint facility is enabled using the following option:
+
+@table @option
+@xopindex{checkpoint, defined}
+@item --checkpoint[=@var{n}]
+Schedule checkpoints before writing or reading each @var{n}th record.
+The default value for @var{n} is 10.
+@end table
+
+A list of arbitrary @dfn{actions} can be executed at each checkpoint.
+These actions include: pausing, displaying textual messages, and
+executing arbitrary external programs. Actions are defined using
+the @option{--checkpoint-action} option.
+
+@table @option
+@xopindex{checkpoint-action, defined}
+@item --checkpoint-action=@var{action}
+Execute an @var{action} at each checkpoint.
+@end table
+
+@cindex @code{echo}, checkpoint action
+The simplest value of @var{action} is @samp{echo}. It instructs
+@command{tar} to display the default message on the standard error
+stream upon arriving at each checkpoint. The default message is (in
+@acronym{POSIX} locale) @samp{Write checkpoint @var{n}}, for write
+checkpoints, and @samp{Read checkpoint @var{n}}, for read checkpoints.
+Here, @var{n} represents ordinal number of the checkpoint.
+
+In another locales, translated versions of this message are used.
+
+This is the default action, so running:
+
+@smallexample
+$ @kbd{tar -c --checkpoint=1000 --checkpoint-action=echo} /var
+@end smallexample
+
+@noindent
+is equivalent to:
+
+@smallexample
+$ @kbd{tar -c --checkpoint=1000} /var
+@end smallexample
+
+The @samp{echo} action also allows to supply a customized message.
+You do so by placing an equals sign and the message right after it,
+e.g.:
+
+@smallexample
+--checkpoint-action="echo=Hit %s checkpoint #%u"
+@end smallexample
+
+The @samp{%s} and @samp{%u} in the above example are
+@dfn{meta-characters}. The @samp{%s} meta-character is replaced with
+the @dfn{type} of the checkpoint: @samp{write} or
+@samp{read} (or a corresponding translated version in locales other
+than @acronym{POSIX}). The @samp{%u} meta-character is replaced with
+the ordinal number of the checkpoint. Thus, the above example could
+produce the following output when used with the @option{--create}
+option:
+
+@smallexample
+tar: Hit write checkpoint #10
+tar: Hit write checkpoint #20
+tar: Hit write checkpoint #30
+@end smallexample
+
+Aside from meta-character expansion, the message string is subject to
+@dfn{unquoting}, during which the backslash @dfn{escape sequences} are
+replaced with their corresponding @acronym{ASCII} characters
+(@pxref{escape sequences}). E.g. the following action will produce an
+audible bell and the message described above at each checkpoint:
+
+@smallexample
+--checkpoint-action='echo=\aHit %s checkpoint #%u'
+@end smallexample
+
+@cindex @code{bell}, checkpoint action
+There is also a special action which produces an audible signal:
+@samp{bell}. It is not equivalent to @samp{echo='\a'}, because
+@samp{bell} sends the bell directly to the console (@file{/dev/tty}),
+whereas @samp{echo='\a'} sends it to the standard error.
+
+@cindex @code{ttyout}, checkpoint action
+The @samp{ttyout=@var{string}} action outputs @var{string} to
+@file{/dev/tty}, so it can be used even if the standard output is
+redirected elsewhere. The @var{string} is subject to the same
+modifications as with @samp{echo} action. In contrast to the latter,
+@samp{ttyout} does not prepend @command{tar} executable name to the
+string, nor does it output a newline after it. For example, the
+following action will print the checkpoint message at the same screen
+line, overwriting any previous message:
+
+@smallexample
+--checkpoint-action="ttyout=\rHit %s checkpoint #%u"
+@end smallexample
+
+@cindex @code{dot}, checkpoint action
+Another available checkpoint action is @samp{dot} (or @samp{.}). It
+instructs @command{tar} to print a single dot on the standard listing
+stream, e.g.:
+
+@smallexample
+$ @kbd{tar -c --checkpoint=1000 --checkpoint-action=dot} /var
+...
+@end smallexample
+
+For compatibility with previous @GNUTAR{} versions, this action can
+be abbreviated by placing a dot in front of the checkpoint frequency,
+as shown in the previous section.
+
+@cindex @code{sleep}, checkpoint action
+Yet another action, @samp{sleep}, pauses @command{tar} for a specified
+amount of seconds. The following example will stop for 30 seconds at each
+checkpoint:
+
+@smallexample
+$ @kbd{tar -c --checkpoint=1000 --checkpoint-action=sleep=30}
+@end smallexample
+
+@cindex @code{exec}, checkpoint action
+Finally, the @code{exec} action executes a given external program.
+For example:
+
+@smallexample
+$ @kbd{tar -c --checkpoint=1000 --checkpoint-action=exec=/sbin/cpoint}
+@end smallexample
+
+This program is executed using @command{/bin/sh -c}, with no
+additional arguments. Its exit code is ignored. It gets a copy of
+@command{tar}'s environment plus the following variables:
+
+@table @env
+@vrindex TAR_VERSION, checkpoint script environment
+@item TAR_VERSION
+@GNUTAR{} version number.
+
+@vrindex TAR_ARCHIVE, checkpoint script environment
+@item TAR_ARCHIVE
+The name of the archive @command{tar} is processing.
+
+@vrindex TAR_BLOCKING_FACTOR, checkpoint script environment
+@item TAR_BLOCKING_FACTOR
+Current blocking factor (@pxref{Blocking}.
+
+@vrindex TAR_CHECKPOINT, checkpoint script environment
+@item TAR_CHECKPOINT
+The checkpoint number.
+
+@vrindex TAR_SUBCOMMAND, checkpoint script environment
+@item TAR_SUBCOMMAND
+A short option describing the operation @command{tar} is executing
+@xref{Operations}, for a complete list of subcommand options.
+
+@vrindex TAR_FORMAT, checkpoint script environment
+@item TAR_FORMAT
+Format of the archive being processed. @xref{Formats}, for a complete
+list of archive format names.
+@end table
+
+Any number of actions can be defined, by supplying several
+@option{--checkpoint-action} options in the command line. For
+example, the command below displays two messages, pauses
+execution for 30 seconds and executes the @file{/sbin/cpoint} script:
+
+@example
+@group
+$ @kbd{tar -c -f arc.tar \
+ --checkpoint-action='\aecho=Hit %s checkpoint #%u' \
+ --checkpoint-action='echo=Sleeping for 30 seconds' \
+ --checkpoint-action='sleep=30' \
+ --checkpoint-action='exec=/sbin/cpoint'}
+@end group
+@end example
+
+This example also illustrates the fact that
+@option{--checkpoint-action} can be used without
+@option{--checkpoint}. In this case, the default checkpoint frequency
+(at each 10th record) is assumed.
+
@node interactive
@section Asking for Confirmation During Operations
@cindex Interactive operation
file, which was meant to be saved, is rather destroyed.
@end enumerate
-So, recognizing the likelihood and the catastrophical nature of these
+So, recognizing the likelihood and the catastrophic nature of these
errors, @GNUTAR{} now takes some distance from elegance, and
cowardly refuses to create an archive when @option{--create} option is
given, there are no arguments besides options, and
archive is extracted, a file archived later in time will replace a
file of the same name which was archived earlier, even though the
older version of the file will remain in the archive unless you delete
-all versions of the file.
+all versions of the file.
Supposing you change the file @file{blues} and then append the changed
version to @file{collection.tar}. As you saw above, the original
The spirit behind the @option{--compare} (@option{--diff},
@option{-d}) option is to check whether the archive represents the
current state of files on disk, more than validating the integrity of
-the archive media. For this later goal, @xref{verify}.
+the archive media. For this later goal, @xref{verify}.
@node create options
@section Options Used by @option{--create}
Specifies that @command{tar} should use @var{user} as the owner of members
when creating archives, instead of the user associated with the source
file. The argument @var{user} can be either an existing user symbolic
-name, or a decimal numeric user ID.
+name, or a decimal numeric user @acronym{ID}.
There is no value indicating a missing number, and @samp{0} usually means
@code{root}. Some people like to force @samp{0} as the value to offer in
@item --group=@var{group}
@opindex group
-Files added to the @command{tar} archive will have a group id of @var{group},
+Files added to the @command{tar} archive will have a group @acronym{ID} of @var{group},
rather than the group from the source file. The argument @var{group}
-can be either an existing group symbolic name, or a decimal numeric group ID.
+can be either an existing group symbolic name, or a decimal numeric group @acronym{ID}.
@end table
@node Ignore Failed Read
The @option{--read-full-records} (@option{-B}) option is turned on by default when
@command{tar} reads an archive from standard input, or from a remote
-machine. This is because on BSD Unix systems, attempting to read a
+machine. This is because on @acronym{BSD} Unix systems, attempting to read a
pipe returns however much happens to be in the pipe, even if it is
less than was requested. If this option were not enabled, @command{tar}
would fail as soon as it read an incomplete record from the pipe.
versions of @command{tar} write garbage after the end-of-archive entry,
since that part of the media is never supposed to be read. @GNUTAR{}
does not write after the end of an archive, but seeks to
-maintain compatiblity among archiving utilities.
+maintain compatibility among archiving utilities.
@table @option
@item --ignore-zeros
To set the modes (access permissions) of extracted files to those
recorded for those files in the archive, use @option{--same-permissions}
in conjunction with the @option{--extract} (@option{--get},
-@option{-x}) operation.
+@option{-x}) operation.
@table @option
@opindex preserve-permissions
@node Directory Modification Times and Permissions
@unnumberedsubsubsec Directory Modification Times and Permissions
-After sucessfully extracting a file member, @GNUTAR{} normally
+After successfully extracting a file member, @GNUTAR{} normally
restores its permissions and modification times, as described in the
previous sections. This cannot be done for directories, because
after extracting a directory @command{tar} will almost certainly
an incremental archive is reversed: first all directory members are
stored, followed by other (non-directory) members. So, when extracting
from incremental archives, @GNUTAR{} alters the above procedure. It
-remebers all restored directories, and restores their meta-data
+remembers all restored directories, and restores their meta-data
only after the entire archive has been processed. Notice, that you do
-not need to specity any special options for that, as @GNUTAR{}
+not need to specify any special options for that, as @GNUTAR{}
automatically detects archives in incremental format.
There may be cases, when such processing is required for normal archives
tar -xOzf foo.tgz bigfile1 bigfile2 | process
@end smallexample
-Hovewer, @option{--to-command} may be more convenient for use with
+However, @option{--to-command} may be more convenient for use with
multiple files. See the next section.
@node Writing to an External Program
The command can obtain the information about the file it processes
from the following environment variables:
-@table @var
+@table @env
@vrindex TAR_FILETYPE, to-command environment
@item TAR_FILETYPE
Type of the file. It is a single letter with the following meaning:
with the @option{--atime-preserve=replace} option), or if you set the clock
backwards.
+@anchor{device numbers}
+@cindex Device numbers, using in incremental backups
Metadata stored in snapshot files include device numbers, which,
-obviously is supposed to be a non-volatile value. However, it turns
-out that NFS devices have undependable values when an automounter
+obviously are supposed to be a non-volatile values. However, it turns
+out that @acronym{NFS} devices have undependable values when an automounter
gets in the picture. This can lead to a great deal of spurious
redumping in incremental dumps, so it is somewhat useless to compare
-two NFS devices numbers over time. The solution implemented currently
-is to considers all NFS devices as being equal when it comes to
-comparing directories; this is fairly gross, but there does not seem
-to be a better way to go.
+two @acronym{NFS} devices numbers over time. The solution implemented
+currently is to considers all @acronym{NFS} devices as being equal
+when it comes to comparing directories; this is fairly gross, but
+there does not seem to be a better way to go.
+
+Apart from using @acronym{NFS}, there are a number of cases where
+relying on device numbers can cause spurious redumping of unmodified
+files. For example, this occurs when archiving @acronym{LVM} snapshot
+volumes. To avoid this, use @option{--no-check-device} option:
+
+@table @option
+@xopindex{no-check-device, described}
+@item --no-check-device
+Do not rely on device numbers when preparing a list of changed files
+for an incremental dump.
+
+@xopindex{check-device, described}
+@item --check-device
+Use device numbers when preparing a list of changed files
+for an incremental dump. This is the default behavior. The purpose
+of this option is to undo the effect of the @option{--no-check-device}
+if it was given in @env{TAR_OPTIONS} environment variable
+(@pxref{TAR_OPTIONS}).
+@end table
+
+There is also another way to cope with changing device numbers. It is
+described in detail in @ref{Fixing Snapshot Files}.
Note that incremental archives use @command{tar} extensions and may
not be readable by non-@acronym{GNU} versions of the @command{tar} program.
the last level was created, you will need to restore from all backups
in turn. Continuing our example, to restore the state of @file{/usr}
file system, one would do@footnote{Notice, that since both archives
-were created withouth @option{-P} option (@pxref{absolute}), these
+were created without @option{-P} option (@pxref{absolute}), these
commands should be run from the root file system.}:
@smallexample
contents of the DUMPDIR header (with terminating nulls) when
@option{--incremental} or @option{--listed-incremental} option was
given, no matter what the verbosity level. This behavior, and,
-especially, the binary output it produced were considered incovenient
+especially, the binary output it produced were considered inconvenient
and were changed in version 1.16}:
@smallexample
the host machine must have @GNUTAR{} installed, and
must be able to access the directory containing the backup scripts and
their support files using the same file name that is used on the
-machine where the scripts are run (i.e. what @command{pwd} will print
+machine where the scripts are run (i.e., what @command{pwd} will print
when in that directory on that machine). If the host that contains
the file system does not have this capability, you can specify another
-host as long as it can access the file system through NFS.
+host as long as it can access the file system through @acronym{NFS}.
If the list of file systems is very long you may wish to put it
in a separate file. This file is usually named
@defvr {Backup variable} DIRLIST
-A path to the file containing the list of the file systems to backup
+The name of the file that contains a list of file systems to backup
or restore. By default it is @file{/etc/backup/dirs}.
@end defvr
@defvr {Backup variable} FILELIST
-A path to the file containing the list of the individual files to backup
+The name of the file that contains a list of individual files to backup
or restore. By default it is @file{/etc/backup/files}.
@end defvr
@defvr {Backup variable} RSH_COMMAND
-Full file name of @command{rsh} binary on remote mashines. This will
+Full file name of @command{rsh} binary on remote machines. This will
be passed via @option{--rsh-command} option to the remote invocation
of @GNUTAR{}.
@end defvr
Name or IP address of the host machine being dumped or restored.
@item fs
-Full path name to the file system being dumped or restored.
+Full file name of the file system being dumped or restored.
@item fsname
File system name with directory separators replaced with colons. This
@item -v[@var{level}]
@itemx --verbose[=@var{level}]
Set verbosity level. The higher the level is, the more debugging
-information will be output during execution. Devault @var{level}
+information will be output during execution. Default @var{level}
is 100, which means the highest debugging level.
@item -t @var{start-time}
@item -v[@var{level}]
@itemx --verbose[=@var{level}]
Set verbosity level. The higher the level is, the more debugging
-information will be output during execution. Devault @var{level}
+information will be output during execution. Default @var{level}
is 100, which means the highest debugging level.
@item -h
If you do not name the archive, @command{tar} uses the value of the
environment variable @env{TAPE} as the file name for the archive. If
that is not available, @command{tar} uses a default, compiled-in archive
-name, usually that for tape unit zero (i.e. @file{/dev/tu00}).
+name, usually that for tape unit zero (i.e., @file{/dev/tu00}).
@cindex Standard input and output
@cindex tar to standard input and output
When the archive is being created to @file{/dev/null}, @GNUTAR{}
tries to minimize input and output operations. The Amanda backup
system, when used with @GNUTAR{}, has an initial sizing pass which
-uses this feature.
+uses this feature.
@node Selecting Archive Members
@section Selecting Archive Members
table:
@multitable @columnfractions 0.20 0.60
-@headitem Escape @tab Replaced with
-@item \a @tab Audible bell (ASCII 7)
-@item \b @tab Backspace (ASCII 8)
-@item \f @tab Form feed (ASCII 12)
-@item \n @tab New line (ASCII 10)
-@item \r @tab Carriage return (ASCII 13)
-@item \t @tab Horizontal tabulation (ASCII 9)
-@item \v @tab Vertical tabulation (ASCII 11)
-@item \? @tab ASCII 127
-@item \@var{n} @tab ASCII @var{n} (@var{n} should be an octal number
+@headitem Escape @tab Replaced with
+@item \a @tab Audible bell (@acronym{ASCII} 7)
+@item \b @tab Backspace (@acronym{ASCII} 8)
+@item \f @tab Form feed (@acronym{ASCII} 12)
+@item \n @tab New line (@acronym{ASCII} 10)
+@item \r @tab Carriage return (@acronym{ASCII} 13)
+@item \t @tab Horizontal tabulation (@acronym{ASCII} 9)
+@item \v @tab Vertical tabulation (@acronym{ASCII} 11)
+@item \? @tab @acronym{ASCII} 127
+@item \@var{n} @tab @acronym{ASCII} @var{n} (@var{n} should be an octal number
of up to 3 digits)
@end multitable
there are other ways to specify file or member names, or to modify the
manner in which @command{tar} selects the files or members upon which to
operate. In general, these methods work both for specifying the names
-of files and archive members.
+of files and archive members.
@node files
@section Reading Names from a File
line, you can put the names into a file, and then use the
@option{--files-from=@var{file-of-names}} (@option{-T
@var{file-of-names}}) option to @command{tar}. Give the name of the
-file which contains the list of files to include as the argument to
+file which contains the list of files to include as the argument to
@option{--files-from}. In the list, the file names should be separated by
newlines. You will frequently use this option when you have generated
the list of files to archive with the @command{find} utility.
@option{--files-from=@var{file-of-names}} (@option{-T @var{file-of-names}})
to read file names terminated by a @code{NUL} instead of a newline, so
files whose names contain newlines can be archived using
-@option{--files-from}.
+@option{--files-from}.
@table @option
@opindex null
@findex exclude
The @option{--exclude=@var{pattern}} option prevents any file or
member whose name matches the shell wildcard (@var{pattern}) from
-being operated on.
+being operated on.
For example, to create an archive with all the contents of the directory
@file{src} except for files whose names end in @file{.o}, use the
command @samp{tar -cf src.tar --exclude='*.o' src}.
single line @file{*.o}, no files whose names end in @file{.o} will be
added to the archive.
+Notice, that lines from @var{file} are read verbatim. One of the
+frequent errors is leaving some extra whitespace after a file name,
+which is difficult to catch using text editors.
+
+However, empty lines are OK.
+
+@cindex version control system, excluding files
+@cindex VCS, excluding files
+@cindex SCCS, excluding files
+@cindex RCS, excluding files
+@cindex CVS, excluding files
+@cindex SVN, excluding files
+@cindex git, excluding files
+@cindex Bazaar, excluding files
+@cindex Arch, excluding files
+@cindex Mercurial, excluding files
+@cindex Darcs, excluding files
@table @option
-@opindex exclude-caches
-@item --exclude-caches
-Causes @command{tar} to ignore directories containing a cache directory tag.
+@opindex exclude-vcs
+@item --exclude-vcs
+Exclude files and directories used by following version control
+systems: @samp{CVS}, @samp{RCS}, @samp{SCCS}, @samp{SVN}, @samp{Arch},
+@samp{Bazaar}, @samp{Mercurial}, and @samp{Darcs}.
@end table
+As of version @value{VERSION}, the following files are excluded:
+
+@itemize @bullet
+@item @file{CVS/}, and everything under it
+@item @file{RCS/}, and everything under it
+@item @file{SCCS/}, and everything under it
+@item @file{.git/}, and everything under it
+@item @file{.gitignore}
+@item @file{.cvsignore}
+@item @file{.svn/}, and everything under it
+@item @file{.arch-ids/}, and everything under it
+@item @file{@{arch@}/}, and everything under it
+@item @file{=RELEASE-ID}
+@item @file{=meta-update}
+@item @file{=update}
+@item @file{.bzr}
+@item @file{.bzrignore}
+@item @file{.bzrtags}
+@item @file{.hg}
+@item @file{.hgignore}
+@item @file{.hgrags}
+@item @file{_darcs}
+@end itemize
+
@findex exclude-caches
-When creating an archive, the @option{--exclude-caches} option causes
-@command{tar} to exclude all directories that contain a @dfn{cache
+When creating an archive, the @option{--exclude-caches} option family
+causes @command{tar} to exclude all directories that contain a @dfn{cache
directory tag}. A cache directory tag is a short file with the
well-known name @file{CACHEDIR.TAG} and having a standard header
specified in @url{http://www.brynosaurus.com/cachedir/spec.html}.
use to hold regenerable, non-precious data, so that such data can be
more easily excluded from backups.
-@menu
-* problems with exclude::
-@end menu
+There are three @samp{exclude-caches} options, each providing a different
+exclusion semantics:
-@node problems with exclude
-@unnumberedsubsec Problems with Using the @code{exclude} Options
+@table @option
+@opindex exclude-caches
+@item --exclude-caches
+Do not archive the contents of the directory, but archive the
+directory itself and the @file{CACHEDIR.TAG} file.
-@xopindex{exclude, potential problems with}
-Some users find @samp{exclude} options confusing. Here are some common
-pitfalls:
+@opindex exclude-caches-under
+@item --exclude-caches-under
+Do not archive the contents of the directory, nor the
+@file{CACHEDIR.TAG} file, archive only the directory itself.
-@itemize @bullet
+@opindex exclude-caches-all
+@item --exclude-caches-all
+Omit directories containing @file{CACHEDIR.TAG} file entirely.
+@end table
+
+@findex exclude-tag
+Another option family, @option{--exclude-tag}, provides a generalization of
+this concept. It takes a single argument, a file name to look for.
+Any directory that contains this file will be excluded from the dump.
+Similarly to @samp{exclude-caches}, there are three options in this
+option family:
+
+@table @option
+@opindex exclude-tag
+@item --exclude-tag=@var{file}
+Do not dump the contents of the directory, but dump the
+directory itself and the @var{file}.
+
+@opindex exclude-tag-under
+@item --exclude-tag-under=@var{file}
+Do not dump the contents of the directory, nor the
+@var{file}, archive only the directory itself.
+
+@opindex exclude-tag-all
+@item --exclude-tag-all=@var{file}
+Omit directories containing @var{file} file entirely.
+@end table
+
+Multiple @option{--exclude-tag*} options can be given.
+
+For example, given this directory:
+
+@smallexample
+@group
+$ @kbd{find dir}
+dir
+dir/blues
+dir/jazz
+dir/folk
+dir/folk/tagfile
+dir/folk/sanjuan
+dir/folk/trote
+@end group
+@end smallexample
+
+The @option{--exclude-tag} will produce the following:
+
+@smallexample
+$ @kbd{tar -cf archive.tar --exclude-tag=tagfile -v dir}
+dir/
+dir/blues
+dir/jazz
+dir/folk/
+tar: dir/folk/: contains a cache directory tag tagfile;
+ contents not dumped
+dir/folk/tagfile
+@end smallexample
+
+Both the @file{dir/folk} directory and its tagfile are preserved in
+the archive, however the rest of files in this directory are not.
+
+Now, using the @option{--exclude-tag-under} option will exclude
+@file{tagfile} from the dump, while still preserving the directory
+itself, as shown in this example:
+
+@smallexample
+$ @kbd{tar -cf archive.tar --exclude-tag-under=tagfile -v dir}
+dir/
+dir/blues
+dir/jazz
+dir/folk/
+./tar: dir/folk/: contains a cache directory tag tagfile;
+ contents not dumped
+@end smallexample
+
+Finally, using @option{--exclude-tag-all} omits the @file{dir/folk}
+directory entirely:
+
+@smallexample
+$ @kbd{tar -cf archive.tar --exclude-tag-all=tagfile -v dir}
+dir/
+dir/blues
+dir/jazz
+./tar: dir/folk/: contains a cache directory tag tagfile;
+ directory not dumped
+@end smallexample
+
+@menu
+* problems with exclude::
+@end menu
+
+@node problems with exclude
+@unnumberedsubsec Problems with Using the @code{exclude} Options
+
+@xopindex{exclude, potential problems with}
+Some users find @samp{exclude} options confusing. Here are some common
+pitfalls:
+
+@itemize @bullet
@item
-The main operating mode of @command{tar} does not act on a path name
-explicitly listed on the command line if one of its file name
+The main operating mode of @command{tar} does not act on a file name
+explicitly listed on the command line, if one of its file name
components is excluded. In the example above, if
you create an archive and exclude files that end with @samp{*.o}, but
explicitly name the file @samp{dir.o/foo} after all the options have been
@item
@FIXME{The change in semantics must have occurred before 1.11,
so I doubt if it is worth mentioning at all. Anyway, should at
-least specify in which version the semantics changed.}
+least specify in which version the semantics changed.}
In earlier versions of @command{tar}, what is now the
@option{--exclude-from} option was called @option{--exclude} instead.
Now, @option{--exclude} applies to patterns listed on the command
command line refer to @emph{files}, not archive members.
By default, inclusion members are compared with archive members
-literally @footnote{Notice that earlier @GNUTAR{} versions used
+literally @footnote{Notice that earlier @GNUTAR{} versions used
globbing for inclusion members, which contradicted to UNIX98
specification and was not documented. @xref{Changes}, for more
information on this and other changes.} and exclusion members are
@table @option
@opindex wildcards
@item --wildcards
-Treat all member names as wildcards.
+Treat all member names as wildcards.
@opindex no-wildcards
@item --no-wildcards
Notice quoting of the pattern to prevent the shell from interpreting
it.
-The effect of @option{--wildcards} option is cancelled by
+The effect of @option{--wildcards} option is canceled by
@option{--no-wildcards}. This can be used to pass part of
the command line arguments verbatim and other part as globbing
patterns. For example, the following invocation:
@itemize @bullet
@item Non-printable control characters:
-
+@anchor{escape sequences}
@multitable @columnfractions 0.20 0.10 0.60
-@headitem Character @tab ASCII @tab Character name
+@headitem Character @tab @acronym{ASCII} @tab Character name
@item \a @tab 7 @tab Audible bell
@item \b @tab 8 @tab Backspace
@item \f @tab 12 @tab Form feed
@item \v @tab 11 @tab Vertical tabulation
@end multitable
-@item Space (ASCII 32)
+@item Space (@acronym{ASCII} 32)
@item Single and double quotes (@samp{'} and @samp{"})
No quoting, display each character as is:
@smallexample
-@group
+@group
$ @kbd{tar tf arch.tar --quoting-style=literal}
./
./a space
./a'single'quote
./a"double"quote
./a\backslash
-./a tab
+./a tab
./a
newline
@end group
'./a'\''single'\''quote'
'./a"double"quote'
'./a\backslash'
-'./a tab'
+'./a tab'
'./a
newline'
@end group
'./a'\''single'\''quote'
'./a"double"quote'
'./a\backslash'
-'./a tab'
+'./a tab'
'./a
newline'
@end group
@end table
For example, using @samp{escape} quoting (compare with the usual
-escape listing above):
+escape listing above):
@smallexample
@group
The option @option{--strip=2} instructs @command{tar} to strip the
two leading components (@file{usr/} and @file{include/}) off the file
-name.
+name.
If you add to the above invocation @option{--verbose} (@option{-v})
option, you will note that the verbose listing still contains the
@var{regexp} and @var{replace} are described in detail in
@ref{The "s" Command, The "s" Command, The `s' Command, sed, GNU sed}.
+As in @command{sed}, you can give several replace expressions,
+separated by a semicolon.
+
Supported @var{flags} are:
@table @samp
Note: the @var{posix} standard does not specify what should happen
when you mix the @samp{g} and @var{number} modifiers. @GNUTAR{}
follows the GNU @command{sed} implementation in this regard, so
-the the interaction is defined to be: ignore matches before the
+the interaction is defined to be: ignore matches before the
@var{number}th, and then match and replace all matches from the
@var{number}th on.
-
+
@end table
Any delimiter can be used in lieue of @samp{/}, the only requirement being
If both @option{--strip-components} and @option{--transform} are used
together, then @option{--transform} is applied first, and the required
number of components is then stripped from its result.
-
+
+You can use as many @option{--transform} options in a single command
+line as you want. The specified expressions will then be applied in
+order of their appearance. For example, the following two invocations
+are equivalent:
+
+@smallexample
+$ @kbd{tar -cf arch.tar --transform='s,/usr/var,/var/' \
+ --transform='s,/usr/local,/usr/,'}
+$ @kbd{tar -cf arch.tar \
+ --transform='s,/usr/var,/var/;s,/usr/local,/usr/,'}
+@end smallexample
+
@node after
@section Operating Only on New Files
@UNREVISED
@node directory
@subsection Changing the Working Directory
-@UNREVISED
@FIXME{need to read over this node now for continuity; i've switched
things around some.}
@smallexample
@group
--C
-/etc
+-C/etc
passwd
hosts
--C
-/lib
+--directory=/lib
libc.a
@end group
@end smallexample
$ @kbd{tar -c -f foo.tar --files-from list}
@end smallexample
-Notice also that you can only use the short option variant in the file
-list, i.e., always use @option{-C}, not @option{--directory}.
-
The interpretation of @option{--directory} is disabled by
@option{--null} option.
For example:
@smallexample
-$ @kbd{(cd / && tar -c -f archive.tar home)}
-# @i{or}:
-$ @kbd{tar -c -f archive.tar -C / home}
+$ @kbd{tar -c -f archive.tar -C / home}
@end smallexample
@include getdate.texi
features were implemented in a way incompatible with other archive
formats.
-Archives in @samp{gnu} format are able to hold pathnames of unlimited
+Archives in @samp{gnu} format are able to hold file names of unlimited
length.
@item oldgnu
@item The maximum length of a symbolic link is limited to 99 characters.
@item It is impossible to store special files (block and character
devices, fifos etc.)
-@item Maximum value of user or group ID is limited to 2097151 (7777777
+@item Maximum value of user or group @acronym{ID} is limited to 2097151 (7777777
octal)
@item V7 archives do not contain symbolic ownership information (user
and group name of the file owner).
This format has traditionally been used by Automake when producing
Makefiles. This practice will change in the future, in the meantime,
-however this means that projects containing filenames more than 99
+however this means that projects containing file names more than 99
characters long will not be able to use @GNUTAR{} @value{VERSION} and
Automake prior to 1.9.
@enumerate
@item The maximum length of a file name is limited to 256 characters,
-provided that the filename can be split at directory separator in
+provided that the file name can be split at a directory separator in
two parts, first of them being at most 155 bytes long. So, in most
cases the maximum file name length will be shorter than 256
characters.
@item The maximum length of a symbolic link name is limited to
100 characters.
-@item Maximum size of a file the archive is able to accomodate
+@item Maximum size of a file the archive is able to accommodate
is 8GB
@item Maximum value of UID/GID is 2097151.
@item Maximum number of bits in device major and minor numbers is 21.
@item posix
Archive format defined by @acronym{POSIX.1-2001} specification. This is the
most flexible and feature-rich format. It does not impose any
-restrictions on file sizes or filename lengths. This format is quite
+restrictions on file sizes or file name lengths. This format is quite
recent, so not all tar implementations are able to handle it properly.
However, this format is designed in such a way that any tar
implementation able to read @samp{ustar} archives will be able to read
formats:
@multitable @columnfractions .10 .20 .20 .20 .20
-@headitem Format @tab UID @tab File Size @tab Path Name @tab Devn
+@headitem Format @tab UID @tab File Size @tab File Name @tab Devn
@item gnu @tab 1.8e19 @tab Unlimited @tab Unlimited @tab 63
@item oldgnu @tab 1.8e19 @tab Unlimited @tab Unlimited @tab 63
@item v7 @tab 2097151 @tab 8GB @tab 99 @tab n/a
switch to @samp{posix}.
@menu
-* Portability:: Making @command{tar} Archives More Portable
* Compression:: Using Less Space through Compression
* Attributes:: Handling File Attributes
+* Portability:: Making @command{tar} Archives More Portable
* cpio:: Comparison of @command{tar} and @command{cpio}
@end menu
-@node Portability
-@section Making @command{tar} Archives More Portable
-
-Creating a @command{tar} archive on a particular system that is meant to be
-useful later on many other machines and with other versions of @command{tar}
-is more challenging than you might think. @command{tar} archive formats
-have been evolving since the first versions of Unix. Many such formats
-are around, and are not always compatible with each other. This section
-discusses a few problems, and gives some advice about making @command{tar}
-archives more portable.
-
-One golden rule is simplicity. For example, limit your @command{tar}
-archives to contain only regular files and directories, avoiding
-other kind of special files. Do not attempt to save sparse files or
-contiguous files as such. Let's discuss a few more problems, in turn.
-
-@FIXME{Discuss GNU extensions (incremental backups, multi-volume
-archives and archive labels) in GNU and PAX formats.}
+@node Compression
+@section Using Less Space through Compression
@menu
-* Portable Names:: Portable Names
-* dereference:: Symbolic Links
-* old:: Old V7 Archives
-* ustar:: Ustar Archives
-* gnu:: GNU and old GNU format archives.
-* posix:: @acronym{POSIX} archives
-* Checksumming:: Checksumming Problems
-* Large or Negative Values:: Large files, negative time stamps, etc.
-* Other Tars:: How to Extract GNU-Specific Data Using
- Other @command{tar} Implementations
+* gzip:: Creating and Reading Compressed Archives
+* sparse:: Archiving Sparse Files
@end menu
-@node Portable Names
-@subsection Portable Names
+@node gzip
+@subsection Creating and Reading Compressed Archives
+@cindex Compressed archives
+@cindex Storing archives in compressed format
-Use portable file and member names. A name is portable if it contains
-only ASCII letters and digits, @samp{/}, @samp{.}, @samp{_}, and
-@samp{-}; it cannot be empty, start with @samp{-} or @samp{//}, or
-contain @samp{/-}. Avoid deep directory nesting. For portability to
-old Unix hosts, limit your file name components to 14 characters or
-less.
+@cindex gzip
+@cindex bzip2
+@cindex lzma
+@cindex lzop
+@cindex compress
+@GNUTAR{} is able to create and read compressed archives. It supports
+@command{gzip}, @command{bzip2}, @command{lzma} and @command{lzop} compression
+programs. For backward compatibility, it also supports
+@command{compress} command, although we strongly recommend against
+using it, because it is by far less effective than other compression
+programs@footnote{It also had patent problems in the past.}.
-If you intend to have your @command{tar} archives to be read under
-MSDOS, you should not rely on case distinction for file names, and you
-might use the @acronym{GNU} @command{doschk} program for helping you
-further diagnosing illegal MSDOS names, which are even more limited
-than System V's.
+Creating a compressed archive is simple: you just specify a
+@dfn{compression option} along with the usual archive creation
+commands. The compression option is @option{-z} (@option{--gzip}) to
+create a @command{gzip} compressed archive, @option{-j}
+(@option{--bzip2}) to create a @command{bzip2} compressed archive,
+@option{-J} (@option{--lzma}) to create an @asis{LZMA} compressed
+archive, @option{--lzop} to create an @asis{LSOP} archive, and
+@option{-Z} (@option{--compress}) to use @command{compress} program.
+For example:
-@node dereference
-@subsection Symbolic Links
-@cindex File names, using symbolic links
-@cindex Symbolic link as file name
+@smallexample
+$ @kbd{tar cfz archive.tar.gz .}
+@end smallexample
-@opindex dereference
-Normally, when @command{tar} archives a symbolic link, it writes a
-block to the archive naming the target of the link. In that way, the
-@command{tar} archive is a faithful record of the file system contents.
-@option{--dereference} (@option{-h}) is used with @option{--create} (@option{-c}), and causes
-@command{tar} to archive the files symbolic links point to, instead of
-the links themselves. When this option is used, when @command{tar}
-encounters a symbolic link, it will archive the linked-to file,
-instead of simply recording the presence of a symbolic link.
+You can also let @GNUTAR{} select the compression program basing on
+the suffix of the archive file name. This is done using
+@option{--auto-compress} (@option{-a}) command line option. For
+example, the following invocation will use @command{bzip2} for
+compression:
-The name under which the file is stored in the file system is not
-recorded in the archive. To record both the symbolic link name and
-the file name in the system, archive the file under both names. If
-all links were recorded automatically by @command{tar}, an extracted file
-might be linked to a file name that no longer exists in the file
-system.
+@smallexample
+$ @kbd{tar cfa archive.tar.bz2 .}
+@end smallexample
-If a linked-to file is encountered again by @command{tar} while creating
-the same archive, an entire second copy of it will be stored. (This
-@emph{might} be considered a bug.)
+@noindent
+whereas the following one will use @command{lzma}:
-So, for portable archives, do not archive symbolic links as such,
-and use @option{--dereference} (@option{-h}): many systems do not support
-symbolic links, and moreover, your distribution might be unusable if
-it contains unresolved symbolic links.
+@smallexample
+$ @kbd{tar cfa archive.tar.lzma .}
+@end smallexample
-@node old
-@subsection Old V7 Archives
-@cindex Format, old style
-@cindex Old style format
-@cindex Old style archives
-@cindex v7 archive format
+For a complete list of file name suffixes recognized by @GNUTAR{},
+@ref{auto-compress}.
-Certain old versions of @command{tar} cannot handle additional
-information recorded by newer @command{tar} programs. To create an
-archive in V7 format (not ANSI), which can be read by these old
-versions, specify the @option{--format=v7} option in
-conjunction with the @option{--create} (@option{-c}) (@command{tar} also
-accepts @option{--portability} or @samp{op-old-archive} for this
-option). When you specify it,
-@command{tar} leaves out information about directories, pipes, fifos,
-contiguous files, and device files, and specifies file ownership by
-group and user IDs instead of group and user names.
+Reading compressed archive is even simpler: you don't need to specify
+any additional options as @GNUTAR{} recognizes its format
+automatically. Thus, the following commands will list and extract the
+archive created in previous example:
-When updating an archive, do not use @option{--format=v7}
-unless the archive was created using this option.
+@smallexample
+# List the compressed archive
+$ @kbd{tar tf archive.tar.gz}
+# Extract the compressed archive
+$ @kbd{tar xf archive.tar.gz}
+@end smallexample
-In most cases, a @emph{new} format archive can be read by an @emph{old}
-@command{tar} program without serious trouble, so this option should
-seldom be needed. On the other hand, most modern @command{tar}s are
-able to read old format archives, so it might be safer for you to
-always use @option{--format=v7} for your distributions.
+The format recognition algorithm is based on @dfn{signatures}, a
+special byte sequences in the beginning of file, that are specific for
+certain compression formats. If this approach fails, @command{tar}
+falls back to using archive name suffix to determine its format
+(@xref{auto-compress}, for a list of recognized suffixes).
-@node ustar
-@subsection Ustar Archive Format
+The only case when you have to specify a decompression option while
+reading the archive is when reading from a pipe or from a tape drive
+that does not support random access. However, in this case @GNUTAR{}
+will indicate which option you should use. For example:
-@cindex ustar archive format
-Archive format defined by @acronym{POSIX}.1-1988 specification is called
-@code{ustar}. Although it is more flexible than the V7 format, it
-still has many restrictions (@xref{Formats,ustar}, for the detailed
-description of @code{ustar} format). Along with V7 format,
-@code{ustar} format is a good choice for archives intended to be read
-with other implementations of @command{tar}.
+@smallexample
+$ @kbd{cat archive.tar.gz | tar tf -}
+tar: Archive is compressed. Use -z option
+tar: Error is not recoverable: exiting now
+@end smallexample
-To create archive in @code{ustar} format, use @option{--format=ustar}
-option in conjunction with the @option{--create} (@option{-c}).
+If you see such diagnostics, just add the suggested option to the
+invocation of @GNUTAR{}:
-@node gnu
-@subsection @acronym{GNU} and old @GNUTAR{} format
+@smallexample
+$ @kbd{cat archive.tar.gz | tar tfz -}
+@end smallexample
-@cindex GNU archive format
-@cindex Old GNU archive format
-@GNUTAR{} was based on an early draft of the
-@acronym{POSIX} 1003.1 @code{ustar} standard. @acronym{GNU} extensions to
-@command{tar}, such as the support for file names longer than 100
-characters, use portions of the @command{tar} header record which were
-specified in that @acronym{POSIX} draft as unused. Subsequent changes in
-@acronym{POSIX} have allocated the same parts of the header record for
-other purposes. As a result, @GNUTAR{} format is
-incompatible with the current @acronym{POSIX} specification, and with
-@command{tar} programs that follow it.
+Notice also, that there are several restrictions on operations on
+compressed archives. First of all, compressed archives cannot be
+modified, i.e., you cannot update (@option{--update} (@option{-u}))
+them or delete (@option{--delete}) members from them or
+add (@option{--append} (@option{-r})) members to them. Likewise, you
+cannot append another @command{tar} archive to a compressed archive using
+@option{--concatenate} (@option{-A})). Secondly, multi-volume
+archives cannot be compressed.
-In the majority of cases, @command{tar} will be configured to create
-this format by default. This will change in the future releases, since
-we plan to make @samp{posix} format the default.
+The following table summarizes compression options used by @GNUTAR{}.
-To force creation a @GNUTAR{} archive, use option
-@option{--format=gnu}.
+@table @option
+@anchor{auto-compress}
+@opindex auto-compress
+@item --auto-compress
+@itemx -a
+Select a compression program to use by the archive file name
+suffix. The following suffixes are recognized:
+
+@multitable @columnfractions 0.3 0.6
+@headitem Suffix @tab Compression program
+@item @samp{.gz} @tab @command{gzip}
+@item @samp{.tgz} @tab @command{gzip}
+@item @samp{.taz} @tab @command{gzip}
+@item @samp{.Z} @tab @command{compress}
+@item @samp{.taZ} @tab @command{compress}
+@item @samp{.bz2} @tab @command{bzip2}
+@item @samp{.tz2} @tab @command{bzip2}
+@item @samp{.tbz2} @tab @command{bzip2}
+@item @samp{.tbz} @tab @command{bzip2}
+@item @samp{.lzma} @tab @command{lzma}
+@item @samp{.tlz} @tab @command{lzma}
+@item @samp{.lzo} @tab @command{lzop}
+@end multitable
-@node posix
-@subsection @GNUTAR{} and @acronym{POSIX} @command{tar}
+@opindex gzip
+@opindex ungzip
+@item -z
+@itemx --gzip
+@itemx --ungzip
+Filter the archive through @command{gzip}.
-@cindex POSIX archive format
-@cindex PAX archive format
-Starting from version 1.14 @GNUTAR{} features full support for
-@acronym{POSIX.1-2001} archives.
+You can use @option{--gzip} and @option{--gunzip} on physical devices
+(tape drives, etc.) and remote files as well as on normal files; data
+to or from such devices or remote files is reblocked by another copy
+of the @command{tar} program to enforce the specified (or default) record
+size. The default compression parameters are used; if you need to
+override them, set @env{GZIP} environment variable, e.g.:
-A @acronym{POSIX} conformant archive will be created if @command{tar}
-was given @option{--format=posix} (@option{--format=pax}) option. No
-special option is required to read and extract from a @acronym{POSIX}
-archive.
+@smallexample
+$ @kbd{GZIP=--best tar cfz archive.tar.gz subdir}
+@end smallexample
-@menu
-* PAX keywords:: Controlling Extended Header Keywords.
-@end menu
+@noindent
+Another way would be to avoid the @option{--gzip} (@option{--gunzip}, @option{--ungzip}, @option{-z}) option and run
+@command{gzip} explicitly:
-@node PAX keywords
-@subsubsection Controlling Extended Header Keywords
+@smallexample
+$ @kbd{tar cf - subdir | gzip --best -c - > archive.tar.gz}
+@end smallexample
-@table @option
-@opindex pax-option
-@item --pax-option=@var{keyword-list}
-Handle keywords in @acronym{PAX} extended headers. This option is
-equivalent to @option{-o} option of the @command{pax} utility.
+@cindex corrupted archives
+About corrupted compressed archives: @command{gzip}'ed files have no
+redundancy, for maximum compression. The adaptive nature of the
+compression scheme means that the compression tables are implicitly
+spread all over the archive. If you lose a few blocks, the dynamic
+construction of the compression tables becomes unsynchronized, and there
+is little chance that you could recover later in the archive.
+
+There are pending suggestions for having a per-volume or per-file
+compression in @GNUTAR{}. This would allow for viewing the
+contents without decompression, and for resynchronizing decompression at
+every volume or file, in case of corrupted archives. Doing so, we might
+lose some compressibility. But this would have make recovering easier.
+So, there are pros and cons. We'll see!
+
+@opindex bzip2
+@item -j
+@itemx --bzip2
+Filter the archive through @code{bzip2}. Otherwise like @option{--gzip}.
+
+@opindex lzma
+@item --lzma
+@itemx -J
+Filter the archive through @command{lzma}. Otherwise like @option{--gzip}.
+
+@opindex lzop
+@item --lzop
+Filter the archive through @command{lzop}. Otherwise like
+@option{--gzip}.
+
+@opindex compress
+@opindex uncompress
+@item -Z
+@itemx --compress
+@itemx --uncompress
+Filter the archive through @command{compress}. Otherwise like @option{--gzip}.
+
+@opindex use-compress-program
+@item --use-compress-program=@var{prog}
+Use external compression program @var{prog}. Use this option if you
+have a compression program that @GNUTAR{} does not support. There
+are two requirements to which @var{prog} should comply:
+
+First, when called without options, it should read data from standard
+input, compress it and output it on standard output.
+
+Secondly, if called with @option{-d} argument, it should do exactly
+the opposite, i.e., read the compressed data from the standard input
+and produce uncompressed data on the standard output.
@end table
-@var{Keyword-list} is a comma-separated
-list of keyword options, each keyword option taking one of
-the following forms:
+@cindex gpg, using with tar
+@cindex gnupg, using with tar
+@cindex Using encrypted archives
+The @option{--use-compress-program} option, in particular, lets you
+implement your own filters, not necessarily dealing with
+compression/decompression. For example, suppose you wish to implement
+PGP encryption on top of compression, using @command{gpg} (@pxref{Top,
+gpg, gpg ---- encryption and signing tool, gpg, GNU Privacy Guard
+Manual}). The following script does that:
-@table @code
-@item delete=@var{pattern}
-When used with one of archive-creation commands,
-this option instructs @command{tar} to omit from extended header records
-that it produces any keywords matching the string @var{pattern}.
+@smallexample
+@group
+#! /bin/sh
+case $1 in
+-d) gpg --decrypt - | gzip -d -c;;
+'') gzip -c | gpg -s ;;
+*) echo "Unknown option $1">&2; exit 1;;
+esac
+@end group
+@end smallexample
-When used in extract or list mode, this option instructs tar
-to ignore any keywords matching the given @var{pattern} in the extended
-header records. In both cases, matching is performed using the pattern
-matching notation described in @acronym{POSIX 1003.2}, 3.13
-(@pxref{wildcards}). For example:
+Suppose you name it @file{gpgz} and save it somewhere in your
+@env{PATH}. Then the following command will create a compressed
+archive signed with your private key:
@smallexample
---pax-option delete=security.*
+$ @kbd{tar -cf foo.tar.gpgz --use-compress=gpgz .}
@end smallexample
-would suppress security-related information.
+@noindent
+Likewise, the following command will list its contents:
-@item exthdr.name=@var{string}
+@smallexample
+$ @kbd{tar -tf foo.tar.gpgz --use-compress=gpgz .}
+@end smallexample
-This keyword allows user control over the name that is written into the
-ustar header blocks for the extended headers. The name is obtained
-from @var{string} after making the following substitutions:
+@ignore
+The above is based on the following discussion:
-@multitable @columnfractions .25 .55
-@headitem Meta-character @tab Replaced By
-@item %d @tab The directory name of the file, equivalent to the
-result of the @command{dirname} utility on the translated pathname.
-@item %f @tab The filename of the file, equivalent to the result
-of the @command{basename} utility on the translated pathname.
-@item %p @tab The process ID of the @command{tar} process.
-@item %% @tab A @samp{%} character.
-@end multitable
+ I have one question, or maybe it's a suggestion if there isn't a way
+ to do it now. I would like to use @option{--gzip}, but I'd also like
+ the output to be fed through a program like @acronym{GNU}
+ @command{ecc} (actually, right now that's @samp{exactly} what I'd like
+ to use :-)), basically adding ECC protection on top of compression.
+ It seems as if this should be quite easy to do, but I can't work out
+ exactly how to go about it. Of course, I can pipe the standard output
+ of @command{tar} through @command{ecc}, but then I lose (though I
+ haven't started using it yet, I confess) the ability to have
+ @command{tar} use @command{rmt} for it's I/O (I think).
-Any other @samp{%} characters in @var{string} produce undefined
-results.
+ I think the most straightforward thing would be to let me specify a
+ general set of filters outboard of compression (preferably ordered,
+ so the order can be automatically reversed on input operations, and
+ with the options they require specifiable), but beggars shouldn't be
+ choosers and anything you decide on would be fine with me.
-If no option @samp{exthdr.name=string} is specified, @command{tar}
-will use the following default value:
+ By the way, I like @command{ecc} but if (as the comments say) it can't
+ deal with loss of block sync, I'm tempted to throw some time at adding
+ that capability. Supposing I were to actually do such a thing and
+ get it (apparently) working, do you accept contributed changes to
+ utilities like that? (Leigh Clayton @file{loc@@soliton.com}, May 1995).
-@smallexample
-%d/PaxHeaders.%p/%f
-@end smallexample
+ Isn't that exactly the role of the
+ @option{--use-compress-prog=@var{program}} option?
+ I never tried it myself, but I suspect you may want to write a
+ @var{prog} script or program able to filter stdin to stdout to
+ way you want. It should recognize the @option{-d} option, for when
+ extraction is needed rather than creation.
-@item globexthdr.name=@var{string}
-This keyword allows user control over the name that is written into
-the ustar header blocks for global extended header records. The name
-is obtained from the contents of @var{string}, after making
-the following substitutions:
+ It has been reported that if one writes compressed data (through the
+ @option{--gzip} or @option{--compress} options) to a DLT and tries to use
+ the DLT compression mode, the data will actually get bigger and one will
+ end up with less space on the tape.
+@end ignore
-@multitable @columnfractions .25 .55
-@headitem Meta-character @tab Replaced By
-@item %n @tab An integer that represents the
-sequence number of the global extended header record in the archive,
-starting at 1.
-@item %p @tab The process ID of the @command{tar} process.
-@item %% @tab A @samp{%} character.
-@end multitable
+@node sparse
+@subsection Archiving Sparse Files
+@cindex Sparse Files
-Any other @samp{%} characters in @var{string} produce undefined results.
+Files in the file system occasionally have @dfn{holes}. A @dfn{hole}
+in a file is a section of the file's contents which was never written.
+The contents of a hole reads as all zeros. On many operating systems,
+actual disk storage is not allocated for holes, but they are counted
+in the length of the file. If you archive such a file, @command{tar}
+could create an archive longer than the original. To have @command{tar}
+attempt to recognize the holes in a file, use @option{--sparse}
+(@option{-S}). When you use this option, then, for any file using
+less disk space than would be expected from its length, @command{tar}
+searches the file for consecutive stretches of zeros. It then records
+in the archive for the file where the consecutive stretches of zeros
+are, and only archives the ``real contents'' of the file. On
+extraction (using @option{--sparse} is not needed on extraction) any
+such files have holes created wherever the continuous stretches of zeros
+were found. Thus, if you use @option{--sparse}, @command{tar} archives
+won't take more space than the original.
-If no option @samp{globexthdr.name=string} is specified, @command{tar}
-will use the following default value:
+@table @option
+@opindex sparse
+@item -S
+@itemx --sparse
+This option instructs @command{tar} to test each file for sparseness
+before attempting to archive it. If the file is found to be sparse it
+is treated specially, thus allowing to decrease the amount of space
+used by its image in the archive.
-@smallexample
-$TMPDIR/GlobalHead.%p.%n
-@end smallexample
+This option is meaningful only when creating or updating archives. It
+has no effect on extraction.
+@end table
-@noindent
-where @samp{$TMPDIR} represents the value of the @var{TMPDIR}
-environment variable. If @var{TMPDIR} is not set, @command{tar}
-uses @samp{/tmp}.
+Consider using @option{--sparse} when performing file system backups,
+to avoid archiving the expanded forms of files stored sparsely in the
+system.
-@item @var{keyword}=@var{value}
-When used with one of archive-creation commands, these keyword/value pairs
-will be included at the beginning of the archive in a global extended
-header record. When used with one of archive-reading commands,
-@command{tar} will behave as if it has encountered these keyword/value
-pairs at the beginning of the archive in a global extended header
-record.
+Even if your system has no sparse files currently, some may be
+created in the future. If you use @option{--sparse} while making file
+system backups as a matter of course, you can be assured the archive
+will never take more space on the media than the files take on disk
+(otherwise, archiving a disk filled with sparse files might take
+hundreds of tapes). @xref{Incremental Dumps}.
-@item @var{keyword}:=@var{value}
-When used with one of archive-creation commands, these keyword/value pairs
-will be included as records at the beginning of an extended header for
-each file. This is effectively equivalent to @var{keyword}=@var{value}
-form except that it creates no global extended header records.
+However, be aware that @option{--sparse} option presents a serious
+drawback. Namely, in order to determine if the file is sparse
+@command{tar} has to read it before trying to archive it, so in total
+the file is read @strong{twice}. So, always bear in mind that the
+time needed to process all files with this option is roughly twice
+the time needed to archive them without it.
+@FIXME{A technical note:
-When used with one of archive-reading commands, @command{tar} will
-behave as if these keyword/value pairs were included as records at the
-end of each extended header; thus, they will override any global or
-file-specific extended header record keywords of the same names.
-For example, in the command:
+Programs like @command{dump} do not have to read the entire file; by
+examining the file system directly, they can determine in advance
+exactly where the holes are and thus avoid reading through them. The
+only data it need read are the actual allocated data blocks.
+@GNUTAR{} uses a more portable and straightforward
+archiving approach, it would be fairly difficult that it does
+otherwise. Elizabeth Zwicky writes to @file{comp.unix.internals}, on
+1990-12-10:
-@smallexample
-tar --format=posix --create \
- --file archive --pax-option gname:=user .
-@end smallexample
+@quotation
+What I did say is that you cannot tell the difference between a hole and an
+equivalent number of nulls without reading raw blocks. @code{st_blocks} at
+best tells you how many holes there are; it doesn't tell you @emph{where}.
+Just as programs may, conceivably, care what @code{st_blocks} is (care
+to name one that does?), they may also care where the holes are (I have
+no examples of this one either, but it's equally imaginable).
-the group name will be forced to a new value for all files
-stored in the archive.
+I conclude from this that good archivers are not portable. One can
+arguably conclude that if you want a portable program, you can in good
+conscience restore files with as many holes as possible, since you can't
+get it right.
+@end quotation
+}
+
+@cindex sparse formats, defined
+When using @samp{POSIX} archive format, @GNUTAR{} is able to store
+sparse files using in three distinct ways, called @dfn{sparse
+formats}. A sparse format is identified by its @dfn{number},
+consisting, as usual of two decimal numbers, delimited by a dot. By
+default, format @samp{1.0} is used. If, for some reason, you wish to
+use an earlier format, you can select it using
+@option{--sparse-version} option.
+
+@table @option
+@opindex sparse-version
+@item --sparse-version=@var{version}
+
+Select the format to store sparse files in. Valid @var{version} values
+are: @samp{0.0}, @samp{0.1} and @samp{1.0}. @xref{Sparse Formats},
+for a detailed description of each format.
@end table
-@node Checksumming
-@subsection Checksumming Problems
+Using @option{--sparse-format} option implies @option{--sparse}.
-SunOS and HP-UX @command{tar} fail to accept archives created using
-@GNUTAR{} and containing non-ASCII file names, that
-is, file names having characters with the eight bit set, because they
-use signed checksums, while @GNUTAR{} uses unsigned
-checksums while creating archives, as per @acronym{POSIX} standards. On
-reading, @GNUTAR{} computes both checksums and
-accept any. It is somewhat worrying that a lot of people may go
-around doing backup of their files using faulty (or at least
-non-standard) software, not learning about it until it's time to
-restore their missing files with an incompatible file extractor, or
-vice versa.
+@node Attributes
+@section Handling File Attributes
+@UNREVISED
-@GNUTAR{} compute checksums both ways, and accept
-any on read, so @acronym{GNU} tar can read Sun tapes even with their
-wrong checksums. @GNUTAR{} produces the standard
-checksum, however, raising incompatibilities with Sun. That is to
-say, @GNUTAR{} has not been modified to
-@emph{produce} incorrect archives to be read by buggy @command{tar}'s.
-I've been told that more recent Sun @command{tar} now read standard
-archives, so maybe Sun did a similar patch, after all?
+When @command{tar} reads files, it updates their access times. To
+avoid this, use the @option{--atime-preserve[=METHOD]} option, which can either
+reset the access time retroactively or avoid changing it in the first
+place.
-The story seems to be that when Sun first imported @command{tar}
-sources on their system, they recompiled it without realizing that
-the checksums were computed differently, because of a change in
-the default signing of @code{char}'s in their compiler. So they
-started computing checksums wrongly. When they later realized their
-mistake, they merely decided to stay compatible with it, and with
-themselves afterwards. Presumably, but I do not really know, HP-UX
-has chosen that their @command{tar} archives to be compatible with Sun's.
-The current standards do not favor Sun @command{tar} format. In any
-case, it now falls on the shoulders of SunOS and HP-UX users to get
-a @command{tar} able to read the good archives they receive.
+Handling of file attributes
-@node Large or Negative Values
-@subsection Large or Negative Values
-@cindex large values
-@cindex future time stamps
-@cindex negative time stamps
-@UNREVISED{}
+@table @option
+@opindex atime-preserve
+@item --atime-preserve
+@itemx --atime-preserve=replace
+@itemx --atime-preserve=system
+Preserve the access times of files that are read. This works only for
+files that you own, unless you have superuser privileges.
-The above sections suggest to use @samp{oldest possible} archive
-format if in doubt. However, sometimes it is not possible. If you
-attempt to archive a file whose metadata cannot be represented using
-required format, @GNUTAR{} will print error message and ignore such a
-file. You will than have to switch to a format that is able to
-handle such values. The format summary table (@pxref{Formats}) will
-help you to do so.
+@option{--atime-preserve=replace} works on most systems, but it also
+restores the data modification time and updates the status change
+time. Hence it doesn't interact with incremental dumps nicely
+(@pxref{Incremental Dumps}), and it can set access or data modification times
+incorrectly if other programs access the file while @command{tar} is
+running.
-In particular, when trying to archive files larger than 8GB or with
-timestamps not in the range 1970-01-01 00:00:00 through 2242-03-16
-12:56:31 @sc{utc}, you will have to chose between @acronym{GNU} and
-@acronym{POSIX} archive formats. When considering which format to
-choose, bear in mind that the @acronym{GNU} format uses
-two's-complement base-256 notation to store values that do not fit
-into standard @acronym{ustar} range. Such archives can generally be
-read only by a @GNUTAR{} implementation. Moreover, they sometimes
-cannot be correctly restored on another hosts even by @GNUTAR{}. For
-example, using two's complement representation for negative time
-stamps that assumes a signed 32-bit @code{time_t} generates archives
-that are not portable to hosts with differing @code{time_t}
-representations.
+@option{--atime-preserve=system} avoids changing the access time in
+the first place, if the operating system supports this.
+Unfortunately, this may or may not work on any given operating system
+or file system. If @command{tar} knows for sure it won't work, it
+complains right away.
-On the other hand, @acronym{POSIX} archives, generally speaking, can
-be extracted by any tar implementation that understands older
-@acronym{ustar} format. The only exception are files larger than 8GB.
+Currently @option{--atime-preserve} with no operand defaults to
+@option{--atime-preserve=replace}, but this is intended to change to
+@option{--atime-preserve=system} when the latter is better-supported.
-@FIXME{Describe how @acronym{POSIX} archives are extracted by non
-POSIX-aware tars.}
+@opindex touch
+@item -m
+@itemx --touch
+Do not extract data modification time.
-@node Other Tars
-@subsection How to Extract GNU-Specific Data Using Other @command{tar} Implementations
+When this option is used, @command{tar} leaves the data modification times
+of the files it extracts as the times when the files were extracted,
+instead of setting it to the times recorded in the archive.
+
+This option is meaningless with @option{--list} (@option{-t}).
+
+@opindex same-owner
+@item --same-owner
+Create extracted files with the same ownership they have in the
+archive.
+
+This is the default behavior for the superuser,
+so this option is meaningful only for non-root users, when @command{tar}
+is executed on those systems able to give files away. This is
+considered as a security flaw by many people, at least because it
+makes quite difficult to correctly account users for the disk space
+they occupy. Also, the @code{suid} or @code{sgid} attributes of
+files are easily and silently lost when files are given away.
+
+When writing an archive, @command{tar} writes the user @acronym{ID} and user name
+separately. If it can't find a user name (because the user @acronym{ID} is not
+in @file{/etc/passwd}), then it does not write one. When restoring,
+it tries to look the name (if one was written) up in
+@file{/etc/passwd}. If it fails, then it uses the user @acronym{ID} stored in
+the archive instead.
+
+@opindex no-same-owner
+@item --no-same-owner
+@itemx -o
+Do not attempt to restore ownership when extracting. This is the
+default behavior for ordinary users, so this option has an effect
+only for the superuser.
+
+@opindex numeric-owner
+@item --numeric-owner
+The @option{--numeric-owner} option allows (ANSI) archives to be written
+without user/group name information or such information to be ignored
+when extracting. It effectively disables the generation and/or use
+of user/group name information. This option forces extraction using
+the numeric ids from the archive, ignoring the names.
+
+This is useful in certain circumstances, when restoring a backup from
+an emergency floppy with different passwd/group files for example.
+It is otherwise impossible to extract files with the right ownerships
+if the password file in use during the extraction does not match the
+one belonging to the file system(s) being extracted. This occurs,
+for example, if you are restoring your files after a major crash and
+had booted from an emergency floppy with no password file or put your
+disk into another machine to do the restore.
+
+The numeric ids are @emph{always} saved into @command{tar} archives.
+The identifying names are added at create time when provided by the
+system, unless @option{--old-archive} (@option{-o}) is used. Numeric ids could be
+used when moving archives between a collection of machines using
+a centralized management for attribution of numeric ids to users
+and groups. This is often made through using the NIS capabilities.
+
+When making a @command{tar} file for distribution to other sites, it
+is sometimes cleaner to use a single owner for all files in the
+distribution, and nicer to specify the write permission bits of the
+files as stored in the archive independently of their actual value on
+the file system. The way to prepare a clean distribution is usually
+to have some Makefile rule creating a directory, copying all needed
+files in that directory, then setting ownership and permissions as
+wanted (there are a lot of possible schemes), and only then making a
+@command{tar} archive out of this directory, before cleaning
+everything out. Of course, we could add a lot of options to
+@GNUTAR{} for fine tuning permissions and ownership.
+This is not the good way, I think. @GNUTAR{} is
+already crowded with options and moreover, the approach just explained
+gives you a great deal of control already.
+
+@xopindex{same-permissions, short description}
+@xopindex{preserve-permissions, short description}
+@item -p
+@itemx --same-permissions
+@itemx --preserve-permissions
+Extract all protection information.
+
+This option causes @command{tar} to set the modes (access permissions) of
+extracted files exactly as recorded in the archive. If this option
+is not used, the current @code{umask} setting limits the permissions
+on extracted files. This option is by default enabled when
+@command{tar} is executed by a superuser.
+
+
+This option is meaningless with @option{--list} (@option{-t}).
+
+@opindex preserve
+@item --preserve
+Same as both @option{--same-permissions} and @option{--same-order}.
+
+The @option{--preserve} option has no equivalent short option name.
+It is equivalent to @option{--same-permissions} plus @option{--same-order}.
+
+@FIXME{I do not see the purpose of such an option. (Neither I. FP.)
+Neither do I. --Sergey}
+
+@end table
+
+@node Portability
+@section Making @command{tar} Archives More Portable
+
+Creating a @command{tar} archive on a particular system that is meant to be
+useful later on many other machines and with other versions of @command{tar}
+is more challenging than you might think. @command{tar} archive formats
+have been evolving since the first versions of Unix. Many such formats
+are around, and are not always compatible with each other. This section
+discusses a few problems, and gives some advice about making @command{tar}
+archives more portable.
+
+One golden rule is simplicity. For example, limit your @command{tar}
+archives to contain only regular files and directories, avoiding
+other kind of special files. Do not attempt to save sparse files or
+contiguous files as such. Let's discuss a few more problems, in turn.
+
+@FIXME{Discuss GNU extensions (incremental backups, multi-volume
+archives and archive labels) in GNU and PAX formats.}
+
+@menu
+* Portable Names:: Portable Names
+* dereference:: Symbolic Links
+* hard links:: Hard Links
+* old:: Old V7 Archives
+* ustar:: Ustar Archives
+* gnu:: GNU and old GNU format archives.
+* posix:: @acronym{POSIX} archives
+* Checksumming:: Checksumming Problems
+* Large or Negative Values:: Large files, negative time stamps, etc.
+* Other Tars:: How to Extract GNU-Specific Data Using
+ Other @command{tar} Implementations
+@end menu
+
+@node Portable Names
+@subsection Portable Names
+
+Use portable file and member names. A name is portable if it contains
+only @acronym{ASCII} letters and digits, @samp{/}, @samp{.}, @samp{_}, and
+@samp{-}; it cannot be empty, start with @samp{-} or @samp{//}, or
+contain @samp{/-}. Avoid deep directory nesting. For portability to
+old Unix hosts, limit your file name components to 14 characters or
+less.
+
+If you intend to have your @command{tar} archives to be read under
+MSDOS, you should not rely on case distinction for file names, and you
+might use the @acronym{GNU} @command{doschk} program for helping you
+further diagnosing illegal MSDOS names, which are even more limited
+than System V's.
+
+@node dereference
+@subsection Symbolic Links
+@cindex File names, using symbolic links
+@cindex Symbolic link as file name
+
+@opindex dereference
+Normally, when @command{tar} archives a symbolic link, it writes a
+block to the archive naming the target of the link. In that way, the
+@command{tar} archive is a faithful record of the file system contents.
+@option{--dereference} (@option{-h}) is used with @option{--create} (@option{-c}), and causes
+@command{tar} to archive the files symbolic links point to, instead of
+the links themselves. When this option is used, when @command{tar}
+encounters a symbolic link, it will archive the linked-to file,
+instead of simply recording the presence of a symbolic link.
-In previous sections you became acquainted with various quircks
-necessary to make your archives portable. Sometimes you may need to
-extract archives containing GNU-specific members using some
-third-party @command{tar} implementation or an older version of
-@GNUTAR{}. Of course your best bet is to have @GNUTAR{} installed,
-but if it is for some reason impossible, this section will explain
-how to cope without it.
+The name under which the file is stored in the file system is not
+recorded in the archive. To record both the symbolic link name and
+the file name in the system, archive the file under both names. If
+all links were recorded automatically by @command{tar}, an extracted file
+might be linked to a file name that no longer exists in the file
+system.
-When we speak about @dfn{GNU-specific} members we mean two classes of
-them: members split between the volumes of a multi-volume archive and
-sparse members. You will be able to always recover such members if
-the archive is in PAX format. In addition split members can be
-recovered from archives in old GNU format. The following subsections
-describe the required procedures in detail.
+If a linked-to file is encountered again by @command{tar} while creating
+the same archive, an entire second copy of it will be stored. (This
+@emph{might} be considered a bug.)
-@menu
-* Split Recovery:: Members Split Between Volumes
-* Sparse Recovery:: Sparse Members
-@end menu
+So, for portable archives, do not archive symbolic links as such,
+and use @option{--dereference} (@option{-h}): many systems do not support
+symbolic links, and moreover, your distribution might be unusable if
+it contains unresolved symbolic links.
-@node Split Recovery
-@subsubsection Extracting Members Split Between Volumes
+@node hard links
+@subsection Hard Links
+@UNREVISED{}
+@cindex File names, using hard links
+@cindex hard links, dereferencing
+@cindex dereferencing hard links
-If a member is split between several volumes of an old GNU format archive
-most third party @command{tar} implementation will fail to extract
-it. To extract it, use @command{tarcat} program (@pxref{Tarcat}).
-This program is available from
-@uref{http://www.gnu.org/@/software/@/tar/@/utils/@/tarcat, @GNUTAR{}
-home page}. It concatenates several archive volumes into a single
-valid archive. For example, if you have three volumes named from
-@file{vol-1.tar} to @file{vol-2.tar}, you can do the following to
-extract them using a third-party @command{tar}:
+Normally, when @command{tar} archives a hard link, it writes a
+block to the archive naming the target of the link (a @samp{1} type
+block). In that way, the actual file contents is stored in file only
+once. For example, consider the following two files:
@smallexample
-$ @kbd{tarcat vol-1.tar vol-2.tar vol-3.tar | tar xf -}
+@group
+$ ls
+-rw-r--r-- 2 gray staff 4 2007-10-30 15:11 one
+-rw-r--r-- 2 gray staff 4 2007-10-30 15:11 jeden
+@end group
@end smallexample
-You could use this approach for many (although not all) PAX
-format archives as well. However, extracting split members from a PAX
-archive is a much easier task, because PAX volumes are constructed in
-such a way that each part of a split member is extracted as a
-different file by @command{tar} implementations that are not aware of
-GNU extensions. More specifically, the very first part retains its
-original name, and all subsequent parts are named using the pattern:
+Here, @file{jeden} is a link to @file{one}. When archiving this
+directory with a verbose level 2, you will get an output similar to
+the following:
@smallexample
-%d/GNUFileParts.%p/%f.%n
+$ tar cfvv ../archive.tar .
+drwxr-xr-x gray/staff 0 2007-10-30 15:13 ./
+-rw-r--r-- gray/staff 4 2007-10-30 15:11 ./jeden
+hrw-r--r-- gray/staff 0 2007-10-30 15:11 ./one link to ./jeden
@end smallexample
-@noindent
-where symbols preceeded by @samp{%} are @dfn{macro characters} that
-have the following meaning:
-
-@multitable @columnfractions .25 .55
-@headitem Meta-character @tab Replaced By
-@item %d @tab The directory name of the file, equivalent to the
-result of the @command{dirname} utility on its full name.
-@item %f @tab The file name of the file, equivalent to the result
-of the @command{basename} utility on its full name.
-@item %p @tab The process ID of the @command{tar} process that
-created the archive.
-@item %n @tab Ordinal number of this particular part.
-@end multitable
+The last line shows that, instead of storing two copies of the file,
+@command{tar} stored it only once, under the name @file{jeden}, and
+stored file @file{one} as a hard link to this file.
-For example, if, a file @file{var/longfile} was split during archive
-creation between three volumes, and the creator @command{tar} process
-had process ID @samp{27962}, then the member names will be:
+It may be important to know that all hard links to the given file are
+stored in the archive. For example, this may be necessary for exact
+reproduction of the file system. The following option does that:
-@smallexample
-var/longfile
-var/GNUFileParts.27962/longfile.1
-var/GNUFileParts.27962/longfile.2
-@end smallexample
+@table @option
+@xopindex{check-links, described}
+@item --check-links
+@itemx -l
+Check the number of links dumped for each processed file. If this
+number does not match the total number of hard links for the file, print
+a warning message.
+@end table
-When you extract your archive using a third-party @command{tar}, these
-files will be created on your disk, and the only thing you will need
-to do to restore your file in its original form is concatenate them in
-the proper order, for example:
+For example, trying to archive only file @file{jeden} with this option
+produces the following diagnostics:
@smallexample
-@group
-$ @kbd{cd var}
-$ @kbd{cat GNUFileParts.27962/longfile.1 \
- GNUFileParts.27962/longfile.2 >> longfile}
-$ rm -f GNUFileParts.27962
-@end group
+$ tar -c -f ../archive.tar jeden
+tar: Missing links to `jeden'.
@end smallexample
-Notice, that if the @command{tar} implementation you use supports PAX
-format archives, it will probably emit warnings about unknown keywords
-during extraction. They will lool like this:
+Although creating special records for hard links helps keep a faithful
+record of the file system contents and makes archives more compact, it
+may present some difficulties when extracting individual members from
+the archive. For example, trying to extract file @file{one} from the
+archive created in previous examples produces, in the absense of file
+@file{jeden}:
@smallexample
-@group
-Tar file too small
-Unknown extended header keyword 'GNU.volume.filename' ignored.
-Unknown extended header keyword 'GNU.volume.size' ignored.
-Unknown extended header keyword 'GNU.volume.offset' ignored.
-@end group
+$ tar xf archive.tar ./one
+tar: ./one: Cannot hard link to `./jeden': No such file or directory
+tar: Error exit delayed from previous errors
@end smallexample
-@noindent
-You can safely ignore these warnings.
+The reason for this behavior is that @command{tar} cannot seek back in
+the archive to the previous member (in this case, @file{one}), to
+extract it@footnote{There are plans to fix this in future releases.}.
+If you wish to avoid such problems at the cost of a bigger archive,
+use the following option:
-If your @command{tar} implementation is not PAX-aware, you will get
-more warnigns and more files generated on your disk, e.g.:
+@table @option
+@xopindex{hard-dereference, described}
+@item --hard-dereference
+Dereference hard links and store the files they refer to.
+@end table
+
+For example, trying this option on our two sample files, we get two
+copies in the archive, each of which can then be extracted
+independently of the other:
@smallexample
@group
-$ @kbd{tar xf vol-1.tar}
-var/PaxHeaders.27962/longfile: Unknown file type 'x', extracted as
-normal file
-Unexpected EOF in archive
-$ @kbd{tar xf vol-2.tar}
-tmp/GlobalHead.27962.1: Unknown file type 'g', extracted as normal file
-GNUFileParts.27962/PaxHeaders.27962/sparsefile.1: Unknown file type
-'x', extracted as normal file
+$ tar -c -vv -f ../archive.tar --hard-dereference .
+drwxr-xr-x gray/staff 0 2007-10-30 15:13 ./
+-rw-r--r-- gray/staff 4 2007-10-30 15:11 ./jeden
+-rw-r--r-- gray/staff 4 2007-10-30 15:11 ./one
@end group
@end smallexample
-Ignore these warnings. The @file{PaxHeaders.*} directories created
-will contain files with @dfn{extended header keywords} describing the
-extracted files. You can delete them, unless they describe sparse
-members. Read further to learn more about them.
+@node old
+@subsection Old V7 Archives
+@cindex Format, old style
+@cindex Old style format
+@cindex Old style archives
+@cindex v7 archive format
-@node Sparse Recovery
-@subsubsection Extracting Sparse Members
+Certain old versions of @command{tar} cannot handle additional
+information recorded by newer @command{tar} programs. To create an
+archive in V7 format (not ANSI), which can be read by these old
+versions, specify the @option{--format=v7} option in
+conjunction with the @option{--create} (@option{-c}) (@command{tar} also
+accepts @option{--portability} or @option{--old-archive} for this
+option). When you specify it,
+@command{tar} leaves out information about directories, pipes, fifos,
+contiguous files, and device files, and specifies file ownership by
+group and user IDs instead of group and user names.
-Any @command{tar} implementation will be able to extract sparse members from a
-PAX archive. However, the extracted files will be @dfn{condensed},
-i.e. any zero blocks will be removed from them. When we restore such
-a condensed file to its original form, by adding zero bloks (or
-@dfn{holes}) back to their original locations, we call this process
-@dfn{expanding} a compressed sparse file.
+When updating an archive, do not use @option{--format=v7}
+unless the archive was created using this option.
-To expand a file, you will need a simple auxiliary program called
-@command{xsparse}. It is available in source form from
-@uref{http://www.gnu.org/@/software/@/tar/@/utils/@/xsparse, @GNUTAR{}
-home page}.
+In most cases, a @emph{new} format archive can be read by an @emph{old}
+@command{tar} program without serious trouble, so this option should
+seldom be needed. On the other hand, most modern @command{tar}s are
+able to read old format archives, so it might be safer for you to
+always use @option{--format=v7} for your distributions. Notice,
+however, that @samp{ustar} format is a better alternative, as it is
+free from many of @samp{v7}'s drawbacks.
-Let's begin with archive members in @dfn{sparse format
-version 1.0}@footnote{@xref{PAX 1}.}, which are the easiest to expand.
-The condensed file will contain both file map and file data, so no
-additional data will be needed to restore it. If the original file
-name was @file{@var{dir}/@var{name}}, then the condensed file will be
-named @file{@var{dir}/@/GNUSparseFile.@var{n}/@/@var{name}}, where
-@var{n} is a decimal number@footnote{technically speaking, @var{n} is a
-@dfn{process ID} of the @command{tar} process which created the
-archive (@pxref{PAX keywords}).}.
+@node ustar
+@subsection Ustar Archive Format
-To expand a version 1.0 file, run @command{xsparse} as follows:
+@cindex ustar archive format
+Archive format defined by @acronym{POSIX}.1-1988 specification is called
+@code{ustar}. Although it is more flexible than the V7 format, it
+still has many restrictions (@xref{Formats,ustar}, for the detailed
+description of @code{ustar} format). Along with V7 format,
+@code{ustar} format is a good choice for archives intended to be read
+with other implementations of @command{tar}.
-@smallexample
-$ @kbd{xsparse @file{cond-file}}
-@end smallexample
+To create archive in @code{ustar} format, use @option{--format=ustar}
+option in conjunction with the @option{--create} (@option{-c}).
-@noindent
-where @file{cond-file} is the name of the condensed file. The utility
-will deduce the name for the resulting expanded file using the
-following algorithm:
+@node gnu
+@subsection @acronym{GNU} and old @GNUTAR{} format
-@enumerate 1
-@item If @file{cond-file} does not contain any directories,
-@file{../cond-file} will be used;
+@cindex GNU archive format
+@cindex Old GNU archive format
+@GNUTAR{} was based on an early draft of the
+@acronym{POSIX} 1003.1 @code{ustar} standard. @acronym{GNU} extensions to
+@command{tar}, such as the support for file names longer than 100
+characters, use portions of the @command{tar} header record which were
+specified in that @acronym{POSIX} draft as unused. Subsequent changes in
+@acronym{POSIX} have allocated the same parts of the header record for
+other purposes. As a result, @GNUTAR{} format is
+incompatible with the current @acronym{POSIX} specification, and with
+@command{tar} programs that follow it.
-@item If @file{cond-file} has the form
-@file{@var{dir}/@var{t}/@var{name}}, where both @var{t} and @var{name}
-are simple names, with no @samp{/} characters in them, the output file
-name will be @file{@var{dir}/@var{name}}.
+In the majority of cases, @command{tar} will be configured to create
+this format by default. This will change in future releases, since
+we plan to make @samp{POSIX} format the default.
-@item Otherwise, if @file{cond-file} has the form
-@file{@var{dir}/@var{name}}, the output file name will be
-@file{@var{name}}.
-@end enumerate
+To force creation a @GNUTAR{} archive, use option
+@option{--format=gnu}.
-In the unlikely case when this algorithm does not suite your needs,
-you can explicitely specify output file name as a second argument to
-the command:
+@node posix
+@subsection @GNUTAR{} and @acronym{POSIX} @command{tar}
-@smallexample
-$ @kbd{xsparse @file{cond-file}}
-@end smallexample
+@cindex POSIX archive format
+@cindex PAX archive format
+Starting from version 1.14 @GNUTAR{} features full support for
+@acronym{POSIX.1-2001} archives.
-It is often a good idea to run @command{xsparse} in @dfn{dry run} mode
-first. In this mode, the command does not actually expand the file,
-but verbosely lists all actions it would be taking to do so. The dry
-run mode is enabled by @option{-n} command line argument:
+A @acronym{POSIX} conformant archive will be created if @command{tar}
+was given @option{--format=posix} (@option{--format=pax}) option. No
+special option is required to read and extract from a @acronym{POSIX}
+archive.
-@smallexample
-@group
-$ @kbd{xsparse -n /home/gray/GNUSparseFile.6058/sparsefile}
-Reading v.1.0 sparse map
-Expanding file `/home/gray/GNUSparseFile.6058/sparsefile' to
-`/home/gray/sparsefile'
-Finished dry run
-@end group
-@end smallexample
+@menu
+* PAX keywords:: Controlling Extended Header Keywords.
+@end menu
-To actually expand the file, you would run:
+@node PAX keywords
+@subsubsection Controlling Extended Header Keywords
-@smallexample
-$ @kbd{xsparse /home/gray/GNUSparseFile.6058/sparsefile}
-@end smallexample
+@table @option
+@opindex pax-option
+@item --pax-option=@var{keyword-list}
+Handle keywords in @acronym{PAX} extended headers. This option is
+equivalent to @option{-o} option of the @command{pax} utility.
+@end table
-@noindent
-The program behaves the same way all UNIX utilities do: it will keep
-quiet unless it has simething important to tell you (e.g. an error
-condition or something). If you wish it to produce verbose output,
-similar to that from the dry run mode, give it @option{-v} option:
+@var{Keyword-list} is a comma-separated
+list of keyword options, each keyword option taking one of
+the following forms:
-@smallexample
-@group
-$ @kbd{xsparse -v /home/gray/GNUSparseFile.6058/sparsefile}
-Reading v.1.0 sparse map
-Expanding file `/home/gray/GNUSparseFile.6058/sparsefile' to
-`/home/gray/sparsefile'
-Done
-@end group
-@end smallexample
+@table @code
+@item delete=@var{pattern}
+When used with one of archive-creation commands,
+this option instructs @command{tar} to omit from extended header records
+that it produces any keywords matching the string @var{pattern}.
-Additionally, if your @command{tar} implementation has extracted the
-@dfn{extended headers} for this file, you can instruct @command{xstar}
-to use them in order to verify the integrity of the expanded file.
-The option @option{-x} sets the name of the extended header file to
-use. Continuing our example:
+When used in extract or list mode, this option instructs tar
+to ignore any keywords matching the given @var{pattern} in the extended
+header records. In both cases, matching is performed using the pattern
+matching notation described in @acronym{POSIX 1003.2}, 3.13
+(@pxref{wildcards}). For example:
@smallexample
-@group
-$ @kbd{xsparse -v -x /home/gray/PaxHeaders.6058/sparsefile \
- /home/gray/GNUSparseFile.6058/sparsefile}
-Reading extended header file
-Found variable GNU.sparse.major = 1
-Found variable GNU.sparse.minor = 0
-Found variable GNU.sparse.name = sparsefile
-Found variable GNU.sparse.realsize = 217481216
-Reading v.1.0 sparse map
-Expanding file `/home/gray/GNUSparseFile.6058/sparsefile' to
-`/home/gray/sparsefile'
-Done
-@end group
+--pax-option delete=security.*
@end smallexample
-An @dfn{extended header} is a special @command{tar} archive header
-that precedes an archive member and contains a set of
-@dfn{variables}, describing the member properties that cannot be
-stored in the standard @code{ustar} header. While optional for
-expanding sparse version 1.0 members, use of extended headers is
-mandatory when expanding sparse members in older sparse formats: v.0.0
-and v.0.1 (The sparse formats are described in detail in @pxref{Sparse
-Formats}). So, for this format, the question is: how to obtain
-extended headers from the archive?
+would suppress security-related information.
-If you use a @command{tar} implementation that does not support PAX
-format, extended headers for each member will be extracted as a
-separate file. If we represent the member name as
-@file{@var{dir}/@var{name}}, then the extended header file will be
-named @file{@var{dir}/@/PaxHeaders.@var{n}/@/@var{name}}, where
-@var{n} is an integer number.
+@item exthdr.name=@var{string}
-Things become more difficult if your @command{tar} implementation
-does support PAX headers, because in this case you will have to
-manually extract the headers. We recommend the following algorithm:
+This keyword allows user control over the name that is written into the
+ustar header blocks for the extended headers. The name is obtained
+from @var{string} after making the following substitutions:
-@enumerate 1
-@item
-Consult the documentation for your @command{tar} implementation for an
-option that will print @dfn{block numbers} along with the archive
-listing (analogous to @GNUTAR{}'s @option{-R} option). For example,
-@command{star} has @option{-block-number}.
+@multitable @columnfractions .25 .55
+@headitem Meta-character @tab Replaced By
+@item %d @tab The directory name of the file, equivalent to the
+result of the @command{dirname} utility on the translated file name.
+@item %f @tab The name of the file with the directory information
+stripped, equivalent to the result of the @command{basename} utility
+on the translated file name.
+@item %p @tab The process @acronym{ID} of the @command{tar} process.
+@item %% @tab A @samp{%} character.
+@end multitable
-@item
-Obtain the verbose listing using the @samp{block number} option, and
-find the position of the sparse member in question and the member
-immediately following it. For example, running @command{star} on our
-archive we obtain:
+Any other @samp{%} characters in @var{string} produce undefined
+results.
+
+If no option @samp{exthdr.name=string} is specified, @command{tar}
+will use the following default value:
@smallexample
-@group
-$ @kbd{star -t -v -block-number -f arc.tar}
-@dots{}
-star: Unknown extended header keyword 'GNU.sparse.size' ignored.
-star: Unknown extended header keyword 'GNU.sparse.numblocks' ignored.
-star: Unknown extended header keyword 'GNU.sparse.name' ignored.
-star: Unknown extended header keyword 'GNU.sparse.map' ignored.
-block 56: 425984 -rw-r--r-- gray/users Jun 25 14:46 2006 GNUSparseFile.28124/sparsefile
-block 897: 65391 -rw-r--r-- gray/users Jun 24 20:06 2006 README
-@dots{}
-@end group
+%d/PaxHeaders.%p/%f
@end smallexample
-@noindent
-(as usual, ignore the warnings about unknown keywords.)
-
-@item
-Let the size of the sparse member be @var{size}, its block number be
-@var{Bs} and the block number of the next member be @var{Bn}.
-Compute:
+@item globexthdr.name=@var{string}
+This keyword allows user control over the name that is written into
+the ustar header blocks for global extended header records. The name
+is obtained from the contents of @var{string}, after making
+the following substitutions:
-@smallexample
-@var{N} = @var{Bs} - @var{Bn} - @var{size}/512 - 2
-@end smallexample
+@multitable @columnfractions .25 .55
+@headitem Meta-character @tab Replaced By
+@item %n @tab An integer that represents the
+sequence number of the global extended header record in the archive,
+starting at 1.
+@item %p @tab The process @acronym{ID} of the @command{tar} process.
+@item %% @tab A @samp{%} character.
+@end multitable
-@noindent
-This number gives the size of the extended header part in tar @dfn{blocks}.
-In our example, this formula gives: @code{897 - 56 - 425984 / 512 - 2
-= 7}.
+Any other @samp{%} characters in @var{string} produce undefined results.
-@item
-Use @command{dd} to extract the headers:
+If no option @samp{globexthdr.name=string} is specified, @command{tar}
+will use the following default value:
@smallexample
-@kbd{dd if=@var{archive} of=@var{hname} bs=512 skip=@var{Bs} count=@var{N}}
+$TMPDIR/GlobalHead.%p.%n
@end smallexample
@noindent
-where @var{archive} is the archive name, @var{hname} is a name of the
-file to store the extended header in, @var{Bs} and @var{N} are
-computed in previous steps.
+where @samp{$TMPDIR} represents the value of the @var{TMPDIR}
+environment variable. If @var{TMPDIR} is not set, @command{tar}
+uses @samp{/tmp}.
-In our example, this command will be
+@item @var{keyword}=@var{value}
+When used with one of archive-creation commands, these keyword/value pairs
+will be included at the beginning of the archive in a global extended
+header record. When used with one of archive-reading commands,
+@command{tar} will behave as if it has encountered these keyword/value
+pairs at the beginning of the archive in a global extended header
+record.
-@smallexample
-$ @kbd{dd if=arc.tar of=xhdr bs=512 skip=56 count=7}
-@end smallexample
-@end enumerate
+@item @var{keyword}:=@var{value}
+When used with one of archive-creation commands, these keyword/value pairs
+will be included as records at the beginning of an extended header for
+each file. This is effectively equivalent to @var{keyword}=@var{value}
+form except that it creates no global extended header records.
-Finally, you can expand the condensed file, using the obtained header:
+When used with one of archive-reading commands, @command{tar} will
+behave as if these keyword/value pairs were included as records at the
+end of each extended header; thus, they will override any global or
+file-specific extended header record keywords of the same names.
+For example, in the command:
@smallexample
-@group
-$ @kbd{xsparse -v -x xhdr GNUSparseFile.6058/sparsefile}
-Reading extended header file
-Found variable GNU.sparse.size = 217481216
-Found variable GNU.sparse.numblocks = 208
-Found variable GNU.sparse.name = sparsefile
-Found variable GNU.sparse.map = 0,2048,1050624,2048,@dots{}
-Expanding file `GNUSparseFile.28124/sparsefile' to `sparsefile'
-Done
-@end group
+tar --format=posix --create \
+ --file archive --pax-option gname:=user .
@end smallexample
-@node Compression
-@section Using Less Space through Compression
+the group name will be forced to a new value for all files
+stored in the archive.
+@end table
-@menu
-* gzip:: Creating and Reading Compressed Archives
-* sparse:: Archiving Sparse Files
-@end menu
+@node Checksumming
+@subsection Checksumming Problems
-@node gzip
-@subsection Creating and Reading Compressed Archives
-@cindex Compressed archives
-@cindex Storing archives in compressed format
+SunOS and HP-UX @command{tar} fail to accept archives created using
+@GNUTAR{} and containing non-@acronym{ASCII} file names, that
+is, file names having characters with the eight bit set, because they
+use signed checksums, while @GNUTAR{} uses unsigned
+checksums while creating archives, as per @acronym{POSIX} standards. On
+reading, @GNUTAR{} computes both checksums and
+accept any. It is somewhat worrying that a lot of people may go
+around doing backup of their files using faulty (or at least
+non-standard) software, not learning about it until it's time to
+restore their missing files with an incompatible file extractor, or
+vice versa.
-@GNUTAR{} is able to create and read compressed archives. It supports
-@command{gzip} and @command{bzip2} compression programs. For backward
-compatibilty, it also supports @command{compress} command, although
-we strongly recommend against using it, since there is a patent
-covering the algorithm it uses and you could be sued for patent
-infringement merely by running @command{compress}! Besides, it is less
-effective than @command{gzip} and @command{bzip2}.
+@GNUTAR{} compute checksums both ways, and accept
+any on read, so @acronym{GNU} tar can read Sun tapes even with their
+wrong checksums. @GNUTAR{} produces the standard
+checksum, however, raising incompatibilities with Sun. That is to
+say, @GNUTAR{} has not been modified to
+@emph{produce} incorrect archives to be read by buggy @command{tar}'s.
+I've been told that more recent Sun @command{tar} now read standard
+archives, so maybe Sun did a similar patch, after all?
-Creating a compressed archive is simple: you just specify a
-@dfn{compression option} along with the usual archive creation
-commands. The compression option is @option{-z} (@option{--gzip}) to
-create a @command{gzip} compressed archive, @option{-j}
-(@option{--bzip2}) to create a @command{bzip2} compressed archive, and
-@option{-Z} (@option{--compress}) to use @command{compress} program.
-For example:
+The story seems to be that when Sun first imported @command{tar}
+sources on their system, they recompiled it without realizing that
+the checksums were computed differently, because of a change in
+the default signing of @code{char}'s in their compiler. So they
+started computing checksums wrongly. When they later realized their
+mistake, they merely decided to stay compatible with it, and with
+themselves afterwards. Presumably, but I do not really know, HP-UX
+has chosen that their @command{tar} archives to be compatible with Sun's.
+The current standards do not favor Sun @command{tar} format. In any
+case, it now falls on the shoulders of SunOS and HP-UX users to get
+a @command{tar} able to read the good archives they receive.
-@smallexample
-$ @kbd{tar cfz archive.tar.gz .}
-@end smallexample
+@node Large or Negative Values
+@subsection Large or Negative Values
+@cindex large values
+@cindex future time stamps
+@cindex negative time stamps
+@UNREVISED{}
-Reading compressed archive is even simpler: you don't need to specify
-any additional options as @GNUTAR{} recognizes its format
-automatically. Thus, the following commands will list and extract the
-archive created in previous example:
+The above sections suggest to use @samp{oldest possible} archive
+format if in doubt. However, sometimes it is not possible. If you
+attempt to archive a file whose metadata cannot be represented using
+required format, @GNUTAR{} will print error message and ignore such a
+file. You will than have to switch to a format that is able to
+handle such values. The format summary table (@pxref{Formats}) will
+help you to do so.
-@smallexample
-# List the compressed archive
-$ @kbd{tar tf archive.tar.gz}
-# Extract the compressed archive
-$ @kbd{tar xf archive.tar.gz}
-@end smallexample
+In particular, when trying to archive files larger than 8GB or with
+timestamps not in the range 1970-01-01 00:00:00 through 2242-03-16
+12:56:31 @sc{utc}, you will have to chose between @acronym{GNU} and
+@acronym{POSIX} archive formats. When considering which format to
+choose, bear in mind that the @acronym{GNU} format uses
+two's-complement base-256 notation to store values that do not fit
+into standard @acronym{ustar} range. Such archives can generally be
+read only by a @GNUTAR{} implementation. Moreover, they sometimes
+cannot be correctly restored on another hosts even by @GNUTAR{}. For
+example, using two's complement representation for negative time
+stamps that assumes a signed 32-bit @code{time_t} generates archives
+that are not portable to hosts with differing @code{time_t}
+representations.
-The only case when you have to specify a decompression option while
-reading the archive is when reading from a pipe or from a tape drive
-that does not support random access. However, in this case @GNUTAR{}
-will indicate which option you should use. For example:
+On the other hand, @acronym{POSIX} archives, generally speaking, can
+be extracted by any tar implementation that understands older
+@acronym{ustar} format. The only exception are files larger than 8GB.
-@smallexample
-$ @kbd{cat archive.tar.gz | tar tf -}
-tar: Archive is compressed. Use -z option
-tar: Error is not recoverable: exiting now
-@end smallexample
+@FIXME{Describe how @acronym{POSIX} archives are extracted by non
+POSIX-aware tars.}
-If you see such diagnostics, just add the suggested option to the
-invocation of @GNUTAR{}:
+@node Other Tars
+@subsection How to Extract GNU-Specific Data Using Other @command{tar} Implementations
-@smallexample
-$ @kbd{cat archive.tar.gz | tar tfz -}
-@end smallexample
+In previous sections you became acquainted with various quirks
+necessary to make your archives portable. Sometimes you may need to
+extract archives containing GNU-specific members using some
+third-party @command{tar} implementation or an older version of
+@GNUTAR{}. Of course your best bet is to have @GNUTAR{} installed,
+but if it is for some reason impossible, this section will explain
+how to cope without it.
-Notice also, that there are several restrictions on operations on
-compressed archives. First of all, compressed archives cannot be
-modified, i.e., you cannot update (@option{--update} (@option{-u})) them or delete
-(@option{--delete}) members from them. Likewise, you cannot append
-another @command{tar} archive to a compressed archive using
-@option{--append} (@option{-r})). Secondly, multi-volume archives cannot be
-compressed.
+When we speak about @dfn{GNU-specific} members we mean two classes of
+them: members split between the volumes of a multi-volume archive and
+sparse members. You will be able to always recover such members if
+the archive is in PAX format. In addition split members can be
+recovered from archives in old GNU format. The following subsections
+describe the required procedures in detail.
-The following table summarizes compression options used by @GNUTAR{}.
+@menu
+* Split Recovery:: Members Split Between Volumes
+* Sparse Recovery:: Sparse Members
+@end menu
-@table @option
-@opindex gzip
-@opindex ungzip
-@item -z
-@itemx --gzip
-@itemx --ungzip
-Filter the archive through @command{gzip}.
+@node Split Recovery
+@subsubsection Extracting Members Split Between Volumes
-You can use @option{--gzip} and @option{--gunzip} on physical devices
-(tape drives, etc.) and remote files as well as on normal files; data
-to or from such devices or remote files is reblocked by another copy
-of the @command{tar} program to enforce the specified (or default) record
-size. The default compression parameters are used; if you need to
-override them, set @env{GZIP} environment variable, e.g.:
+@cindex Mutli-volume archives, extracting using non-GNU tars
+If a member is split between several volumes of an old GNU format archive
+most third party @command{tar} implementation will fail to extract
+it. To extract it, use @command{tarcat} program (@pxref{Tarcat}).
+This program is available from
+@uref{http://www.gnu.org/@/software/@/tar/@/utils/@/tarcat.html, @GNUTAR{}
+home page}. It concatenates several archive volumes into a single
+valid archive. For example, if you have three volumes named from
+@file{vol-1.tar} to @file{vol-3.tar}, you can do the following to
+extract them using a third-party @command{tar}:
@smallexample
-$ @kbd{GZIP=--best tar cfz archive.tar.gz subdir}
+$ @kbd{tarcat vol-1.tar vol-2.tar vol-3.tar | tar xf -}
@end smallexample
-@noindent
-Another way would be to avoid the @option{--gzip} (@option{--gunzip}, @option{--ungzip}, @option{-z}) option and run
-@command{gzip} explicitly:
+@cindex Mutli-volume archives in PAX format, extracting using non-GNU tars
+You could use this approach for most (although not all) PAX
+format archives as well. However, extracting split members from a PAX
+archive is a much easier task, because PAX volumes are constructed in
+such a way that each part of a split member is extracted to a
+different file by @command{tar} implementations that are not aware of
+GNU extensions. More specifically, the very first part retains its
+original name, and all subsequent parts are named using the pattern:
@smallexample
-$ @kbd{tar cf - subdir | gzip --best -c - > archive.tar.gz}
+%d/GNUFileParts.%p/%f.%n
@end smallexample
-@cindex corrupted archives
-About corrupted compressed archives: @command{gzip}'ed files have no
-redundancy, for maximum compression. The adaptive nature of the
-compression scheme means that the compression tables are implicitly
-spread all over the archive. If you lose a few blocks, the dynamic
-construction of the compression tables becomes unsynchronized, and there
-is little chance that you could recover later in the archive.
-
-There are pending suggestions for having a per-volume or per-file
-compression in @GNUTAR{}. This would allow for viewing the
-contents without decompression, and for resynchronizing decompression at
-every volume or file, in case of corrupted archives. Doing so, we might
-lose some compressibility. But this would have make recovering easier.
-So, there are pros and cons. We'll see!
-
-@opindex bzip2
-@item -j
-@itemx --bzip2
-Filter the archive through @code{bzip2}. Otherwise like @option{--gzip}.
-
-@opindex compress
-@opindex uncompress
-@item -Z
-@itemx --compress
-@itemx --uncompress
-Filter the archive through @command{compress}. Otherwise like @option{--gzip}.
-
-The @acronym{GNU} Project recommends you not use
-@command{compress}, because there is a patent covering the algorithm it
-uses. You could be sued for patent infringement merely by running
-@command{compress}.
+@noindent
+where symbols preceeded by @samp{%} are @dfn{macro characters} that
+have the following meaning:
-@opindex use-compress-program
-@item --use-compress-program=@var{prog}
-Use external compression program @var{prog}. Use this option if you
-have a compression program that @GNUTAR{} does not support. There
-are two requirements to which @var{prog} should comply:
+@multitable @columnfractions .25 .55
+@headitem Meta-character @tab Replaced By
+@item %d @tab The directory name of the file, equivalent to the
+result of the @command{dirname} utility on its full name.
+@item %f @tab The file name of the file, equivalent to the result
+of the @command{basename} utility on its full name.
+@item %p @tab The process @acronym{ID} of the @command{tar} process that
+created the archive.
+@item %n @tab Ordinal number of this particular part.
+@end multitable
-First, when called without options, it should read data from standard
-input, compress it and output it on standard output.
+For example, if the file @file{var/longfile} was split during archive
+creation between three volumes, and the creator @command{tar} process
+had process @acronym{ID} @samp{27962}, then the member names will be:
-Secondly, if called with @option{-d} argument, it should do exactly
-the opposite, i.e., read the compressed data from the standard input
-and produce uncompressed data on the standard output.
-@end table
+@smallexample
+var/longfile
+var/GNUFileParts.27962/longfile.1
+var/GNUFileParts.27962/longfile.2
+@end smallexample
-@cindex gpg, using with tar
-@cindex gnupg, using with tar
-@cindex Using encrypted archives
-The @option{--use-compress-program} option, in particular, lets you
-implement your own filters, not necessarily dealing with
-compression/decomression. For example, suppose you wish to implement
-PGP encryption on top of compression, using @command{gpg} (@pxref{Top,
-gpg, gpg ---- encryption and signing tool, gpg, GNU Privacy Guard
-Manual}). The following script does that:
+When you extract your archive using a third-party @command{tar}, these
+files will be created on your disk, and the only thing you will need
+to do to restore your file in its original form is concatenate them in
+the proper order, for example:
@smallexample
@group
-#! /bin/sh
-case $1 in
--d) gpg --decrypt - | gzip -d -c;;
-'') gzip -c | gpg -s ;;
-*) echo "Unknown option $1">&2; exit 1;;
-esac
+$ @kbd{cd var}
+$ @kbd{cat GNUFileParts.27962/longfile.1 \
+ GNUFileParts.27962/longfile.2 >> longfile}
+$ rm -f GNUFileParts.27962
@end group
@end smallexample
-Suppose you name it @file{gpgz} and save it somewhere in your
-@env{PATH}. Then the following command will create a commpressed
-archive signed with your private key:
+Notice, that if the @command{tar} implementation you use supports PAX
+format archives, it will probably emit warnings about unknown keywords
+during extraction. They will look like this:
@smallexample
-$ @kbd{tar -cf foo.tar.gpgz --use-compress=gpgz .}
+@group
+Tar file too small
+Unknown extended header keyword 'GNU.volume.filename' ignored.
+Unknown extended header keyword 'GNU.volume.size' ignored.
+Unknown extended header keyword 'GNU.volume.offset' ignored.
+@end group
@end smallexample
@noindent
-Likewise, the following command will list its contents:
+You can safely ignore these warnings.
+
+If your @command{tar} implementation is not PAX-aware, you will get
+more warnings and more files generated on your disk, e.g.:
@smallexample
-$ @kbd{tar -tf foo.tar.gpgz --use-compress=gpgz .}
+@group
+$ @kbd{tar xf vol-1.tar}
+var/PaxHeaders.27962/longfile: Unknown file type 'x', extracted as
+normal file
+Unexpected EOF in archive
+$ @kbd{tar xf vol-2.tar}
+tmp/GlobalHead.27962.1: Unknown file type 'g', extracted as normal file
+GNUFileParts.27962/PaxHeaders.27962/sparsefile.1: Unknown file type
+'x', extracted as normal file
+@end group
@end smallexample
-@ignore
-The above is based on the following discussion:
-
- I have one question, or maybe it's a suggestion if there isn't a way
- to do it now. I would like to use @option{--gzip}, but I'd also like
- the output to be fed through a program like @acronym{GNU}
- @command{ecc} (actually, right now that's @samp{exactly} what I'd like
- to use :-)), basically adding ECC protection on top of compression.
- It seems as if this should be quite easy to do, but I can't work out
- exactly how to go about it. Of course, I can pipe the standard output
- of @command{tar} through @command{ecc}, but then I lose (though I
- haven't started using it yet, I confess) the ability to have
- @command{tar} use @command{rmt} for it's I/O (I think).
-
- I think the most straightforward thing would be to let me specify a
- general set of filters outboard of compression (preferably ordered,
- so the order can be automatically reversed on input operations, and
- with the options they require specifiable), but beggars shouldn't be
- choosers and anything you decide on would be fine with me.
-
- By the way, I like @command{ecc} but if (as the comments say) it can't
- deal with loss of block sync, I'm tempted to throw some time at adding
- that capability. Supposing I were to actually do such a thing and
- get it (apparently) working, do you accept contributed changes to
- utilities like that? (Leigh Clayton @file{loc@@soliton.com}, May 1995).
-
- Isn't that exactly the role of the
- @option{--use-compress-prog=@var{program}} option?
- I never tried it myself, but I suspect you may want to write a
- @var{prog} script or program able to filter stdin to stdout to
- way you want. It should recognize the @option{-d} option, for when
- extraction is needed rather than creation.
-
- It has been reported that if one writes compressed data (through the
- @option{--gzip} or @option{--compress} options) to a DLT and tries to use
- the DLT compression mode, the data will actually get bigger and one will
- end up with less space on the tape.
-@end ignore
-
-@node sparse
-@subsection Archiving Sparse Files
-@cindex Sparse Files
-@UNREVISED
-
-@table @option
-@opindex sparse
-@item -S
-@itemx --sparse
-Handle sparse files efficiently.
-@end table
-
-This option causes all files to be put in the archive to be tested for
-sparseness, and handled specially if they are. The @option{--sparse}
-(@option{-S}) option is useful when many @code{dbm} files, for example, are being
-backed up. Using this option dramatically decreases the amount of
-space needed to store such a file.
-
-In later versions, this option may be removed, and the testing and
-treatment of sparse files may be done automatically with any special
-@acronym{GNU} options. For now, it is an option needing to be specified on
-the command line with the creation or updating of an archive.
+Ignore these warnings. The @file{PaxHeaders.*} directories created
+will contain files with @dfn{extended header keywords} describing the
+extracted files. You can delete them, unless they describe sparse
+members. Read further to learn more about them.
-Files in the file system occasionally have @dfn{holes}. A @dfn{hole} in a file
-is a section of the file's contents which was never written. The
-contents of a hole read as all zeros. On many operating systems,
-actual disk storage is not allocated for holes, but they are counted
-in the length of the file. If you archive such a file, @command{tar}
-could create an archive longer than the original. To have @command{tar}
-attempt to recognize the holes in a file, use @option{--sparse} (@option{-S}). When
-you use this option, then, for any file using less disk space than
-would be expected from its length, @command{tar} searches the file for
-consecutive stretches of zeros. It then records in the archive for
-the file where the consecutive stretches of zeros are, and only
-archives the ``real contents'' of the file. On extraction (using
-@option{--sparse} is not needed on extraction) any such
-files have holes created wherever the continuous stretches of zeros
-were found. Thus, if you use @option{--sparse}, @command{tar} archives
-won't take more space than the original.
+@node Sparse Recovery
+@subsubsection Extracting Sparse Members
-A file is sparse if it contains blocks of zeros whose existence is
-recorded, but that have no space allocated on disk. When you specify
-the @option{--sparse} option in conjunction with the @option{--create}
-(@option{-c}) operation, @command{tar} tests all files for sparseness
-while archiving. If @command{tar} finds a file to be sparse, it uses a
-sparse representation of the file in the archive. @xref{create}, for
-more information about creating archives.
+@cindex sparse files, extracting with non-GNU tars
+Any @command{tar} implementation will be able to extract sparse members from a
+PAX archive. However, the extracted files will be @dfn{condensed},
+i.e., any zero blocks will be removed from them. When we restore such
+a condensed file to its original form, by adding zero blocks (or
+@dfn{holes}) back to their original locations, we call this process
+@dfn{expanding} a compressed sparse file.
-@option{--sparse} is useful when archiving files, such as dbm files,
-likely to contain many nulls. This option dramatically
-decreases the amount of space needed to store such an archive.
+@pindex xsparse
+To expand a file, you will need a simple auxiliary program called
+@command{xsparse}. It is available in source form from
+@uref{http://www.gnu.org/@/software/@/tar/@/utils/@/xsparse.html, @GNUTAR{}
+home page}.
-@quotation
-@strong{Please Note:} Always use @option{--sparse} when performing file
-system backups, to avoid archiving the expanded forms of files stored
-sparsely in the system.
+@cindex sparse files v.1.0, extracting with non-GNU tars
+Let's begin with archive members in @dfn{sparse format
+version 1.0}@footnote{@xref{PAX 1}.}, which are the easiest to expand.
+The condensed file will contain both file map and file data, so no
+additional data will be needed to restore it. If the original file
+name was @file{@var{dir}/@var{name}}, then the condensed file will be
+named @file{@var{dir}/@/GNUSparseFile.@var{n}/@/@var{name}}, where
+@var{n} is a decimal number@footnote{technically speaking, @var{n} is a
+@dfn{process @acronym{ID}} of the @command{tar} process which created the
+archive (@pxref{PAX keywords}).}.
-Even if your system has no sparse files currently, some may be
-created in the future. If you use @option{--sparse} while making file
-system backups as a matter of course, you can be assured the archive
-will never take more space on the media than the files take on disk
-(otherwise, archiving a disk filled with sparse files might take
-hundreds of tapes). @xref{Incremental Dumps}.
-@end quotation
+To expand a version 1.0 file, run @command{xsparse} as follows:
-@command{tar} ignores the @option{--sparse} option when reading an archive.
+@smallexample
+$ @kbd{xsparse @file{cond-file}}
+@end smallexample
-@table @option
-@item --sparse
-@itemx -S
-Files stored sparsely in the file system are represented sparsely in
-the archive. Use in conjunction with write operations.
-@end table
+@noindent
+where @file{cond-file} is the name of the condensed file. The utility
+will deduce the name for the resulting expanded file using the
+following algorithm:
-However, users should be well aware that at archive creation time,
-@GNUTAR{} still has to read whole disk file to
-locate the @dfn{holes}, and so, even if sparse files use little space
-on disk and in the archive, they may sometimes require inordinate
-amount of time for reading and examining all-zero blocks of a file.
-Although it works, it's painfully slow for a large (sparse) file, even
-though the resulting tar archive may be small. (One user reports that
-dumping a @file{core} file of over 400 megabytes, but with only about
-3 megabytes of actual data, took about 9 minutes on a Sun Sparcstation
-ELC, with full CPU utilization.)
-
-This reading is required in all cases and is not related to the fact
-the @option{--sparse} option is used or not, so by merely @emph{not}
-using the option, you are not saving time@footnote{Well! We should say
-the whole truth, here. When @option{--sparse} is selected while creating
-an archive, the current @command{tar} algorithm requires sparse files to be
-read twice, not once. We hope to develop a new archive format for saving
-sparse files in which one pass will be sufficient.}.
+@enumerate 1
+@item If @file{cond-file} does not contain any directories,
+@file{../cond-file} will be used;
-Programs like @command{dump} do not have to read the entire file; by
-examining the file system directly, they can determine in advance
-exactly where the holes are and thus avoid reading through them. The
-only data it need read are the actual allocated data blocks.
-@GNUTAR{} uses a more portable and straightforward
-archiving approach, it would be fairly difficult that it does
-otherwise. Elizabeth Zwicky writes to @file{comp.unix.internals}, on
-1990-12-10:
+@item If @file{cond-file} has the form
+@file{@var{dir}/@var{t}/@var{name}}, where both @var{t} and @var{name}
+are simple names, with no @samp{/} characters in them, the output file
+name will be @file{@var{dir}/@var{name}}.
-@quotation
-What I did say is that you cannot tell the difference between a hole and an
-equivalent number of nulls without reading raw blocks. @code{st_blocks} at
-best tells you how many holes there are; it doesn't tell you @emph{where}.
-Just as programs may, conceivably, care what @code{st_blocks} is (care
-to name one that does?), they may also care where the holes are (I have
-no examples of this one either, but it's equally imaginable).
+@item Otherwise, if @file{cond-file} has the form
+@file{@var{dir}/@var{name}}, the output file name will be
+@file{@var{name}}.
+@end enumerate
-I conclude from this that good archivers are not portable. One can
-arguably conclude that if you want a portable program, you can in good
-conscience restore files with as many holes as possible, since you can't
-get it right.
-@end quotation
+In the unlikely case when this algorithm does not suit your needs,
+you can explicitly specify output file name as a second argument to
+the command:
-@node Attributes
-@section Handling File Attributes
-@UNREVISED
+@smallexample
+$ @kbd{xsparse @file{cond-file} @file{out-file}}
+@end smallexample
-When @command{tar} reads files, it updates their access times. To
-avoid this, use the @option{--atime-preserve[=METHOD]} option, which can either
-reset the access time retroactively or avoid changing it in the first
-place.
+It is often a good idea to run @command{xsparse} in @dfn{dry run} mode
+first. In this mode, the command does not actually expand the file,
+but verbosely lists all actions it would be taking to do so. The dry
+run mode is enabled by @option{-n} command line argument:
-Handling of file attributes
+@smallexample
+@group
+$ @kbd{xsparse -n /home/gray/GNUSparseFile.6058/sparsefile}
+Reading v.1.0 sparse map
+Expanding file `/home/gray/GNUSparseFile.6058/sparsefile' to
+`/home/gray/sparsefile'
+Finished dry run
+@end group
+@end smallexample
-@table @option
-@opindex atime-preserve
-@item --atime-preserve
-@itemx --atime-preserve=replace
-@itemx --atime-preserve=system
-Preserve the access times of files that are read. This works only for
-files that you own, unless you have superuser privileges.
+To actually expand the file, you would run:
-@option{--atime-preserve=replace} works on most systems, but it also
-restores the data modification time and updates the status change
-time. Hence it doesn't interact with incremental dumps nicely
-(@pxref{Backups}), and it can set access or data modification times
-incorrectly if other programs access the file while @command{tar} is
-running.
+@smallexample
+$ @kbd{xsparse /home/gray/GNUSparseFile.6058/sparsefile}
+@end smallexample
-@option{--atime-preserve=system} avoids changing the access time in
-the first place, if the operating system supports this.
-Unfortunately, this may or may not work on any given operating system
-or file system. If @command{tar} knows for sure it won't work, it
-complains right away.
+@noindent
+The program behaves the same way all UNIX utilities do: it will keep
+quiet unless it has simething important to tell you (e.g. an error
+condition or something). If you wish it to produce verbose output,
+similar to that from the dry run mode, use @option{-v} option:
-Currently @option{--atime-preserve} with no operand defaults to
-@option{--atime-preserve=replace}, but this is intended to change to
-@option{--atime-preserve=system} when the latter is better-supported.
+@smallexample
+@group
+$ @kbd{xsparse -v /home/gray/GNUSparseFile.6058/sparsefile}
+Reading v.1.0 sparse map
+Expanding file `/home/gray/GNUSparseFile.6058/sparsefile' to
+`/home/gray/sparsefile'
+Done
+@end group
+@end smallexample
-@opindex touch
-@item -m
-@itemx --touch
-Do not extract data modification time.
+Additionally, if your @command{tar} implementation has extracted the
+@dfn{extended headers} for this file, you can instruct @command{xstar}
+to use them in order to verify the integrity of the expanded file.
+The option @option{-x} sets the name of the extended header file to
+use. Continuing our example:
-When this option is used, @command{tar} leaves the data modification times
-of the files it extracts as the times when the files were extracted,
-instead of setting it to the times recorded in the archive.
+@smallexample
+@group
+$ @kbd{xsparse -v -x /home/gray/PaxHeaders.6058/sparsefile \
+ /home/gray/GNUSparseFile.6058/sparsefile}
+Reading extended header file
+Found variable GNU.sparse.major = 1
+Found variable GNU.sparse.minor = 0
+Found variable GNU.sparse.name = sparsefile
+Found variable GNU.sparse.realsize = 217481216
+Reading v.1.0 sparse map
+Expanding file `/home/gray/GNUSparseFile.6058/sparsefile' to
+`/home/gray/sparsefile'
+Done
+@end group
+@end smallexample
-This option is meaningless with @option{--list} (@option{-t}).
+@anchor{extracting sparse v.0.x}
+@cindex sparse files v.0.1, extracting with non-GNU tars
+@cindex sparse files v.0.0, extracting with non-GNU tars
+An @dfn{extended header} is a special @command{tar} archive header
+that precedes an archive member and contains a set of
+@dfn{variables}, describing the member properties that cannot be
+stored in the standard @code{ustar} header. While optional for
+expanding sparse version 1.0 members, the use of extended headers is
+mandatory when expanding sparse members in older sparse formats: v.0.0
+and v.0.1 (The sparse formats are described in detail in @ref{Sparse
+Formats}.) So, for these formats, the question is: how to obtain
+extended headers from the archive?
-@opindex same-owner
-@item --same-owner
-Create extracted files with the same ownership they have in the
-archive.
+If you use a @command{tar} implementation that does not support PAX
+format, extended headers for each member will be extracted as a
+separate file. If we represent the member name as
+@file{@var{dir}/@var{name}}, then the extended header file will be
+named @file{@var{dir}/@/PaxHeaders.@var{n}/@/@var{name}}, where
+@var{n} is an integer number.
-This is the default behavior for the superuser,
-so this option is meaningful only for non-root users, when @command{tar}
-is executed on those systems able to give files away. This is
-considered as a security flaw by many people, at least because it
-makes quite difficult to correctly account users for the disk space
-they occupy. Also, the @code{suid} or @code{sgid} attributes of
-files are easily and silently lost when files are given away.
+Things become more difficult if your @command{tar} implementation
+does support PAX headers, because in this case you will have to
+manually extract the headers. We recommend the following algorithm:
-When writing an archive, @command{tar} writes the user id and user name
-separately. If it can't find a user name (because the user id is not
-in @file{/etc/passwd}), then it does not write one. When restoring,
-it tries to look the name (if one was written) up in
-@file{/etc/passwd}. If it fails, then it uses the user id stored in
-the archive instead.
+@enumerate 1
+@item
+Consult the documentation of your @command{tar} implementation for an
+option that prints @dfn{block numbers} along with the archive
+listing (analogous to @GNUTAR{}'s @option{-R} option). For example,
+@command{star} has @option{-block-number}.
-@opindex no-same-owner
-@item --no-same-owner
-@itemx -o
-Do not attempt to restore ownership when extracting. This is the
-default behavior for ordinary users, so this option has an effect
-only for the superuser.
+@item
+Obtain verbose listing using the @samp{block number} option, and
+find block numbers of the sparse member in question and the member
+immediately following it. For example, running @command{star} on our
+archive we obtain:
-@opindex numeric-owner
-@item --numeric-owner
-The @option{--numeric-owner} option allows (ANSI) archives to be written
-without user/group name information or such information to be ignored
-when extracting. It effectively disables the generation and/or use
-of user/group name information. This option forces extraction using
-the numeric ids from the archive, ignoring the names.
+@smallexample
+@group
+$ @kbd{star -t -v -block-number -f arc.tar}
+@dots{}
+star: Unknown extended header keyword 'GNU.sparse.size' ignored.
+star: Unknown extended header keyword 'GNU.sparse.numblocks' ignored.
+star: Unknown extended header keyword 'GNU.sparse.name' ignored.
+star: Unknown extended header keyword 'GNU.sparse.map' ignored.
+block 56: 425984 -rw-r--r-- gray/users Jun 25 14:46 2006 GNUSparseFile.28124/sparsefile
+block 897: 65391 -rw-r--r-- gray/users Jun 24 20:06 2006 README
+@dots{}
+@end group
+@end smallexample
-This is useful in certain circumstances, when restoring a backup from
-an emergency floppy with different passwd/group files for example.
-It is otherwise impossible to extract files with the right ownerships
-if the password file in use during the extraction does not match the
-one belonging to the file system(s) being extracted. This occurs,
-for example, if you are restoring your files after a major crash and
-had booted from an emergency floppy with no password file or put your
-disk into another machine to do the restore.
+@noindent
+(as usual, ignore the warnings about unknown keywords.)
-The numeric ids are @emph{always} saved into @command{tar} archives.
-The identifying names are added at create time when provided by the
-system, unless @option{--old-archive} (@option{-o}) is used. Numeric ids could be
-used when moving archives between a collection of machines using
-a centralized management for attribution of numeric ids to users
-and groups. This is often made through using the NIS capabilities.
+@item
+Let @var{size} be the size of the sparse member, @var{Bs} be its block number
+and @var{Bn} be the block number of the next member.
+Compute:
-When making a @command{tar} file for distribution to other sites, it
-is sometimes cleaner to use a single owner for all files in the
-distribution, and nicer to specify the write permission bits of the
-files as stored in the archive independently of their actual value on
-the file system. The way to prepare a clean distribution is usually
-to have some Makefile rule creating a directory, copying all needed
-files in that directory, then setting ownership and permissions as
-wanted (there are a lot of possible schemes), and only then making a
-@command{tar} archive out of this directory, before cleaning
-everything out. Of course, we could add a lot of options to
-@GNUTAR{} for fine tuning permissions and ownership.
-This is not the good way, I think. @GNUTAR{} is
-already crowded with options and moreover, the approach just explained
-gives you a great deal of control already.
+@smallexample
+@var{N} = @var{Bs} - @var{Bn} - @var{size}/512 - 2
+@end smallexample
-@xopindex{same-permissions, short description}
-@xopindex{preserve-permissions, short description}
-@item -p
-@itemx --same-permissions
-@itemx --preserve-permissions
-Extract all protection information.
+@noindent
+This number gives the size of the extended header part in tar @dfn{blocks}.
+In our example, this formula gives: @code{897 - 56 - 425984 / 512 - 2
+= 7}.
-This option causes @command{tar} to set the modes (access permissions) of
-extracted files exactly as recorded in the archive. If this option
-is not used, the current @code{umask} setting limits the permissions
-on extracted files. This option is by default enabled when
-@command{tar} is executed by a superuser.
+@item
+Use @command{dd} to extract the headers:
+@smallexample
+@kbd{dd if=@var{archive} of=@var{hname} bs=512 skip=@var{Bs} count=@var{N}}
+@end smallexample
-This option is meaningless with @option{--list} (@option{-t}).
+@noindent
+where @var{archive} is the archive name, @var{hname} is a name of the
+file to store the extended header in, @var{Bs} and @var{N} are
+computed in previous steps.
-@opindex preserve
-@item --preserve
-Same as both @option{--same-permissions} and @option{--same-order}.
+In our example, this command will be
-The @option{--preserve} option has no equivalent short option name.
-It is equivalent to @option{--same-permissions} plus @option{--same-order}.
+@smallexample
+$ @kbd{dd if=arc.tar of=xhdr bs=512 skip=56 count=7}
+@end smallexample
+@end enumerate
-@FIXME{I do not see the purpose of such an option. (Neither I. FP.)
-Neither do I. --Sergey}
+Finally, you can expand the condensed file, using the obtained header:
-@end table
+@smallexample
+@group
+$ @kbd{xsparse -v -x xhdr GNUSparseFile.6058/sparsefile}
+Reading extended header file
+Found variable GNU.sparse.size = 217481216
+Found variable GNU.sparse.numblocks = 208
+Found variable GNU.sparse.name = sparsefile
+Found variable GNU.sparse.map = 0,2048,1050624,2048,@dots{}
+Expanding file `GNUSparseFile.28124/sparsefile' to `sparsefile'
+Done
+@end group
+@end smallexample
@node cpio
@section Comparison of @command{tar} and @command{cpio}
@FIXME{Reorganize the following material}
The @command{cpio} archive formats, like @command{tar}, do have maximum
-pathname lengths. The binary and old ASCII formats have a max path
-length of 256, and the new ASCII and CRC ASCII formats have a max
-path length of 1024. @acronym{GNU} @command{cpio} can read and write archives
-with arbitrary pathname lengths, but other @command{cpio} implementations
+file name lengths. The binary and old @acronym{ASCII} formats have a maximum file
+length of 256, and the new @acronym{ASCII} and @acronym{CRC ASCII} formats have a max
+file length of 1024. @acronym{GNU} @command{cpio} can read and write archives
+with arbitrary file name lengths, but other @command{cpio} implementations
may crash unexplainedly trying to read them.
-@command{tar} handles symbolic links in the form in which it comes in BSD;
+@command{tar} handles symbolic links in the form in which it comes in @acronym{BSD};
@command{cpio} doesn't handle symbolic links in the form in which it comes
in System V prior to SVR4, and some vendors may have added symlinks
to their system without enhancing @command{cpio} to know about them.
Others may have enhanced it in a way other than the way I did it
at Sun, and which was adopted by AT&T (and which is, I think, also
present in the @command{cpio} that Berkeley picked up from AT&T and put
-into a later BSD release---I think I gave them my changes).
+into a later @acronym{BSD} release---I think I gave them my changes).
(SVR4 does some funny stuff with @command{tar}; basically, its @command{cpio}
can handle @command{tar} format input, and write it on output, and it
@command{cpio} handles special files; traditional @command{tar} doesn't.
-@command{tar} comes with V7, System III, System V, and BSD source;
-@command{cpio} comes only with System III, System V, and later BSD
+@command{tar} comes with V7, System III, System V, and @acronym{BSD} source;
+@command{cpio} comes only with System III, System V, and later @acronym{BSD}
(4.3-tahoe and later).
@command{tar}'s way of handling multiple hard links to a file can handle
-file systems that support 32-bit inumbers (e.g., the BSD file system);
-@command{cpio}s way requires you to play some games (in its "binary"
-format, i-numbers are only 16 bits, and in its "portable ASCII" format,
-they're 18 bits---it would have to play games with the "file system ID"
-field of the header to make sure that the file system ID/i-number pairs
+file systems that support 32-bit inumbers (e.g., the @acronym{BSD} file system);
+@command{cpio}s way requires you to play some games (in its ``binary''
+format, i-numbers are only 16 bits, and in its ``portable @acronym{ASCII}'' format,
+they're 18 bits---it would have to play games with the "file system @acronym{ID}"
+field of the header to make sure that the file system @acronym{ID}/i-number pairs
of different files were always different), and I don't know which
@command{cpio}s, if any, play those games. Those that don't might get
confused and think two files are the same file when they're not, and
This means that the @option{--append}, @option{--concatenate}, and
@option{--delete} commands will not work on any other kind of file.
Some media simply cannot be backspaced, which means these commands and
-options will never be able to work on them. These non-backspacing
+options will never be able to work on them. These non-backspacing
media include pipes and cartridge tape drives.
Some other media can be backspaced, and @command{tar} will work on them
@opindex blocking-factor
The data in an archive is grouped into blocks, which are 512 bytes.
Blocks are read and written in whole number multiples called
-@dfn{records}. The number of blocks in a record (i.e. the size of a
+@dfn{records}. The number of blocks in a record (i.e., the size of a
record in units of 512 bytes) is called the @dfn{blocking factor}.
The @option{--blocking-factor=@var{512-size}} (@option{-b
@var{512-size}}) option specifies the blocking factor of an archive.
blocking factor (particularly if you're not sure what the blocking factor
is), you can usually use the @option{--read-full-records} (@option{-B}) option while
specifying a blocking factor larger then the blocking factor of the archive
-(i.e. @samp{tar --extract --read-full-records --blocking-factor=300}.
+(i.e., @samp{tar --extract --read-full-records --blocking-factor=300}.
@xref{list}, for more information on the @option{--list} (@option{-t})
operation. @xref{Reading}, for a more detailed explanation of that option.
@xopindex{read-full-records, short description}
@item -B
@itemx --read-full-records
-Reblock as we read (for reading 4.2BSD pipes).
+Reblock as we read (for reading 4.2@acronym{BSD} pipes).
If @option{--read-full-records} is used, @command{tar}
will not panic if an attempt to read a record from the archive does
-not return a full record. Instead, @command{tar} will keep reading
+not return a full record. Instead, @command{tar} will keep reading
until it has obtained a full
record.
This option is turned on by default when @command{tar} is reading
an archive from standard input, or from a remote machine. This is
-because on BSD Unix systems, a read of a pipe will return however
+because on @acronym{BSD} Unix systems, a read of a pipe will return however
much happens to be in the pipe, even if it is less than @command{tar}
requested. If this option was not used, @command{tar} would fail as
soon as it read an incomplete record from the pipe.
often call @samp{volume} a @dfn{tape}, there is absolutely no
requirement for multi-volume archives to be stored on tapes. Instead,
they can use whatever media type the user finds convenient, they can
-even be located on files.
+even be located on files.
-When creating a multi-volume arvhive, @GNUTAR{} continues to fill
+When creating a multi-volume archive, @GNUTAR{} continues to fill
current volume until it runs out of space, then it switches to
next volume (usually the operator is queried to replace the tape on
this point), and continues working on the new volume. This operation
-continues untill all requested files are dumped. If @GNUTAR{} detects
+continues until all requested files are dumped. If @GNUTAR{} detects
end of media while dumping a file, such a file is archived in split
-form. Some very big files can even be split across several volumes.
+form. Some very big files can even be split across several volumes.
Each volume is itself a valid @GNUTAR{} archive, so it can be read
without any special options. Consequently any file member residing
When @GNUTAR{} comes to the end of a storage media, it asks you to
change the volume. The built-in prompt for POSIX locale
is@footnote{If you run @GNUTAR{} under a different locale, the
-translation to the locale's language will be used.}:
+translation to the locale's language will be used.}:
@smallexample
Prepare volume #@var{n} for `@var{archive}' and hit return:
The volume number used by @command{tar} in its tape-changing prompt
can be changed; if you give the
@option{--volno-file=@var{file-of-number}} option, then
-@var{file-of-number} should be an unexisting file to be created, or
+@var{file-of-number} should be an non-existing file to be created, or
else, a file already containing a decimal number. That number will be
used as the volume number of the first volume written. When
@command{tar} is finished, it will rewrite the file with the
If you want more elaborate behavior than this, you can write a special
@dfn{new volume script}, that will be responsible for changing the
volume, and instruct @command{tar} to use it instead of its normal
-prompting procedure:
+prompting procedure:
@table @option
@item --info-script=@var{script-name}
@item TAR_ARCHIVE
The name of the archive @command{tar} is processing.
+@vrindex TAR_BLOCKING_FACTOR, info script environment variable
+@item TAR_BLOCKING_FACTOR
+Current blocking factor (@pxref{Blocking}.
+
@vrindex TAR_VOLUME, info script environment variable
@item TAR_VOLUME
Ordinal number of the volume @command{tar} is about to start.
@vrindex TAR_SUBCOMMAND, info script environment variable
@item TAR_SUBCOMMAND
-Short option describing the operation @command{tar} is executing
+A short option describing the operation @command{tar} is executing
@xref{Operations}, for a complete list of subcommand options.
@vrindex TAR_FORMAT, info script environment variable
@item TAR_FORMAT
Format of the archive being processed. @xref{Formats}, for a complete
list of archive format names.
+
+@vrindex TAR_FD, info script environment variable
+@item TAR_FD
+File descriptor which can be used to communicate the new volume
+name to @command{tar}.
@end table
The volume script can instruct @command{tar} to use new archive name,
-by writing in to file descriptor 3 (see below for an example).
+by writing in to file descriptor @env{$TAR_FD} (see below for an example).
If the info script fails, @command{tar} exits; otherwise, it begins
writing the next volume.
@end smallexample
The second method is to use the @samp{n} response to the tape-change
-prompt.
+prompt.
Finally, the most flexible approach is to use a volume script, that
-writes new archive name to the file descriptor #3. For example, the
+writes new archive name to the file descriptor @env{$TAR_FD}. For example, the
following volume script will create a series of archive files, named
@file{@var{archive}-@var{vol}}, where @var{archive} is the name of the
archive being created (as given by @option{--file} option) and
case $TAR_SUBCOMMAND in
-c) ;;
-d|-x|-t) test -r $@{name:-$TAR_ARCHIVE@}-$TAR_VOLUME || exit 1
- ;;
+ ;;
*) exit 1
esac
-echo $@{name:-$TAR_ARCHIVE@}-$TAR_VOLUME >&3
+echo $@{name:-$TAR_ARCHIVE@}-$TAR_VOLUME >&$TAR_FD
@end group
@end smallexample
-The same script cant be used while listing, comparing or extracting
+The same script can be used while listing, comparing or extracting
from the created archive. For example:
@smallexample
that volume), use @option{--extract}, again without
@option{--multi-volume}.
-If an archive member is split across volumes (i.e. its entry begins on
+If an archive member is split across volumes (i.e., its entry begins on
one volume of the media and ends on another), you need to specify
@option{--multi-volume} to extract it successfully. In this case, you
should load the volume where the archive member starts, and use
@option{--label=@var{archive-label}} again in conjunction with the
@option{--append}, @option{--update} or @option{--concatenate} operation.
-@FIXME{This is no longer true: Multivolume archives in @samp{POSIX}
-format can be extracted using any posix-compliant tar
-implementation. The split members can then be recreated from parts
-using a simple shell script. Provide more information about it:}
-Beware that there is @emph{no} real standard about the proper way, for
-a @command{tar} archive, to span volume boundaries. If you have a
-multi-volume created by some vendor's @command{tar}, there is almost
-no chance you could read all the volumes with @GNUTAR{}.
-The converse is also true: you may not expect
-multi-volume archives created by @GNUTAR{} to be
-fully recovered by vendor's @command{tar}. Since there is little
-chance that, in mixed system configurations, some vendor's
-@command{tar} will work on another vendor's machine, and there is a
-great chance that @GNUTAR{} will work on most of
-them, your best bet is to install @GNUTAR{} on all
-machines between which you know exchange of files is possible.
+Notice that multi-volume support is a GNU extension and the archives
+created in this mode should be read only using @GNUTAR{}. If you
+absolutely have to process such archives using a third-party @command{tar}
+implementation, read @ref{Split Recovery}.
@node Tape Files
@subsection Tape Files
@cindex Listing volume label
The volume label will be displayed by @option{--list} along with
the file contents. If verbose display is requested, it will also be
-explicitely marked as in the example below:
+explicitly marked as in the example below:
@smallexample
@group
the archive label matches the one specified and will refuse to proceed
if it does not. Use this as a safety precaution to avoid accidentally
overwriting existing archives. For example, if you wish to add files
-to @file{archive}, presumably labelled with string @samp{My volume},
+to @file{archive}, presumably labeled with string @samp{My volume},
you will get:
@smallexample
@noindent
in case its label does not match. This will work even if
-@file{archive} is not labelled at all.
+@file{archive} is not labeled at all.
Similarly, @command{tar} will refuse to list or extract the
archive if its label doesn't match the @var{archive-label}
@appendix Configuring Help Summary
Running @kbd{tar --help} displays the short @command{tar} option
-summary (@pxref{help}). This summary is organised by @dfn{groups} of
+summary (@pxref{help}). This summary is organized by @dfn{groups} of
semantically close options. The options within each group are printed
in the following order: a short option, eventually followed by a list
of corresponding long option names, followed by a short description of
Right margin of the text output. Used for wrapping.
@end deftypevr
+@node Fixing Snapshot Files
+@appendix Fixing Snapshot Files
+@include tar-snapshot-edit.texi
+
@node Tar Internals
@appendix Tar Internals
@include intern.texi
@appendix Index of Command Line Options
This appendix contains an index of all @GNUTAR{} long command line
-options. The options are listed without the preceeding double-dash.
+options. The options are listed without the preceding double-dash.
For a cross-reference of short command line options, @ref{Short Option Summary}.
@printindex op
projects / chaz / tar / blobdiff