@quotation
Permission is granted to copy, distribute and/or modify this document
-under the terms of the GNU Free Documentation License, Version 1.1 or
+under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
-Invariant Sections, with the Front-Cover Texts being ``A GNU Manual,''
+Invariant Sections, with the Front-Cover Texts being ``A GNU Manual'',
and with the Back-Cover Texts as in (a) below. A copy of the license
-is included in the section entitled "GNU Free Documentation License".
+is included in the section entitled ``GNU Free Documentation
+License''.
(a) The FSF's Back-Cover Text is: ``You have the freedom to
copy and modify this GNU manual. Buying copies from the FSF
* Date input formats::
* Formats::
* Media::
+* Reliability and security::
Appendices
* Tar Internals::
* Genfile::
* Free Software Needs Free Documentation::
-* Copying This Manual::
+* GNU Free Documentation License::
* Index of Command Line Options::
* Index::
* Pure numbers in date strings:: 19931219, 1440.
* Seconds since the Epoch:: @@1078100502.
* Specifying time zone rules:: TZ="America/New_York", TZ="UTC0".
-* Authors of get_date:: Bellovin, Eggert, Salz, Berets, et al.
+* Authors of parse_datetime:: Bellovin, Eggert, Salz, Berets, et al.
Controlling the Archive Format
@item --check-device
Check device numbers when creating a list of modified files for
incremental archiving. This is the default. @xref{device numbers},
-for a detailed description.
+for a detailed description.
@opsummary{checkpoint}
@item --checkpoint[=@var{number}]
@item --dereference
@itemx -h
-When creating a @command{tar} archive, @command{tar} will archive the
-file that a symbolic link points to, rather than archiving the
-symlink. @xref{dereference}.
+When reading or writing a file to be archived, @command{tar} accesses
+the file that a symbolic link points to, rather than the symlink
+itself. @xref{dereference}.
@opsummary{directory}
@item --directory=@var{dir}
@item --no-check-device
Do not check device numbers when creating a list of modified files
for incremental archiving. @xref{device numbers}, for
-a detailed description.
+a detailed description.
@opsummary{no-delay-directory-restore}
@item --no-delay-directory-restore
from pipes on systems with buggy implementations. @xref{Reading}.
@opsummary{record-size}
-@item --record-size=@var{size}
+@item --record-size=@var{size}[@var{suf}]
Instructs @command{tar} to use @var{size} bytes per record when accessing the
-archive. @xref{Blocking Factor}.
+archive. The argument can be suffixed with a @dfn{size suffix}, e.g.
+@option{--record-size=10K} for 10 Kilobytes. @xref{size-suffixes},
+for a list of valid suffixes. @xref{Blocking Factor}, for a detailed
+description of this option.
@opsummary{recursion}
@item --recursion
@samp{~}. @xref{backup}.
@opsummary{tape-length}
-@item --tape-length=@var{num}
-@itemx -L @var{num}
+@item --tape-length=@var{num}[@var{suf}]
+@itemx -L @var{num}[@var{suf}]
Specifies the length of tapes that @command{tar} is writing as being
-@w{@var{num} x 1024} bytes long. @xref{Using Multiple Tapes}.
+@w{@var{num} x 1024} bytes long. If optional @var{suf} is given, it
+specifies a multiplicative factor to be used instead of 1024. For
+example, @samp{-L2M} means 2 megabytes. @xref{size-suffixes}, for a
+list of allowed suffixes. @xref{Using Multiple Tapes}, for a detailed
+discussion of this option.
@opsummary{test-label}
@item --test-label
@samp{Current %s is newer or same age}
@kwindex unknown-keyword
@cindex @samp{Ignoring unknown extended header keyword `%s'}, warning message
-@item unknown-keyword
+@item unknown-keyword
@samp{Ignoring unknown extended header keyword `%s'}
@end table
Other operations don't deal with these members as perfectly as you might
prefer; if you were to use @option{--extract} to extract the archive,
-only the most recently added copy of a member with the same name as
+only the most recently added copy of a member with the same name as
other members would end up in the working directory. This is because
@option{--extract} extracts an archive in the order the members appeared
in the archive; the most recently archived members will be extracted
@group
$ @kbd{tar --directory sourcedir --create --file=- . \
| tar --directory targetdir --extract --file=-}
-@end group
+@end group
@end smallexample
@noindent
Some users are enthusiastic about @code{Amanda} (The Advanced Maryland
Automatic Network Disk Archiver), a backup system developed by James
da Silva @file{jds@@cs.umd.edu} and available on many Unix systems.
-This is free software, and it is available from @uref{http://www.amanda.org}.
+This is free software, and it is available from @uref{http://www.amanda.org}.
@FIXME{
for an incremental dump. This is the default behavior. The purpose
of this option is to undo the effect of the @option{--no-check-device}
if it was given in @env{TAR_OPTIONS} environment variable
-(@pxref{TAR_OPTIONS}).
+(@pxref{TAR_OPTIONS}).
@end table
There is also another way to cope with changing device numbers. It is
$ @kbd{tar -c -f archive.tar -C / home}
@end smallexample
-@include getdate.texi
+@xref{Integrity}, for some of the security-related implications
+of using this option.
+
+@include parse-datetime.texi
@node Formats
@chapter Controlling the Archive Format
a wide variety of compression programs, namely: @command{gzip},
@command{bzip2}, @command{lzip}, @command{lzma}, @command{lzop},
@command{xz} and traditional @command{compress}. The latter is
-supported mostly for backward compatibility, and we recommend
+supported mostly for backward compatibility, and we recommend
against using it, because it is by far less effective than the other
compression programs@footnote{It also had patent problems in the past.}.
create a @command{gzip} compressed archive, @option{-j}
(@option{--bzip2}) to create a @command{bzip2} compressed archive,
@option{--lzip} to create an @asis{lzip} compressed archive,
-@option{-J} (@option{--xz}) to create an @asis{XZ} archive,
+@option{-J} (@option{--xz}) to create an @asis{XZ} archive,
@option{--lzma} to create an @asis{LZMA} compressed
archive, @option{--lzop} to create an @asis{LSOP} archive, and
@option{-Z} (@option{--compress}) to use @command{compress} program.
You can use any of these options on physical devices (tape drives,
etc.) and remote files as well as on normal files; data to or from
such devices or remote files is reblocked by another copy of the
-@command{tar} program to enforce the specified (or default) record
+@command{tar} program to enforce the specified (or default) record
size. The default compression parameters are used. Most compression
programs allow to override these by setting a program-specific
environment variable. For example, when using @command{gzip} you can
-j, --bzip2 filter the archive through lbzip2
@end group
@end smallexample
-
+
@noindent
which means that running @command{tar --bzip2} will invoke @command{lbzip2}.
Normally, when @command{tar} archives a symbolic link, it writes a
block to the archive naming the target of the link. In that way, the
@command{tar} archive is a faithful record of the file system contents.
-@option{--dereference} (@option{-h}) is used with @option{--create} (@option{-c}), and causes
-@command{tar} to archive the files symbolic links point to, instead of
-the links themselves. When this option is used, when @command{tar}
-encounters a symbolic link, it will archive the linked-to file,
-instead of simply recording the presence of a symbolic link.
-
-The name under which the file is stored in the file system is not
-recorded in the archive. To record both the symbolic link name and
-the file name in the system, archive the file under both names. If
-all links were recorded automatically by @command{tar}, an extracted file
-might be linked to a file name that no longer exists in the file
-system.
+When @option{--dereference} (@option{-h}) is used with
+@option{--create} (@option{-c}), @command{tar} archives the files
+symbolic links point to, instead of
+the links themselves.
-If a linked-to file is encountered again by @command{tar} while creating
-the same archive, an entire second copy of it will be stored. (This
-@emph{might} be considered a bug.)
-
-So, for portable archives, do not archive symbolic links as such,
-and use @option{--dereference} (@option{-h}): many systems do not support
+When creating portable archives, use @option{--dereference}
+(@option{-h}): some systems do not support
symbolic links, and moreover, your distribution might be unusable if
it contains unresolved symbolic links.
+When reading from an archive, the @option{--dereference} (@option{-h})
+option causes @command{tar} to follow an already-existing symbolic
+link when @command{tar} writes or reads a file named in the archive.
+Ordinarily, @command{tar} does not follow such a link, though it may
+remove the link before writing a new file. @xref{Dealing with Old
+Files}.
+
+The @option{--dereference} option is unsafe if an untrusted user can
+modify directories while @command{tar} is running. @xref{Security}.
+
@node hard links
@subsection Hard Links
@cindex File names, using hard links
@xopindex{tape-length, short description}
@item -L @var{num}
-@itemx --tape-length=@var{num}
-Change tape after writing @var{num} x 1024 bytes.
+@itemx --tape-length=@var{size}[@var{suf}]
+Change tape after writing @var{size} units of data. Unless @var{suf} is
+given, @var{size} is treated as kilobytes, i.e. @samp{@var{size} x
+1024} bytes. The following suffixes alter this behavior:
+
+@float Table, size-suffixes
+@caption{Size Suffixes}
+@multitable @columnfractions 0.2 0.3 0.3
+@headitem Suffix @tab Units @tab Byte Equivalent
+@item b @tab Blocks @tab @var{size} x 512
+@item B @tab Kilobytes @tab @var{size} x 1024
+@item c @tab Bytes @tab @var{size}
+@item G @tab Gigabytes @tab @var{size} x 1024^3
+@item K @tab Kilobytes @tab @var{size} x 1024
+@item k @tab Kilobytes @tab @var{size} x 1024
+@item M @tab Megabytes @tab @var{size} x 1024^2
+@item P @tab Petabytes @tab @var{size} x 1024^5
+@item T @tab Terabytes @tab @var{size} x 1024^4
+@item w @tab Words @tab @var{size} x 2
+@end multitable
+@end float
This option might be useful when your tape drivers do not properly
detect end of physical tapes. By being slightly conservative on the
@anchor{tape-length}
@table @option
@opindex tape-length
-@item --tape-length=@var{size}
-@itemx -L @var{size}
-Set maximum length of a volume. The @var{size} argument should then
-be the usable size of the tape in units of 1024 bytes. This option
-selects @option{--multi-volume} automatically. For example:
+@item --tape-length=@var{size}[@var{suf}]
+@itemx -L @var{size}[@var{suf}]
+Set maximum length of a volume. The @var{suf}, if given, specifies
+units in which @var{size} is expressed, e.g. @samp{2M} mean 2
+megabytes (@pxref{size-suffixes}, for a list of allowed size
+suffixes). Without @var{suf}, units of 1024 bytes (kilobyte) are
+assumed.
+
+This option selects @option{--multi-volume} automatically. For example:
@smallexample
$ @kbd{tar --create --tape-length=41943040 --file=/dev/tape @var{files}}
@end smallexample
+
+@noindent
+or, which is equivalent:
+
+@smallexample
+$ @kbd{tar --create --tape-length=4G --file=/dev/tape @var{files}}
+@end smallexample
@end table
@anchor{change volume prompt}
which can be removed from the center of a tape reel, or some other
changeable feature.
+@node Reliability and security
+@chapter Reliability and Security
+
+The @command{tar} command reads and writes files as any other
+application does, and is subject to the usual caveats about
+reliability and security. This section contains some commonsense
+advice on the topic.
+
+@menu
+* Reliability::
+* Security::
+@end menu
+
+@node Reliability
+@section Reliability
+
+Ideally, when @command{tar} is creating an archive, it reads from a
+file system that is not being modified, and encounters no errors or
+inconsistencies while reading and writing. If this is the case, the
+archive should faithfully reflect what was read. Similarly, when
+extracting from an archive, ideally @command{tar} ideally encounters
+no errors and the extracted files faithfully reflect what was in the
+archive.
+
+However, when reading or writing real-world file systems, several
+things can go wrong; these include permissions problems, corruption of
+data, and race conditions.
+
+@menu
+* Permissions problems::
+* Data corruption and repair::
+* Race conditions::
+@end menu
+
+@node Permissions problems
+@subsection Permissions Problems
+
+If @command{tar} encounters errors while reading or writing files, it
+normally reports an error and exits with nonzero status. The work it
+does may therefore be incomplete. For example, when creating an
+archive, if @command{tar} cannot read a file then it cannot copy the
+file into the archive.
+
+@node Data corruption and repair
+@subsection Data Corruption and Repair
+
+If an archive becomes corrupted by an I/O error, this may corrupt the
+data in an extracted file. Worse, it may corrupt the file's metadata,
+which may cause later parts of the archive to become misinterpreted.
+An tar-format archive contains a checksum that most likely will detect
+errors in the metadata, but it will not detect errors in the data.
+
+If data corruption is a concern, you can compute and check your own
+checksums of an archive by using other programs, such as
+@command{cksum}.
+
+When attempting to recover from a read error or data corruption in an
+archive, you may need to skip past the questionable data and read the
+rest of the archive. This requires some expertise in the archive
+format and in other software tools.
+
+@node Race conditions
+@subsection Race conditions
+
+If some other process is modifying the file system while @command{tar}
+is reading or writing files, the result may well be inconsistent due
+to race conditions. For example, if another process creates some
+files in a directory while @command{tar} is creating an archive
+containing the directory's files, @command{tar} may see some of the
+files but not others, or it may see a file that is in the process of
+being created. The resulting archive may not be a snapshot of the
+file system at any point in time. If an application such as a
+database system depends on an accurate snapshot, restoring from the
+@command{tar} archive of a live file system may therefore break that
+consistency and may break the application. The simplest way to avoid
+the consistency issues is to avoid making other changes to the file
+system while tar is reading it or writing it.
+
+When creating an archive, several options are available to avoid race
+conditions. Some hosts have a way of snapshotting a file system, or
+of temporarily suspending all changes to a file system, by (say)
+suspending the only virtual machine that can modify a file system; if
+you use these facilities and have @command{tar -c} read from a
+snapshot when creating an archive, you can avoid inconsistency
+problems. More drastically, before starting @command{tar} you could
+suspend or shut down all processes other than @command{tar} that have
+access to the file system, or you could unmount the file system and
+then mount it read-only.
+
+When extracting from an archive, one approach to avoid race conditions
+is to create a directory that no other process can write to, and
+extract into that.
+
+@node Security
+@section Security
+
+In some cases @command{tar} may be used in an adversarial situation,
+where an untrusted user is attempting to gain information about or
+modify otherwise-inaccessible files. Dealing with untrusted data
+(that is, data generated by an untrusted user) typically requires
+extra care, because even the smallest mistake in the use of
+@command{tar} is more likely to be exploited by an adversary than by a
+race condition.
+
+@menu
+* Privacy::
+* Integrity::
+* Live untrusted data::
+* Security rules of thumb::
+@end menu
+
+@node Privacy
+@subsection Privacy
+
+Standard privacy concerns apply when using @command{tar}. For
+example, suppose you are archiving your home directory into a file
+@file{/archive/myhome.tar}. Any secret information in your home
+directory, such as your SSH secret keys, are copied faithfully into
+the archive. Therefore, if your home directory contains any file that
+should not be read by some other user, the archive itself should be
+not be readable by that user. And even if the archive's data are
+inaccessible to untrusted users, its metadata (such as size or
+last-modified date) may reveal some information about your home
+directory; if the metadata are intended to be private, the archive's
+parent directory should also be inaccessible to untrusted users.
+
+One precaution is to create @file{/archive} so that it is not
+accessible to any user, unless that user also has permission to access
+all the files in your home directory.
+
+Similarly, when extracting from an archive, take care that the
+permissions of the extracted files are not more generous than what you
+want. Even if the archive itself is readable only to you, files
+extracted from it have their own permissions that may differ.
+
+@node Integrity
+@subsection Integrity
+
+When creating archives, take care that they are not writable by a
+untrusted user; otherwise, that user could modify the archive, and
+when you later extract from the archive you will get incorrect data.
+
+When @command{tar} extracts from an archive, by default it writes into
+files relative to the working directory. If the archive was generated
+by an untrusted user, that user therefore can write into any file
+under the working directory. If the working directory contains a
+symbolic link to another directory, the untrusted user can also write
+into any file under the referenced directory. When extracting from an
+untrusted archive, it is therefore good practice to create an empty
+directory and run @command{tar} in that directory.
+
+When extracting from two or more untrusted archives, each one should
+be extracted independently, into different empty directories.
+Otherwise, the first archive could create a symbolic link into an area
+outside the working directory, and the second one could follow the
+link and overwrite data that is not under the working directory. For
+example, when restoring from a series of incremental dumps, the
+archives should have been created by a trusted process, as otherwise
+the incremental restores might alter data outside the working
+directory.
+
+If you use the @option{--absolute-names} (@option{-P}) option when
+extracting, @command{tar} respects any file names in the archive, even
+file names that begin with @file{/} or contain @file{..}. As this
+lets the archive overwrite any file in your system that you can write,
+the @option{--absolute-names} (@option{-P}) option should be used only
+for trusted archives.
+
+Conversely, with the @option{--keep-old-files} (@option{-k}) option,
+@command{tar} refuses to replace existing files when extracting; and
+with the @option{--no-overwrite-dir} option, @command{tar} refuses to
+replace the permissions or ownership of already-existing directories.
+These options may help when extracting from untrusted archives.
+
+@node Live untrusted data
+@subsection Dealing with Live Untrusted Data
+
+Extra care is required when creating from or extracting into a file
+system that is accessible to untrusted users. For example, superusers
+who invoke @command{tar} must be wary about its actions being hijacked
+by an adversary who is reading or writing the file system at the same
+time that @command{tar} is operating.
+
+When creating an archive from a live file system, @command{tar} is
+vulnerable to denial-of-service attacks. For example, an adversarial
+user could create the illusion of an indefinitely-deep directory
+hierarchy @file{d/e/f/g/...} by creating directories one step ahead of
+@command{tar}, or the illusion of an indefinitely-long file by
+creating a sparse file but arranging for blocks to be allocated just
+before @command{tar} reads them. There is no easy way for
+@command{tar} to distinguish these scenarios from legitimate uses, so
+you may need to monitor @command{tar}, just as you'd need to monitor
+any other system service, to detect such attacks.
+
+While a superuser is extracting from an archive into a live file
+system, an untrusted user might replace a directory with a symbolic
+link, in hopes that @command{tar} will follow the symbolic link and
+extract data into files that the untrusted user does not have access
+to. Even if the archive was generated by the superuser, it may
+contain a file such as @file{d/etc/passwd} that the untrusted user
+earlier created in order to break in; if the untrusted user replaces
+the directory @file{d/etc} with a symbolic link to @file{/etc} while
+@command{tar} is running, @command{tar} will overwrite
+@file{/etc/passwd}. This attack can be prevented by extracting into a
+directory that is inaccessible to untrusted users.
+
+Similar attacks via symbolic links are also possible when creating an
+archive, if the untrusted user can modify an ancestor of a top-level
+argument of @command{tar}. For example, an untrusted user that can
+modify @file{/home/eve} can hijack a running instance of @samp{tar -cf
+- /home/eve/Documents/yesterday} by replacing
+@file{/home/eve/Documents} with a symbolic link to some other
+location. Attacks like these can be prevented by making sure that
+untrusted users cannot modify any files that are top-level arguments
+to @command{tar}, or any ancestor directories of these files.
+
+@node Security rules of thumb
+@subsection Security Rules of Thumb
+
+This section briefly summarizes rules of thumb for avoiding security
+pitfalls.
+
+@itemize @bullet
+
+@item
+Protect archives at least as much as you protect any of the files
+being archived.
+
+@item
+Extract from an untrusted archive only into an otherwise-empty
+directory. This directory and its parent should be accessible only to
+trusted users. For example:
+
+@example
+@group
+$ @kbd{chmod go-rwx .}
+$ @kbd{mkdir -m go-rwx dir}
+$ @kbd{cd dir}
+$ @kbd{tar -xvf /archives/got-it-off-the-net.tar.gz}
+@end group
+@end example
+
+As a corollary, do not do an incremental restore from an untrusted archive.
+
+@item
+Do not let untrusted users access files extracted from untrusted
+archives without checking first for problems such as setuid programs.
+
+@item
+Do not let untrusted users modify directories that are ancestors of
+top-level arguments of @command{tar}. For example, while you are
+executing @samp{tar -cf /archive/u-home.tar /u/home}, do not let an
+untrusted user modify @file{/}, @file{/archive}, or @file{/u}.
+
+@item
+Pay attention to the diagnostics and exit status of @command{tar}.
+
+@item
+When archiving live file systems, monitor running instances of
+@command{tar} to detect denial-of-service attacks.
+
+@item
+Avoid unusual options such as @option{--absolute-names} (@option{-P}),
+@option{--dereference} (@option{-h}), @option{--overwrite},
+@option{--recursive-unlink}, and @option{--remove-files} unless you
+understand their security implications.
+
+@end itemize
+
@node Changes
@appendix Changes
@appendix Free Software Needs Free Documentation
@include freemanuals.texi
-@node Copying This Manual
-@appendix Copying This Manual
-
-@menu
-* GNU Free Documentation License:: License for copying this manual
-@end menu
+@node GNU Free Documentation License
+@appendix GNU Free Documentation License
@include fdl.texi
projects / chaz / tar / blobdiff