X-Git-Url: https://git.dogcows.com/gitweb?p=chaz%2Ftar;a=blobdiff_plain;f=doc%2Fsparse.texi;h=1bbccfc49872e658e521b0ca99dfeb86e02dcef6;hp=7b9145d8ab78ce020c79c3d156c71643f2d1d889;hb=45ccda119355a1087450039a250359c1d0de0d08;hpb=9588a106a7192cf276e8db0d51c7818be286bf41 diff --git a/doc/sparse.texi b/doc/sparse.texi index 7b9145d..1bbccfc 100644 --- a/doc/sparse.texi +++ b/doc/sparse.texi @@ -1,8 +1,10 @@ @c This is part of the paxutils manual. -@c Copyright (C) 2006 Free Software Foundation, Inc. +@c Copyright (C) 2006, 2014 Free Software Foundation, Inc. @c This file is distributed under GFDL 1.1 or any later version @c published by the Free Software Foundation. +@cindex sparse formats +@cindex sparse versions The notion of sparse file, and the ways of handling it from the point of view of @GNUTAR{} user have been described in detail in @ref{sparse}. This chapter describes the internal format @GNUTAR{} @@ -12,12 +14,12 @@ The support for sparse files in @GNUTAR{} has a long history. The earliest version featuring this support that I was able to find was 1.09, released in November, 1990. The format introduced back then is called @dfn{old GNU} sparse format and in spite of the fact that its design -contained many flaws, it was the only format @GNUTAR{} supported +contained many flaws, it was the only format @GNUTAR{} supported until version 1.14 (May, 2004), which introduced initial support for sparse archives in @acronym{PAX} archives (@pxref{posix}). This -format was not free from design flows, either and it was subsequently +format was not free from design flaws, either and it was subsequently improved in versions 1.15.2 (November, 2005) and 1.15.92 (June, -2006). +2006). In addition to GNU sparse format, @GNUTAR{} is able to read and extract sparse files archived by @command{star}. @@ -33,7 +35,9 @@ The following subsections describe each format in detail. @node Old GNU Format @appendixsubsec Old GNU Format -The format introduced some time around 1990 (v. 1.09). It was +@cindex sparse formats, Old GNU +@cindex Old GNU sparse format +The format introduced in November 1990 (v. 1.09) was designed on top of standard @code{ustar} headers in such an unfortunate way that some of its fields overwrote fields required by POSIX. @@ -57,7 +61,7 @@ extension sparse header follows, @code{0} otherwise. @end multitable Each of @code{sparse_header} object at offset 386 describes a single -data chunk. It has the following structure: +data chunk. It has the following structure: @multitable @columnfractions 0.10 0.10 0.20 0.60 @headitem Offset @tab Size @tab Data type @tab Contents @@ -74,7 +78,7 @@ the following structure: @multitable @columnfractions 0.10 0.10 0.20 0.20 0.40 @headitem Offset @tab Size @tab Name @tab Data type @tab Contents @item 0 @tab 21 @tab sp @tab @code{sparse_header} @tab -(21 entires) File map. +(21 entries) File map. @item 504 @tab 1 @tab isextended @tab Bool @tab @code{1} if an extension sparse header follows, or @code{0} otherwise. @end multitable @@ -83,25 +87,29 @@ A header with @code{isextended=0} ends the map. @node PAX 0 @appendixsubsec PAX Format, Versions 0.0 and 0.1 -@UNREVISED{} +@cindex sparse formats, v.0.0 There are two formats available in this branch. The version @code{0.0} is the initial version of sparse format used by @command{tar} versions 1.14--1.15.1. The sparse file map is kept in extended (@code{x}) PAX header variables: @table @code +@vrindex GNU.sparse.size, extended header variable @item GNU.sparse.size -Real size of the stored file +Real size of the stored file; @item GNU.sparse.numblocks -Number of blocks in the sparse map +@vrindex GNU.sparse.numblocks, extended header variable +Number of blocks in the sparse map; @item GNU.sparse.offset -Offset of the data block +@vrindex GNU.sparse.offset, extended header variable +Offset of the data block; @item GNU.sparse.numbytes -Size of the data block +@vrindex GNU.sparse.numbytes, extended header variable +Size of the data block. @end table The latter two variables repeat for each data block, so the overall @@ -109,11 +117,11 @@ structure is like this: @smallexample @group -GNU.sparse.size=@var{size} -GNU.sparse.numblocks=@var{numblocks} +GNU.sparse.size=@var{size} +GNU.sparse.numblocks=@var{numblocks} repeat @var{numblocks} times - GNU.sparse.offset=@var{offset} - GNU.sparse.numbytes=@var{numbytes} + GNU.sparse.offset=@var{offset} + GNU.sparse.numbytes=@var{numbytes} end repeat @end group @end smallexample @@ -124,20 +132,23 @@ This format presented the following two problems: @item Whereas the POSIX specification allows a variable to appear multiple times in a header, it requires that only the last occurrence be -meaningful. Thus, multiple ocurrences of @code{GNU.sparse.offset} and -@code{GNU.sparse.numbytes} are conficting with the POSIX specs. +meaningful. Thus, multiple occurrences of @code{GNU.sparse.offset} and +@code{GNU.sparse.numbytes} are conflicting with the POSIX specs. @item -Attempting to extract such archives using a third-party @command{tar}s -results in extraction of sparse files in @emph{compressed form}. If +Attempting to extract such archives using a third-party's @command{tar} +results in extraction of sparse files in @emph{condensed form}. If the @command{tar} implementation in question does not support POSIX format, it will also extract a file containing extension header attributes. This file can be used to expand the file to its original state. However, posix-aware @command{tar}s will usually ignore the -unknown variables, which makes restoring the file much more -difficult@FIXME-xref{how to extract sparse file using third-party @command{tar}s}. +unknown variables, which makes restoring the file more +difficult. @xref{extracting sparse v.0.x, Extraction of sparse +members in v.0.0 format}, for the detailed description of how to +restore such members using non-GNU @command{tar}s. @end enumerate +@cindex sparse formats, v.0.1 @GNUTAR{} 1.15.2 introduced sparse format version @code{0.1}, which attempted to solve these problems. As its predecessor, this format stores sparse map in the extended POSIX header. It retains @@ -147,8 +158,9 @@ it uses a single variable: @table @code @item GNU.sparse.map +@vrindex GNU.sparse.map, extended header variable Map of non-null data chunks. It is a string consisting of -comma-separated values "@var{offset},@var{size}[,@var{offset-1},@var{size-1}...]" +comma-separated values "@var{offset},@var{size}[,@var{offset-1},@var{size-1}...]" @end table To address the 2nd problem, the @code{name} field in @code{ustar} @@ -158,21 +170,23 @@ is replaced with a special name, constructed using the following pattern: %d/GNUSparseFile.%p/%f @end smallexample +@vrindex GNU.sparse.name, extended header variable The real name of the sparse file is stored in the variable @code{GNU.sparse.name}. Thus, those @command{tar} implementations that are not aware of GNU extensions will at least extract the files into separate directories, giving the user a possibility to expand it -afterwards @FIXME-ref{how to extract sparse file using third-party -@command{tar}s}. +afterwards. @xref{extracting sparse v.0.x, Extraction of sparse +members in v.0.1 format}, for the detailed description of how to +restore such members using non-GNU @command{tar}s. The resulting @code{GNU.sparse.map} string can be @emph{very} long. Although POSIX does not impose any limit on the length of a @code{x} -header variable, this possibly can confuse some tars. +header variable, this possibly can confuse some @command{tar}s. @node PAX 1 @appendixsubsec PAX Format, Version 1.0 -@UNREVISED{} +@cindex sparse formats, v.1.0 The version @code{1.0} of sparse format was introduced with @GNUTAR{} 1.15.92. Its main objective was to make the resulting file extractable with little effort even by non-posix aware @command{tar} @@ -182,9 +196,11 @@ identify the format being used: @table @code @item GNU.sparse.major +@vrindex GNU.sparse.major, extended header variable Major version @item GNU.sparse.minor +@vrindex GNU.sparse.minor, extended header variable Minor version @end table @@ -195,23 +211,24 @@ constructed using the following pattern: %d/GNUSparseFile.%p/%f @end smallexample +@vrindex GNU.sparse.name, extended header variable, in v.1.0 +@vrindex GNU.sparse.realsize, extended header variable The real name of the sparse file is stored in the variable @code{GNU.sparse.name}. The real size of the file is stored in the variable @code{GNU.sparse.realsize}. The sparse map itself is stored in the file data block, preceding the actual -file data. It consists of a series of octal numbers of arbitrary length, delimited +file data. It consists of a series of octal numbers of arbitrary length, delimited by newlines. The map is padded with nulls to the nearest block boundary. The first number gives the number of entries in the map. Following are map entries, each one consisting of two numbers giving the offset and size of the data block it describes. -The format is designed in such a way that non-posix aware tars and tars not +The format is designed in such a way that non-posix aware @command{tar}s and @command{tar}s not supporting @code{GNU.sparse.*} keywords will extract each sparse file in its condensed form with the file map prepended and will place it into a separate directory. Then, using a simple program it would be -possible to expand the file to its original form even without GNU tar. -@FIXME-xref{how to extract sparse file using third-party -@command{tar}s}. @FIXME{Write the program and give its URL here}. - +possible to expand the file to its original form even without @GNUTAR{}. +@xref{Sparse Recovery}, for the detailed information on how to extract +sparse members without @GNUTAR{}.