.\"-
.\" Copyright (c) 2004-2016 Maxim Sobolev <sobomax@FreeBSD.org>
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
.\" ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
.\" SUCH DAMAGE.
.\"
.\" $FreeBSD$
.\"
.Dd August 9, 2019
.Dt MKUZIP 8
.Os
.Sh NAME
.Nm mkuzip
.Nd compress disk image for use with
.Xr geom_uzip 4
class
.Sh SYNOPSIS
.Nm
.Op Fl dSsvZ
.Op Fl A Ar compression_algorithm
.Op Fl C Ar compression_level
.Op Fl j Ar compression_jobs
.Op Fl o Ar outfile
.Op Fl s Ar cluster_size
.Ar infile
.Sh DESCRIPTION
The
.Nm
utility compresses a disk image file so that the
.Xr geom_uzip 4
class will be able to decompress the resulting image at run-time.
This allows for a significant reduction of size of disk image at
the expense of some CPU time required to decompress the data each
time it is read.
The
.Nm
utility
works in two phases:
.Bl -enum
.It
An
.Ar infile
image is split into clusters; each cluster is compressed.
.It
The resulting set of compressed clusters is written to the output file.
In addition, a
.Dq table of contents
header is written which allows for efficient seeking.
.El
.Pp
The options are:
.Bl -tag -width indent
.It Fl A Op Ar lzma | Ar zlib | Ar zstd
Select a specific compression algorithm.
If this option is not provided, the default is
.Ar zlib .
.Pp
The
.Ar lzma
algorithm provides noticeable better compression levels than zlib on the same
data set.
It has vastly slower compression speed and moderately slower decompression
speed.
.Pp
The
.Ar zstd
algorithm provides better compression levels than zlib on the same data set.
It also has faster compression and decompression speed than zlib.
In the very high compression
.Dq level
settings, it does not offer quite as high a compression ratio as
.Ar lzma .
However, its decompression speed does not suffer at high compression
.Dq levels .
.It Fl C Ar compression_level
Select the integer compression level used to parameterize the chosen
compression algorithm.
.Pp
For any given algorithm, a lesser number selects a faster compression mode.
A greater number selects a slower compression mode.
Typically, for the same algorithm, a greater
.Ar compression_level
provides better final compression ratio.
.Pp
For
.Ar lzma ,
the range of valid compression levels is
.Va 0-9 .
The
.Nm
default for lzma is
.Va 6 .
.Pp
For
.Ar zlib ,
the range of valid compression levels is
.Va 1-9 .
The
.Nm
default for zlib is
.Va 9 .
.Pp
For
.Ar zstd ,
the range of valid compression levels is currently
.Va 1-19 .
The
.Nm
default for zstd is
.Va 9 .
.It Fl d
Enable de-duplication.
When the option is enabled
.Nm
detects identical blocks in the input and replaces each subsequent occurrence
of such block with pointer to the very first one in the output.
Setting this option results is moderate decrease of compressed image size,
typically around 3-5% of a final size of the compressed image.
.It Fl j Ar compression_jobs
Specify the number of compression jobs that
.Nm
runs in parallel to speed up compression.
When option is not specified the number of jobs set to be equal
to the value of
.Va hw.ncpu
.Xr sysctl 8
variable.
.It Op Fl L
Legacy flag that indicates the same thing as
.Dq Fl A Ar lzma .
.It Fl o Ar outfile
Name of the output file
.Ar outfile .
The default is to use the input name with the suffix
.Pa .uzip
for the
.Xr zlib 3
compression or
.Pa .ulzma
for the
.Xr lzma 3 .
.It Fl S
Print summary about the compression ratio as well as output
file size after file has been processed.
.It Fl s Ar cluster_size
Split the image into clusters of
.Ar cluster_size
bytes, 16384 bytes by default.
The
.Ar cluster_size
should be a multiple of 512 bytes.
.It Fl v
Display verbose messages.
.It Fl Z
Disable zero-block detection and elimination.
When this option is set,
.Nm
compresses blocks of zero bytes just as it would any other block.
When the option is not set,
.Nm
detects and compresses zero blocks in a space-efficient way.
Setting
.Fl Z
increases compressed image sizes slightly, typically less than 0.1%.
.El
.Sh IMPLEMENTATION NOTES
The compression ratio largely depends on the compression algorithm, level, and
cluster size used.
For large cluster sizes (16kB and higher), typical overall image compression
ratios with
.Xr zlib 3
are only 1-2% less than those achieved with
.Xr gzip 1
over the entire image.
However, it should be kept in mind that larger cluster sizes lead to higher
overhead in the
.Xr geom_uzip 4
class, as the class has to decompress the whole cluster even if
only a few bytes from that cluster have to be read.
.Pp
Additionally, the threshold at 16-32 kB where a larger cluster size does not
benefit overall compression ratio is an artifact of the
.Xr zlib 3
algorithm in particular.
.Ar Lzma
and
.Ar Zstd will continue to provide better compression ratios as cluster sizes
are increased, at high enough compression levels.
The same tradeoff continues to apply: reads in
.Xr geom_uzip 4
become more expensive the greater the cluster size.
.Pp
The
.Nm
utility
inserts a short shell script at the beginning of the generated image,
which makes it possible to
.Dq run
the image just like any other shell script.
The script tries to load the
.Xr geom_uzip 4
class if it is not loaded, configure the image as an
.Xr md 4
disk device using
.Xr mdconfig 8 ,
and automatically mount it using
.Xr mount_cd9660 8
on the mount point provided as the first argument to the script.
.Pp
The de-duplication is a
.Fx
specific feature and while it does not require any changes to on-disk
compressed image format, however it did require some matching changes to the
.Xr geom_uzip 4
to handle resulting images correctly.
.Pp
To make use of
.Ar zstd
.Nm
images, the kernel must be configured with
.Cd ZSTDIO .
It is enabled by default in many
.Cd GENERIC
kernels provided as binary distributions by
.Fx .
The status on any particular system can be verified by checking
.Xr sysctl 8
.Dv kern.features.geom_uzip_zstd
for
.Dq 1 .
.Sh EXIT STATUS
.Ex -std
.Sh SEE ALSO
.Xr gzip 1 ,
.Xr xz 1 ,
.Xr zstd 1 ,
.Xr zlib 3 ,
.Xr geom 4 ,
.Xr geom_uzip 4 ,
.Xr md 4 ,
.Xr mdconfig 8 ,
.Xr mount_cd9660 8
.Sh AUTHORS
.An Maxim Sobolev Aq Mt sobomax@FreeBSD.org