.\"	$NetBSD: compat_linux.8,v 1.46 2021/11/28 18:08:51 ryo Exp $
.\"
.\" Copyright (c) 1995 Frank van der Linden
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\" 3. All advertising materials mentioning features or use of this software
.\"    must display the following acknowledgement:
.\"      This product includes software developed for the NetBSD Project
.\"      by Frank van der Linden
.\" 4. The name of the author may not be used to endorse or promote products
.\"    derived from this software without specific prior written permission
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
.\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
.\" IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
.\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
.\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
.\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.\"
.Dd September 26, 2021
.Dt COMPAT_LINUX 8
.Os
.Sh NAME
.Nm compat_linux
.Nd setup procedure for running Linux binaries
.Sh DESCRIPTION
.Nx
supports running Linux binaries.
This applies to aarch64, alpha, amd64, arm, i386, m68k, and powerpc systems for now.
Both the a.out and ELF binary formats are supported.
Most programs should work.
.Nx
aarch64 and amd64 can execute both 32-bit and 64-bit Linux programs.
Programs that will not work include some that use
i386-specific calls, such as enabling virtual 8086 mode.
Currently, sound is supported through OSSv3 compat.
.Pp
The Linux compatibility feature is active
for kernels compiled with the
.Dv COMPAT_LINUX
option enabled.
If support for Linux a.out executables is desired, the
.Dv EXEC_AOUT
option should be enabled in addition to option
.Dv COMPAT_LINUX .
Similarly, if support for Linux 32-bit and/or 64-bit ELF executables
is desired, the
.Dv EXEC_ELF32
and/or
.Dv EXEC_ELF64
options (respectively) should be enabled in addition to
.Dv COMPAT_LINUX .
If sound support is desired,
.Dv COMPAT_OSSAUDIO
should be enabled.
.Pp
A lot of programs are dynamically linked.
This means that you will also need the Linux shared libraries that the
program depends on, and the runtime linker.
Also, you will need to create a
.Dq shadow root
directory for Linux binaries on your
.Nx
system.
This directory is named
.Pa /emul/linux
or
.Pa /emul/linux32
for 32-bit emulation on 64-bit systems.
Any file operations done by Linux programs run under
.Nx
will look in this directory first.
So, if a Linux program opens, for example,
.Pa /etc/passwd ,
.Nx
will
first try to open
.Pa /emul/linux/etc/passwd ,
and if that does not exist open the
.Sq real
.Pa /etc/passwd
file.
It is recommended that you install
Linux packages that include configuration files, etc under
.Pa /emul/linux ,
to avoid naming conflicts with possible
.Nx
counterparts.
Shared libraries should also be installed in the shadow tree.
Filenames that start "/../" are only looked up in the real root.
.Pp
Generally, you will need to look for the shared libraries that Linux
binaries depend on only the first few times that you install a Linux
program on your
.Nx
system.
After a while, you will have a sufficient
set of Linux shared libraries on your system to be able to run newly
imported Linux binaries without any extra work.
.Ss Setting up shared libraries
Find the dependencies of a Linux binary using
.Xr readelf 1 :
.Bd -literal
$ readelf -d ./runner | grep Shared
 0x00000001 (NEEDED)                     Shared library: [libstdc++.so.6]
 0x00000001 (NEEDED)                     Shared library: [libz.so.1]
 0x00000001 (NEEDED)                     Shared library: [libXxf86vm.so.1]
 0x00000001 (NEEDED)                     Shared library: [libGL.so.1]
 0x00000001 (NEEDED)                     Shared library: [libopenal.so.1]
 0x00000001 (NEEDED)                     Shared library: [libm.so.6]
 0x00000001 (NEEDED)                     Shared library: [librt.so.1]
 0x00000001 (NEEDED)                     Shared library: [libpthread.so.0]
 0x00000001 (NEEDED)                     Shared library: [libdl.so.2]
 0x00000001 (NEEDED)                     Shared library: [libcrypto.so.1.0.0]
 0x00000001 (NEEDED)                     Shared library: [libXext.so.6]
 0x00000001 (NEEDED)                     Shared library: [libX11.so.6]
 0x00000001 (NEEDED)                     Shared library: [libXrandr.so.2]
 0x00000001 (NEEDED)                     Shared library: [libGLU.so.1]
 0x00000001 (NEEDED)                     Shared library: [libssl.so.1.0.0]
 0x00000001 (NEEDED)                     Shared library: [libgcc_s.so.1]
 0x00000001 (NEEDED)                     Shared library: [libc.so.6]
.Ed
.Pp
For x86, you can simply install the openSUSE shared libraries using the
.Pa pkgsrc/emulators/suse131_*
or
.Pa pkgsrc/emulators/suse131_32_*
packages.
.Pp
For example, an application which requires
.Pa libcrypto.so.1.0.0 ,
.Pa libXext.so.6 ,
and
.Pa libGL.so.1
will require
.Dv openssl ,
.Dv x11 ,
and
.Dv glx ,
in addition to the
.Dv base
SUSE package.
.Pp
Otherwise, you may have to obtain shared libraries from another Linux
system, and copy them to e.g.
.Pa /emul/linux/lib64 .
.Ss Setting up procfs
Some Linux binaries expect procfs to be mounted and that it
contains some Linux-specific extensions.
If it's not the case, they behave unexpectedly or even crash.
.Pp
Mount procfs on
.Nx
using following command:
.Bl -tag -width 123 -offset indent
.It $ mount_procfs procfs /emul/linux/proc
.El
.Pp
You can also set up your system so that procfs is mounted automatically
on system boot, by putting an entry like the one below to
.Pa /etc/fstab .
.Bl -tag -width 123 -offset indent
.It procfs /emul/linux/proc procfs ro
.El
.Pp
Note: 
.Xr mount_procfs 8
defaults to Linux flavored procfs since
.Nx 5.0 .
Ensure you do not mount procfs with
.Ar nolinux .
.Pp
See
.Xr mount_procfs 8
for further information.
.Ss Setting up other files
Newer version of Linux use
.Pa /etc/nsswitch.conf
for network information, such as NIS and DNS.
You must create or get a valid copy of this file and put it in
.Pa /emul/linux/etc .
.Sh CAVEATS
.Nm
is generally not enabled in
.Dv GENERIC
kernels for security reasons,
but is available as a module.
It must be added to
.Xr modules.conf 5
to be used.
It will not be loaded automatically.
.Sh BUGS
The information about Linux distributions will become outdated.
.Pp
Absolute pathnames pointed to by symbolic links are only looked up in the
shadow root when the symbolic link itself was found by an absolute
pathname inside the shadow root.
This is not consistent.
.Pp
Linux executables cannot handle directory offset cookies > 32 bits.
Should such an offset occur, you will see the message
.Dq linux_getdents: dir offset too large for emulated program .
Currently, this can only
happen on NFS mounted file systems, mounted from servers that return
offsets with information in the upper 32 bits.
These errors should rarely happen, but can be avoided by mounting this
file system with offset translation enabled.
See the
.Fl X
option to
.Xr mount_nfs 8 .
The
.Fl 2
option to
.Xr mount_nfs 8
will also have the desired effect, but is less preferable.