.\"	$NetBSD: dmover.9,v 1.15 2017/07/03 21:28:48 wiz Exp $
.\"
.\" Copyright (c) 2002 Wasabi Systems, Inc.
.\" All rights reserved.
.\"
.\" Written by Jason R. Thorpe for Wasabi Systems, Inc.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\" 3. All advertising materials mentioning features or use of this software
.\"    must display the following acknowledgement:
.\"	This product includes software developed for the NetBSD Project by
.\"	Wasabi Systems, Inc.
.\" 4. The name of Wasabi Systems, Inc. may not be used to endorse
.\"    or promote products derived from this software without specific prior
.\"    written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY WASABI SYSTEMS, INC. ``AS IS'' AND
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
.\" PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL WASABI SYSTEMS, INC
.\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
.\" POSSIBILITY OF SUCH DAMAGE.
.\"
.Dd December 4, 2007
.Dt DMOVER 9
.Os
.Sh NAME
.Nm dmover_backend_register ,
.Nm dmover_backend_unregister ,
.Nm dmover_session_create ,
.Nm dmover_session_destroy ,
.Nm dmover_request_alloc ,
.Nm dmover_request_free ,
.Nm dmover_process ,
.Nm dmover_done
.Nd hardware-assisted data mover interface
.Sh SYNOPSIS
.In dev/dmover/dmovervar.h
.Pp
Client interface routines:
.Pp
.Ft int
.Fn "dmover_session_create" "const char *" "struct dmover_session **"
.Ft void
.Fn "dmover_session_destroy" "struct dmover_session *"
.Ft "struct dmover_request *"
.Fn "dmover_request_alloc" "struct dmover_session *" "dmover_buffer *"
.Ft void
.Fn "dmover_request_free" "struct dmover_request *"
.Ft void
.Fn "dmover_process" "struct dmover_request *"
.Pp
Back-end interface routines:
.Pp
.Ft void
.Fn "dmover_backend_register" "struct dmover_backend *"
.Ft void
.Fn "dmover_backend_unregister" "struct dmover_backend *"
.Ft void
.Fn "dmover_done" "struct dmover_request *"
.Sh DESCRIPTION
The
.Nm dmover
facility provides an interface to hardware-assisted data movers.
This can be used to copy data from one location in memory to another, clear
a region of memory, fill a region of memory with a pattern, and perform
simple operations on multiple regions of memory, such as an XOR, without
intervention by the CPU.
.Pp
The drivers for hardware-assisted data movers present themselves to
.Nm dmover
by registering their capabilities.
When a client wishes to use a
.Nm dmover
function, it creates a session for that function, which identifies back-ends
capable of performing that function.
The client then enqueues requests on that session, which the back-ends
process asynchronously.
The client may choose to block until the request is completed, or may
have a call-back invoked once the request has been completed.
.Pp
When a client creates a session, the
.Nm dmover
facility identifies back-ends which are capable of handling the requested
function.
When a request is scheduled for processing, the
.Nm dmover
scheduler will identify the best back-end to process the request from
the list of candidate back-ends, in an effort to provide load balancing,
while considering the relative performance of each back-end.
.Pp
A
.Nm dmover
function always has one output region.
A function may have zero or more input regions, or may use an immediate
value as an input.
For functions which use input regions, the lengths of each input region
and the output region must be the same.
All
.Nm dmover
functions with the same name will have the same number of and type inputs.
If a back-end attempts to register a function which violates this invariant,
behavior is undefined.
.Pp
The
.Nm dmover
facility supports several types of buffer descriptors.
For functions which use input regions, each input buffer descriptor and
the output buffer descriptor must be of the same type.
This restriction may be removed in a future revision of the interface.
.Pp
The
.Nm dmover
facility may need to interrupt request processing and restart it.
Clients of the
.Nm dmover
facility should take care to avoid unwanted side-effects should this occur.
In particular, for functions which use input regions, no input region may
overlap with the output region.
.Ss DATA STRUCTURES
The
.Nm dmover
facility shares several data structures between the client and
back-end in order to describe sessions and requests.
.Bd -literal -offset indent
typedef enum {
	DMOVER_BUF_LINEAR,
	DMOVER_BUF_UIO
} dmover_buffer_type;

typedef struct {
	void *l_addr;
	size_t l_len;
} dmover_buf_linear;

typedef union {
	dmover_buf_linear dmbuf_linear;
	struct uio *dmbuf_uio;
} dmover_buffer;
.Ed
.Pp
Together, these data types are used to describe buffer data structures
which the
.Nm dmover
facility understands.
Additional buffer types may be added in future revisions of the
.Nm dmover
interface.
.Pp
The
.Fa dmover_assignment
structure contains the information about the back-end to which a
request is currently assigned.
It contains the following public members:
.Bl -tag -width "XXXX"
.It struct dmover_backend *das_backend
This is a pointer to the back-end.
.It const struct dmover_algdesc *das_algdesc
This is a pointer to the algorithm description provided by
the back-end for the request's function.
.El
.Pp
The
.Fa dmover_session
structure contains the following public members:
.Bl -tag -width "XXXX"
.It void *dses_cookie
This is a pointer to client private data.
.It int dses_ninputs
This is the number of inputs used by the selected function.
.El
.Pp
The
.Fa dmover_request
structure contains the following public members:
.Bl -tag -width "XXXX"
.It TAILQ_ENTRY(dmover_request) dreq_dmbq
Linkage on the back-end's queue of pending requests.
.It struct dmover_session *dreq_session
Pointer to the session with which this request is associated.
This is intended for use by the back-end.
.It struct dmover_assignment *dreq_assignment
Pointer to the
.Fa dmover_assignment
structure which describes the back-end to which the request is
currently assigned.
The back-end is assigned when the request is scheduled with
.Fn dmover_process .
.It void (*dreq_callback)(struct dmover_request *)
This is a pointer to an optional call-back function provided by the
client.
If provided, the call-back is invoked when the request is complete.
This field must be
.Dv NULL
if
.Em DMOVER_REQ_WAIT
is set in
.Em dreq_flags .
.It void *dreq_cookie
This is a pointer to client private data specific to the request.
.It void *dreq_dmbcookie
This is a pointer to back-end private data, for use while the back-end
is actively processing a request.
.It volatile int dreq_flags
The following flags are defined:
.Bl -tag -width "DMOVER_REQ_RUNNINGXX"
.It DMOVER_REQ_DONE
The request has been completed.
If not using a call-back, the client may poll this bit to determine
if a request has been processed.
.It DMOVER_REQ_ERROR
An error has occurred while processing the request.
.It DMOVER_REQ_RUNNING
The request is currently being executed by the back-end.
Once a command is running, it cannot be cancelled, and must run to completion.
.It DMOVER_REQ_WAIT
If set by the client,
.Fn dmover_process
will wait for the request to complete using
.Xr cv_wait 9 .
This flag may only be used if the caller has a valid thread context.
If this flag is set, a callback may not be used.
.El
.It int dreq_error
If the
.Em DMOVER_REQ_ERROR
bit is set, this contains the
.Xr errno 2
value indicating the error that occurred during processing.
.It dmover_buffer_type dreq_outbuf_type
The type of the output buffer.
.It dmover_buffer dreq_outbuf
The output buffer.
.It uint8_t dreq_immediate[8]
This is the input for algorithms which use an immediate value.
Values smaller than 8 bytes should use the least-significant bytes first.
For example, a 32-bit integer would occupy bytes 0, 1, 2, and 3.
.It dmover_buffer_type dreq_inbuf_type
The type of the input buffer.
This is only used if the
.Nm dmover
function has one or more inputs.
.It dmover_buffer *dreq_inbuf
A pointer to an array of input buffers.
This is only used if the
.Nm dmover
function has one or more inputs.
The number of inputs, and thus the number of valid elements in the
array, is specified by the algorithm description for the session.
.El
.Ss CLIENT INTERFACE
The following functions are provided to the client:
.Bl -tag -width "XXXX"
.It Fn dmover_session_create "function" "sessionp"
.Pp
The
.Fn dmover_session_create
function creates a data mover session for the specified data movement
function
.Fa function .
A handle to the new session is returned in
.Fa sessionp .
.Pp
The following are valid data movement function names:
.Bl -tag -width "fill8xx"
.It Dq zero
Fill a memory region with zeros.
This algorithm has an input count of 0.
.It Dq fill8
Fill a memory region with an 8-bit pattern.
This algorithm has an input count of 0.
The pattern is provided in the
.Em dreq_imm8
member of the
.Fa dmover_request
structure.
.It Dq copy
Copy a memory region from one location to another.
This algorithm has an input count of 1.
.It Dq xor2
Perform an XOR operation on 2 inputs.
This algorithm has an input count of 2.
.It Dq xor3
Perform an XOR operation on 3 inputs.
This algorithm has an input count of 3.
.It Dq xor4
Perform an XOR operation on 4 inputs.
This algorithm has an input count of 4.
.It Dq xor5
Perform an XOR operation on 5 inputs.
This algorithm has an input count of 5.
.It Dq xor6
Perform an XOR operation on 6 inputs.
This algorithm has an input count of 6.
.It Dq xor7
Perform an XOR operation on 7 inputs.
This algorithm has an input count of 7.
.It Dq xor8
Perform an XOR operation on 8 inputs.
This algorithm has an input count of 8.
.El
.Pp
Users of the
.Nm dmover
facility are encouraged to use the following aliases for the well-known
function names, as doing so saves space and reduces the chance of programming
errors:
.Bl -tag -width "DMOVER_FUNC_FILL32xx"
.It DMOVER_FUNC_ZERO
.Dq zero
.Pq Va dmover_funcname_zero
.It DMOVER_FUNC_FILL8
.Dq fill8
.Pq Va dmover_funcname_fill8
.It DMOVER_FUNC_COPY
.Dq copy
.Pq Va dmover_funcname_copy
.It DMOVER_FUNC_XOR2
.Dq xor2
.Pq Va dmover_funcname_xor2
.It DMOVER_FUNC_XOR3
.Dq xor3
.Pq Va dmover_funcname_xor3
.It DMOVER_FUNC_XOR4
.Dq xor4
.Pq Va dmover_funcname_xor4
.It DMOVER_FUNC_XOR5
.Dq xor5
.Pq Va dmover_funcname_xor5
.It DMOVER_FUNC_XOR6
.Dq xor6
.Pq Va dmover_funcname_xor6
.It DMOVER_FUNC_XOR7
.Dq xor7
.Pq Va dmover_funcname_xor7
.It DMOVER_FUNC_XOR8
.Dq xor8
.Pq Va dmover_funcname_xor8
.El
.It Fn dmover_session_destroy "session"
.Pp
The
.Fn dmover_session_destroy
function tears down a data mover session and releases all resources
associated with it.
.It Fn dmover_request_alloc "session" "inbuf"
.Pp
The
.Fn dmover_request_alloc
function allocates a
.Nm dmover
request structure and associates it with the specified session.
If the
.Fa inbuf
argument is not
.Dv NULL ,
.Fa inbuf
is used as the array of input buffer descriptors in the request.
Otherwise, if
.Fa inbuf
is
.Dv NULL
and the
.Nm dmover
function requires input buffers, the input buffer descriptors will be
allocated automatically using
.Xr malloc 9 .
.Pp
If the request structure or input buffer descriptors cannot be allocated,
.Fn dmover_request_alloc
return
.Dv NULL
to indicate failure.
.It Fn dmover_request_free "req"
.Pp
The
.Fn dmover_request_free
function frees a
.Nm dmover
request structure.
If the
.Nm dmover
function requires input buffers, and the input buffer descriptors
associated with
.Fa req
were allocated by
.Fn dmover_request_alloc ,
then the input buffer descriptors will also be freed.
.It Fn dmover_process "req"
.Pp
The
.Fn dmover_process
function submits the
.Nm dmover
request
.Fa req
for processing.
The call-back specified by the request is invoked when processing is
complete.
.El
.Pp
The
.Fn dmover_session_create
and
.Fn dmover_session_destroy
functions must not be called from interrupt context.
.Pp
The
.Fn dmover_request_alloc ,
.Fn dmover_request_free ,
and
.Fn dmover_process
functions may be called from interrupt handlers at levels
.Em IPL_VM ,
.Em IPL_SOFTCLOCK ,
and
.Em IPL_SOFTNET ,
or in non-interrupt context.
.Pp
The request completion call-back is called from a software interrupt
handler at
.Em IPL_SOFTCLOCK .
.Ss BACK-END INTERFACE
A back-end describes the
.Nm dmover
functions it can perform using an array of
.Fa dmover_algdesc
structures:
.Bd -literal -offset indent
struct dmover_algdesc {
	const char *dad_name;	/* algorithm name */
	void *dad_data;		/* opaque algorithm description */
	int dad_ninputs;	/* number of inputs */
};
.Ed
.Pp
The
.Em dad_name
member points to a valid
.Nm dmover
function name which the client may specify.
The
.Em dad_data
member points to a back-end-specific description of the algorithm.
.Pp
A back-end presents itself to the
.Nm dmover
facility using the
.Fa dmover_backend
structure.
The back-end must initialize the following members of the structure:
.Bl -tag -width "XXXX"
.It const char *dmb_name
This is the name of the back-end.
.It u_int dmb_speed
This is an estimate of the number of kilobytes/second that the
back-end can process.
.It void *dmb_cookie
This is a pointer to back-end private data.
.It const struct dmover_algdesc *dmb_algdescs
This points to an array of
.Fa dmover_algdesc
structures which describe the functions the data mover can perform.
.It int dmb_nalgdescs
This is the number of elements in the
.Em dmb_algdescs
array.
.It void (*dmb_process)(struct dmover_backend *)
This is the entry point to the back-end used to process requests.
.El
.Pp
When invoked by the
.Nm dmover
facility, the back-end's
.Fn (*dmb_process)
function should examine the pending request queue in its
.Fa dmover_backend
structure:
.Bl -tag -width "XXXX"
.It TAILQ_HEAD(, dmover_request) dmb_pendreqs
This is the queue of pending requests.
.It int dmb_npendreqs
This is the number of requests in the
.Em dmb_pendreqs
queue.
.El
.Pp
If an error occurs when processing the request, the
.Em DMOVER_REQ_ERROR
bit must be set in the
.Em dreq_flags
member of the request, and the
.Em dreq_error
member set to an
.Xr errno 2
value to indicate the error.
.Pp
When the back-end has finished processing the request, it must call
the
.Fn dmover_done
function.
This function eventually invokes the client's call-back routine.
.Pp
If a hardware-assisted data mover uses interrupts, the interrupt handlers
should be registered at IPL_VM.
.Pp
The following functions are provided to the back-ends:
.Bl -tag -width "XXXX"
.It Fn dmover_backend_register "backend"
.Pp
The
.Fn dmover_backend_register
function registers the back-end
.Fa backend
with the
.Nm dmover
facility.
.It Fn dmover_backend_unregister "backend"
.Pp
The
.Fn dmover_backend_unregister
function removes the back-end
.Fa backend
from the
.Nm dmover
facility.
The back-end must already be registered.
.It Fn dmover_done "req"
.Pp
The
.Fn dmover_done
function is called by the back-end when it has finished processing
a request, whether the request completed successfully or not.
.El
.Pp
The
.Fn dmover_backend_register
and
.Fn dmover_backend_unregister
functions must not be called from interrupt context.
.Pp
The
.Fn dmover_done
function may be called at
.Em IPL_VM ,
.Em IPL_SOFTCLOCK ,
.Em IPL_SOFTNET ,
or in non-interrupt context.
.Sh EXAMPLES
The following is an example of a client using
.Nm dmover
to zero-fill a region of memory.
In this example, the CPU will be able to context switch to another
thread and perform work while the hardware-assisted data mover clears
the specified block of memory.
.Bd -literal
int
hw_bzero(void *buf, size_t len)
{
	struct dmover_session *dses;
	struct dmover_request *dreq;
	int error;

	error = dmover_session_create(DMOVER_FUNC_ZERO, &dses);
	if (error)
		return (error);

	dreq = dmover_request_alloc(dses, NULL);
	if (dreq == NULL) {
		dmover_session_destroy(dses);
		return (ENOMEM);
	}

	dreq->dreq_flags = DMOVER_REQ_WAIT;
	dreq->dreq_callback = NULL;
	dreq->dreq_outbuf.dreq_outbuf_type = DMOVER_BUF_LINEAR;
	dreq->dreq_outbuf.dmbuf_linear.l_addr = buf;
	dreq->dreq_outbuf.dmbuf_linear.l_len = len;

	dmover_process(dreq);

	error = (dreq->dreq_flags & DMOVER_REQ_ERROR) ?
	    dreq->dreq_error : 0;

	dmover_request_free(dreq);
	dmover_session_destroy(dses);

	return (error);
}
.Ed
.Sh SEE ALSO
.Xr queue 3 ,
.Xr dmoverio 4
.Sh HISTORY
The
.Nm dmover
facility first appeared in
.Nx 2.0 .
.Sh AUTHORS
The
.Nm dmover
facility was designed and implemented by
.An Jason R. Thorpe
.Aq thorpej@wasabisystems.com
and contributed by Wasabi Systems, Inc.
.Sh BUGS
The mechanism by which a back-end should advertise its performance to
the request scheduler is not well-defined.
Therefore, the load-balancing mechanism within the request scheduler is
also not well-defined.