GpuKernelLaunch¶
Launches a GPU kernel.
Declaration¶
Syntax¶
GpuKernelLaunch(symbol kernel, expr thread_count, expr block_size,
call_arg* args, symbol? component_fit, gpu_scratch_buffer* scratch,
expr? stop_status, stmt* stop_replay, expr* overwritten,
expr* overwritten_if)
gpu_scratch_buffer = (int argument_index, expr size, expr? read_back)
Arguments¶
Argument |
Description |
|---|---|
|
the Function to run. Its signature has |
|
the number of threads the launch numbers: one per iteration of the loop, or, when the nest is launched in tiles, one per place in whole tiles; a count below one runs none. |
|
the number of threads per block. |
|
the arguments passed to every thread. |
|
the Block the host runs before the launch to give the allocatable array components of elements of an array argument the loop writes the storage the kernel writes into, or |
|
the arguments that are buffers only the device holds (below). |
|
the host array passed as the status buffer of a kernel that can |
|
the host statements that run the |
|
array arguments the kernel may assign whole before it reads them (below). |
|
for each of |
Return values¶
None.
Description¶
A launch is not a call: it starts thread_count threads, rounded up to whole
blocks of block_size, and returns without waiting for them. The threads are
numbered from zero, block after block; the kernel sends every thread numbered
thread_count or higher back before its body, so the host sizes what the
launch holds per thread -- a workspace slice -- by thread_count, not by the
threads it starts.
On a device that asks for tiles (CUDA), a nest of two or more dimensions
without a reduction can be launched in tiles: a block covers a tile of 32
iterations of the dimension its stores are contiguous in by the rest of the
block along a second dimension, the one the stores step through next in
memory (or, if they do not agree on one, the next one in loop order). The
block index numbers the tiles, and thread_count counts whole tiles. Where
whole tiles would pad the nest by more than a quarter, as a short dimension
does, the host tells the kernel to number the iterations flat instead, and
thread_count counts iterations. Padding to whole tiles can take
thread_count past the kind of the loop indices, so it is an integer(8).
The kernel checks the zero-based tile coordinates against the extents before
adding the loop lower bounds. A padded thread returns before constructing a
source index that could overflow even when every valid index is representable.
GpuSync is what waits; the gpu_launch_sync
pass places it where the host next needs what a kernel wrote (see
GPU offloading).
The execution configuration is part of the node rather than of the kernel, because the same kernel is normally launched with different sizes.
The kernel is an ordinary Function; what makes
it launchable is that its signature has exec_space = Kernel, and a kernel
has no result. See exec_space.
The gpu_offload pass replaces a parallel loop assigned to the device with its
launch, together with the copies the host makes before and after it; nothing
of the original loop is kept. gpu_kernel_finalize records the kernel's
Function.gpu layout after shared lowering. device_launch_expand and both
device emitters consume that layout. Workspace sizes are evaluated once on the host and passed as
explicit scalar arguments, so host allocation and device indexing use the
same values.
Before creating a launch, GPU offload checks the allocation of allocatable array results in callees that remain out of line. Nested or multiple allocation sites must agree with a buffer shape established by an unconditional fixed allocation. Otherwise the loop is rejected while the original loop is still available, which is an error: it is a lowering LFortran does not have yet. Reallocation and conditional assignments remain eligible when they preserve every extent, not just the total element count.
A loop reaches GPU offload only after the unsupported-construct check in the
parallel_dispatch pass has assigned it to the device (see
GPU offloading), so a loop the offload cannot
lower is a compile-time error, whatever its reason, whether or not
--gpu-allow-cpu-fallback is given, and also with --show-gpu-kernel-source.
A scratch entry makes the argument argument_index a buffer of size
elements of the kernel argument's element type that only the device holds,
such as an arena of the device heap: the host allocates nothing for it and
copies nothing in or out. The actual argument is an allocatable array of the
launching procedure, not allocated before the launch. When read_back is
present, the host allocates the actual with read_back elements after the
launch, at most size, and copies the leading elements of the buffer into
it; otherwise the actual stays unallocated.
A kernel that can stop records the statement a thread reached in the
status buffer stop_status. The statements of stop_replay read it and run
that statement on the host. They run after the first GpuSync that follows
the launch, which the gpu_launch_sync pass places, copies of them after
each GpuSync the launch may still be pending at; that pass also clears the
first element of stop_status once, on entry to the procedure.
When overwritten_if[i], a scalar logical the host evaluates at the launch,
is true, the kernel assigns every element of the array overwritten[i], an
actual argument of the launch, before it reads any of them. What the host
holds there then does not reach the kernel, so device_launch_expand hands
the buffer to the runtime as LFORTRAN_GPU_WRITE_DISCARD, and a runtime
with memory of its own only makes room for it instead of copying it in.
gpu_offload finds such arrays in the loop as the source wrote it: an
assignment a(i, j, ...) = value subscripted by the loop variables, one per
loop, that each iteration reaches before anything else of it names a, at
the top of the iteration or inside IFs that compare loop variables with
values the loop cannot change. The condition is that those loops, narrowed
by the IFs, run over exactly the bounds of a.
This proof does not imply that the buffer is unaliased. If any other buffer
argument of the launch overlaps it, the runtime treats the discard as an
ordinary write so that the other argument can still read its previous
contents. The kernel layout's gpu_kernel_argument.noalias is a separate
shared-ASR proof; it is not inferred from overwritten.
Private array storage is placed after shared lowering by gpu_kernel_storage,
before address-space specialization. Fixed-size arrays that fit the device's
stack budget have Thread memory space. Larger arrays have Global memory
space and the kernel layout records their per-thread workspace slices. This
placement never turns an iteration-private value into a shared value.
Examples¶
(GpuKernelLaunch
:kernel (SymbolRef 1 "zero")
:thread_count (IntegerConstant
:n 1
:type (Integer
:kind 4
)
:intboz_type :Decimal
)
:block_size (IntegerConstant
:n 256
:type (Integer
:kind 4
)
:intboz_type :Decimal
)
:args [
(call_arg
:value (Var
:v (SymbolRef 2 "a")
)
)
]
:component_fit nil
:scratch []
:stop_status nil
:stop_replay []
:overwritten []
:overwritten_if []
)
It comes from this complete ASR text document:
(TranslationUnit
:symtab (SymbolTable
:id 0
:symbols {
"m" (Module
:symtab (SymbolTable
:id 1
:symbols {
"launch" (Function
:symtab (SymbolTable
:id 2
:symbols {
"a" (Variable
:parent_symtab 2
:name "a"
:dependencies []
:intent :InOut
:symbolic_value nil
:value nil
:storage :Default
:type (Array
:type (Real
:kind 4
)
:dims [
(dimension
:start (IntegerConstant
:n 1
:type (Integer
:kind 4
)
:intboz_type :Decimal
)
:length (IntegerConstant
:n 256
:type (Integer
:kind 4
)
:intboz_type :Decimal
)
)
]
:physical_type :FixedSizeArray
:memory_space :Global
)
:type_declaration nil
:abi :Source
:access :Public
:presence :Required
:value_attr false
:target_attr false
:contiguous_attr false
:bindc_name nil
:is_volatile false
:is_protected false
:pass_attr :NotMethod
:self_argument nil
:codims []
:common_block nil
)
}
)
:name "launch"
:function_signature (FunctionType
:arg_types [
(Array
:type (Real
:kind 4
)
:dims [
(dimension
:start (IntegerConstant
:n 1
:type (Integer
:kind 4
)
:intboz_type :Decimal
)
:length (IntegerConstant
:n 256
:type (Integer
:kind 4
)
:intboz_type :Decimal
)
)
]
:physical_type :FixedSizeArray
:memory_space :Global
)
]
:return_var_type nil
:abi :Source
:deftype :Implementation
:bindc_name nil
:elemental false
:pure false
:module false
:inline false
:static false
:restrictions []
:is_restriction false
:exec_space :Host
)
:dependencies []
:args [
(Var
:v (SymbolRef 2 "a")
)
]
:body [
(GpuKernelLaunch
:kernel (SymbolRef 1 "zero")
:thread_count (IntegerConstant
:n 1
:type (Integer
:kind 4
)
:intboz_type :Decimal
)
:block_size (IntegerConstant
:n 256
:type (Integer
:kind 4
)
:intboz_type :Decimal
)
:args [
(call_arg
:value (Var
:v (SymbolRef 2 "a")
)
)
]
:component_fit nil
:scratch []
:stop_status nil
:stop_replay []
:overwritten []
:overwritten_if []
)
(GpuSync)
]
:return_var nil
:access :Public
:deterministic false
:side_effect_free false
:module_file nil
)
"zero" (Function
:symtab (SymbolTable
:id 3
:symbols {
"a" (Variable
:parent_symtab 3
:name "a"
:dependencies []
:intent :InOut
:symbolic_value nil
:value nil
:storage :Default
:type (Array
:type (Real
:kind 4
)
:dims [
(dimension
:start (IntegerConstant
:n 1
:type (Integer
:kind 4
)
:intboz_type :Decimal
)
:length (IntegerConstant
:n 256
:type (Integer
:kind 4
)
:intboz_type :Decimal
)
)
]
:physical_type :FixedSizeArray
:memory_space :Global
)
:type_declaration nil
:abi :Source
:access :Public
:presence :Required
:value_attr false
:target_attr false
:contiguous_attr false
:bindc_name nil
:is_volatile false
:is_protected false
:pass_attr :NotMethod
:self_argument nil
:codims []
:common_block nil
)
}
)
:name "zero"
:function_signature (FunctionType
:arg_types [
(Array
:type (Real
:kind 4
)
:dims [
(dimension
:start (IntegerConstant
:n 1
:type (Integer
:kind 4
)
:intboz_type :Decimal
)
:length (IntegerConstant
:n 256
:type (Integer
:kind 4
)
:intboz_type :Decimal
)
)
]
:physical_type :FixedSizeArray
:memory_space :Global
)
]
:return_var_type nil
:abi :Source
:deftype :Implementation
:bindc_name nil
:elemental false
:pure false
:module false
:inline false
:static false
:restrictions []
:is_restriction false
:exec_space :Kernel
)
:dependencies []
:args [
(Var
:v (SymbolRef 3 "a")
)
]
:body []
:return_var nil
:access :Public
:deterministic false
:side_effect_free false
:module_file nil
)
}
)
:name "m"
:parent_module nil
:dependencies []
:loaded_from_mod false
:intrinsic false
:has_submodules false
:global_init nil
)
"main" (Program
:symtab (SymbolTable
:id 4
:symbols {}
)
:name "main"
:dependencies []
:body []
:global_init nil
)
}
)
:items []
:global_init nil
)