
CUDA is a parallel programming architecture and model, which includes a C complier plus support for OpenCL and DirectCompute, see Figure \ref{cudaarc}. It is designed such that it naively supports multiple computational interfaces such as standard languages and APIs. The main intention of this programming model is to simplify programming on the GPU. When using CUDA to program one will avoid the overhead of programming using a graphical API such as CG. Another feature worth mentioning here is how CUDA enables the programmer to directly program to the GPUs components such as direct memory access, using the shared memory and specifically manipulating threads.

\begin{figure}[H]
	\centering
		\includegraphics[width=9cm]{report/background/cudaarchitecture.PNG}
		\caption{Image illustrating CUDA architecture from \cite{nvidia_cuda_prog_guide}, with permission from NVIDIA}
		\label{cudaarc}
\end{figure}

One has the option of running CUDA as a C extension, such that those familiar with C programming can easily use it to accelerate their code. As mentioned earlier the GPU functions as an accelerator and is therefore used with the CPU where one can run non parallelizable code in a program on the CPU and the highly parallelizable code on the GPU to accelerate execution and distribute the workload. The NVIDIA CUDA programming model consists of a "host" that is a traditional CPU, and one or more compute devices that are massively data-parallel co-processors (GPUs). Each device is equipped with a large number of arithmetic execution units that has its own DRAM, and runs many threads in parallel.

\begin{figure}[H]
	\centering
	\includegraphics[width=7cm]{report/background/cudamem.png}
	\caption{Illustration of the CUDA memory hierarchy from \cite{nvidia_cuda_prog_guide}, with permission NVIDIA}
	\label{cudamem}
\end{figure}

To invoke calculations on the GPU one has to perform a kernel launch, which is basically a function written with the intent of what each thread on the GPU is to perform. The GPU has a specific architecture of threads where they are divided into blocks and where blocks are divided into a grid, see Figure \ref{cudamem}. The grid has two dimensions and can contain up to 65536 blocks in each dimension. While each block contains threads in three dimensions, and can contain up to 512 threads in two dimensions and 64 in the third. When executing a kernel one specifies the dimensions of the grid and blocks to specify how many threads will be executing the kernel. 

A mention has already been made about the GPU having its own DRAM and that this is used in communicating with the host system. To accelerate calculations within the GPU itself there are several other layers of memory such as constant, shared and texture. Table \ref{memhi} shows the relation between these. Here the ocus is on showing the different memory options available, and to map their attributes. For more details regarding CUDA and how to program on it read the NVIDIA programming guide \cite{nvidia_cuda_prog_guide}

\begin{table}[H]
	\centering
\label{memhi}
\caption{Table showing CUDA Memory Hierarchy, with data from \cite{nvidia_cuda_prog_guide}}
\begin{tabular}{|c|c|c|c|c|}
\hline
\textbf{Memory}				&	\textbf{Location}		&	\textbf{Cached} 		&\textbf{Access}				&\textbf{Scope in Architecture} \\
\hline
Register			&	On-chip			&	No 				&Read/Write		&Single thread \\
\hline
Local					&	Off-chip		&	No 				&Read/Write		&Single thread \\
\hline
Shared				&	On-chip			&	No 			&Read/Write		&Threads in a Block \\
\hline
Global				&	On-chip			&	No 				&Read/Write		&All \\
\hline
Constant			&	On-chip			&	Yes				&Read					&All \\
\hline
Texture				&	On-chip			&	Yes				&Read					&All \\
\hline

\end{tabular}
\end{table}