DMA Access Synchronization ========================== - At driver level, few types of buffers are supported: * SIMPLE - non-reusable buffers, the use infomation can be used for cleanup after crashed applications. * EXCLUSIVE - reusable buffers which can be mmaped by a single appliction only. There is two modes of these buffers: + Buffers in a STANDARD mode are created for a single DMA operation and if such buffer is detected while trying to reuse, the last operation has failed and reset is needed. + Buffers in a PERSISTENT mode are preserved between invocations of control application and cleaned up only after the PERSISTENT flag is removed * SHARED - reusable buffers shared by multiple processes. Not really needed at the moment. KMEM_FLAG_HW - indicates that buffer can be used by hardware, acually this means that DMA will be enabled afterwards. The driver is not able to check if it really was enable and therefore will block any attempt to release buffer until KMEM_HW_FLAG is passed to kmem_free routine as well. The later should only called with KMEM_HW_FLAG after the DMA engine is stopped. Then, the driver can be realesd by kmem_free if ref count reaches 0. KMEM_FLAG_EXCLUSIVE - prevents multiple processes mmaping the buffer simultaneously. This is used to prevent multiple processes use the same DMA engine at the same time. KMEM_FLAG_REUSE - requires reuse of existing buffer. If reusable buffer is found (non-reusable buffers, i.e. allocated without KMEM_FLAG_REUSE are ignored), it is returned instead of allocation. Three types of usage counters are used. At moment of allocation, the HW reference is set if neccessary. The usage counter is increased by kmem_alloc function and decreased by kmem_free. Finally, the reference is obtained at returned during mmap/munmap. So, on kmem_free, we do not clean a) buffers with reference count above zero or hardware reference set. REUSE flag should be supplied, overwise the error is returned b) PERSISTENT buffer. REUSE flash should be supplied, overwise the error is returned c) non-exclusive buffers with usage counter above zero (For exclusive buffer the value of usage counter above zero just means that application have failed without cleaning buffers first. There is no easy way to detect that for shared buffers, so it is left as manual operation in this case) d) any buffer if KMEM_FLAG_REUSE was provided to function During module unload, only buffers with references can prevent cleanup. In this case the only possiblity to free the driver is to call kmem_free passing FORCE flags. KMEM_FLAG_PERSISTENT - if passed to allocation routine, changes mode of buffer to PERSISTENT, if passed to free routine, vice-versa changes mode of buffer to NORMAL. Basically, if we call 'pci --dma-start' this flag should be passed to alloc and if we call 'pci --dma-stop' it should be passed to free. In other case, the flag should not be present. If application crashed, the munmap while be still called cleaning software references. However, the hardware reference will stay since it is not clear if hardware channel was closed or not. To lift hardware reference, the application can be re-executed (or dma_stop called, for instance). * If there is no hardware reference, the buffers will be reused by next call to application and for EXCLUSIVE buffer cleaned at the end. For SHARED buffers they will be cleaned during module cleanup only (no active references). * The buffer will be reused by next call which can result in wrong behaviour if buffer left in incoherent stage. This should be handled on upper level. - At pcilib/kmem level synchronization of multiple buffers is performed * The HW reference and following modes should be consistent between member parts: REUSABLE, PERSISTENT, EXCLUSIVE (only HW reference and PERSISTENT mode should be checked, others are handled on dirver level) * It is fine if only part of buffers are reused and others are newly allocated. However, on higher level this can be checked and resulting in failure. Treatment of inconsistencies: * Buffers are in PRESISTENT mode, but newly allocated, OK * Buffers are reused, but are not in PERSISTENT mode (for EXCLUSIVE buffers this means that application has crashed during the last execution), OK * Some of buffers are reused (not just REUSABLE, but actually reused), others - not, OK until a) either PERSISTENT flag is set or reused buffers are non-PERSISTENT b) either HW flag is set or reused buffers does not hold HW reference * PERSISTENT mode inconsistency, FAIL (even if we are going to set PERSISTENT mode anyway) * HW reference inconsistency, FAIL (even if we are going to set HW flag anyway) On allocation error at some of the buffer, call clean routine and * Preserve HW reference if buffers held HW reference before * Preserve PERSISTENT mode if buffers were in PERSISTENT mode before * Remove REUSE flag, we want to clean if it is allowed by current buffer status * EXCLUSIVE flag is not important for kmem_free routine. - At DMA level There is 4 components of DMA access: * DMA engine enabled/disabled * DMA engine IRQs enabled/disabled - always enabled at startup * Memory buffers * Ring start/stop pointers To prevent multiple processes accessing DMA engine in parallel, the first action is buffer initialization * Always with REUSE, EXCLUSIVE, and HW flags * Optionally with PERSISTENT flag (if DMA_PERSISTENT flag is set) If another DMA app is running, the buffer allocation will fail (no dma_stop is executed in this case) Depending on PRESERVE flag, kmem_free will be called with REUSE flag keeping buffer in memory (this is redundant since HW flag is enough) or HW flag indicating that DMA engine is stopped and buffer could be cleaned. PERSISTENT flag is defined by DMA_PERSISTENT flag passed to stop routine. PRESERVE flag is enforced if DMA_PERSISTENT is not passed to dma_stop routine and either it: a) Explicitely set by DMA_PERMANENT flag passed to dma_start function b) Implicitely set if DMA engine is already enabled during dma_start, all buffers are reused, and are in persistent mode. If PRESERVE flag is on, the engine will not be stopped at the end of execution (and buffers will stay because of HW flag). If buffers are reused and in PERSISTENT mode, DMA engine was on before dma_start (we not basing on PRESERVE flag, because it can be enforced), ring pointers are calculated from LAST_BD and states of ring elements. If previous application crashed (i.e. buffers may be corrupted). Two cases are possible: * If during the call buffers were in non-PERSISTENT mode, it can be easily detected - buffers are reused, but are not in PERSISTENT mode (or at least was not before we set them to). In this case we just reinitialize all buffers. * If during the call buffers were in PERSISTENT mode, it is up to user to check their consistency and restart DMA engine.] IRQs are enabled and disabled at each call Register Access Synchronization =============================== We need to serialize access to the registers by the different running applications and handle case when registers are accessed indirectly by writting PCI BARs (DMA implementations, for instance). * An option would be to serialize at least access to CMOSIS registers which are always accessed over register functions. Register/DMA Configuration ========================== - XML description of registers - Formal XML-based (or non XML-based) language for DMA implementation. a) Writting/Reading register values b) Wait until = on = report error c) ... ?