簡單瞭解 iOS CVPixelBuffer （上）
theme: smartblue
「這是我參與2022首次更文挑戰的第8天，活動詳情查看：2022首次更文挑戰」。
前言：
在iOS中，我們會常常看到 CVPixelBufferRef 這個類型，最常見到的場景是在Camera 採集的時候，返回的數據中有一個CMSampleBufferRef，而每個CMSampleBufferRef則包含一個 CVPixelBufferRef，在視頻硬解碼的返回數據裏也是一個 CVPixelBufferRef（裏面包含了所有的壓縮的圖片信息）。在瞭解了CVPixelBufferRef之後，我們將能夠掌握並且運用CVPixelBufferRef的使用；
本篇我們主要熟悉下CVPixelBuffer的使用；
CVPixelBuffer 簡介
CVPixelBuffer：核心視頻像素緩衝區是在主存儲器中保存像素的圖像緩衝區。生成幀、壓縮或解壓縮視頻或使用 Core Image 的應用程序都可以使用 CVPixelBuffer。
CVPixelBufferRef：是像素緩衝區類型，對CVPixelBuffer對象的引用。像素緩衝區類型基於圖像緩衝區類型；像素緩衝器實現了圖像緩衝器的存儲器存儲。
```
/*!
@typedef CVPixelBufferRef
@abstract Based on the image buffer type. The pixel buffer implements the memory storage for an image buffer.
*/ typedef CVImageBufferRef CVPixelBufferRef; ``` 由於CVPixelBufferRef是C中的對象，所以沒有ARC內存管理，必須由開發者自己去管理引用計數，控制對象生命週期；
首先，我們先來了解有關CVPixelBufferRef的基礎方法，例如：CVPixelBufferRef的創建、持有、釋放；
創建
創建的方法有四種，前三種常用到： - CVPixelBufferCreate() - CVPixelBufferCreateWithBytes() - CVPixelBufferCreateWithPlanarBytes() - CVPixelBufferCreateWithIOSurface()
普通的創建方法
/*!
為給定大小和像素格式創建單個像素緩衝區
@function CVPixelBufferCreate
@abstract Call to create a single PixelBuffer for a given size and pixelFormatType.
@discussion Creates a single PixelBuffer for a given size and pixelFormatType. It allocates the necessary memory based on the pixel dimensions, pixelFormatType and extended pixels described in the pixelBufferAttributes. Not all parameters of the pixelBufferAttributes will be used here.
@param width Width of the PixelBuffer in pixels.
@param height Height of the PixelBuffer in pixels.
@param pixelFormatType Pixel format indentified by its respective OSType.
@param pixelBufferAttributes A dictionary with additional attributes for a pixel buffer. This parameter is optional. See BufferAttributeKeys for more details.
@param pixelBufferOut The new pixel buffer will be returned here
@result returns kCVReturnSuccess on success.
*/
CV_EXPORT CVReturn CVPixelBufferCreate(
CFAllocatorRef CV_NULLABLE allocator,
size_t width,
size_t height,
OSType pixelFormatType,
CFDictionaryRef CV_NULLABLE pixelBufferAttributes,
CV_RETURNS_RETAINED_PARAMETER CVPixelBufferRef CV_NULLABLE * CV_NONNULL pixelBufferOut) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
創建一個由內存位置指定數據的像素緩衝區
``` /*! 為給定大小和像素格式創建一個像素緩衝區，其中包含由內存位置指定的數據 @function CVPixelBufferCreateWithBytes @abstract Call to create a single PixelBuffer for a given size and pixelFormatType based on a passed in piece of memory. @discussion Creates a single PixelBuffer for a given size and pixelFormatType. Not all parameters of the pixelBufferAttributes will be used here. It requires a release callback function that will be called, when the PixelBuffer gets destroyed so that the owner of the pixels can free the memory. @param width Width of the PixelBuffer in pixels @param height Height of the PixelBuffer in pixels @param pixelFormatType Pixel format indentified by its respective OSType. @param baseAddress Address of the memory storing the pixels. @param bytesPerRow Row bytes of the pixel storage memory. @param releaseCallback CVPixelBufferReleaseBytePointerCallback function that gets called when the PixelBuffer gets destroyed. @param releaseRefCon User data identifying the PixelBuffer for the release callback. @param pixelBufferAttributes A dictionary with additional attributes for a a pixel buffer. This parameter is optional. See PixelBufferAttributes for more details. @param pixelBufferOut The new pixel buffer will be returned here @result returns kCVReturnSuccess on success.
*/
CV_EXPORT CVReturn CVPixelBufferCreateWithBytes( CFAllocatorRef CV_NULLABLE allocator, size_t width, size_t height, OSType pixelFormatType, void * CV_NONNULL baseAddress, size_t bytesPerRow, CVPixelBufferReleaseBytesCallback CV_NULLABLE releaseCallback, void * CV_NULLABLE releaseRefCon, CFDictionaryRef CV_NULLABLE pixelBufferAttributes, CV_RETURNS_RETAINED_PARAMETER CVPixelBufferRef CV_NULLABLE * CV_NONNULL pixelBufferOut) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0); ```
創建一個平面格式的CVPixelBuffer
/*!
@function CVPixelBufferCreateWithPlanarBytes
@abstract Call to create a single PixelBuffer in planar format for a given size and pixelFormatType based on a passed in piece of memory.
@discussion Creates a single PixelBuffer for a given size and pixelFormatType. Not all parameters of the pixelBufferAttributes will be used here. It requires a release callback function that will be called, when the PixelBuffer gets destroyed so that the owner of the pixels can free the memory.
@param width Width of the PixelBuffer in pixels
@param height Height of the PixelBuffer in pixels
@param pixelFormatType Pixel format indentified by its respective OSType.
@param dataPtr Pass a pointer to a plane descriptor block, or NULL.
@param dataSize pass size if planes are contiguous, NULL if not.
@param numberOfPlanes Number of planes.
@param planeBaseAddress Array of base addresses for the planes.
@param planeWidth Array of plane widths.
@param planeHeight Array of plane heights.
@param planeBytesPerRow Array of plane bytesPerRow values.
@param releaseCallback CVPixelBufferReleaseBytePointerCallback function that gets called when the PixelBuffer gets destroyed.
@param releaseRefCon User data identifying the PixelBuffer for the release callback.
@param pixelBufferAttributes A dictionary with additional attributes for a a pixel buffer. This parameter is optional. See PixelBufferAttributes for more details.
@param pixelBufferOut The new pixel buffer will be returned here
@result returns kCVReturnSuccess on success.
*/
CV_EXPORT CVReturn CVPixelBufferCreateWithPlanarBytes(
CFAllocatorRef CV_NULLABLE allocator,
size_t width,
size_t height,
OSType pixelFormatType,
void * CV_NULLABLE dataPtr, // pass a pointer to a plane descriptor block, or NULL
size_t dataSize, // pass size if planes are contiguous, NULL if not
size_t numberOfPlanes,
void * CV_NULLABLE planeBaseAddress[CV_NONNULL ],
size_t planeWidth[CV_NONNULL ],
size_t planeHeight[CV_NONNULL ],
size_t planeBytesPerRow[CV_NONNULL ],
CVPixelBufferReleasePlanarBytesCallback CV_NULLABLE releaseCallback,
void * CV_NULLABLE releaseRefCon,
CFDictionaryRef CV_NULLABLE pixelBufferAttributes,
CV_RETURNS_RETAINED_PARAMETER CVPixelBufferRef CV_NULLABLE * CV_NONNULL pixelBufferOut) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
修改
在使用 CPU 訪問像素數據之前必須調用CVPixelBufferLockBaseAddress,然後再調用CVPixelBufferUnlockBaseAddress。
而使用 GPU 訪問像素數據時，就沒有這個必要了。
填充擴展的PixelBuffer
``` /*! 填充擴展的PixelBuffer @function CVPixelBufferFillExtendedPixels @abstract Fills the extended pixels of the PixelBuffer. This function replicates edge pixels to fill the entire extended region of the image. @param pixelBuffer Target PixelBuffer.
*/
CV_EXPORT CVReturn CVPixelBufferFillExtendedPixels( CVPixelBufferRef CV_NONNULL pixelBuffer ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0); ```
鎖定PixelBuffer地址
``` /*! 鎖定PixelBuffer地址 @function CVPixelBufferLockBaseAddress @abstract Description Locks the BaseAddress of the PixelBuffer to ensure that the memory is accessible. @discussion This API ensures that the CVPixelBuffer is accessible in system memory. This should only be called if the base address is going to be used and the pixel data will be accessed by the CPU. @param pixelBuffer Target PixelBuffer. @param lockFlags See CVPixelBufferLockFlags. @result kCVReturnSuccess if the lock succeeded, or error code on failure
*/
CV_EXPORT CVReturn CVPixelBufferLockBaseAddress( CVPixelBufferRef CV_NONNULL pixelBuffer, CVPixelBufferLockFlags lockFlags ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
```
解鎖PixelBuffer地址
``` /*!
@function CVPixelBufferUnlockBaseAddress
@abstract Description Unlocks the BaseAddress of the PixelBuffer.
@param pixelBuffer Target PixelBuffer.
@param unlockFlags See CVPixelBufferLockFlags.
@result kCVReturnSuccess if the unlock succeeded, or error code on failure
*/
CV_EXPORT CVReturn CVPixelBufferUnlockBaseAddress( CVPixelBufferRef CV_NONNULL pixelBuffer, CVPixelBufferLockFlags unlockFlags ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0); ```
內存管理（保留或者釋放）
這裏在開篇的時候也提到過，由於CVPixelBufferRef是C中的對象，所以不存在ARC內存管理，由開發者自己去管理引用計數，從而控制對象的生命週期；
持有
/*!
引用計數+1
@function CVPixelBufferRetain
@abstract Retains a CVPixelBuffer object
@discussion Equivalent to CFRetain, but NULL safe
@param buffer A CVPixelBuffer object that you want to retain.
@result A CVPixelBuffer object that is the same as the passed in buffer.
*/
CV_EXPORT CVPixelBufferRef CV_NULLABLE CVPixelBufferRetain( CVPixelBufferRef CV_NULLABLE texture ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
釋放
/*!
引用計數-1
@function CVPixelBufferRelease
@abstract Releases a CVPixelBuffer object
@discussion Equivalent to CFRelease, but NULL safe
@param buffer A CVPixelBuffer object that you want to release.
*/
CV_EXPORT void CVPixelBufferRelease( CV_RELEASES_ARGUMENT CVPixelBufferRef CV_NULLABLE texture ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
檢查
CVPixelBufferGetHeight 返回像素緩衝區的高度
CVPixelBufferGetWidth 返回像素緩衝區的寬度
這裏方法較多，就不一一舉例，可以查看官方文檔
像素格式類型 kCVPixelFormatType
RGB :
kCVPixelFormatType_32BGRA = 'BGRA'
kCVPixelFormatType_32BGRA = 'BGRA',
kCVPixelFormatType_32ABGR = 'ABGR',
kCVPixelFormatType_32RGBA = 'RGBA',
NV12 :
kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange = '420v',
kCVPixelFormatType_420YpCbCr8BiPlanarFullRange = '420f',
YUV420P :
kCVPixelFormatType_420YpCbCr8Planar = 'y420',
kCVPixelFormatType_{長度|序列}{顏色空間}{Planar|BiPlanar}{VideoRange|FullRange}
從這裏類型可以看出，YUV格式和RGB格式都是可以創建成CVPixelBuffer；
那麼拿到CVPixelBufferRef後，如果要顯示的話， 我們可以通過CVPixelBufferRef轉成UIImage，或者通過繪製紋理的形式展示。
想要更清楚的瞭解kCVPixelFormatType這些類型，那麼就需要我們對YUV格式和RGB格式有更深入的瞭解，所以在下一篇中，我們將對兩種格式進行詳細的説明，然後再來看kCVPixelFormatType，那樣將會加深我們對此的瞭解程度。
- LeetCode 初級算法之數組（上），看看你都學會了嗎？
- LeetCode 初級算法之鏈表，看看你都學會了嗎？
- LeetCode 初級算法之字符串（上），看看你都學會了嗎？
- 純代碼佈局，也可以一樣的簡潔
- UIStackView之一問一答
- 使用UIStackView來簡化iOS的界面佈局
- 夏天來了，iOS開發者們該如何減少App耗電？（上）
- 夏天來了，App開發者們如何看待手機發燙問題？
- 聊聊iOS中UITableView複用的那些事
- 曾經經典的微信打飛機遊戲還有人記得嗎？
- iOS 原生渲染與 Flutter 有什麼區別 （上）
- 瞭解 Mach-O文件
- CocoaPods中podsepc文件設置詳解
- iOS 原生渲染與 Flutter 有什麼區別 （下）
- 簡單瞭解 iOS CVPixelBuffer （上）
- 談談 iOS 包瘦身方案
- 播放器重構的探索之路
- 如何使用CocoaPods製作私有庫
- iOS 組件化方案