简单了解 iOS CVPixelBuffer （上）
theme: smartblue
「这是我参与2022首次更文挑战的第8天，活动详情查看：2022首次更文挑战」。
前言：
在iOS中，我们会常常看到 CVPixelBufferRef 这个类型，最常见到的场景是在Camera 采集的时候，返回的数据中有一个CMSampleBufferRef，而每个CMSampleBufferRef则包含一个 CVPixelBufferRef，在视频硬解码的返回数据里也是一个 CVPixelBufferRef（里面包含了所有的压缩的图片信息）。在了解了CVPixelBufferRef之后，我们将能够掌握并且运用CVPixelBufferRef的使用；
本篇我们主要熟悉下CVPixelBuffer的使用；
CVPixelBuffer 简介
CVPixelBuffer：核心视频像素缓冲区是在主存储器中保存像素的图像缓冲区。生成帧、压缩或解压缩视频或使用 Core Image 的应用程序都可以使用 CVPixelBuffer。
CVPixelBufferRef：是像素缓冲区类型，对CVPixelBuffer对象的引用。像素缓冲区类型基于图像缓冲区类型；像素缓冲器实现了图像缓冲器的存储器存储。
```
/*!
@typedef CVPixelBufferRef
@abstract Based on the image buffer type. The pixel buffer implements the memory storage for an image buffer.
*/ typedef CVImageBufferRef CVPixelBufferRef; ``` 由于CVPixelBufferRef是C中的对象，所以没有ARC内存管理，必须由开发者自己去管理引用计数，控制对象生命周期；
首先，我们先来了解有关CVPixelBufferRef的基础方法，例如：CVPixelBufferRef的创建、持有、释放；
创建
创建的方法有四种，前三种常用到： - CVPixelBufferCreate() - CVPixelBufferCreateWithBytes() - CVPixelBufferCreateWithPlanarBytes() - CVPixelBufferCreateWithIOSurface()
普通的创建方法
/*!
为给定大小和像素格式创建单个像素缓冲区
@function CVPixelBufferCreate
@abstract Call to create a single PixelBuffer for a given size and pixelFormatType.
@discussion Creates a single PixelBuffer for a given size and pixelFormatType. It allocates the necessary memory based on the pixel dimensions, pixelFormatType and extended pixels described in the pixelBufferAttributes. Not all parameters of the pixelBufferAttributes will be used here.
@param width Width of the PixelBuffer in pixels.
@param height Height of the PixelBuffer in pixels.
@param pixelFormatType Pixel format indentified by its respective OSType.
@param pixelBufferAttributes A dictionary with additional attributes for a pixel buffer. This parameter is optional. See BufferAttributeKeys for more details.
@param pixelBufferOut The new pixel buffer will be returned here
@result returns kCVReturnSuccess on success.
*/
CV_EXPORT CVReturn CVPixelBufferCreate(
CFAllocatorRef CV_NULLABLE allocator,
size_t width,
size_t height,
OSType pixelFormatType,
CFDictionaryRef CV_NULLABLE pixelBufferAttributes,
CV_RETURNS_RETAINED_PARAMETER CVPixelBufferRef CV_NULLABLE * CV_NONNULL pixelBufferOut) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
创建一个由内存位置指定数据的像素缓冲区
``` /*! 为给定大小和像素格式创建一个像素缓冲区，其中包含由内存位置指定的数据 @function CVPixelBufferCreateWithBytes @abstract Call to create a single PixelBuffer for a given size and pixelFormatType based on a passed in piece of memory. @discussion Creates a single PixelBuffer for a given size and pixelFormatType. Not all parameters of the pixelBufferAttributes will be used here. It requires a release callback function that will be called, when the PixelBuffer gets destroyed so that the owner of the pixels can free the memory. @param width Width of the PixelBuffer in pixels @param height Height of the PixelBuffer in pixels @param pixelFormatType Pixel format indentified by its respective OSType. @param baseAddress Address of the memory storing the pixels. @param bytesPerRow Row bytes of the pixel storage memory. @param releaseCallback CVPixelBufferReleaseBytePointerCallback function that gets called when the PixelBuffer gets destroyed. @param releaseRefCon User data identifying the PixelBuffer for the release callback. @param pixelBufferAttributes A dictionary with additional attributes for a a pixel buffer. This parameter is optional. See PixelBufferAttributes for more details. @param pixelBufferOut The new pixel buffer will be returned here @result returns kCVReturnSuccess on success.
*/
CV_EXPORT CVReturn CVPixelBufferCreateWithBytes( CFAllocatorRef CV_NULLABLE allocator, size_t width, size_t height, OSType pixelFormatType, void * CV_NONNULL baseAddress, size_t bytesPerRow, CVPixelBufferReleaseBytesCallback CV_NULLABLE releaseCallback, void * CV_NULLABLE releaseRefCon, CFDictionaryRef CV_NULLABLE pixelBufferAttributes, CV_RETURNS_RETAINED_PARAMETER CVPixelBufferRef CV_NULLABLE * CV_NONNULL pixelBufferOut) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0); ```
创建一个平面格式的CVPixelBuffer
/*!
@function CVPixelBufferCreateWithPlanarBytes
@abstract Call to create a single PixelBuffer in planar format for a given size and pixelFormatType based on a passed in piece of memory.
@discussion Creates a single PixelBuffer for a given size and pixelFormatType. Not all parameters of the pixelBufferAttributes will be used here. It requires a release callback function that will be called, when the PixelBuffer gets destroyed so that the owner of the pixels can free the memory.
@param width Width of the PixelBuffer in pixels
@param height Height of the PixelBuffer in pixels
@param pixelFormatType Pixel format indentified by its respective OSType.
@param dataPtr Pass a pointer to a plane descriptor block, or NULL.
@param dataSize pass size if planes are contiguous, NULL if not.
@param numberOfPlanes Number of planes.
@param planeBaseAddress Array of base addresses for the planes.
@param planeWidth Array of plane widths.
@param planeHeight Array of plane heights.
@param planeBytesPerRow Array of plane bytesPerRow values.
@param releaseCallback CVPixelBufferReleaseBytePointerCallback function that gets called when the PixelBuffer gets destroyed.
@param releaseRefCon User data identifying the PixelBuffer for the release callback.
@param pixelBufferAttributes A dictionary with additional attributes for a a pixel buffer. This parameter is optional. See PixelBufferAttributes for more details.
@param pixelBufferOut The new pixel buffer will be returned here
@result returns kCVReturnSuccess on success.
*/
CV_EXPORT CVReturn CVPixelBufferCreateWithPlanarBytes(
CFAllocatorRef CV_NULLABLE allocator,
size_t width,
size_t height,
OSType pixelFormatType,
void * CV_NULLABLE dataPtr, // pass a pointer to a plane descriptor block, or NULL
size_t dataSize, // pass size if planes are contiguous, NULL if not
size_t numberOfPlanes,
void * CV_NULLABLE planeBaseAddress[CV_NONNULL ],
size_t planeWidth[CV_NONNULL ],
size_t planeHeight[CV_NONNULL ],
size_t planeBytesPerRow[CV_NONNULL ],
CVPixelBufferReleasePlanarBytesCallback CV_NULLABLE releaseCallback,
void * CV_NULLABLE releaseRefCon,
CFDictionaryRef CV_NULLABLE pixelBufferAttributes,
CV_RETURNS_RETAINED_PARAMETER CVPixelBufferRef CV_NULLABLE * CV_NONNULL pixelBufferOut) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
修改
在使用 CPU 访问像素数据之前必须调用CVPixelBufferLockBaseAddress,然后再调用CVPixelBufferUnlockBaseAddress。
而使用 GPU 访问像素数据时，就没有这个必要了。
填充扩展的PixelBuffer
``` /*! 填充扩展的PixelBuffer @function CVPixelBufferFillExtendedPixels @abstract Fills the extended pixels of the PixelBuffer. This function replicates edge pixels to fill the entire extended region of the image. @param pixelBuffer Target PixelBuffer.
*/
CV_EXPORT CVReturn CVPixelBufferFillExtendedPixels( CVPixelBufferRef CV_NONNULL pixelBuffer ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0); ```
锁定PixelBuffer地址
``` /*! 锁定PixelBuffer地址 @function CVPixelBufferLockBaseAddress @abstract Description Locks the BaseAddress of the PixelBuffer to ensure that the memory is accessible. @discussion This API ensures that the CVPixelBuffer is accessible in system memory. This should only be called if the base address is going to be used and the pixel data will be accessed by the CPU. @param pixelBuffer Target PixelBuffer. @param lockFlags See CVPixelBufferLockFlags. @result kCVReturnSuccess if the lock succeeded, or error code on failure
*/
CV_EXPORT CVReturn CVPixelBufferLockBaseAddress( CVPixelBufferRef CV_NONNULL pixelBuffer, CVPixelBufferLockFlags lockFlags ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
```
解锁PixelBuffer地址
``` /*!
@function CVPixelBufferUnlockBaseAddress
@abstract Description Unlocks the BaseAddress of the PixelBuffer.
@param pixelBuffer Target PixelBuffer.
@param unlockFlags See CVPixelBufferLockFlags.
@result kCVReturnSuccess if the unlock succeeded, or error code on failure
*/
CV_EXPORT CVReturn CVPixelBufferUnlockBaseAddress( CVPixelBufferRef CV_NONNULL pixelBuffer, CVPixelBufferLockFlags unlockFlags ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0); ```
内存管理（保留或者释放）
这里在开篇的时候也提到过，由于CVPixelBufferRef是C中的对象，所以不存在ARC内存管理，由开发者自己去管理引用计数，从而控制对象的生命周期；
持有
/*!
引用计数+1
@function CVPixelBufferRetain
@abstract Retains a CVPixelBuffer object
@discussion Equivalent to CFRetain, but NULL safe
@param buffer A CVPixelBuffer object that you want to retain.
@result A CVPixelBuffer object that is the same as the passed in buffer.
*/
CV_EXPORT CVPixelBufferRef CV_NULLABLE CVPixelBufferRetain( CVPixelBufferRef CV_NULLABLE texture ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
释放
/*!
引用计数-1
@function CVPixelBufferRelease
@abstract Releases a CVPixelBuffer object
@discussion Equivalent to CFRelease, but NULL safe
@param buffer A CVPixelBuffer object that you want to release.
*/
CV_EXPORT void CVPixelBufferRelease( CV_RELEASES_ARGUMENT CVPixelBufferRef CV_NULLABLE texture ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
检查
CVPixelBufferGetHeight 返回像素缓冲区的高度
CVPixelBufferGetWidth 返回像素缓冲区的宽度
这里方法较多，就不一一举例，可以查看官方文档
像素格式类型 kCVPixelFormatType
RGB :
kCVPixelFormatType_32BGRA = 'BGRA'
kCVPixelFormatType_32BGRA = 'BGRA',
kCVPixelFormatType_32ABGR = 'ABGR',
kCVPixelFormatType_32RGBA = 'RGBA',
NV12 :
kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange = '420v',
kCVPixelFormatType_420YpCbCr8BiPlanarFullRange = '420f',
YUV420P :
kCVPixelFormatType_420YpCbCr8Planar = 'y420',
kCVPixelFormatType_{长度|序列}{颜色空间}{Planar|BiPlanar}{VideoRange|FullRange}
从这里类型可以看出，YUV格式和RGB格式都是可以创建成CVPixelBuffer；
那么拿到CVPixelBufferRef后，如果要显示的话， 我们可以通过CVPixelBufferRef转成UIImage，或者通过绘制纹理的形式展示。
想要更清楚的了解kCVPixelFormatType这些类型，那么就需要我们对YUV格式和RGB格式有更深入的了解，所以在下一篇中，我们将对两种格式进行详细的说明，然后再来看kCVPixelFormatType，那样将会加深我们对此的了解程度。
- LeetCode 初级算法之数组（上），看看你都学会了吗？
- LeetCode 初级算法之链表，看看你都学会了吗？
- LeetCode 初级算法之字符串（上），看看你都学会了吗？
- 纯代码布局，也可以一样的简洁
- UIStackView之一问一答
- 使用UIStackView来简化iOS的界面布局
- 夏天来了，iOS开发者们该如何减少App耗电？（上）
- 夏天来了，App开发者们如何看待手机发烫问题？
- 聊聊iOS中UITableView复用的那些事
- 曾经经典的微信打飞机游戏还有人记得吗？
- iOS 原生渲染与 Flutter 有什么区别 （上）
- 了解 Mach-O文件
- CocoaPods中podsepc文件设置详解
- iOS 原生渲染与 Flutter 有什么区别 （下）
- 简单了解 iOS CVPixelBuffer （上）
- 谈谈 iOS 包瘦身方案
- 播放器重构的探索之路
- 如何使用CocoaPods制作私有库
- iOS 组件化方案