a Live Graphics Framework for iOS
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
Assets
Pixels.xcodeproj
Shaders
Source
.gitignore
DOCS.md
Info.plist
LICENSE
README.md

README.md

Pixels

a Live Graphics Framework for iOS and macOS (beta)
powered by Metal

ContentPIXs: CameraPIX - ImagePIX - VideoPIX - ColorPIX - CirclePIX - RectanglePIX - PolygonPIX - GradientPIX - NoisePIX - TextPIX
EffectPIXs: LevelsPIX - BlurPIX - EdgePIX - ThresholdPIX - QuantizePIX - TransformPIX - KaleidoscopePIX - TwirlPIX - FeedbackPIX - ChannelMixPIX - ChromaKeyPIX - CornerPinPIX - HueSaturationPIX - CropPIX - FlipFlopPIX - RangePIX - SharpenPIX - SlopePIX - CrossPIX - BlendPIX - LookupPIX - DisplacePIX - RemapPIX - ReorderPIX - BlendsPIX

Examples: Camera Effects - Green Screen
Info: Coordinate Space - Blend Operators - Effect Convenience Funcs - File IO - High Bit Mode - Apps

Under development.


Note that Pixels does not have simulator support. Metal for iOS can only run on a physical device.

Docs

Classes, Delegates and Properties of:
Pixels - PIX - PIXContent - PIXEffect

Installing

Pod coming soon!

To contribute or test, follow these three steps:

  1. Drag Pixels.xcodeproj in to your Xcode project.
  2. Drag Pixels.framework under Products to your projects embedded binaries section of the general tab of your app target.
  3. Drag PixelsShaders.metallib under Products (from the nested PixelsShaders.xcodeproj) to your projects copy bundle resources under build phases of your app target.

Tutorial

High Quality (1,5 GB) - Mid Quality (0,5 GB) - Low Quality (200 MB) - Screen Lapse x4 (100 MB)
Video used: warm neon birth by BEEPLE.

Example: Camera Effects

import Pixels

let camera = CameraPIX()

let levels = LevelsPIX()
levels.inPix = camera
levels.gamma = 2.0
levels.inverted = true

let hueSaturation = HueSaturationPIX()
hueSaturation.inPix = levels
hueSaturation.hue = 0.5
hueSaturation.saturation = 0.5

let blur = BlurPIX()
blur.inPix = hueSaturation
blur.radius = 0.25

let finalPix: PIX = blur
finalPix.view.frame = view.bounds
view.addSubview(finalPix.view)

This can also be done with Effect Convenience Funcs:

let pix = CameraPIX()._gamma(2.0)._invert()._hue(0.5)._saturation(0.5)._blur(0.25)

Though this is not as efficient as a single copy of LevelsPIX and HueSaturationPIX.

Remember to add NSCameraUsageDescription to your info.plist

Example: Green Screen

import Pixels

let cityImage = ImagePIX()
cityImage.image = UIImage(named: "city")

let supermanVideo = VideoPIX()
supermanVideo.load(fileNamed: "superman", withExtension: "mov")

let supermanKeyed = ChromaKeyPIX()
supermanKeyed.inPix = supermanVideo
supermanKeyed.keyColor = .green

let blendPix = BlendPIX()
blendPix.blendingMode = .over
blendPix.inPixA = cityImage
blendPix.inPixB = supermanKeyed

let finalPix: PIX = blendPix
finalPix.view.frame = view.bounds
view.addSubview(finalPix.view)

This can also be done with Blend Operators and Effect Convenience Funcs:

let pix = ImagePIX("city") & VideoPIX("superman.mov")._chromaKey(.green)

This is a representation of the Pixel Nodes Green Screen project.

Coordinate Space

Pixels coordinate space is normalized to the vertical axis with the origin in the center.
Note that compared to native UIKit views the vertical axis is flipped.

Center: CGPoint(x: 0, y: 0)
Bottom Left: CGPoint(x: -0.5 * aspectRatio, y: -0.5)
Top Right: CGPoint(x: 0.5 * aspectRatio, y: 0.5)

Tip: PIX.Res has an .aspect property:
let aspectRatio: CGFloat = PIX.Res._1080p.aspect

Blend Operators

A quick and convenient way to blend PIXs
These are the supported PIX.BlendingMode operators:

& !& + - * ** *** % <> >< --
.over .under .add .subtract .multiply .power .gamma .difference .minimum .maximum .subtractWithAlpha
let blendPix = (CameraPIX() *** NoisePIX(res: .fullHD(.portrait))) * CirclePIX(res: .fullHD(.portrait))

The default global blend operator fill mode is .aspectFit, change it like this:
PIX.blendOperators.globalFillMode = .aspectFill

Effect Convenience Funcs

  • pix._reRes(to: ._1080p * 0.5) -> ResPIX
  • pix._reRes(by: 0.5) -> ResPIX
  • pix._brightness(0.5) -> LevelsPIX
  • pix._darkness(0.5) -> LevelsPIX
  • pix._contrast(0.5) -> LevelsPIX
  • pix._gamma(0.5) -> LevelsPIX
  • pix._invert() -> LevelsPIX
  • pix._opacity(0.5) -> LevelsPIX
  • pix._blur(0.5) -> BlurPIX
  • pix._edge() -> EdgePIX
  • pix._threshold(at: 0.5) -> ThresholdPIX
  • pix._quantize(by: 0.5) -> QuantizePIX
  • pix._position(at: CGPoint(x: 0.5, y: 0.5)) -> TransformPIX
  • pix._rotate(to: .pi) -> TransformPIX
  • pix._scale(by: 0.5) -> TransformPIX
  • pix._kaleidoscope() -> KaleidoscopePIX
  • pix._twirl(0.5) -> TwirlPIX
  • pix._swap(.red, .blue) -> ChannelMixPIX
  • pix._key(.green) -> ChromaKeyPIX
  • pix._hue(0.5) -> HueSaturationPIX
  • pix._saturation(0.5) -> HueSaturationPIX
  • pix._crop(CGRect(x: 0.5, y 0.5, width: 0.5, height: 0.5)) -> CropPIX
  • pix._flipX() -> FlipFlopPIX
  • pix._flipY() -> FlipFlopPIX
  • pix._flopLeft() -> FlipFlopPIX
  • pix._flopRight() -> FlipFlopPIX
  • pix._range(inLow: 0.0, inHigh: 0.5, outLow: 0.5, outHigh: 1.0) -> RangePIX
  • pix._range(inLow: .clear, inHigh: .gray, outLow: .gray, outHigh: .white) -> RangePIX
  • pix._sharpen() -> SharpenPIX
  • pix._slope() - > SlopePIX
  • pixA._lookup(pix: pixB, axis: .x) -> LookupPIX
  • pixA._lumaBlur(pix: pixB, radius: 0.5) -> LumaBlurPIX
  • pixA._displace(pix: pixB, distance: 0.5) -> DisplacePIX
  • pixA._remap(pix: pixB) -> RemapPIX

Keep in mind that these funcs will create new PIXs.
Be careful of overloading GPU memory if in a loop.

File IO

You can find example files here.

import Pixels

let url = Bundle.main.url(forResource: "test", withExtension: "json")!
let json = try! String(contentsOf: url)
let project = try! Pixels.main.import(json: json)

let finalPix: PIX = project.pixs.last!
finalPix.view.frame = view.bounds
view.addSubview(finalPix.view)

To export just run Pixels.main.export() once you've created your PIXs.

Note that exporting resources like image and video are not yet supported.

High Bit Mode

Some effects like DisplacePIX and SlopePIX can benefit from a higher bit depth.
The default is 8 bits. Change it like this: Pixels.main.colorBits = ._16

Enable high bit mode before you create any PIXs.

Note resources do not support higher bits yet.
There is currently there is some gamma offset with resources.

MetalPIXs

let metalPix = MetalPIX(res: ._1080p, code:
    """
    pix = float4(u, v, 0.0, 1.0);
    """
)
let metalEffectPix = MetalEffectPIX(code:
    """
    float gamma = 0.25;
    pix = pow(inPix, 1.0 / gamma);
    """
)
metalEffectPix.inPix = CameraPIX()
let metalMergerEffectPix = MetalMergerEffectPIX(code:
    """
    pix = pow(inPixA, 1.0 / inPixB);
    """
)
metalMergerEffectPix.inPixA = CameraPIX()
metalMergerEffectPix.inPixB = ImagePIX("img_name")
let metalMultiEffectPix = MetalMultiEffectPIX(code:
    """
    float4 inPixA = inTexs.sample(s, uv, 0);
    float4 inPixB = inTexs.sample(s, uv, 1);
    float4 inPixC = inTexs.sample(s, uv, 2);
    pix = inPixA + inPixB + inPixC;
    """
)
metalMultiEffectPix.inPixs = [ImagePIX("img_a"), ImagePIX("img_b"), ImagePIX("img_c")]

Uniforms:

var lumUniform = MetalUniform(name: "lum")
let metalPix = MetalPIX(res: ._1080p, code:
    """
    pix = float4(in.lum, in.lum, in.lum, 1.0);
    """,
    uniforms: [lumUniform]
)
lumUniform.value = 0.5

Apps

Pixel Nodes

a Live Graphics Node Editor for iPad
powered by Pixels


by Anton Heestand, Hexagons