Android quick start

This guide takes you from an empty project to beautified camera frames. It assumes you already have a camera preview (CameraX, Camera2 or an RTC SDK’s capturer).

1. Add the dependency

Facevity ships as an AAR, com.knight98:facevity:0.3.2, with a transitive dependency on com.google.mediapipe:tasks-vision:0.10.32 from Google’s Maven repository. The Maven repository URL (or the AAR file) is provided with your trial licence.

// settings.gradle.kts
dependencyResolutionManagement {
    repositories {
        google()
        mavenCentral()
        maven { url = uri("<Facevity repository URL from your trial>") }
    }
}

// app/build.gradle.kts
android {
    defaultConfig { minSdk = 24 }
    // Keeps the face and segmentation models memory-mappable (recommended).
    androidResources { noCompress += listOf("task", "tflite") }
}

dependencies {
    implementation("com.knight98:facevity:0.3.2")
}

R8 / ProGuard. The AAR ships consumer rules that keep the MediaPipe, protobuf and Flogger classes MediaPipe needs at runtime. You don’t need to add anything; if you maintain your own rules, don’t strip those packages or the face model will fail to load in minified builds.

2. Permissions

The SDK itself needs no permissions. Your app still requests the camera as usual, and INTERNET is needed only for online licence activation:

<!-- AndroidManifest.xml -->
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.INTERNET" /> <!-- online licence activation only -->

Facevity never sends camera data anywhere.

3. Set up the licence

Pick one of three ways (details in Licensing):

// A. Online activation: once per install; the token is cached and refreshed about daily.
val fv = Facevity.initialize(context, FacevityOptions.Builder()
    .licenseServerUrl("https://<licence server from your trial>")
    .build())
fv.activateLicense("FV-XXXX-XXXX-XXXX-XXXX") { info ->
    Log.i("App", "Facevity licence: ${info.status}")
}

// B. Offline licence file: put license.fvl in src/main/assets/facevity/ (loaded automatically).
//    Or load it yourself:
fv.setLicenseFile(context.assets.open("my_license.fvl"))

// C. Token from your own backend
fv.setLicenseToken(tokenFromYourServer)

Without a valid licence the SDK keeps working in evaluation mode and draws a “Facevity · unlicensed” watermark (UnlicensedBehavior.WATERMARK, the default). Choose UnlicensedBehavior.PASSTHROUGH to return frames unchanged instead.

4. Create the engine and choose a look

val fv = Facevity.initialize(context)        // cheap; the face model loads on a background thread
fv.setParams(BeautyParams.NATURAL)           // presets: NATURAL, GLAM, DEFAULT, NONE
fv.setSmoothIntensity(55)                    // 0 off, 50 natural, 100 strong
fv.setBrightness(40)                         // 0..100
fv.setSkinTone(-10)                          // -100..100 (cooler .. warmer)
fv.setOverallIntensity(100)                  // scales every effect
fv.setBeautyEnabled(true)

Use one engine per camera pipeline. Setters are thread-safe and take effect on the next frame. In Java, use the builder: new BeautyParams.Builder().smooth(60).brightness(30).build().

Slider scale. Since 0.3.0 every 0..100 level means the same thing: 0 = off, 50 = natural, 100 = strong but still realistic. Wire a slider straight to a level, or map a 0..1 value with BeautyParams.fromUnit(x). If your app stored levels with 0.2, convert them once with the scale table; the presets were adjusted for you and look the same.

Backlight compensation is on by default (backlightFix = 50, automatic). It only acts when the face is clearly darker than its surround. Set it to 0 to turn it off, or up to 100 for a stronger lift.

5. Makeup and accessories

// Lipstick: a shade from BeautyColors or any RGB, then the finish
fv.setLipstick(BeautyColors.LIP_ROSE, 60)             // licence feature beauty.makeup
fv.setParams(fv.params.toBuilder().lipGloss(70).build()) // 0 matte .. 100 gloss

// Teeth: acts only while teeth show (open jaw, parted lips or a broad smile)
fv.setTeethWhitening(50)

// AR accessories (licence feature ar.props)
val props = fv.availableProps()                       // id, name, anchor
fv.setProp("round_glasses")
fv.clearProp()

Check capabilities before showing controls. fv.capabilities() reflects your licence: lipstickSupported, teethWhiteningSupported, backlightSupported, reshapeSupported and propsSupported. These effects work from the face landmarks on every device. Call prepareCapabilities() early and update your UI from a listener:

fv.prepareCapabilities()                               // starts loading the segmenter early
fv.addCapabilitiesListener { caps ->
    runOnUiThread {
        lipstickButton.isVisible = caps.lipstickSupported
        propsButton.isVisible = caps.propsSupported
    }
}

Your own accessories. A prop is a folder with prop.json, prop.png and icon.png. Download a pack and add it with fv.addPropsDirectory(dir); no code changes are needed for new props.

6. Texture path (preferred)

Use the texture path for camera previews and for RTC SDKs that hand you a GL texture. Call it on the thread that owns the GL context: your renderer thread, or the RTC SDK’s GL thread.

surfaceTexture.updateTexImage()
surfaceTexture.getTransformMatrix(matrix)

val out = fv.processTexture(
    TextureFrame(
        oesTextureId, /* isOes = */ true,
        bufferWidth, bufferHeight,
        rotation,                         // degrees clockwise to make the raw frame upright
        mirrored = isFrontCamera,
        surfaceTexture.timestamp,
        matrix,
    )
)

if (out == oesTextureId) {
    // Nothing was done (beauty off, no licence feature, …): draw the OES texture as before.
} else {
    // GL_TEXTURE_2D, identity matrix, same size as the input, raw orientation.
    // Valid until the call after next.
    drawTexture2D(out, FrameOrientation.displayMatrix(rotation, isFrontCamera))
}

Rotation. rotation always refers to the raw buffer: CameraX ImageInfo.rotationDegrees, WebRTC or Agora VideoFrame.rotation, or, for any device orientation, FrameOrientation.uprightRotation(sensorOrientation, displayRotation, front). The SDK strips the camera transform from OES matrices and works in raw orientation.

Showing the output. Rotate it clockwise by rotation and mirror it for the front camera. FrameOrientation.displayMatrix(rotation, front) is a ready-made texture matrix. If your renderer draws the camera with the plain SurfaceTexture matrix, pass outputLayout = OutputLayout.SAME_AS_INPUT and keep drawing the output with that matrix (as a sampler2D).

Mirroring is a display concern: the output is never mirrored. Mirror your local preview as usual.

GL state. The SDK saves and restores the GL state it touches (framebuffer, viewport, program, texture bindings, VAO, blending).

7. Buffer path (NV21 / NV12 / I420)

Use the buffer path when you only have CPU frames, for example from an RTC SDK’s frame observer.

val frame = BufferFrame(bytes, PixelFormat.NV21, width, height, rotation, mirrored = false, timestampNs)
val changed = fv.processBuffer(frame)   // true: bytes now hold the processed frame (in place)

• Call it from any thread. The SDK copies the input, renders on its own EGL thread and waits at most 75% of the measured frame interval (49 ms at 15 fps, 25 ms at 30 fps; bufferBudgetMs, 40 ms, until the interval is known; capped by maxBufferBudgetMs, 80 ms).
• If the GPU is still busy or the budget is missed, the frame is filled with the last retouched frame (up to 250 ms old) instead of the raw camera image, so calls do not flicker. These are counted in stats().framesRepeated. The camera is never blocked.
• On slow devices (bufferPipelining, on by default) each frame receives the previous frame’s result, one frame of latency instead of a wait for the GPU.
• Width must be a multiple of 4 and height even.
• For strided I420 planes use YuvPlanes.packI420(...) and YuvPlanes.unpackI420(...).

The buffer path costs two copies and a GPU readback per frame, so prefer textures when you can.

8. Lifecycle

// Camera switched (front <-> rear): faces fade back in within ~0.2 s
fv.resetTracking()

// On the GL thread, before the surface / context goes away
fv.onGlContextDestroyed()

// When the camera session ends: frees the buffer thread and the face model
fv.release()

• Create one engine per camera session and release() it when done.
• If a GL context dies without onGlContextDestroyed(), the next frame from a new context starts clean.
• Rapid toggles of setBeautyEnabled and parameter changes are safe.

9. Diagnostics

fv.stats() returns frames in, processed, passed through, dropped and failed; average and maximum processing time; face-detection time and rate; processing FPS; faces; detector state and the last error. Log tag: Facevity.

val s = fv.stats()
Log.d("App", "fps=${s.processFps} avg=${s.avgProcessMs} ms dropped=${s.framesDropped} faces=${fv.facesDetected}")

FacevityOptions.Builder().measureGpuTime(true) waits for the GPU each frame so the times include GPU work. Use it for profiling only.

Minimal CameraX example

The demo app shipped with your trial contains a complete GLSurfaceView + CameraX renderer: about twenty lines between updateTexImage() and drawing the result, plus provideSurface() for CameraX. Next: video-call integrations.