Best Practices

Best Practices

What works, learned mostly by building Piko — a PikPak client for Android, Windows and macOS whose player, random-clip feed, downloads and uploads all run on this SDK. Each item says why, so it can be weighed rather than obeyed.

Accounts and sessions

Give the client a store that persists. The refresh token lives only in the session; without it every cold start is a password sign-in, with a captcha handshake in front. On Android pass a FileSessionStore under context.filesDir or your own store — there is no default there.

Pass a password supplier, not a password, when the password is in a keychain or has to be asked for. It is called only for a full sign-in.

Do not call login() before every request. Every call logs in by itself when it has to. Call it once at start-up if you want the failure early, on a screen that can show it.

Do not clear sessions yourself on errors. The client clears a dead refresh token before it falls back to the password. A caller that also clears on its own guesses — an IOException is the network, not the session.

Tell "offline" from "refused" by type. PikPakException is PikPak saying no; the engine's IO exception is no answer at all. Piko keeps a client whose login failed on the network and retries it with backoff, and signs the user out only on a PikPakException.

Identity

Persist the gcid, not the file id. A file id dies when the file is trashed, swept or rebuilt; the gcid names the content forever. Store the gcid, the size, the name and — for a transcode — the mediaId, and build a PikPakFileHandle from them later. Keep the file id beside it only to save the handle its first rebuild.

Store gcids as the SDK hands them out. Every gcid the SDK returns is upper case — listings, magnet resolution, gcidByCid, PikPakHash — so == is a content comparison. Before 1.0 the local hashes were lower case; normalise any you stored then with uppercase().

Reading files

Build handles from the detail you already have. client.fileHandle(detail, mediaId) takes the first link from it, so the first read makes no detail request. Piko lists a folder, checks each candidate's variants with one getFile, and keeps that detail for five minutes — the link inside lasts far longer — so preparing a clip costs nothing more.

One handle and one cache per file, many streams on it. Streams on one PikPakFileCache share its blocks, so a second stream for a seek, a second connection from the player, a prefetch or a download all read what the others fetched. A second cache for the same file starts from nothing. Close the cache and then the handle when the file is done; closing a stream only gives up its position.

Put a player behind a loopback HTTP proxy with one stream per request. Players want a URL; the link expires and caps at eight connections, so give them http://127.0.0.1/... and serve each HTTP request from its own stream. Overlapping requests — mpv opening a new connection before closing the old one, an interleaved MP4 reading two places — then never cancel each other. Piko's proxy is PikoMediaProxy in its shared/media/proxy.

Fetch the tail with the head. When a request starts at offset 0, prefetch the last 512 KiB at INDEX_PRIORITY: a Matroska file's Cues and a trailing MP4 moov are there, and the demuxer jumps to them before the first frame. A faststart MP4 wastes 512 KiB, which is cheaper than probing the format first.

Lower readAheadLimit for excerpts. The default 32 MiB is for a film. A 30-second clip needs about ten seconds ahead, and only once someone is watching it: every block beyond that is taken from the budget the next clip needs. Piko holds a clip's read-ahead at the minimum (one block) and the player's own buffer at two seconds until the clip has played three, then widens both to ten seconds, sized from the stream's average bitrate × 1.5.

Keep short-lived streams short. A stream opened to read a few kilobytes — a timestamp, a keyframe search — still gets the 32 MiB window and keeps asking for blocks until it is closed. Set readAheadLimit to the minimum on it first. And open it in the role of the work it serves: a helper read that gates whether a clip is ready, run as a background stream after the foreground prefetch finished, is capped at two requests and queued behind every prefetch — in Piko it made clips finish one after another at a fraction of the line.

Mark what is on screen. StreamRole.FOREGROUND for the stream the user watches, BACKGROUND for anything preloaded; switch as the user moves. The cache survives the switch.

Say when someone is waiting. Set urgent on the streams of the item on screen while a seek has not shown its frame, while the player reports buffering, and before its first frame; clear it afterwards. Not every player reports buffering on a seek (Piko's desktop backend does not), so track the seek yourself. See Playback.

Warming what comes next

Prefetch in the order things will be played, and cancel what is skipped. Ties go to the older request, so the next item finishes first instead of everything sharing the line. Cancelling a prefetch withdraws it; left standing, a skipped item's prefetch would be the oldest demand and win against the one now on screen.

Prefetch in the foreground band, not as background streams. A background file keeps two single-block requests in flight while anything plays: fine for idle warming, too slow for the next item in a feed — Piko measured five to six seconds per clip that way, and one to two seconds as a foreground prefetch, which still ranks below the playing stream's own reads.

Prefetch enough to play, not more. An item admitted to the queue after only its first frame was fetched starts playing and immediately competes with the next prefetch; one that waits for thirteen seconds of video leaves the queue thin when the user swipes fast. Piko fetches five seconds from the first keyframe, admits the clip, and fetches the rest only once it has played three (see readAheadLimit above). Keep the player's own probing inside what was fetched, too: FFmpeg probes up to 5 MB when it opens a stream, past a short head and onto a cold edge host — Piko caps it at 1 MB and one second for previews.

Warm a few, not many. Twenty-odd clips warming at once split the line and mostly go unwatched. Piko keeps eight ready ahead of the furthest point reached and prepares the detail and length of twelve (the API calls are cheap, the bytes are not); with that, cold start was 1.9 s and nearly every swipe landed on a playable clip.

Transcodes

Use a transcode for previews and feeds, the original for watching. A 720P transcode is a fraction of the original's bytes. It is MPEG-TS, so it has no index: seeking into it makes a player bisect by timestamp, which Piko measured at 4 to 18 s per start. Since any 188-byte-aligned slice of a TS plays from its next keyframe, cut the slice you need (offset from the average bitrate, rounded to 188) and serve it as a short file starting at 0 — no seek at all.

A transcode's length costs a probe. Pass streamSize when you stored it; the client remembers probes for the process.

Keeping bytes across restarts

Store the start, not the stream. A BlockStore is offered every block the cache fetches; keep only what makes the next start instant. Piko keeps the first seconds and the last 512 KiB of each clip slice, 300 MB least-recently-used, plus a small record of gcid, mediaId, length and slice position. A clip seen before then starts from disk with no API call at all — 0.25 s to playing — while the link for the rest is fetched in the background with handle.prewarm().

Download through the cache when the file is also played. A download into a DurableBlockStore shares the cache with the streams, so nothing is fetched twice — on a free account's 20 GiB of downstream a day, that is the difference between one episode and two. Ask for the container's head and tail first, then the middle, and call download again after a failure rather than retrying inside: the store remembers what it holds. downloadTo is for a file nobody plays.

Size a lease budget from the free space. On an account that leases, set client.leaseBudget = LeaseBudget(limit - usage) from getQuota() once at start-up, after sweeping the lease folder, so the sweep's leftovers are not counted as used. about trails writes by more than a lease lasts, so re-reading it while leases run measures nothing useful.

Observing transfers

Log RangeAttempts, but only the interesting ones. Pass onRangeAttempt and log attempts that failed or took more than a second to first byte, with the host. Skip Cancelled — players cancel constantly — and skip the fast Complete ones, which arrive by the dozen per second. That one line tells a slow host from a slow line from a throttled link.

Do not add host switching of your own. The reader moves off a silent host within two seconds and off one measured four times slower than a sibling, and tells the account; a wrapper that also reopens handles on slowness fights it. Log RangeAttempt.host to see where reads went.

Drive operations

Wait for listings after changing them. A listing trails a trash, move or restore by up to a second. After a mutation, trust the call's answer or poll the listing briefly; do not assert on it at once.

Poll tasks with getTask, never with the listing. The listing's default filter hides new and finished tasks, and the listing overlays the produced file's state onto the task's.

Retry an offline task by submitting its URL again. The server's RETRY does not work.

Choose instant or offline by count and size. Instant is a second and 15 % of the size against the upload allowance; offline is minutes and the full size against the offline allowance, six times cheaper per byte. For a file or two, instant; for a pack, offline and pruneOfflineOutput.

Uploads

Hash cheaply first. XunleiCid reads 60 KB; gcidByCid finds content PikPak already knows, which makes the upload instant. Hash the whole file only on a miss — a wrong gcid is accepted and poisons later instant uploads.

Persist the session for large files. startUpload returns a serializable session; save it, and continueUpload sends only missing parts, from any process, for twelve hours.

Budgets

Leave the budgets alone on a good line; lower the account budget on a bad one. On a line slower than about 0.8 MB/s × 16, more connections only make each slower and the slowest much slower: set accountConnectionBudget to roughly your bandwidth over 0.8 MB/s. Never raise it past 16 — the account then admits fewer, not more (Measurements).

Hold one BandwidthLimiter for the process and pass it to every downloadTo, so the ceiling a user sets is a ceiling on the total. Change bytesPerSecond in place; there is no need to restart downloads.

Pick a root domain in the background, not on the first request. Probe the four roots with probeDomain after start-up, compare warmRequest among the usable ones, and switch client.domain only for a clear margin; roots measured the same outside the evening peak, so switching on noise only churns.

Give the SDK its own HTTP clients. Your app-wide client probably has ContentNegotiation and HttpRequestRetry, which break the CDN and compound the retries.