Encoder settings explained: x264, NVENC, AMF and Apple VideoToolbox

Pick the encoder that uses the hardware you can spare

  • x264 (software) — encodes on the CPU. Best quality per bit at a given bitrate, but it competes with the game for CPU time. Sensible on a dedicated streaming PC or a CPU with plenty of spare cores.
  • NVIDIA NVENC — a dedicated block on GeForce and RTX cards that is separate from the shader cores, so it barely affects game performance. Recent generations also expose HEVC and AV1 encoding for platforms that accept them.
  • AMD AMF — the equivalent hardware encoder on Radeon cards.
  • Intel Quick Sync — hardware encoding on Intel integrated graphics, useful when the discrete GPU is fully occupied by the game.
  • Apple VideoToolbox — hardware encoding on macOS, the practical default on Apple silicon.

Rate control

CBR holds a constant bitrate and is what live platforms expect, because their ingest and player stacks are built around a predictable stream. CQP and CRF target a constant quality level and let the bitrate float, which is what you want for local recordings: quiet scenes cost little, busy scenes get the bits they need. VBR sits between the two and is mainly useful for file output with an average target.

Bitrate

Bitrate is the single biggest quality lever for live streaming, but the ceiling is set by the destination platform and by your real upload capacity, not by OBS. Check the current ingest guidance for the service you stream to, then stay comfortably below your measured upload speed so other traffic on the connection cannot push you into dropped frames.

Keyframe interval

Live platforms generally expect a fixed keyframe interval of 2 seconds, because segmented delivery cuts the stream on keyframes. Leave it at 2 for streaming. For local recording, a longer interval or the automatic setting produces smaller files.

Preset

The preset trades encoding effort for quality. On x264, slower presets look better at the same bitrate and cost more CPU; veryfast is the usual starting point for live use. On NVENC, the quality presets (P5 to P7) with a slower tuning give a visible improvement over the default at almost no cost to game frame rate, because the work happens on the dedicated encoder block.

Profile and look-ahead

Use the high profile for H.264. Look-ahead and psycho-visual tuning on NVENC improve quality on fast-moving content but add a small amount of latency and, in some driver versions, extra VRAM use. Turn them on, record a two-minute test of your busiest content, and keep them only if the encoder lag statistic stays at zero.

Measure, do not guess

Open View → Stats while testing. If frames missed due to rendering lag climbs, the GPU is saturated. If skipped frames due to encoding lag climbs, the encoder is overloaded. If dropped frames (network) climbs, the problem is the connection to the ingest server, not the encoder.