Av1an

Av1an is a video encoding framework. It can increase your encoding speed and improve cpu utilization by running multiple encoder processes in parallel. Target quality, VMAF plotting, and more, available to take advantage for video encoding.

For help with av1an, please reach out to us on Discord or file a GitHub issue

General

-i <INPUT>
		Input file to encode

		Can be a video or vapoursynth (.py, .vpy) script.

-o <OUTPUT_FILE>
		Video output file

	--temp <TEMP>
		Temporary directory to use

		If not specified, the temporary directory name is a hash of the input file name.

-q, --quiet
		Disable printing progress to the terminal

	--verbose
		Print extra progress info and stats to terminal

-l, --log-file <LOG_FILE>
		Log file location [default: <temp dir>/log.log]

	--log-level <LOG_LEVEL>
		Set log level for log file (does not affect command-line log level)

		error: Designates very serious errors.

		warn: Designates hazardous situations.

		info: Designates useful information.

		debug: Designates lower priority information.

		trace: Designates very low priority, often extremely verbose, information. Includes
		rav1e scenechange decision info.

		[default: DEBUG]
		[possible values: error, warn, info, debug, trace]

-r, --resume
		Resume previous session from temporary directory

-k, --keep
		Do not delete the temporary folder after encoding has finished

	--force
		Do not check if the encoder arguments specified by -v/--video-params are valid

-y
		Overwrite output file, without confirmation

-n
		Never overwrite output file, without confirmation

	--max-tries <MAX_TRIES>
		Maximum number of chunk restarts for an encode

		[default: 3]

-w, --workers <WORKERS>
		Number of workers to spawn [0 = automatic]

		[default: 0]

	--set-thread-affinity <SET_THREAD_AFFINITY>
		Pin each worker to a specific set of threads of this size (disabled by default)

		This is currently only supported on Linux and Windows, and does nothing on unsupported
		platforms. Leaving this option unspecified allows the OS to schedule all processes
		spawned.

	--scaler <SCALER>
		Scaler used for scene detection (if --sc-downscale-height XXXX is used) and VMAF
        calculation

		Valid scalers are based on the scalers available in ffmpeg, including lanczos[1-9] with [1-9]
        defining the width of the lanczos scaler.

-h, --help
		Print help information

-V, --version
		Print version information

Scene detection

-s, --scenes <SCENES>
		File location for scenes

	--split-method <SPLIT_METHOD>
		Method used to determine chunk boundaries

		"av-scenechange" uses an algorithm to analyze which frames of the video are the start
		of new scenes, while "none" disables scene detection entirely (and only relies on
		-x/--extra-split to add extra scenecuts).

		[default: av-scenechange]
		[possible values: av-scenechange, none]

	--sc-method <SC_METHOD>
		Scene detection algorithm to use for av-scenechange

		Standard: Most accurate, still reasonably fast. Uses a cost-based algorithm to determine
		keyframes.

		Fast: Very fast, but less accurate. Determines keyframes based on the raw difference
		between pixels.

		[default: standard]
		[possible values: standard, fast]

	--sc-only
		Run the scene detection only before exiting

		Requires a scene file with --scenes.

	--sc-pix-format <SC_PIX_FORMAT>
		Perform scene detection with this pixel format

	--sc-downscale-height <SC_DOWNSCALE_HEIGHT>
		Optional downscaling for scene detection

		Specify as the desired maximum height to scale to (e.g. "720" to downscale to 720p
		— this will leave lower resolution content untouched). Downscaling improves scene
		detection speed but lowers accuracy, especially when scaling to very low resolutions.

		By default, no downscaling is performed.

-x, --extra-split <EXTRA_SPLIT>
		Maximum scene length, in frames

		When a scenecut is found whose distance to the previous scenecut is greater than the
		value specified by this option, one or more extra splits (scenecuts) are added. Set this
		option to 0 to disable adding extra splits.

    --extra-split-sec <EXTRA_SPLIT_SEC>
		Maximum scene length, in seconds

        If both frames and seconds are specified, then the number of frames will take priority.

		[default: 10]

	--min-scene-len <MIN_SCENE_LEN>
		Minimum number of frames for a scenecut

		[default: 24]

    --ignore-frame-mismatch
        Ignore any detected mismatch between scene frame count and encoder frame count

Encoding

-e, --encoder <ENCODER>
		Video encoder to use

		[default: aom]
		[possible values: aom, rav1e, vpx, svt-av1, x264, x265]

-v, --video-params <VIDEO_PARAMS>
		Parameters for video encoder

		These parameters are for the encoder binary directly, so the ffmpeg syntax cannot be
		used. For example, CRF is specified in ffmpeg via "-crf <crf>", but the x264 binary
		takes this value with double dashes, as in "--crf <crf>". See the --help output of each
		encoder for a list of valid options.

-p, --passes <PASSES>
		Number of encoder passes

		Since aom and vpx benefit from two-pass mode even with constant quality mode (unlike
		other encoders in which two-pass mode is used for more accurate VBR rate control), two-
		pass mode is used by default for these encoders.

		When using aom or vpx with RT mode (--rt), one-pass mode is always used regardless
		of the value specified by this flag (as RT mode in aom and vpx only supports one-pass
		encoding).

		[possible values: 1, 2]

-a, --audio-params <AUDIO_PARAMS>
		Audio encoding parameters (ffmpeg syntax)

		If not specified, "-c:a copy" is used.

		Do not use ffmpeg's -map syntax with this option. Instead, use the colon syntax with
		each parameter you specify.

		Subtitles are always copied by default.

		Example to encode all audio tracks with libopus at 128k:

		-a="-c:a libopus -b:a 128k"

		Example to encode the first audio track with libopus at 128k, and the second audio track
		with aac at 24k, where only the second track is downmixed to a single channel:

		-a="-c:a:0 libopus -b:a:0 128k -c:a:1 aac -ac:a:1 1 -b:a:1 24k"

-f, --ffmpeg <FFMPEG_FILTER_ARGS>
		FFmpeg filter options

-m, --chunk-method <CHUNK_METHOD>
		Method used for piping exact ranges of frames to the encoder

		Methods that require an external vapoursynth plugin:

		lsmash - Generally the best and most accurate method. Does not require intermediate
		files. Errors generally only occur if the input file itself is broken (for example,
		if the video bitstream is invalid in some way, video players usually try to recover
		from the errors as much as possible even if it results in visible artifacts, while
		lsmash will instead throw an error). Requires the lsmashsource vapoursynth plugin to
		be installed.

		ffms2 - Accurate and does not require intermediate files. Can sometimes have bizarre
		bugs that are not present in lsmash (that can cause artifacts in the piped output).
		Slightly faster than lsmash for y4m input. Requires the ffms2 vapoursynth plugin to
		be installed.

        dgdecnv - Very fast, but only decodes AVC, HEVC, MPEG-2, and VC1. Does not require intermediate files.
	    Requires dgindexnv to be present in system path, NVIDIA GPU that support CUDA video decoding, and dgdecnv vapoursynth plugin
        to be installed.

	    bestsource - Very slow but accurate. Linearly decodes input files. Does not require intermediate files, requires the BestSource vapoursynth plugin
        to be installed.

		Methods that only require ffmpeg:

		hybrid - Uses a combination of segment and select. Usually accurate but requires
		intermediate files (which can be large). Avoids decoding irrelevant frames by seeking to
		the first keyframe before the requested frame and decoding only a (usually very small)
		number of irrelevant frames until relevant frames are decoded and piped to the encoder.

		select - Extremely slow, but accurate. Does not require intermediate files. Decodes
		from the first frame to the requested frame, without skipping irrelevant frames (causing
		quadratic decoding complexity).

		segment - Create chunks based on keyframes in the source. Not frame exact, as it can
		only split on keyframes in the source. Requires intermediate files (which can be large).

		Default: lsmash (if available), otherwise ffms2 (if available), otherwise DGDecNV (if available), otherwise bestsource (if available), otherwise hybrid.

		[possible values: segment, select, ffms2, lsmash, dgdecnv, bestsource, hybrid]

	--chunk-order <CHUNK_ORDER>
		The order in which av1an will encode chunks

		Available methods:

		long-to-short - The longest chunks will be encoded first. This method results in the
		smallest amount of time with idle cores, as the encode will not be waiting on a very
		long chunk to finish at the end of the encode after all other chunks have finished.

		short-to-long - The shortest chunks will be encoded first.

		sequential - The chunks will be encoded in the order they appear in the video.

		random - The chunks will be encoded in a random order. This will provide a more accurate
		estimated filesize sooner in the encode.

		[default: long-to-short]
		[possible values: long-to-short, short-to-long, sequential, random]

	--photon-noise <PHOTON_NOISE>
		Generates a photon noise table and applies it using grain synthesis [strength: 0-64]
		(disabled by default)

		Photon noise tables are more visually pleasing than the film grain generated by aomenc,
		and provide a consistent level of grain regardless of the level of grain in the source.
		Strength values correlate to ISO values, e.g. 1 = ISO 100, and 64 = ISO 6400. This
		option currently only supports aomenc and rav1e.

		An encoder's grain synthesis will still work without using this option, by specifying
		the correct parameter to the encoder. However, the two should not be used together, and
		specifying this option will disable the encoder's internal grain synthesis.

-c, --concat <CONCAT>
		Determines method used for concatenating encoded chunks and audio into output file

		ffmpeg - Uses ffmpeg for concatenation. Unfortunately, ffmpeg sometimes produces files
		with partially broken audio seeking, so mkvmerge should generally be preferred if
		available. ffmpeg concatenation also produces broken files with the --enable-keyframe-
		filtering=2 option in aomenc, so it is disabled if that option is used. However, ffmpeg
		can mux into formats other than matroska (.mkv), such as WebM. To output WebM, use
		a .webm extension in the output file.

		mkvmerge - Generally the best concatenation method (as it does not have either of the
		aforementioned issues that ffmpeg has), but can only produce matroska (.mkv) files.
		Requires mkvmerge to be installed.

		ivf - Experimental concatenation method implemented in av1an itself to concatenate to an
		ivf file (which only supports VP8, VP9, and AV1, and does not support audio).

		[default: ffmpeg]
		[possible values: ffmpeg, mkvmerge, ivf]

	--pix-format <PIX_FORMAT>
		FFmpeg pixel format

		[default: yuv420p10le]

	--zones <ZONES>
		Path to a file specifying zones within the video with differing encoder settings.

		The zones file should include one zone per line,
		with each arg within a zone space-separated.
		No quotes or escaping are needed around the encoder args,
		as these are assumed to be the last argument.

		The zone args on each line should be in this order:

		```
		start_frame end_frame encoder reset(opt) video_params
		```

		For example:

		```
		136 169 aom --photon-noise 4 --cq-level=32
		169 1330 rav1e reset -s 3 -q 42
		```

		Example line 1 will encode frames 136-168 using aomenc
		with the argument `--cq-level=32` and enable av1an's `--photon-noise` option.
		Note that the end frame number is *exclusive*.
		The start and end frame will both be forced to be scenecuts.
		Additional scene detection will still be applied within the zones.
		`-1` can be used to refer to the last frame in the video.

		The default behavior as shown on line 1 is to preserve
		any options passed to `--video-params` or `--photon-noise`
		in av1an, and append or overwrite the additional zone settings.

		Example line 2 will encode frames 169-1329 using rav1e.
		The `reset` keyword instructs av1an to ignore any settings
		which affect the encoder, and use only the parameters from this zone.

		For segments where no zone is specified,
		the settings passed to av1an itself will be used.

		The video params which may be specified include any parameters
		that are allowed by the encoder, as well as the following av1an options:

		- `-x`/`--extra-split`
		- `--min-scene-len`
		- `--passes`
		- `--photon-noise` (aomenc/rav1e only)

VMAF

	--vmaf
		Plot an SVG of the VMAF for the encode

		This option is independent of --target-quality, i.e. it can be used with or without it.
		The SVG plot is created in the same directory as the output file.

	--vmaf-path <VMAF_PATH>
		Path to VMAF model (used by --vmaf and --target-quality)

		If not specified, ffmpeg's default is used.

	--vmaf-res <VMAF_RES>
		Resolution used for VMAF calculation

        If set to inputres, the output video will be scaled to the resolution of the input video.

		[default: 1920x1080]

	--vmaf-threads <VMAF_THREADS>
		Number of threads to use for VMAF calculation

	--vmaf-filter <VMAF_FILTER>
		Filter applied to source at VMAF calcualation

		This option should be specified if the source is cropped, for example.

Target Quality

	--target-quality <TARGET_QUALITY>
		Target a VMAF score for encoding (disabled by default)

		For each chunk, target quality uses an algorithm to find the quantizer/crf needed to
		achieve a certain VMAF score. Target quality mode is much slower than normal encoding,
		but can improve the consistency of quality in some cases.

		The VMAF score range is 0-100 (where 0 is the worst quality, and 100 is the best).
		Floating-point values are allowed.

	--probes <PROBES>
		Maximum number of probes allowed for target quality

		[default: 4]

	--probing-rate <PROBING_RATE>
		Framerate for probes, 1 - original

		[default: 1]

	--probe-slow
		Use encoding settings for probes specified by --video-params rather than faster, less
		accurate settings

		Note that this always performs encoding in one-pass mode, regardless of --passes.

	--min-q <MIN_Q>
		Lower bound for target quality Q-search early exit

		If min_q is tested and the probe's VMAF score is lower than target_quality, the Q-search
		early exits and min_q is used for the chunk.

		If not specified, the default value is used (chosen per encoder).

	--max-q <MAX_Q>
		Upper bound for target quality Q-search early exit

		If max_q is tested and the probe's VMAF score is higher than target_quality, the Q-
		search early exits and max_q is used for the chunk.

		If not specified, the default value is used (chosen per encoder).

Target Quality

Table of Contents

1. Description
2. Requirements
3. Commands
4. Example of usage

Description

Target Quality has a really simple goal, instead of guessing what the CQ/CRF value to choose for desired level of video quality we set quality level we want, quality goal is set in value of VMAF score we want to achieve and let the algorithm find CRF/CQ value that will result in that score, for each segment. Which simultaneously achieve 3 things, if compared to usual, single value CRF/CQ encode.

• Ensuring better level of visual consistency than default rate controls
• Give enough bitrate to complex segments to match target quality.
• Save bitrate by not overspending on scenes, which saves bit rate.

Requirements

• Working VMAF setup

  • FFMPEG with libvmaf (It's de facto default configuration from 2020)
  • Installed or manually selected VMAF models
    • by default it grabs /usr/share/model/vmaf_v0.6.1.pkl

• Supported encoder

  • aomenc
  • rav1e
  • svt-av1
  • x265
  • x264
  • vpx

• Quality/Constant Rate control (Target quality change crf/cq value for each segment). Which means that encoders must be in mode that use CRF/CQ and have those options specified ( --crf 30, --cq-level=30) those values get replaced for each segment

Commands

• --target-quality FLOAT - enables target quality with default settings for that encoder, targets FLOAT value

• --probes INT - Overrides maximum amount of probes to make for each segment (Default 4)

• --min_q INT --max_q INT - Overrides default CRF/CQ boundaries for search

Example of usage

av1an -i file --target-quality 90 - Will run aomenc with default settings of target-quality

av1an -i file --target-quality 95 --vmaf_path "vmaf_v.0.6.3.pkl" --probes 6 - With specified path to vmaf model and 6 probes per segment

Scaling

By default vmaf calculation is done at 1920x1080 with default model. VMAF calculation resolution can be changed

--vmaf-res 3840x2160

Cropping with target quality

Filter with crop should be supplied for both ffmpeg options and vmaf filter

--ffmpeg "-vf crop=3840:1900:0:0" --vmaf-filter "crop=3840:1900:0:0" --vmaf-res "3840x1900"

or cropping and resizing could be done with vapoursynth script -i 4k_crop.vpy --vmaf-res "3840x1600" --target-quality 90 -o test.mkv

Compiling Av1an

You can natively build Av1an on Linux and Windows. Cross-compilation is not supported.

Compiling on Linux

To compile Av1an from source, you need the following dependencies:

• Rust (version 1.70.0 or higher)
• NASM
• clang/LLVM
• FFmpeg
• VapourSynth

On Arch Linux, you can install these dependencies by running

pacman -S --needed rust nasm clang ffmpeg vapoursynth

Installation instructions on other distros will vary.

After installing the dependencies, you need to clone the repository and start the build process:

git clone https://github.com/master-of-zen/Av1an && cd Av1an
cargo build --release

The resulting binary will be the file ./target/release/av1an.

Compiling on Windows

If you just want a current build of Av1an that is newer than the last official release, you can find a pre-built binary of the current master branch at https://github.com/master-of-zen/Av1an/releases/tag/latest.

If you want to build the binary yourself, you will need the following dependencies:

• Microsoft Visual C++ Build Tools - this is a dependency for Rust
• The Rust toolchain
• Python (version 3.8 or 3.10) - this is a dependency for VapourSynth. Recommended to install for all users
• VapourSynth
• NASM
• FFmpeg (thanks to gyan for providing these builds)

Extract the file ffmpeg-5.0.1-full_build-shared.7z to a directory, then create a new environment variable called FFMPEG_DIR (this can be done with with the "Edit environment variables for your account" function available in the control panel), and set it to the directory that you extracted the original file to (for example, set it to C:\Users\Username\Downloads\ffmpeg-5.0.1-full_build-shared).

Then, either clone the repository by running

git clone https://github.com/master-of-zen/Av1an

Or download and extract the source code manually.

Open a command prompt or PowerShell window inside the cloned repository/extracted ZIP folder and run the command cargo build --release. If this command executes successfully with no errors, av1an.exe will be in the folder target\release.

To use av1an.exe, copy all the .dll files from ffmpeg-5.0.1-full_build-shared\bin to the same directory as av1an.exe, and ensure that ffmpeg.exe is in a folder accessible via the PATH environment variable.

Docker

The docker image is frequently updated and includes all supported encoders and all optional components. It is based on Arch Linux and provides recent versions of encoders and libraries.

The image provides three types of tags that you can use:

• masterofzen/av1an:master for the latest commit from master
• masterofzen/av1an:sha-####### for a specific git commit (short hash)

Examples

The following examples assume the file you want to encode is in your current working directory.

Linux

docker run --privileged -v "$(pwd):/videos" --user $(id -u):$(id -g) -it --rm masterofzen/av1an:master -i S01E01.mkv {options}

Windows

docker run --privileged -v "${PWD}:/videos" -it --rm masterofzen/av1an:master -i S01E01.mkv {options}

The image can also be manually built by running

docker build -t "av1an" .

in the root directory of this repository. The dependencies will automatically be installed into the image, no manual installations necessary.

To specify a different directory to use you would replace $(pwd) with the directory

docker run --privileged -v "/c/Users/masterofzen/Videos":/videos --user $(id -u):$(id -g) -it --rm masterofzen/av1an:master -i S01E01.mkv {options}

The --user flag is required on linux to avoid permission issues with the docker container not being able to write to the location, if you get permission issues ensure your user has access to the folder that you are using to encode.

Aomenc

GIT: AOM

Table of Contents

• Important command line options
• Example settings and explanation

Important command line options

Example settings and explanation

Constant quality

--end-usage=q --cq-level=30 --cpu-used=4 --threads=64

It is recommended to set it the rate control --end-usage=q to get the highest quality rate control method possible. Only use CQ if you are planning to stream with a maximum bitrate, and CBR/VBR for livestreaming.

It is recommended to the --cq-level in range 20-40 depending on your source.

Target Bitrate

--end-usage=vbr --target-bitrate=1000 --cpu-used=4 --threads=64

To get good efficiency with VBR, it is strongly recommended to use aomenc in 2-pass mode(which is the default in av1an).

Tiles(tile columns and rows)

... --tile-columns=2 --tile-rows=1 ...

If tiles required to improve playback on old devices or high framerates, set --tile-columns=2 and --tile-rows=1 at 1080p. For higher resolution/higher framerate encoding, set it to --tile-columns=3 and --tile-rows=2.

CPU preset

... --cpu-used=6 ...

--cpu-used=6 requires ~30% less bitrate than x265 for same quality, gains by preset increments are relatively smaller than x264/x265.

Bit-depth

... --bit-depth=10 ... .. -b 8.. .. -b 10.. .. -b 12.. It is recommended to set it to 10-bit even for 8-bit content for higher efficiency (due to better compression efficiency) and less banding.

Lag-in-frames

... --lag-in-frames=48 ...

More is better, up to a limit of 48 (default is 35).

Content tune

... --tune-content=default ...

default: tuned for most content. screen: tuned for screen recordings

Flags used for most native 10-bit HDR content

... --color-primaries=bt2020 --transfer-characteristics=smpte2084 --matrix-coefficients=bt2020ncl ...

Example command line for good quality

--end-usage=q --cq-level=22 --cpu-used=4 --threads=8 --bit-depth=10 --lag-in-frames=35 --enable-fwd-kf=1 --enable-qm=1 --enable-chroma-deltaq=1 --quant-b-adapt=1 --mv-cost-upd-freq=2

Example command line for fast speed

--end-usage=q --cq-level=22 --cpu-used=6 --threads=64 --tile-columns=2 --tile-rows=1 --bit-depth=8

SVT-AV1

This will be a quick guide for setting options svt-av1 when using with Av1an. For more complete documentation, you should read the official documentation:

• SVT-AV1 Docs - GitLab

Make sure your svt-av1 encoder is up-to-date.

Rate control

--rc

The --rc option selects what rate control strategy you want to use.

• --rc 0 - Constant rate factor
• --rc 1 - Variable bit rate
• --rc 2 - Constant bit rate

Constant rate factor

--rc 0 --crf

Constant rate factor, a.k.a. constant quality. This is the most common way of determining video quality. You set --crf to be a number between 1-63, and the encoder will work out a bit-rate to keep constant quality. The lower the number, the less compression, the higher it is, the more compression will be used. Anything lower than 20 is considered "hi-fi", 30+ is for mini encodes.

Example:

av1an ... -v " --rc 0 --crf 24 --preset 4 --input-depth 10 --tune 0" ...

Variable bit-rate

--rc 1 --tbr

Variable bit-rate. Requires you to set target bit-rate --tbr.

Example:

av1an ... --passes 2 -v " --rc 1 --tbr 2000 --preset 4 --input-depth 10 --tune 0" ...

Preset

--preset

If RC controls the compression strategy, then the preset determines what optimisation features get enabled (at the cost of encode time). Realistically the range for preset is 0-13, with 0 being the most optimized (and slow). 0-3 is only for the latest and fastest system, or the most patient of people. 4-6 is more common among enthusiasts. You should go as low as you can bare, 4 is a good starting place for most.

Example:

... --preset 4 ...

Tune

--tune

• --tune 0 - VQ
• --tune 1 - PSNR

VQ is subjective quality, while PSNR is an objective measurement. Most will recommend using VQ, it seems to make the image sharper as well. Default is PSNR.

Film grain

--film-grain --film-grain-denoise

Synthesize film grain! --film-grain can be set to 1-50 (default is 0 - off), the higher the number the stronger the effect. The default behaviour is to denoise, then add the synthesized noise. But this can remove fine detail, so it is recommended to disable the denoise stage by setting --film-grain-denoise to 0.

You can also disable the encoders denoise, it is possible to use denoising filters provided by ffmpeg and vaporsynth. These will give better results than any of the encoders internal denoise filter. ffmpeg's hqdn3d, and nlmeans filters should be a good starting point. Common vaporsynth filters include: BM3D, DFTTest, SMDegrain, and KNLMeansCL.

Example:

... --film-grain 10 --film-grain-denoise 0 ...

Input depth

--input-depth 10

You can choose a bit-depth of 8bit or 10bit. It is almost always recommended to use 10bit, even if the source is only 8bit. It is by far the most optimized, and can even fix problems inherent with 8bit.

You might not want to use 10bit if fast encode/decode is more important than video quality.

Lookahead and key frames

--lookahead --keyint

These optimisations come at the cost of increased RAM usage during encode, and worse performance when seeking during playback. But they are worth the compromise.

Lookahead will affect how many future frames the encoder will look forward to. Increases effectiveness of several optimizations. Max is 120.

Modern video files include key frames which are Intra coded pictures, and Inter frames, which store only information changed since the previously encoded reference frames. Setting keyint=24 will give you 1 second in a 24fps video. It is recommended to have 10 seconds of GOP, for a 24fps video this will now be keyint=240.

24fps:

--lookahead 120 --keyint 240

30fps:

--lookahead 120 --keyint 300

rav1e

GIT: rav1e

Table of Contents

• Important command line options
• Example settings and explanation

Important command line options