Encoding Settings - Tips

Overview:
This post contains a few useful tips to help encode better than a usual encode. As you people know that Hi10 Encodes are pretty good in the first place (ignore noob encoders, though), these tips will enhance the encoding with respect to quality and file size.

I'll simply explain what settings to change, not how to change them. It's pretty easy to change settings anyway, whether you use a GUI or CLI.

This post would seem quite long as there are no pics, just text.

Settings:
- CRF: Constant Rate Factor
- AQ Method
- AQ Strength
______________________________________
Rest of the settings can be kept unchanged.

CRF: The lower the value the higher would be the criteria of the quality, consequently, better quality and larger size. You can test it yourself to see what value suits your eyes.

AQ Method: Simply said, it's an algorithm to retain quality. Official built of x264.exe has 2 methods. I recommend to use method 1. You might find over the internet that method 2 is suitable for anime encoding. (Bullshit, IMO). Also, there's an unofficial built of x264.exe that has 4 Methods. If you happen to use that built, then I recommend to use method 3, not method 1. (Method 3 is better than Method 1.)

AQ Strength: It will set the strength for the selected method above. IMO, this (including aq method) is the most important setting that makes a significant difference and if you get the hold of this, you're one step close to be as good as me XD. I'll try to explain it in detail.

The usual strength almost all of the re-encoders use over the internet is (0.6 - 0.7). As far as my experience tells me, a usual BD 720p should have 1.0 strength. If the source is grainy like [Coalgirls] releases or classic BD releases that contains a lot of grains esp. in dark scenes, you can set the value of AQ to (1.35-1.5). Similarly, for BD 1080p you can tweak around a bit.

AQ Functionality and Grains:
What does the AQ Strength do? It retains the grains and, as a result, help a lot to get a very good quality at dark scenes. (Bright scenes usually have minimal grains, so it rarely acts on bright scenes). The higher the strength, the more efficiently grains would be retained and thus a better quality. Also, some (quite a majority) people think grains are just a form of noise. (True noobs, IMO).

Presence of grains help to distinct between outlines / borders, and this fact helps users to see things clearly in dark scenes. Of course, if you try to remove grains inefficiently or if you set the AQ strength such that it retains half grains and half not, it will CERTAINLY become noise. (My experience).

If you want to remove grains, you can't do it with mere settings. You'll have to apply some hi-profile filters and that's where you people will get nothing. (Not related.) So if you think that setting a low AQ value will remove the grains or make them disappear at some level, well... then all you'll get will be a shitty quality with cancerous noise.

Very Important Fact:

Generally speaking for a certain CRF value.

- Comparatively, dark scenes consume less bitrate. And look shitty. (If CRF is as low as 18, then it's fine XD)

- Bright scenes consume considerable bitrate. Bright scenes can look good even at less bitrate. That's a fact. But ironically, bright scenes automatically consume high bitrate. In other words, this is where you can extract the extra bitrate and allocate it to dark scenes. Means an efficient encode, overall.

- Grains, ONLY IF RETAINED, consume considerable amount of bitrate. So when you select a high AQ value, more grains will be there and more bitrate will be used. (Larger file size.)

- Another way to retain grains is just to use a lower CRF. As a lower CRF will raise the quality automatically the grains will be retained automatically and you do not need to set a AQ as high as 1.5.
-BUT-
- If you select a low CRF, the bright scenes will get a high criteria of quality and as I told previously bright scenes consume very high bitrate in the first place. Lowering the CRF will certainly retain grains at dark scenes, but it will also let bright scenes consume a hell lot of bitrate.

- On the other hand if you use a decent CRF and select a high AQ like 1.5, you'll get a awesome quality at dark scenes as expected, and it will not let the bright scenes get a lot of bitrate because bright scenes do not have grains generally and the CRF is not that low. (Believe me, it makes a lot of difference.)

For example, I encoded Rurouni Kenshin - New Kyoto Arc - Part 2. BD 720p. (Source: 45min. Runtime - 4.36 GiB).
- At CRF=16, AQ Mode=3, AQ Strength=1.00 (1.22 GiB - Video Only).
- At CRF=21, AQ Mode=3, AQ Strength=1.50 (743 MiB - Video Only).

Both had same quality. You won't find a difference esp because generally a re-encoded file sucks at dark scenes, but with AQ technique it remains awesome as source even at dark scenes.

That concludes another verbose but awesome tips.
______________________________
Tsk. Another secret of my encoding revealed XD

About that improved x264 with AQ Mode 3, this should be the official site:
https://astrataro.wordpress.com/category/encode/x264/

I've never used it before but, it's never too late to start.. xD

RanDom wrote:About that improved x264 with AQ Mode 3, this should be the official site:
https://astrataro.wordpress.com/category/encode/x264/

That's right. I got my hands on its built with a lot of difficulty.

RanDom wrote:

By the way zeust, x264 default settings for deblocking are something like 1:1 but I've noticed that most of the raw encodes out there use -2:-1, right now I'm using 0:0 on some 1080p sources and the overall image quality seems somewhat a bit sharper and the size is pretty much the same, I wonder is it better to keep the deblocking settings as close as possible to the source's settings?

I don't know how it works with negative values, I mean from what I've read so far the x264 deblock basically adds grain near the borders of the blocks to create an optical effect (I guess higher value = more grain) but at times I think that it also reduces the sharpness a lot, or at least it looks like that on my crappy 768p screen (and I don't have a lot of confidence in my sight too). By any chance, do you know how it works when negative values are applied or how does it affect the overall quality?

I have never tested it because it makes no significant difference. As far as I know, the more the deblocking value be selected, the less would be the grains, thus video would get a bit blurry. (Almost negligible effect, though). However, I don't know how the negative logarithm works. You can always test runs, though as I said it won't affect that notably. It never will be.

zeust wrote:I have never tested it because it makes no significant difference. As far as I know, the more the deblocking value be selected, the less would be the grains, thus video would get a bit blurry. (Almost negligible effect, though). However, I don't know how the negative logarithm works. You can always test runs, though as I said it won't affect that notably. It never will be.

Then I'll just stick with 0:0, if only my cpu were faster I could test more settings but sadly it's not the case..

Anyway, thanks for all the explanation, you gave some really useful tips, I'll be able to improve my next encodes for sure (I've been using only str 0.75 for both 720p and 1080p until now).