A not too serious comparison (hopefully!)
The purpose of this simple page
is to show some interesting results of encoding 720p videos with VP8
and x264, using different settings and bitrate; once videos have been
encoded I've used qpsnr
to compute the PSNR and SSIM of the videos themselves. My english is
far from good, so please I beg your pardon for any mistakes that will
come in this brief comparison ;-)
What are VP8 and x264?
VP8 and x264 are two codecs used to compress (lossy compression) video. Those pieces of software will basically get video streams (a sequence of images) and digitally compress it, discarding information that should be useless and/or which our eye won't (shouldn't) notice difference if it's there or not.
Both of them belong to the specie of DCT/iDCT codecs, but their family is different: x264 is a MPEG standard (as in fully international standards body organization), while VP8 has been recently (May 2010) open sourced and source-code standardised (let's say the standard is less formal than MPEG standard) by Google.
Apparently those codecs should be the best of breed in their respective families and, most of all, at the moment there's a lot of controversy surrounding both of them.
If VP8 is and open source/royalty free/patetnt free codec (and standard after all), x264 instead is a free open source implementation of a publicly available standard which instead is filled software patents which (what a strange coincidence) lead to (a very big amount if I may say) royalty payments.
Controversy is that (clearly in my opinions) a lot of companies behind x264 want to push it as standard for web video, while others companies (and most of all non profit organizations) want instead to push VP8 because they think that all the patents that force every producer of h.264 (the standard of x264) video to pay and be sort of slave of MPEG L.A. is a huge limit to the web and personal freedoms.
Hey I've heard that Steve Jobs has recently bashed VP8...
Yes that's true. In his view VP8 can't be hardware implemented. But apparently Apple owns a lot of patents in h.264 standard!
Could Steve Job's opinion be, should I say it, slightly interested, a bit partisan?
Given the MPEG L.A. royalties payment scheme, would Apple (as any other company which is part of h.264 patents pool holder group) benefit?
Yes...a lot. To quote mr. Rupert Murdoch (owner of Sky) "monopolies are dreadful things until you own one" .
Should we trust mr. Jobs' opinion on this matter?
What are you going to analyze?
I'm going to encode and analyze two scenes of the movie Avatar and a small HD scene salled Shark (ironically this scene comes from Apple HD video website).
The first Avatar scene one is less dynamic (when we see Terrans move their spaceships to start the attack against the Big Tree (of life?), till when they drop their mechs on the ground) while the second scene is much more dynamic (when the protagonist destroys the main spaceship and the colonel drop on land with his mech).
The Shark scene instead is colorful with a lot of details and movements, but even different lighting conditions.
Avatar
Pictures from scene 1
What are VP8 and x264?
VP8 and x264 are two codecs used to compress (lossy compression) video. Those pieces of software will basically get video streams (a sequence of images) and digitally compress it, discarding information that should be useless and/or which our eye won't (shouldn't) notice difference if it's there or not.
Both of them belong to the specie of DCT/iDCT codecs, but their family is different: x264 is a MPEG standard (as in fully international standards body organization), while VP8 has been recently (May 2010) open sourced and source-code standardised (let's say the standard is less formal than MPEG standard) by Google.
Apparently those codecs should be the best of breed in their respective families and, most of all, at the moment there's a lot of controversy surrounding both of them.
If VP8 is and open source/royalty free/patetnt free codec (and standard after all), x264 instead is a free open source implementation of a publicly available standard which instead is filled software patents which (what a strange coincidence) lead to (a very big amount if I may say) royalty payments.
Controversy is that (clearly in my opinions) a lot of companies behind x264 want to push it as standard for web video, while others companies (and most of all non profit organizations) want instead to push VP8 because they think that all the patents that force every producer of h.264 (the standard of x264) video to pay and be sort of slave of MPEG L.A. is a huge limit to the web and personal freedoms.
Hey I've heard that Steve Jobs has recently bashed VP8...
Yes that's true. In his view VP8 can't be hardware implemented. But apparently Apple owns a lot of patents in h.264 standard!
Could Steve Job's opinion be, should I say it, slightly interested, a bit partisan?
Given the MPEG L.A. royalties payment scheme, would Apple (as any other company which is part of h.264 patents pool holder group) benefit?
Yes...a lot. To quote mr. Rupert Murdoch (owner of Sky) "monopolies are dreadful things until you own one" .
Should we trust mr. Jobs' opinion on this matter?
What are you going to analyze?
I'm going to encode and analyze two scenes of the movie Avatar and a small HD scene salled Shark (ironically this scene comes from Apple HD video website).
The first Avatar scene one is less dynamic (when we see Terrans move their spaceships to start the attack against the Big Tree (of life?), till when they drop their mechs on the ground) while the second scene is much more dynamic (when the protagonist destroys the main spaceship and the colonel drop on land with his mech).
The Shark scene instead is colorful with a lot of details and movements, but even different lighting conditions.
Avatar
Pictures from scene 1
Pictures from scene 2
I decided to encode x264 with Handbrake v. svn3392 and VP8 with ffmpeg 0.6; for each scene 6 files have been encoded:
For each encoding I've always executed 2 passes; I don't know how to set avanced VP8 encoding with ffmpeg, so apart the bitrate (and the passes flag) I've set no other options when encoding VP8 videos.
Following are file size and real bitrate (audio has been eliminated from the clips):
Below there's the PSNR for scene 1:
and SSIM for the same scene:
I was really surprised to notice that VP8 at 2 MBps (2k) performs better than x264, and for both measures (both PSNR and SSIM).
Let's have a look at the differences from the most different frame (note that below images are zoomed!) for 4 MBps (4k) files (around frame 415):
From left ro right: Reference, x264 4k HP, VP8 4k
- x264 2k (normal, default settings in Handbrake)
- x264 2k High Profile (default settings in High Profile Handbrake)
- VP8 2k (default settings in ffmpeg)
- x264 4k (normal, default settings in Handbrake)
- x264 4k High Profile (default settings in High Profile Handbrake)
- VP8 4k (default settings in ffmpeg)
For each encoding I've always executed 2 passes; I don't know how to set avanced VP8 encoding with ffmpeg, so apart the bitrate (and the passes flag) I've set no other options when encoding VP8 videos.
Following are file size and real bitrate (audio has been eliminated from the clips):
| Type | File Name | Size (kbytes) | Bitrate (Kbps) |
| x264 2k HP | avatar1_2k_hp.m4v | 21368426 | 2035 |
| x264 2k | avatar1_2k.m4v | 21316971 | 2030 |
| VP8 2k | avatar1_2k.webm | 27444531 | 2614 |
| x264 4k HP | avatar1_4k_hp.m4v | 42626863 | 4060 |
| x264 4k | avatar1_4k.m4v | 42597593 | 4057 |
| VP8 4k | avatar1_4k.webm | 42188123 | 4018 |
| x264 2k HP | avatar2_2k_HP.m4v | 16393251 | 1561 |
| x264 2k | avatar2_2k.m4v | 16341316 | 1556 |
| VP8 2k | avatar2_2k.webm | 24969953 | 2378 |
| x264 4k HP | avatar2_4k_HP.m4v | 32700701 | 3114 |
| x264 4k | avatar2_4k.m4v | 32660315 | 3111 |
| VP8 4k | avatar2_4k.webm | 32471261 | 3093 |
Below there's the PSNR for scene 1:
Let's have a look at the differences from the most different frame (note that below images are zoomed!) for 4 MBps (4k) files (around frame 415):
From left ro right: Reference, x264 4k HP, VP8 4k
As you can easily spot, if the foreground details are well preserved,
the background veins on the rock walls and foliage are less defined and
a lot blurred in VP8.
That said, I have to be honest: while watching the movie, the details where attention is focused are always of very good quality, and plese keep in mind this frame is one where VP8 performs the worst compared to x264.
Now, the second scene PSNR:
That said, I have to be honest: while watching the movie, the details where attention is focused are always of very good quality, and plese keep in mind this frame is one where VP8 performs the worst compared to x264.
Now, the second scene PSNR:
and SSIM:
These charts are extremely surprising: VP8 is performing very good, and VP8 2 MBps (2k) is slightly behind x264 4 MBps (4k)
Let's have a look at the differences from the most different frames for 4 MBps (4k) files (around frame 1310):
From left ro right: Reference, x264 4k HP, VP8 4k
Let's have a look at the differences from the most different frames for 4 MBps (4k) files (around frame 1310):
From left ro right: Reference, x264 4k HP, VP8 4k
In the above pictures we can see that all the differences between x264 4 MBps (4k) High Profile and VP8 are smaller.
And, in my opinion, most of all on the 3rd row, the VP8 images details are sharper than x264. The fact that both PSNR and SSIM say that VP8 is of very good quality, can be seen in the third row: the details and the shapes look much more clear to me in VP8 video.
Shark
Below the pictures of Shark scene:
This time I've used ivfenc as well to encode VP8.
I did specify the following options:
./ivfenc in.yuv out.ivf \
--i420 -w 1280 -h 720 -p 2 -t 4 \
--best --target-bitrate=4000 --end-usage=0 \
--auto-alt-ref=1 --timebase=1000/25000 -v \
--minsection-pct=5 --maxsection-pct=800 \
--lag-in-frames=16 --kf-min-dist=0 --kf-max-dist=360 \
--token-parts=2 --static-thresh=0 --drop-frame=0 \
--min-q=0 --max-q=60
And indeed in the following charts you can find VP8 4k --best as the video encoded with ivfenc (which paradoxally was even faster than ffmpeg), while VP8 4k is as above VP8 encoded with ffmpeg.
PSNR:
SSIM:
The Shark scene shows us that when using a VP8 encoder with options to tune output for more quality, VP8 4k --best competes straight with normal x264 4k, and sometimes it's at level of x264 4k HP (High Profile).
And considering that according to other people this encoder is still raw, badly implemented with poor code, the above charts are definitely not bad, if you know what I mean ;-)
Hey, I've heard about XviD and Theora video codecs, what about them?
Just to show you a comparisno between the new codecs (VP8, x264) and old ones (XivD, Theora), below are PSNR and SSIM of scene one with 4 MBps rate.
I hope the charts speak for themselves:
Now, if for the PSNR chart we can see that x264 and VP8 are mainly
better than XviD and Theora (but not so extreme difference) in the SSIM
chart we can grasp (and measure) that both Theora and XviD lag behind
quite a lot.
And this is not unexpected; when watching the scenes encoded with XviD and/or Theora, you feel you'd need a lot more bitrate to achieve same quality as in VP8 and/or x264.
And yes, for HD content Theora is not good. Simple.
What can I say?
Well, given that in many occasions PSNR and SSIM aren't far away between x264 and VP8, VP8 is a fair competitor for x264.
Indeed, currently (Jun 2010) the badly implemented VP8 encoder (--best option in ivfenc) offers same quality as x264 normal. Which is pretty good considering the fact that there's room for improvement.
Would I use VP8 now to save my HD videos? Not really, because ffmpeg 0.6 is extremely slow (6+ FPS on my 4 cores AMD x4 965 B.E.), and doesn't use all of my 4 cores, while Handbrake is faster ( 60+ FPS) and uses 4 corse 100% each. And even if ivfenc is faster than ffmpeg there is still the question about playing videos on other devices which at the moment don't support VP8.
So, from my viewpoint, VP8 is not usable in production right now, but once it'll be better implemented (and maybe the encoder will manage to better save background details) VP8, given the patent free nature of this format, could become the real standard.
Final words
At the moment x264 is having the edge because is a very good encoder, stable since some years and really implements the h.264 standard inside out. In a nice way indeed.
VP8 is like rough gold, it needs to be primed, in the code and perhaps some details algorithm search as well.
But give VP8 some time, I feel that this one could be a real liberation (and evolution if I may say) from a heavily patented codec.
If you want to have a word, please contact me at ema at _fast_web_net.it (remove the _ otherwise email address won't work).
And, in my opinion, most of all on the 3rd row, the VP8 images details are sharper than x264. The fact that both PSNR and SSIM say that VP8 is of very good quality, can be seen in the third row: the details and the shapes look much more clear to me in VP8 video.
Shark
Below the pictures of Shark scene:
I did specify the following options:
./ivfenc in.yuv out.ivf \
--i420 -w 1280 -h 720 -p 2 -t 4 \
--best --target-bitrate=4000 --end-usage=0 \
--auto-alt-ref=1 --timebase=1000/25000 -v \
--minsection-pct=5 --maxsection-pct=800 \
--lag-in-frames=16 --kf-min-dist=0 --kf-max-dist=360 \
--token-parts=2 --static-thresh=0 --drop-frame=0 \
--min-q=0 --max-q=60
And indeed in the following charts you can find VP8 4k --best as the video encoded with ivfenc (which paradoxally was even faster than ffmpeg), while VP8 4k is as above VP8 encoded with ffmpeg.
PSNR:
And considering that according to other people this encoder is still raw, badly implemented with poor code, the above charts are definitely not bad, if you know what I mean ;-)
Hey, I've heard about XviD and Theora video codecs, what about them?
Just to show you a comparisno between the new codecs (VP8, x264) and old ones (XivD, Theora), below are PSNR and SSIM of scene one with 4 MBps rate.
I hope the charts speak for themselves:
And this is not unexpected; when watching the scenes encoded with XviD and/or Theora, you feel you'd need a lot more bitrate to achieve same quality as in VP8 and/or x264.
And yes, for HD content Theora is not good. Simple.
What can I say?
Well, given that in many occasions PSNR and SSIM aren't far away between x264 and VP8, VP8 is a fair competitor for x264.
Indeed, currently (Jun 2010) the badly implemented VP8 encoder (--best option in ivfenc) offers same quality as x264 normal. Which is pretty good considering the fact that there's room for improvement.
Would I use VP8 now to save my HD videos? Not really, because ffmpeg 0.6 is extremely slow (6+ FPS on my 4 cores AMD x4 965 B.E.), and doesn't use all of my 4 cores, while Handbrake is faster ( 60+ FPS) and uses 4 corse 100% each. And even if ivfenc is faster than ffmpeg there is still the question about playing videos on other devices which at the moment don't support VP8.
So, from my viewpoint, VP8 is not usable in production right now, but once it'll be better implemented (and maybe the encoder will manage to better save background details) VP8, given the patent free nature of this format, could become the real standard.
- Compression speed: in this case the real winner is x264; VP8 code needs much more improvement to be comptetitive (at least using ffmpeg, ivfenc needs improvements as well).
- Patents and royalties: well clearly VP8 is the winner here, without chance for x264 to ever come near. Unless MPEG L.A. decides to make it become a royalty free standard (which I definitely doubt as seen as they make money out of patents).
- Quality: they are sort of tied now, x264 is slightly better in more static scenes while VP8 is better for highly dynamic ones. But when proper encoders will be available for VP8 probably we'll see it fill (at least some) the gap in less dynamic scenes.
- Hardware accelleration: this was one of the major publicity stun that was lacking for other video formats than x264 (h.264 to be specific). But now a lot of companies have announced hardware support for VP8.
Final words
At the moment x264 is having the edge because is a very good encoder, stable since some years and really implements the h.264 standard inside out. In a nice way indeed.
VP8 is like rough gold, it needs to be primed, in the code and perhaps some details algorithm search as well.
But give VP8 some time, I feel that this one could be a real liberation (and evolution if I may say) from a heavily patented codec.
If you want to have a word, please contact me at ema at _fast_web_net.it (remove the _ otherwise email address won't work).
Disclaimer
This comparison has been done just for the purpose of it. The author is not affiliated with any cited organization.
Avatar is © 2009 Twentieth Century Fox Film Corporation and Dune Entertainment LLC. All Rights Reserved, © 2010 Twentieth Century Fox Home Entertainment LLC. All Rights Reserved.
Shark is Copyright © 2010 Apple Inc. All rights reserved.
The above images have been used for citing purpose; in case the owner of the above images wants to remove them, please contact the above email address.
This comparison has been done just for the purpose of it. The author is not affiliated with any cited organization.
Avatar is © 2009 Twentieth Century Fox Film Corporation and Dune Entertainment LLC. All Rights Reserved, © 2010 Twentieth Century Fox Home Entertainment LLC. All Rights Reserved.
Shark is Copyright © 2010 Apple Inc. All rights reserved.
The above images have been used for citing purpose; in case the owner of the above images wants to remove them, please contact the above email address.