Actually it WAS said upstream that lossless is the same file, bit-for-bit, that’s why I mentioned this.
When you remove data from a file, that data is lost EVEN IF it is being accurately redescribed elsewhere. NO ONE is arguing whether or not the resulting video is visually identical to the original, but data is being removed and then redescribed in short-hand. How is that not data loss? Honestly, I don’t really care to debate the semantics of whether removing data from a file is “loss” if there is no negative impact, all that matters (as far as this thread is concerned) is what conditions should be met when choosing one codec over another.
Lossless codecs rely on being able to toss/redefine data in order to keep file sizes small which is the ONLY value in using a lossless codec instead of uncompressed. Lossless codecs can perform very well but ONLY if appropriate source material is being used.
Aye, it’s semantics indeed. But the definition is clear, check the Wikipedia article. Or any paper on lossless compression algorithms.
No, that’s exactly what’s false. If you re-describe it elsewhere in the same file, the data is, by definition, not lost. How can data be lost when it’s still there??
In the example of “1000 pixels”, the data of the 999 instances of “black pixels” is not lost at all, it’s right there in the phrase “1000 pixels”. Reducing it to “1 black pixel” would be data loss, because then the information about the 999 pixels is entirely gone.
If your definition of “loss” was true, then the very term “lossless compression” would be a contradiction in terms, because a “lossless” file wouldn’t be compressed at all. (because you say if the file is smaller, something is lost. Which goes against the very idea of compression like ZIP)
[li]COMPRESSED means smaller file size[/li][li]LOSSY means when you unpack the file, you don’t get the same result as to begin with (JPG, MPG, DIVX etc)[/li][li]LOSSLESS means when you unpack the file, you get the same data you started with (ZIP, RAR etc.)[/li][/ul]I quote Wikipedia again:
Lossless data compression is a class of data compression algorithms that allows the exact original data to be reconstructed from the compressed data
I can see where you’re coming from. You’re saying the losslessly compressed file does not contain the same data as the original uncompressed file. And you’re right!
What WE are saying is that the losslessly compressed file DOES contain the same data as the original uncompressed file. And we’re right too!
I suppose this is really a matter of perception; depending on how you want to view the compressed file.
But coming to the crux of the matter, a properly written lossless codec performs EXACTLY the same as an uncompressed file. This cannot be disputed.
The zip/rar analogy holds as well because the behaviour is the same if the appropriate front-end is provided. This can be best described when you consider Windows XP’s ZIP folders. All the decompression/recompression happens behind-the-scenes when you edit the zip folder, but the contents (while still being compressed) will match the original data exactly
This is where we’re coming from by saying that there is NO loss because while there is the lossless wrapper around the original file, the required data is decompressed/decoded on the fly, edited and then recompressed and put back in the lossless wrapper without any interaction required.
Not to keep arguing but this is wrong. Lossless codecs are not some magic suitcase that can stuff 1,000 lbs. of s…t into a 500lb sack. There is no lossless “wrapper” and there is no “original file” once it has been compressed to lossless. The video information fed into a lossless codec is permanently, irrevocably altered and redefined to a smaller, compacted state. On decompression the codec CAN translate this information into an expanded state so that the video player understands it, but your original file no longer exists, it has been redefined. In the case of zip files, the decompressor can translate the compacted file back into a 1:1 duplicate, but NOT because the original file was “stored” inside the codec - it merely expands the compacted definitions back into full definitions and creates a new file based on this.
As for this whole argument about whether some form of data tossing/loss is happening, have either of you even considered the notion that the Animation codec, for example, has a quality slider ranging from "best" on down? Think about it. The codec in its "best" state delivers a 1:1 quality because it only tosses/redefines what data it can without affecting quality. At the lower settings, it begins tossing more data, resulting in quality loss. As such, my point stands that the codec is selectively removing data. In its best form it is just limiting the degree to which it does this.
For the purposes of this thread, all that matters is for people to understand that lossless codecs rely on certain types of video data being fed to them to work well. People often complain that Animation codec files are huge, and my response was that this particular codec performs very well when the appropriate subject matter is fed into it. Give it a video file and the file will be remarkably huge. Give it the type of subject matter it was designed to efficiently compress (such as blocky graphics) and the file will be very small while still looking great. What is at issue here is when to use which codec, depending on the circumstances. Upstream someone claimed that lossless files do not throw away any data, and by my definition they DO, that is how the file size is lowered - by removing entries and replacing them with more compact definitions. When people understand how the codec works, they will know when to use it vs. uncompressed and that was the only point of my comments upstream.
I suppose we can end this by agreeing on the following - lossless set to "best" quality is visually identical to the original file, although data has been removed and consolidated in the interest of reducing the file size. Agreed?
Hi again. I’m sorry, I like continuing this. I respect you very much Beenies, and I certainly don’t want to continue till we rip each other’s heads off. I might stop soon. But for now, I’ll have another go. Sorry.
Let me begin again by saying that if your definition of lossless was true, there would be no such thing as “lossless compression”, because “compression” by your definition means “tossing” and “losing” data. So compression, by your definition, can’t be lossless… explain that?
Whenever possible, that’s exactly what it is, yes. JUST LIKE ZIP. It’s a means of discribing THE SAME DATA in a shorter way, so no loss is happening.
You can also ZIP your video files to a smaller file size. That’s also lossless compression, the exact same principle. The difference between ZIP and, for instance, Quicktime Animation at 100% is that the Animation algorithm is specialized for image data, so you will get better results than with ZIP.
And with that, I’m beginning to understand your misunderstanding… I have a feeling I know what you’re going to say to the ZIP example: You’re going to say that it can’t be compared, because with ZIP you unpack the original EXACT file, right? Whereas with animation codec you never return to the original file?
Well, that’s because the original IMAGE DATA is not a file. IMAGE DATA is just a series of pixels, no matter if you put it in a gif, png, targa, in your mind, or on post-it notes. THE PIXEL DATA is the original master. The uncompressed (huge) file is alraedy just a container, it only REPRESENTS the master (proof: you can convert it to another uncompressed image format, which gives you completely different bit sequence, but the exact same image data - that’s also lossless, even though all the bits are different). Put that file through lossless compression, and you get a much smaller file, which is also just a container. Play that file, or load it into compositing or editing, and you restore THE ORIGINAL PIXEL DATA on your screen, compositing app or whereever. The original file? No, you don’t restore that, but who cares, your source data is pixels, not a file. If you can restore your original pixels, you have had NO LOSS. If you had loss, there would be “damage” to your image (compression artifacts).
It works like this:
Master data -> Compression algorithm -> Compressed file -> Uncompressed algorithm -> Master data
With ZIP and program files, that would be:
QUAKE.EXE -> ZIP algorithm -> QUAKE.ZIP file (smaller) -> UNZIP algorithm -> QUAKE.EXE (perfect match to original file)
And, with video data:
ORIGINAL PIXEL DATA (in any container!) -> Lossless codec -> VIDEO FILE (smaller than if uncompressed) -> Decoding (playback) -> ORIGINAL PIXEL DATA
Of course, as with the ZIP example, lossless compression only offers a limited effect, depending on the data. As you said, a live video signal will be much less compressed that a blank image or one with large blocks of solid color. And that is paramount in the understanding of lossless compression. It’s called entropy - a certain amount of data can only be compressed so much without loss. That’s why lossless compression doesn’t have a constant bitrate, it’s inevitably impossible.
Yes, the animation codec is only lossless at 100%.
OK, reading through the whole thing again it strikes me that we are actually very close to total agreement. The only thing we disagree about is, as I was in on in my last post, whether or not the SOURCE DATA for video compression is a file or abstract pixel data.
My case is, obviously, that it’s pixel data. To prove that I present two claims:[ol]
[li](as in the above post) You can convert one uncompressed master file (or file sequence) to another uncompressed format, and still have a master. Ie, even if the bit sequence is totally different (100% data loss, according to you), you still have a master just as perfect as the original.[/li]So, surely, the original file’s bits doesn’t matter, what matters is pixel data and nothing else.
[li]You don’t even need a master file to create a lossless compressed file, you can create it directly from your 3D, compositing or editing package.[/li][/ol]I rest my case. It’s been fun.
Fair enough. In closing here’s my final point - lossless codecs can reproduce images identical to the original, but behind the scenes the image data has been consolidated and redundant information is removed to reduce the file size. I call this “tossing data” which I still find an accurate description of what is happening given that redundant data is literally removed and replaced with compacted descriptions. Unnecessary data being removed and the resulting file being smaller is “tossing data,” I see no other way to describe it.
In the case of video compression, the original video data is lost for good in the sense that only a compacted description of the video exists within the file. Without the codec that understands how to expand this data, the file cannot be viewed because it is not a complete video file. Data HAS been lost in favor of a new description that utterly requires the decompressor to interpret it.
In the context of how this discussion started, I only intended to point out that video footage files will not benefit from lossless compression anywhere near as much as solid graphics. This was backed up almost verbatim in the Wikipedia thread you linked to, and is all that really matters here.
Anyway, I'm sure we all learned something new in these exchanges, certainly it was fun debating this issue but I doubt future visitors to this sticky will agree! ;) Peace
Jonas was right. Lossless codecs compress the data, but don’t throw away anything. The advantage is they save space, the disadvantage is they have to compress/uncompress every time they’re accessed. This happens so quickly, though, you almost don’t notice. Go ahead and render a completely uncompressed frame, then render the same frame in .png. Open them both up, with one in difference mode on top of the other. They will precisely cancel each other out, because they are precisely identical. In fact, you can apply any kind of wacky post effect you want on them, and they will respond in identical ways.
Hey there. I´ve being reading a looot of threads lately about codecs and compression and stuff like that, cause im finally finishing an animated short i´ve been doing and want to know whats the best posible way to render it,and i just want to say that the discussion between beenyweenies and jussing has been one of the most inspiring and informative things i´ve ever read on cg talk,seriously, i understood so many stuff that has troubled me for so much time that its no funny,and the way they behave and debated each other arguments back and forth without resorting to sarcasm and insults was imo admirable.
Thanks to both of you for taking the time to write those explanations,really.
For virtually all situations, rendering to the Quicktime Animation codec is advised. Uncompressed Quicktime or AVI is also okay except that the files will be needlessly huge compared to Animation codec, with the exact same visual result. The reasons for this are discussed in great detail upstream if you want more info.
You could also render to a PNG frame sequence if you frequently suffer from failed renders or bad frames (also discussed upstream).
I was wondering if you could help me to solve a tricky mistery, at least for me, I have a none compressed file in HD(1080p) I would like to distribute to a couple of friends and maybe other people, but I don’t know what kind of codec or tools I need to make it playable for almost averyone who has mac or pc without the lossing of so much quality, Maybe I going to burn the video as a file, and if necessary, add the right tools to be seen.
What is what you suggest that I should use? I heard something about H.264, MPG4, Procoder, Rhozet Carbon Coder, Cinema Craft Encoder,among others, but it is harder to take a good decision, maybe burning it with 2 or 3 formats.