To FLAC or NOT: A lossless vs. MP3 comparison (UPDATED)
UPDATE (9.9.07): Corrections have been made… I did this again paying closer attention to detail. I found that the “Delete Silence” feature doesn’t really work. At least, it’s not always consistent. I ran it twice on the same file and it got different readings. It turns out that my original result files were affected by this.
With the new Apple iPod Classic available (almost) in 160GB, I’ve decided to freshen up my data with new CD rips. Yes, I still buy CDs. I like the idea of pure 16-bit/44.1 kHz audio when I want it. I began ripping CDs in 1998-1999, when Iomega offered me the opportunity to beta test their new HipZip MP3 player, which used 40MB Clik storage. I have to admit, those disks were cool, and the player was the best thing around. I ripped MP3s at 192 kbps, which back then was pretty high. Napster was just getting started and everything was usually 128 kbps. Even then, I thought those were mediocre. Since then, I’ve stuck with the 192 kbps bitrate simply for consistency. I’ve maintained all of the MP3s since then, never once losing any data. Now, I’ve been trying to decide whether to do 256 kbps AAC (which is what Apple iTunes Plus is) or to do a lossless codec. No, I wouldn’t do Apple Lossless.. I don’t want Apple owning the format of my music. I’ve been reading about FLAC. Read on for a simple, quick comparison.
Simply put…
From the FLAC homepage:
FLAC stands for Free Lossless Audio Codec, an audio format similar to MP3, but lossless, meaning that audio is compressed in FLAC without any loss in quality. This is similar to how Zip works, except with FLAC you will get much better compression because it is designed specifically for audio, and you can play back compressed FLAC files in your favorite player (or your car or home stereo, see supported devices) just like you would an MP3 file.
The compression is pretty similar to Apple Lossless. I won’t get into the differences here, but both approximate 2:1 compression ratio. The idea behind ripping everything again lossless is that I would transcode the files to something like 256 kbps when copying them to my iPod. They’ll reside on my hard drive as FLAC. This way, I can always transcode at different rates later. Or, perhaps I want the songs I listen to most at a high quality. I could probably leave albums released prior to 2000 as 192 kbps, as I don’t listen to them as much (but still do some). The final advantage is that I won’t have to rip the albums ever again, at least until a superior lossless codec is introduced. Hopefully, that won’t be for some years. The one disadvantage here is disk space. My current MP3 folder measures in around 45GB or so. Perhaps more since there’s CDs on my shelves I haven’t ripped. If I were to estimate that in FLAC, I’m looking at 225GB of lossless audio. That seems like a lot, except that storage is remarkable cheap these days. I just recently added two 250GB hard drives to my desktop for less than $90. My concept of “a lot of storage” is still a bit dated I think. I currently own approximately 1.75TB of storage spread across 9 hard drives on two computers. I think I can spare the drive space. Maybe I’ll invest in another hard drive just for FLAC.
Anyways, I wanted to be sure FLAC is worth the time and disk space. So, I did a little test. The procedure goes like this:
- Rip the same song in FLAC (Compression Level 5, which is default) and MP3
192 kbps CBR. I use Easy CD-DA Extractor for this. Update (9.9.07): I redid this using the LAME Standard Preset at the suggestion of others. Easy CD-DA Extractor’s 192 kbps preset has “Automatic” for high and low pass settings. - I found some lovely filters for Adobe Audition that enable me to open FLAC right in the program. Sweet! They’re free. Sweet!
First, I ran the “Delete Silence” feature to ensure the two rips are the same length.Update (9.9.07): It turns out that this feature doesn’t work the way you expect it to. For some reason, ripping the same track from a CD into two formats changes the amplitude of “silence”. Even opening an existing FLAC or WAV and saving it as MP3 from within Audition does some odd stuff to the beginning and ends of files. I ended up analyzing and editing to the sample level to ensure that the first peak above -30dB occurs at the same sample number in each file.
Copy the right channel of the FLAC file, and do a Mix Paste on the LEFT channel of the MP3, but choose Replace as the method. This makes the resulting file the right channel from the two rips. The screenshot to the right shows the same clip in the two formats. The display mode is spectral. you can see where the MP3 started cutting off the high frequencies. Ouch!- Then, I did a Channel Mixer, and used the vocal cut preset. This takes one channel and mixes it with the inverse of the other. The result is the difference of the two channels! I expected to hear a lot of highs, but I did NOT expect what it sounds like. Update (9.9.07): The resulting difference is actually slightly different than if you read this prior this edit. The misalignment of samples from the earlier step caused this.
- I repeated steps 4 and 5 on the left channel, and then combined the two results together again in a new stereo WAV. I’ve uploaded the WAV file for you to listen to here. I think you’ll be quite amazed. You might want to right click and choose “Save Target”. It’s about 7.5MB. Update (9.9.07): I also did this to one channel of the file that has the high and low pass set to “Automatic”. Listen to this.
So, that’s a rough, but very eye opening comparison of what MP3s are doing to original CD quality. To verify this comparison method, I did the same thing but mixed with a WAV extracted right from the CD. The result of the Channel Mixer was flat line audio… NOTHING. This tells me that this method works.
I’ll continue to buy CDs until I can digitally buy DRM-free lossless music. This little test just made up my mind really. I think I may do this test on another song as well. Let me know if you found this helpful!
EDIT: I just found this comparison of different lossless formats.
Update: Thanks for the comments. Yes, I am aware that MP3s take advantage of masking frequencies. I should have mentioned for my visitors that I am a full time pro-audio engineer, doing live audio and recording on a regular basis. So, these subtle differences are things I can tell. When I A/B’d FLAC vs. 192 kbps MP3, I noticed that the upper mids of 4 kHz - 5 kHz were still there, but a tad bit “padded” compared to the FLAC rip. There is also a slight roll-off of bass frequencies, that most people won’t ever notice without a high quality sub.
Another reason for me to stay away from these MP3s is that, when doing audio recordings, it’s a recommended practice to A/B your mix with something you listen to. What good is this if you’re A naturally sounds better than these MP3s because of poor compression?
Update (9.9.07): So, the samples being off a little bit skewed the test. I was suspicious of this from the beginning, but no one picked up on the results. I got enough crap about the “method” that I decided to do it again, paying closer attention to details. The first time around, it took me no more than 2 minutes. This time, I spent a whole hour.
I also want to make it very clear that I am not doing this to replace ABX tests. I’m not measuring psychoacoustics. I’m merely measuring differences in audio files.