Personal Media Players…. Are they good enough?
Audio Practitioner needs a high quality signal source. Is personal media players good enough? Pat Brown performed several tests and provided the data from the tests.
Introduction
One of the staples of the audio technician is a high quality signal source. Speech and music tracks are needed, along with some pure tones and various types of noise. The personal media player (PMP for the rest of this article) has radically changed the way that many people listen to music. Their widespread use has created a competitive marketplace where prices have fallen while capabilities have increased. The Ipod™ and devices like it have reached commodity status. They are small, cheap and readily available. This makes them a logical consideration as a signal source for audio system evaluation.
But how good are these little devices? Some herald their sound quality, while others scoff at their consumer audio origins. Technically they are somewhat shrouded in mystery. I failed at finding any serious technical data on any of the PMPs that I tested for this article. Some relevant questions include:
- 1. What is the maximum output level?
- 2. Should I turn it all the way up?
- 3. What is the minimum load impedance?
- 4. How much distortion is present?
PMPs are nearly always judged subjectively, but something can be very inaccurate and still sound good. I decided to run a battery of tests on some popular models to provide some actual technical specifications for our readers.
As I set out to devise some tests, the monthly Audio Precision e-news letter showed up in my Inbox. The engineers at AP have devised some automated tests for PMPs for use with their analyzers. This provided a great nucleus of tests, which I augmented with a few others to round out the project.
I presented the concept to the Syn-Aud-Con listserv for some input from the Heard. I got many suggestions for PMP models as well as tests to run on them. It quickly became apparent that this was going to expand beyond PMPs and spill over into sound cards, USB and Firewire audio interfaces and a few more. It also became apparent that I could not test every device on the market. The model line-up I used is my best attempt at including the requested units or similar devices, subject to availability. Not surprisingly this mushroomed into a big project, but one that I think provides some tangible and useful information.
Every PMP tested had an unbalanced output topology with a 3.5mm output jack. The exceptions were the USBPre™ and the EASERA Gateway™. Of course, these are not PMPs, so the IO topology was a bit different. The USBPre has phono outputs as well as a headphone jack. I tested the headphone output to be consistent with the other devices. The Gateway is TRS balanced. I tested it in an unbalanced configuration (by shorting ring to sleeve) after determining that the ring was resistor driven.
Test Overview 
The main test procedure I used was developed specifically for this application by Audio Precision for use with their analyzers. It involves playing a special multi-tone signal from the audio device into the ATS-2 analyzer. The reference tones are full-scale sine waves. The ATS-2 analyzes this signal to derive a number of performance specifications. These are listed in Table 1. The Pass/Fail criteria (Figure 1) for each can be determined by the user, but I found that the default values are consistent with what one would expect from a field reference playback device.
Each PMP had the test stimuli transferred to it digitally, with no AD/DA conversion (a possible source of errors). The exception was the Sony Minidisc, which may explain why it fared poorly on some of the tests.
Some of the devices tested had sharp increases in distortion with their output level at their maximum volume setting. In most cases this setting produced about 0dBV (1Vrms) for a sine wave reference tone for each PMP. As would be expected the USBPre and Gateway had much higher output levels. I elected to test each device at its highest level setting that produced less than 1% distortion, which means that a few of the devices were not operated at the max level setting for the test. The matrix indicates the maximum output level of each device as well as the level at which it was tested. This yields a valuable piece of information – the maximum playback level setting of the device for system testing. Most of the units had to be turned down a bit to reduce the distortion of a full-scale sine wave.
Additional Tests
The STIPa test for speech intelligibility can now be carried out on a number of hand-held analyzers, including the Goldline DSP-30™ and the Ivie IE-35™. Since the stimulus for the test must be played from a portable audio device, I also tested each unit to see if it would work for this purpose. After considering a number of approaches for this test – some rather sophisticated – I decided to simply play each device through a test loudspeaker and compare scores to a reference audio source. I used the EASERA Gateway as the reference device. Assuming no changes to the measurement setup, the scores should be identical for each DUT. The IE- 35 was used to collect the signal (see p. 29, this issue). One would expect an exceptional STI-PA score (close to unity) at close proximity to a loudspeaker in a quiet environment.
I also measured the output impedance of each device using the voltage divider method. A pure tone track was adjusted for -10dBV open circuit, and a decade resistance box was decremented until the level was -16dBV. The 6dB level drop indicates that one-half of the voltage is dropped across the output impedance of the PMP, which means that the value of the decade box equals the PMP output impedance. As expected, all of the devices had very low output impedances, but some were much lower than others. A minimum load impedance of 10- times the measured output impedance is suggested. Note that this makes some of the devices unsuitable for driving headphones, but all should be okay driving typical mic and line inputs on sound systems. Figure 3 shows the impedance vs. frequency plot for a set of typical ear buds used with these devices.
File Format
The ideal file format for system testing is the WAV file. Some devices support WAV files and some did not. All of the devices supported MP3. A couple of them supported WMA. A comparison of specifications using MP3 vs. WMA revealed that MP3 scored better in categories where there was a measurable difference. Also, the MP3 format scores were identical to the WAV scores in almost every specification (slightly lower in S/N). For that reason, I used WAV format when supported and MP3 otherwise. The Sony Minidisc was tested with the proprietary MD format.
Output Level
Most of the PMPs output about 0dBV (1Vrms) for a full-scale sine wave. This makes their maximum RMS output level for music and speech about 10-20dB lower. This level is suitable for consumer level (-10dBV) inputs, or professional line level inputs with a preamp. Mic level inputs can be problematic for these devices. First, turning the PMP down to a very low level results in degradation of the signal-to-noise ratio. Second, phantom power may damage the PMP (I did not test for this for obvious reasons). Both problems can be solved by using a step-down transformer or direct box when driving microphone inputs.
It is not surprising that the USBPre and Gateway provided much higher output voltages than the PMPs. Either would likely drive most power amps to clip, while some of the PMPs will not. This is a major advantage that audio interfaces have over PMPs, and maybe sufficient justification for having PC w/interface on hand for serious testing sessions.
Y-Cables
Since many audio practitioners are in the habit of summing left and right outputs with a Y-cable, I thought it would be interesting to test each one to see if it was tolerant of this over-loaded condition. Every device failed this test, so I did not bother including the results on the matrix. It should be noted that a few of the devices literally shut down during this test unless I significantly dropped the output level. Also, after some adjustment all of the devices produced an audio signal when summed through the Y-cable, but with significant degradation of some of the specifications. Figure 2 shows the test results for an Ipod tested with and without a Y-cable.
The results of the Y-cable test serve as a reminder that the constant voltage interface requires at least a 1:10 source-to-load impedance ratio. For more information on the ramifications of incorrect summing methods, see the excellent Rane Note “Why not Wye” at
http://www.rane.com/note109.html
The Toughest Test
By far the toughest test for all of these devices was the reproduction of swept sine waves. Many of the devices produce artifacts as the sweeps approach the Nyquist frequency (one-half the sample rate). If you intend to use the audio device as a source for sweeping systems for the purpose of collecting impulse responses, the field narrows considerably (Figure 4).
Other Factors
There are other factors beside audio quality that should be considered. I checked each for a “repeat single track” option – a must for sound system testing.
Some of the devices were easy to navigate (i.e. the Ipod) while others were down right frustrating (Samsung YP-U2J). All of the PMPs but the Ipods used Windows Media Player™ as the PC interface. The Ipods used Apple’s Itunes™. The USBPre and Gateway have the advantage of allowing any WAV editor to be used to playback files.
Some Observations
There were some not-so-obvious results from the tests. The MP3 format is often thought to be unsuitable as a test stimulus due to the lossy compression algorithm used to reduce the file size. The Samsung Z5 met all benchmarks using the MP3 format.
Some failures were due to driver issues. The USB-Pre had some intermittent clicking that made it unsuitable for the STI-PA and IR tests.
There was a correlation between physical size and performance, with the smallest devices exhibiting poorer performance than their larger siblings. Miniaturization does appear to come at a cost. While I did not test any cell phone sources, I would expect them to do poorly for this reason. This is also the most likely reason that both sound card chips tested poorly.
Only one device – the EASERA Gateway – met all benchmarks without special attention to level settings, etc. It also had the highest output level and the benefit of impedance balanced outputs.
Itunes proved to be a much friendlier interface than Windows Media Player. The Ipod drivers are integral to it so complete management of the Ipod is accomplished from one application.
Conclusions
All of the devices tested were satisfactory for general music listening. Even the bad ones were good enough to take jogging. If your intent is to test audio systems, some definitely fared better than others. The Ipod Nano and Ipod Video provided the best overall experience with a good balance of high audio quality, intuitive user interface and problem-free connection to the host PC.
So, are these devices as good as PCs with sound cards or external interfaces? You have the data, you be the judge. pb
