Okay, yeah you would need an extra adapter, but that's evident. Again, the products I listed were suggestions only. You can do research on your own for headphone amps, but you could buy a cheap desktop audio system with RCA inputs and headphone jack and just be done with it. Check out some yard sales for some if you're so inclined. You should be able to tell exactly what cables and adapters you need by examining the product images/specs.
There are probably a couple of audio outputs from the TV. Most HDTVs have analog and digital outputs. The red and white RCA jacks you typically see are analog, so there is no digital to analog conversion. There will usually be a single digital audio out, but let's not even go there.