I've bought two consoles from junk yards, one for my 08 and then another for my 05 from a local junk yard from '04's and both also had the 12v outlet and harness. Both times I took the outlet, harness and double lid and threw the console away. Replacing the lid is a two screw operation, replacing the entire console is 4 10mm bolt operation.
And after driving my 04 for many years not having the armrest was very noticeable, and I really missed the top level storage for gum, pen, etc.
'04 Base Frosty, '08 Base Stealth, '05 Base Satellite