You're spot on - you're sending an amplified signal straight into the factory amp. The volume will be loud and I've found there will also be an excessive amount of white noise.
A speaker level converter wired in between the head unit's speaker outputs and the speaker wiring on the factory harness adapter will help. You can then adjust the levels accordingly.
I have this one here:
https://www.crutchfield.com/p_142SLC4/S ... erter.html
Adds a lot of bulk to the install but it works well.
Some have went from the line outs to the speaker wiring, but I seem to remember a lot of popping noises/etc.
03 Vibe base. Born 10/14/2002 06:07 AM
Neptune. Auto, Moon & Tunes, power package. 135k mi.
'13 Sonic LT - 1.4L ECOTEC turbo/6A. 116k mi.
'15 Escape SE - 1.6L EcoBoost, FWD. 58k mi. (hers)