The speaker housing things are riveted to the door, so it's pretty tough to get them off without drilling out the rivets. I wouldn't worry too much about breaking them, especially if it's from the rear speakers. You have to either remove them altogether or remove the guts of the speaker (the magnet, cone, and rubber surrounds) and use the plastic part as a spacer for the new speaker. This is necessary for the front door at least; for the rear door you don't really need the old speaker at all.There's a great speaker install how-to by Cocoarl:
http://forums.genvibe.com/zerothread?id=785