350X steering stem bearings

[Lots_Of_Nothing]

I purchased the All Balls racing steering stem bearing kit for the 85 350X (replaces the stock loose ball type with tapered bearings), decided to put it on today, and ran into a little problem... I figured that the piece circled in this picture would come off to get replaced by the tapered bearing (like the top one), but I cant seem to get the dang thing off... (the part circled in yellow)

http://img64.exs.cx/img64/9108/stem10px.jpg

Before I go forcing it and break something, is it even supposed to come off? Here is a picture of the only other way I can think of them mounting,, But it doesnt seem correct (nevermind the loose races on the stem);

http://img180.exs.cx/img180/6291/stem27ss.jpg

So.. What am I doing wrong? :D

[NOS_350X]

It comes off, mine did, try using some heat.

[Lots_Of_Nothing]

It comes off, mine did, try using some heat.

Alright, thanks, thats all I needed to know.. I didnt wanna bust out the torch and hammer untill I was sure it was meant to come off.

[Lots_Of_Nothing]

After lots of severe heating and beating it finally popped off... Now another question.. The bearings came with a washer/spacer, Where does this go?