You'll need one or the other of these medications: Ivermectin or Revolution. There's a great article about treating for lice/mites here:
ectoparasites. If you can see them, they're probably lice. Mites are too small to see without a big magnifying glass or microscope.
You'll also need to throw out the batch of bedding/litter you have and get a new batch. That's generally where lice come from. Unfortunately, freezing won't kill lice eggs. The only way to kill lice eggs in your bedding is to store it through at least one life cycle of lice, which I think is about 60 days. Many folks who've had bad luck with ectoparasites have opted to use fleece for bedding, changing and washing it every couple of days. Fleece doesn't have bugs. You'll also need to scrub your cage very thoroughly to get rid of any bugs that might be hanging out on it or on the bars.
Ectoparasites are a major hassle to deal with, but you need to treat every rat for them. The bugs can cause some severe health problems (in addition to the terrible itching) if left untreated.
