A good tip for future builds is to sand to a higher grit prior to filling. Knocking down those layer lines creates less work for the filler and you don't need to add as much, which reduces the weight for the helmet.
I personally sand up to 150 (starting grit can vary on print quality), do a light coat of UV resin, sand that at ~300, and then spray a light/medium coat of filler primer. Or you can skip the UV resin step, go straight to 300 grit, and spray on a heavier coat of filler primer. With either method, you'd just need to do a light pass of wet sanding of about 600 on the filler primer and you should be left with maybe a few spots that just need to be filled in and wet sanded with spot putty. The results come out incredibly smooth and if your print quality is really good, you can start at the UV resin step and only have to do 2 full passes of sanding.