Hey there, new to the forum, but I may be of significant help. A few years ago I made a full Batman suit out of fiberglass mat, carbon fiber and kevlar. I also am very sensitive to fiberglass so I've found a method that makes it much more tolerable to work with.
First, how impact resistant are we talking? The shell material should be a first step, but foam and appropriate padding would be just as important. If it is for airsoft/ paintball, it isn't as big of a deal, if it is for combat training or anything else more concussion inducing then you'll want to make sure you adequately pad it, because the shock still transfers even if the shell doesn't break. Play safe.
Fiberglass, even the mat stuff, is EXTREMELY itchy when cut and reduced to smaller fibers. You don't want it to touch your skin at all until it has been safely resined. I know that is an annoyance, but fiberglass tends to still be my starting place for reenforcing builds. It helps shape everything nicely and it is relatively cheap and can be easily cut and molded to meet your needs.
I find the key to dealing with fiberglass is to wear long sleeves at all times and tuck your sleeves into mechanic/ surgical gloves to prevent fibers getting in. Also be sure to wear a respirator while cutting the fiberglass, I know it isn't as fine as asbestos or other fibers, but you still don't want to breathe it in or get it up your nose. Trust me, this is the voice of experience, itching the inside of your nose makes you look like you are picking your nose which apparently is inappropriate to do at your wedding (that was a joke, but seriously wear a respirator).
Once you are done working in a space, be sure to clean it thoroughly to pick up any fibers. This is true regardless of what material you end up choosing. All of these fiber based cloths can be irritating to the skin, eyes, and everywhere else if left uncleaned. Working in an area with hard surfaces is your friend for fiber-work.
The same precautions should be taken with the clothes you are changing out of once you are done the work. Put them separately, shake them out outside, then submerge them in water to stop the fibers from getting airborn. Wash your clothes, clean out the washer and dryer prior to putting in new loads of non-fiberglass clothes.
I have combined carbon fiber and kevlar with fiberglass too. Carbon fiber should only really be used for visual impact/ aesthetics. I second it being quite brittle. Kevlar however does a great job of increasing impact resistance/ strength. I second only needing a few layers of it though, it is tough! There is a reason they use it in bulletproof vests and fighter pilot helmets. It also isn't exceptionally cheap.
The other key, is choosing the right resins. Your weave is only as strong as the resin that binds it and not all resins are created equal. You'll have to do your research on resins to determine exactly what you need (strength, clarity, flexibility, etc.). For my non-visual parts I just used automotive resin/ bondo. For my carbon fiber I used special highly clear resin created for that exact purpose (it isn't strong or flexible, just very clear and pretty).
Let me know if you have any questions, looking forward to seeing your progress and sharing mine (I'm building an Infinite style suit with carbon fiber accents... it is going to be like a sports car and John 117 had a baby).