yep, for the specular maps I realized that the bake for the individual specular maps were on a black background rather than no background and when I erased that black background it fixed that issue and for the normal maps I just tweaked the premul in each of the nodes and that seemed to fix it although I have no idea why lol