@emericanized hi!
I am not sure, but I think the issue is in your hi-res material. You are baking the Emit, meaning that you need to connect the dirty vertex color to an emission shader. Something like this:

And the texture you made for the low-res version should be selected in the material for the low-res:

If this doesn’t help, please provide a link to your .blend.