when modelling the dots did you make sure you had a flat surface along the edge and parallel with the door?

and enough verts. High poly.
The texture map I assume you have is 4K and no alpha channel and 32 bit float enabled. and in the UV editor window N panel change the color space to non color.
With the performance issue I don’t know your hardware. but in the render settings CPU is best for baking I find, bring the render samples way down you don’t need high number of samples when baking I go with 64 but 32 is probably enough, under the performance tab select auto detect for No. of threads this will speed things up immensely, my CPU uses 4 threads per core, basically 4 times faster than if it is set to 1 in blender. and save the image after every bake otherwise that previous bake image is held in memory and will slow you down because you have less memory available to process the next bake. and save the file as well.
I could help you further if you want to post a link to the file via google drive.