I also supposed that the solution could maybe have to do with prime numbers
Yes, every time, you get a problem with whole numbers, you should probably look at prime factorization.
Unless you are talking about really large numbers; in that case it might be really hard (even with computers) to find that (always unique!) prime factorization. In fact, that is why SHA cryptography works, They multiply two large primes and make the result known (public key). But to encrypt and decrypt, you will need those primes and unless you know at least one of them, you are in deep trouble 😉
