Alright so I got it to work. But it actually made me ask another question. So if I put that code into the update function, then it worked just find. I teleported right there after pressing releasing the button. Why would I need the update function when pressing a button but not when I was doing it with a collider?
