Well, look at it this way. If you are in a car and you accelerate, your speed increases. If the rate of acceleration is constant, and you have a long enough road ahead of you, you could, in theory, accelerate to somewhere near the speed of light. Of course, in your car, you have the problem that friction on the road, friction against the air, friction in the engine itself will only allow for a limited speed.
Now the earth gravity is a source of constant acceleration. If you let a ball fall 2 seconds, it will have double the speed as if you would let it fall only 1 second. Of course the problems remain the same, friction against the air will reduce its speed, and there is a break even point known as "terminal velocity", where the force from the friction cancels out the gravitational force that would accelerate the ball. Also, you can't have it fall endlessly, at some point it is going to simply crash into the ground.
Now if you are in space, there is no air, so there is no (or far far less) friction to stop you. And with the portals, you can create an endless fall, gathering more speed with every second you spend in there. From that point on, it is just a waiting game, and how long you have to wait depends on the size of the planet/sun you use.