Sounds like the spring in the controller. Sad part is you may have to heat the overlay to get to the screws to take it apart. Not good as it usually ruins it. You can get a ton of different overlays off ebay. But if you can get it apart without damaging the controller overlay, that would be an achievement. So it is pretty easy from there. Take out a couple screws to get to the main part then there are prongs you lift up with a flat head screwdriver. To replace the spring all you need is the spring from an Atari 2600 cart. There are millions of Pac-Mans and Combat carts you can use for pennies. Remove the old spring, which is probably broken in 2 pieces from ware and tare. Insert the Atari 2600 spring into the place of the old Vectrex one. You will have 2 prongs sticking out from the spring so you have to cut them off. Not completely, leave just a small part sticking out. A little less than 1/8 of an inch or so. Just eye ball it. Not a huge deal. Move the controller around and make sure it goes back to center when you move it around. You will notice it is much stiffer than previous. Put everything back together and give it a test. Should have fixed the problem.
Also, when you have it apart there is a copper circle contact part that may have corrosion on it. Clean that off.