How do I get into the 3 pin connector at the end of the N64 pad that plugs into the system?

All I can see is a small hole, I tried pressing in it with a screwdriver and pulling the middle bit, but it wont come out.. does anyone know how to open it?

Im asking as I have a controllers thats not detected when you plug it into the N64, so I think the wires inside may be loose..