You could very gently try to put something else in there, like a needle, safety pin, straight pin, etc, and then use that to try to work the broken bit out.