A couple of things to consider: first, you are likely gluing end grain to both sides of the veneer. Whatever glue you use will be sucked into the end grain of the blank, leaving little at the actual joint. If using titebond, spread a thin layer on the end of the blank, let it soak in/dry, then add a "second coat" and glue to the veneer. Second, a CA joint is inherently brittle. If you want to use CA for the joint, drill cautiously. The twisting action on the blank can easily cause the CA joint to simply crack apart. I've found the best adhesive for this type of joint to be epoxy.

How are you holding your blank to drill it? If you are simply holding the end in a vice or a chuck, you are subjecting the joints to a lot of twisting action. Try to support the joint while drilling.  Keep your speed down and avoid heat.

Whatever glue you decide to use, give it much more time to dry than the label recommends. Over night for any of them is really the way to go.

Dale: If you can drill straight through a 5" or so pen blank by hand, you are definitely a better man than I am.   

Seriously, if you are drilling by hand, you are putting all kinds of forces on that unsupported joint. I would stay away from CA. If you stay with titebond, try the "pre-wetting" process to get some glue soaked into the end grain of the cut ends of the blank - the more porous the wood, the more you need to slather on. Otherwise, epoxy is your friend.