Credit cards fueled a huge surge in airport lounge visitors, and now it affects your odds of being able to visit one — even with the right card.