Don't panic! All sensitive information on the keycard is encrypted with your passcode. But you should make best efforts to keep your keycard and it's content private. So make sure you control who has access to it. If another party has your keycard, they can take your funds.
If your keycard does get exposed or copied in a way that makes you uncomfortable, the best course of action is to empty the corresponding wallets into another account and discontinue the use of your old account.