Why Card Sorting Still Matters
In a world of AI-powered navigation and smart search, card sorting might seem old-fashioned. It's not. Card sorting is the fastest, cheapest way to understand how your users mentally organize information. No amount of AI can replace knowing that your users think "Billing" belongs under "Account" not "Settings."
We run card sorts on every project that involves navigation design, content restructuring, or feature reorganization. The insights are always surprising and always actionable.
1. Choose the Right Sort Type for Your Question
Open Card Sort: Participants create their own categories. Use this when you're starting fresh or redesigning an existing IA. It reveals how users naturally group information without being influenced by your current structure. Best for discovery.
Closed Card Sort: You provide the categories; participants sort cards into them. Use this to validate a proposed structure. It tells you whether your category names make sense and whether items end up where you expect. Best for validation.
Hybrid Sort: You provide some categories, but participants can create new ones. Use this when you have a partial structure and want to fill gaps. It balances structure with flexibility.
2. Get Your Card Labels Right
This is where most card sorts go wrong. The labels on your cards determine the quality of your results. Ambiguous labels produce ambiguous groupings.
Rules we follow:
Use plain language, not internal jargon. "Payment History" not "Transaction Ledger." Keep labels to 2-4 words. Avoid labels that contain the category name. Don't include more than 40-60 cards; beyond that, participants experience fatigue and results degrade.
| Billing | Invoices | Payment | Profile | Security | Notifs | |
|---|---|---|---|---|---|---|
| Billing | 100% | 82% | 78% | 18% | 12% | 8% |
| Invoices | 82% | 100% | 75% | 10% | 5% | 42% |
| Payment | 78% | 75% | 100% | 15% | 38% | 6% |
| Profile | 18% | 10% | 15% | 100% | 68% | 55% |
| Security | 12% | 5% | 38% | 68% | 100% | 35% |
| Notifs | 8% | 42% | 6% | 55% | 35% | 100% |
3. Sample Size and Analysis
How many participants do you need? For an open card sort, 15-20 participants produce statistically reliable groupings. Fewer than 15 and you'll see too much variance. More than 30 and you're over-investing for marginal improvement.
For analysis, we use a similarity matrix to identify which cards were grouped together most frequently. Items that cluster above 70% agreement belong together confidently. Items between 40-70% need further investigation. Below 40% means no consensus, and you may need to reconsider the item's labeling or placement.
Always Run a Pilot
Before running the full study, test with 3 participants. The pilot reveals: cards with confusing labels (participants ask for clarification), missing cards (participants mention items that aren't included), and whether the task is too long (sessions over 20 minutes produce unreliable data).
We've changed card labels after every pilot we've ever run. It takes one afternoon and saves the entire study from flawed data.