Finding and Merging Duplicates
Keep your CRM clean with duplicate detection, smart merge, and data quality tools.
By Sebastian StreiffertPublished Jan 10, 2026Updated May 29, 20267 min read
The Hidden Cost of Duplicate Data
Duplicate contacts and companies don't just clutter your CRM. They actively undermine your sales process. Reps waste time chasing leads that colleagues already contacted. Reports show inflated pipeline numbers. Customers receive duplicate outreach (when using your outreach platform) and lose confidence in your organization.
The problem compounds over time. Every data import, web form submission, and manual entry creates another opportunity for duplicates to slip in. Without regular cleanup, even well-maintained CRMs accumulate redundant records. See Data Import guide for best practices on preventing duplicates during import.
What the Hygiene Hub includes
Core tools
Duplicate Detection
Smart Merge
Prevention Rules
Data Quality Score
How Duplicate Detection Works
Lumenbase uses fuzzy matching algorithms to identify potential duplicates. Unlike simple exact-match searches, fuzzy matching catches variations like "Robert Smith" and "Bob Smith" or "Acme Corp" and "Acme Corporation."
Matching Criteria
| Entity | Primary Match Fields | Secondary Match Fields |
|---|---|---|
| Contacts | Email address (strongest) | Name + company, phone number |
| Companies | Domain/website | Company name, phone number |
| Deals | Deal name + company | Contact + close date |
Email matching is the most reliable signal. Two contacts with identical email addresses are almost certainly the same person. Name matching is trickier. Common names like "John Smith" generate many false positives, so the system requires additional matching fields before flagging these as duplicates.
Confidence Scoring
Each potential duplicate receives a confidence score from 0 to 100 indicating how likely the records are actually duplicates:
| Score Range | Confidence | Typical Match |
|---|---|---|
| 90 to 100 | Very High | Exact email match or exact name + company |
| 70 to 89 | High | Similar name + same domain or phone |
| 50 to 69 | Medium | Similar name + partial field matches |
| Below 50 | Low | Not flagged as potential duplicate |
Finding Duplicates
You can access duplicate detection from two places: the Hygiene Hub or directly within any entity list view.
Open the Hygiene Hub
System Operations → Hygiene Hub → Duplicates. You see a summary of detected duplicates grouped by entity type.Review Duplicate Groups
Filter by Confidence
Quick Duplicate Check
From any Contacts or Companies list, click the "Find Duplicates" button in the toolbar. This runs a focused scan on just the visible records, which is useful when you suspect duplicates in a specific segment.
Merging Duplicate Records
Merging combines two or more duplicate records into a single, unified record. The process preserves all associated data (activities, deals, tasks) by re-linking it to the surviving record.
The Merge Process
- Select which record becomes the 'primary' (the one that survives)
- Review field-by-field which values to keep from each duplicate
- Confirm the merge. Secondary records are deleted and their data re-linked
- All activities, deals, and tasks from deleted records move to the primary
Field Resolution
When duplicates have different values for the same field, you choose which to keep. The system suggests defaults based on data completeness and recency:
- Non-empty values preferred over empty ones
- More recently updated values preferred when both exist
- Email addresses default to the validated one if available
- Manual override available for every field
Bulk Merge Operations
When you face hundreds of duplicates (common after a messy data import), merging one at a time isn't practical. Bulk merge lets you resolve multiple duplicate groups at once.
How Bulk Merge Works
- Filter duplicates to high-confidence matches (90+) for safety
- Select multiple duplicate groups using checkboxes
- Click 'Bulk Merge' and choose your resolution strategy
- Review the summary showing what will happen to each group
- Confirm to process all selected merges
Resolution Strategies
| Strategy | Behavior |
|---|---|
| Keep Oldest | Primary record is the one created first |
| Keep Newest | Primary record is the most recently created |
| Keep Most Complete | Primary record has the most filled fields |
| Keep Most Active | Primary record has the most associated activities |
Preventing Future Duplicates
Cleanup is necessary, but prevention is better. Lumenbase offers several tools to stop duplicates before they enter your database.
Import duplicate handling
During CSV or Excel imports, the system automatically flags potential duplicates. You can choose to:
- Skip: Don't import records that match existing entries
- Update: Merge imported data into existing records
- Create Anyway: Import as new records (creates duplicates intentionally)
- Review: Pause import to manually decide on each match
Real-time duplicate warnings
When manually creating contacts or companies, the system checks for matches in real-time. If a potential duplicate exists, you'll see a warning with the option to view the existing record instead of creating a new one.
Web form deduplication
Forms connected via the API can be configured to update existing records rather than creating duplicates. This is particularly useful for newsletter signups and event registrations where repeat submissions are common.
Data Hygiene Best Practices
Ongoing hygiene
- Run duplicate scans monthly, or quarterly at minimum for smaller databases
- Process high-confidence duplicates immediately; they're almost always correct
- Review medium-confidence matches carefully; name-only matches often aren't true duplicates
- Train your team to check for existing records before creating new ones
- Use consistent data entry standards (e.g., always 'Inc.' not 'Incorporated')
- Clean up before major imports to avoid multiplying existing duplicates
- Consider assigning a 'data steward' responsible for ongoing quality
Was this article helpful?