Maintenance — Duplicate Handling¶
PastPaths can find and merge records that were imported multiple times or registered as separate entries even though they refer to the same person, place, media file, or event. The feature is accessed via Administration → Maintenance or via dedicated buttons on the respective list pages.
The Maintenance page¶
The Administration → Maintenance page is a list with one row per area, in alphabetical order. Each row has a heading, a short description, and the buttons on the right. Hover over the row to read a longer description.
| Row | Buttons |
|---|---|
| Events | Search duplicates |
| Export folder | Open export folder — opens the folder where reports, GEDCOM files, and backups are saved, see Backups |
| Media | Clean up media (orphaned files and missing thumbnails), see Clean up media, and Search duplicates |
| Persons | No connections, see Persons without connections, and Search duplicates |
| Places | Clean up unused (places without links) and Search duplicates |
| Relationships | Check relationships, see Relationships |
| Sources | Go to tab — see Sources, categories, and tags |
| Taxonomy | Go to tab — see Sources, categories, and tags |
Overview¶
Each entity type has its own points-based system that weighs several signals together into a probability score (0–100). The score determines whether a pair is shown as a possible duplicate. You then manually choose which fields to keep, and the system handles all database links automatically.
| Entity | Primary signal | Default threshold |
|---|---|---|
| Person | Phonetic name + date + place of birth | 50 |
| Place | Source text (GEDCOM PLAC) + coordinates | 30 |
| Media | SHA-256 hash | 35 |
| Event | Type + date + place | 60 |
Sources, categories, and tags¶
Unlike persons, places, media, and events, tags, categories, and sources have no automatic scored detection. Tag and category names are already deduplicated technically (the same name can't be created twice), so "duplicates" here are in practice spelling variants or synonyms — Probate and Probate record, for example — which the system can't determine automatically.
Instead, you select what to merge yourself:
- Go to Administration → Taxonomy (for tags/categories) or Administration → Sources
- Check at least two rows in the table
- The Merge selected button appears — click it
- Choose which of the selected items to keep (default: the one with the most links)
- Confirm
All links (persons, places, events, media, blog posts for tags/categories; citations for sources) are automatically moved to the kept record. For categories, any subcategories are moved to the kept category instead of being deleted.
The same shortcut is available on the Maintenance page: the Go to tab button on the Sources and Taxonomy rows takes you directly to the respective tab. Hover over the row for a short description of how merging works there.
Searching for duplicates¶
Automatic search¶
Open the search via Maintenance → Search duplicates or the button on the respective page (Persons, Places, Media, Events). Adjust the threshold with the slider and click Search. The system returns pairs sorted by score, with a color-coded confidence indicator:
- Green (high) — strong match, likely the same record
- Orange (medium) — possible duplicate, review manually
- Gray (low) — weak signal, use your judgment
Under each pair, the score breakdown per signal is shown (e.g. "Exact name: 25/25," "Coordinates: 30/30") to help you understand why the pair was flagged.
Manual search¶
The Manual search tab lets you search freely and select any records for merging — regardless of how similar they are score-wise. Useful when you know two records are the same but the system hasn't found them automatically (e.g. a name typo that the phonetic matching doesn't catch).
Select at least two records and click Merge selected. With more than two, the merge dialog opens sequentially: you merge one pair at a time until all are done.
The merge tab¶
When you click Review on a duplicate pair, the merge dialog opens.
Choosing a keeper¶
At the top, you choose which of the two records will be the keeper — the one that's kept. Click the name chips to switch. This choice affects the default setting for all fields below (the keeper's field is preselected).
Field selection¶
Each data field is shown as a row with the keeper's value on the left and the other record's value on the right. Click the side you want to select which value is kept.
Fields with an image (the person's profile photo, the media file's thumbnail) show the images side by side above the field rows, so you can compare visually before choosing which data to keep.
For text fields (description, notes/biography) and taxonomy (categories, tags), there's a third option:
Combine both — the texts are concatenated with a separator, or the taxonomy is merged as a union. This option is selected automatically when both records have content, to minimize data loss.
The confirmation dialog¶
Before the merge is carried out, a summary is shown:
- Which record will be deleted and which will be kept
- How many links will be moved (relationships, events, citations, etc.)
- A warning if any text field or taxonomy field will be discarded (i.e. you chose only one side's data)
The merge cannot be undone. Check that you've chosen the right keeper before confirming.
What happens in the background¶
Upon confirmation:
- Selected field values are written to the keeper
- All foreign-key links are updated (see table below)
- Taxonomy (categories and tags) is merged according to your choice
- Any "Not duplicates" markings for the removed record are cleared
- The removed record is permanently deleted from the database
| Entity | Links that are moved |
|---|---|
| Person | Relationships, event links, media, citations, bookmarks, research notes |
| Place | Birthplace references, place-of-death references, events, media, citations, bookmarks, research notes |
| Media | Person links, place links, event links, citations |
| Event | Person roles, media links, citations, bookmarks, research notes |
If both records are bookmarked, you get one bookmark, and both notes are kept.
Media files on disk
When merging media records, the database row for the removed record is deleted, but the physical file is left on disk as an orphan. This is a deliberate choice to avoid data loss. Orphan files can be cleaned up manually if needed, via the Clean up media button on the Media row under Maintenance. → Clean up media
Not duplicates¶
If the system suggests a pair that you know aren't duplicates, click Not duplicates. The pair is hidden from future searches, and an Undo button is shown for 5 seconds if you change your mind right away.
The marking is stored in the database and applies until you undo it — either right away via the Undo button, or later via the Ignored tab in the same dialog (next to Automatic search and Manual search). It lists every pair you've marked "not duplicates" for that entity type, with an Undo button on each row, so you're not limited to the 5-second window if you change your mind afterward.
Clean up media¶
Open via the Clean up media button on the Media row under Maintenance. The dialog has two tabs.
Orphaned files¶
Files that are still in the media folder but that no media record points to anymore — for example after merging media records or an interrupted import.
- Click Search. The scan goes through the entire media folder.
- The files are listed with a preview, file size, and last modified date.
- Check the files you want to delete and confirm.
Deleting files is permanent
The files are removed from disk and can't be recovered. If you're unsure — make a backup first.
Missing thumbnails¶
Images that have no generated thumbnail. This is most common for media imported via GEDCOM or WordPress in older versions of PastPaths. Without a thumbnail, a generic icon is shown in places such as the duplicate and merge views.
- Open the Missing thumbnails tab.
- Click Generate thumbnails.
The original images aren't affected — only the missing thumbnails are created.
Persons without connections¶
Open via the No connections button on the Persons row under Maintenance. Finds persons with no family relationships at all — regardless of whether they have events, source citations, or known birth/death dates. Common after large GEDCOM imports where individual records lose their connections.
Click Search to list them. Per row, you can:
- Ignore — marks the person as reviewed, hidden from future searches
- Tag "Missing relations" — same principle as the import tag, for deliberate categorization
- Open — opens the person's edit dialog directly, e.g. to write a note about why the connection is missing
- Delete
The list doesn't refresh automatically after an action — press Search again if you want an updated list.
Relationships¶
Partner and sibling relationships are stored in both directions — one row for "Anna is married to Erik" and one for "Erik is married to Anna". They should have the same details, but older versions of PastPaths could make them differ: a change to a relationship was saved in only one direction, and import from TNG set relationship types the program doesn't recognize.
Open via Maintenance → Relationships → Check relationships and click Check. The result is split into three groups:
- Different details in the two directions — which fields differ: subtype, dates, date precision, or note
- Missing opposite direction — only one direction exists
- Invalid subtype — with what it will be changed to, or a prompt to fix it manually
Click a name to open the person.
Fix automatically first shows a confirmation and then does the following:
- Differing details are taken from the direction that was changed most recently.
- Notes that differ are combined under the heading "Från relationens andra riktning" (From the relationship's other direction), so no text is lost.
- A missing opposite direction is created as a copy of the one that exists.
- Known invalid subtypes are replaced with valid ones (divorced → Ex-partner, common-law → Cohabiting). Subtypes the program doesn't recognize are left unchanged — open the person and choose a subtype for the relationship.
What remains is shown after the fix.
Scoring system per entity¶
Person¶
| Signal | Points |
|---|---|
| Exact name match | 40 |
| Phonetic match (Carlsson/Karlsson, Wahlström/Vahlström, etc.) | 20 |
| Birth year matches (precision: year) | 22 |
| Birth year + month matches | 25 |
| Exact date of birth matches | 30 |
| Birthplace: coordinates < 500 m | 20 |
| Birthplace: parish + country matches | 10 |
| Date of death matches | 15 |
| Gender conflict (both set, different genders) | −30 |
Default threshold: 50. Recommended range: 40–80.
Place¶
| Signal | Points |
|---|---|
| Source text (GEDCOM PLAC string) exact match | 50 |
| Coordinates < 100 m | 30 |
| Exact name match | 25 |
| Normalized name matches (slug) | 20 |
| Parish + country matches | 15 |
| Similar name (fuzzy) | up to 15 |
Default threshold: 30. A lower threshold catches more possible pairs — useful for large GEDCOM imports.
Source text on manually entered places
The source text is the strongest signal (50 points). Previously only imported places had one — places you typed in by hand had none at all, and so could never match an imported place on that signal. PastPaths now works out the source text itself when you save a place in the form, so manual and imported places are compared on equal terms.
Media¶
| Signal | Points |
|---|---|
| Identical file (SHA-256 hash) | 80 |
| Same file size | 10 |
| Similar filename | up to 20 |
| Same media type | 5 |
Hash calculation
The SHA-256 hash is calculated the first time a duplicate search runs and is saved to the database. Searching a large media collection may take a few minutes the first time. Subsequent searches are fast.
Missing thumbnails
Media imported via GEDCOM or WordPress could previously lack a generated thumbnail, which caused the duplicate and merge views to show a generic icon instead of the image (the media library itself always showed the original image correctly). New imports now generate thumbnails automatically. For already-imported media without a thumbnail: go to Maintenance → Clean up media → Missing thumbnails and click Generate thumbnails.
Default threshold: 35. A hash match (80 pts) is always "high" confidence.
Event¶
Only searches among non-trivial event types: marriage, divorce, baptism, confirmation, graduation, military service, emigration, immigration, occupation, retirement, burial, census, residence, education, and other. Birth and death are handled automatically when merging persons.
| Signal | Points |
|---|---|
| Same type + date + place | 80 |
| Same type + date (no place) | 40 |
| Same type + place (no date) | 25 |
| 2 or more shared participants | +15 |
| 1 shared participant | +5 |
| Identical source string | +20 |
| Shared source citations | +15 |
Default threshold: 60. Raise to 80 if you only want near-certain matches.
Sequential merging¶
During a manual search, you can select more than two records. The dialog then opens for the first two, and once you've confirmed that merge, the next pair opens automatically (the keeper from the previous step against the next record in the queue). You can cancel the sequence at any time with Cancel in the dialog.
Event type during sequential merge
For events, the system checks that all records in the queue have the same event type. If a record in the queue has a different type, it's automatically skipped with a message.
Tips¶
- Start with a low threshold, raise it gradually. Run a search at threshold 30 for places to see the volume. Then raise it if the list is overwhelming.
- Taxonomy defaults to "Combine both." If you want to remove duplicated import tags (e.g.
-tng-import-2025-01-01on both), choose "Keep keeper" instead. - The Maintenance page aggregates all entities but is just a shortcut to the same dialogs found on the respective list pages. The results are identical.