Thoughts on Resolving the Family Tree Dilemma

In my last post, I reviewed some of most perplexing issues faced by the FamilySearch.org Family Tree program. See "The Persistent Merging Problem on the FamilySearch Family Tree." In this post, I will conceptually examine some of the solutions to the problems.

The FamilySearch.org Family Tree is a wiki designed to hold genealogical data. As such, the Family Tree is structured so that each person who ever lived on the earth can theoretically occupy only one node in the tree structure. Since each person who has ever lived can only have two biological parents, the tree structure is a lattice of interrelated individuals. A single node is the product of two individual parents, however, a node can also be "related" to additional sets of parents based on social conventions, such as adoption, foster child, guardianships etc. If there were no such alternative relationships, the tree would be 2 dimensional. In order to accommodate these alternative relationships, the tree must become 3 dimensional and allow links from an individual to more than one set of parents. In addition, every individual represented by a node can further be related to other individuals who share the same set of parents. The tree must also represent culturally defined marriage relationships between two or more individuals. An individual may have multiple marriage and marriage-like connections. Even though the tree provides a node for two or more "parent" relationships, some of these individuals may not be identified even though, theoretically, there is a place in the tree for all of these unidentified individuals.

For someone designing such a program, there is a fundamental challenge of adequately representing these possibly complex relationships in a way that can be visualized. The presently designed Family Tree does an adequate job of representing the full, potential lattice of relationships. As a user, I can start with myself and navigate to anyone who is related to me in any way, including all of the non-biological relationships. From the standpoint of processing all this information, computer systems that can adequately store the entire lattice are just now beginning to be available. The adaptation of the Family Tree to a computerized representation is made even more challenging by the need to document every single node (individual) on the tree structure. Because the number of related ancestral nodes increases geometrically, the number of potential nodes is possibly over 100 billion individuals.

The tree structure also has to allow for the reality of pedigree collapse when individuals marry relatives with a partial copy of their own ancestral lattice.

Given these parameters, those who have designed and developed the Family Tree have done an amazingly effective job. Now, I need to consider the data or information incorporated into the tree structure. As I have pointed out in many previous posts, the challenge of incorporating the existing data into the tree dwarfs the challenge of creating the structure in the first place, especially when you consider the fact that the information is duplicative, incomplete, contradictory and some is completely fabricated. In addition, the "users" who are being allowed to contribute even more information to the tree range from extremely sophisticated to entirely clueless. Here the collaborative structure of the wiki excels. The wiki structure allows an infinite number of "corrections" to the initial data entered into the system. In the case of the Family Tree, the combined data presents a variety of selection options. Since only one individual can ultimately reside in his or her respective node, all of the duplicates must be either deleted or ignored by the program. By its nature, the tree will incorporate incorrect information and the node structure will not accurately model reality. But because of the ability of anyone to make changes, the tree structure will inevitably evolve into a more accurate representation of the real relationships between the nodes as the information is corrected and the relationships more correctly defined.

To summarize; the structure of the Family Tree is an adequate representation of historical reality. It has the ability to incorporate the full spectrum of family relationships and adsorb all of the information attached to every individual. As the program evolves, it will inevitably begin to incorporate any additional structural needs that can be identified.

Users of the program come with differing amounts of initially possessed data and opinions. Some users come with little or no knowledge of their family and its relationships. Others possess huge files of information. The Family Tree accommodates both ends of the user experience. The open, collaborative structure of the tree is foreign territory to most users. Genealogists are usually solitary researchers with a fixed, preconceived view of the data they have collected. Getting these individuals to cooperate and accommodate those who they may disagree with is a major challenge. Those users faced with accommodating conflicting entries often become defensive and have the attitude that unless the game can be played their way, they will simply refuse to play at all. Fortunately, the tree structure of the program simply ploughs around these rocks in the field and moves on.

What about the huge pile of duplicates that were entered into the program by seeding it with multiple databases accumulated over more than a hundred years? The mechanism for resolving all of this information is built into the program. Practical limitations on the access to certain parts of the information, i.e. Temple related and membership related information, makes the resolution of the duplicates more complicated than simply opening up the program to a massive merging effort.

What is the most effective methodology for working with the Family Tree? The basis of any progress in sorting out the information in the Family Tree is careful, documented research. As I have added sources and corrected the existing information from those sources, the Family Tree has become locally more reliable. This is a slow, methodical process. Not every data field can be completely established and/or verified. By their nature, historical records are often conflicting and may be inaccurate or incomplete. Some relationships may never be adequately established. Ultimately, the only way the Family Tree overcomes the merging problem and every other challenge to the integrity of the information is step by step and data field by data field.

Every user of the tree needs to closely examine all of the sources added to every individual for relevance. Inappropriate sources need to be detached. The information presented by the sources needs to be evaluated and where appropriate, used to correct the entries in the Family Tree. All unsupported relationships need to be deleted. As the program progresses, all duplicate entries need to be merged. This needs to be done, individual by individual, systematically working backward from the individual user. Any conflicting information needs to be resolved by collaboration. As we collectively work on the Family Tree we will soon find the current limitations of the information and can begin to add more information that is missing. As a byproduct of this methodology, new individuals will be added to the Family Tree. Skipping generations back into the past to do "research" on remote ancestors is counterproductive. Unless all of the intervening individuals are adequately documented, the users who jump generations are almost guaranteed to be working on people to whom they are not related.

We are not so much building a new structure as remodeling an old one. We need to make sure of the adequacy of the construction of the tree at every level before adding on. Patience, accuracy, responsibility and most importantly love for the people in the tree should be the basis for what we do. To quote from President Gordon B. Hinckley from the August 1986 Ensign,

Rather, let us go forward with faith and with the vision of the great and marvelous future that lies ahead as this work grows in strength and gains in momentum. Build faith in the hearts of all those around you.

Quoting further from the same article,

May the Lord bless us as builders of faith. May our testimonies strengthen and become as anchors to which others may secure their faith in hours of doubt and concern. May the candle of learning ever burn in our minds. Above all, may testimony grow in our hearts that this is in reality the church of the living God and that it will continue to move forward to fulfill its divine destiny. May we each do our part faithfully and with thanksgiving to the Lord for all the blessings he so wondrously bestows upon us as we follow his teachings and draw near to him.

Let's get to work.

The Persistent Merging Problem on the FamilySearch Family Tree

Here is a screenshots from the FamilySearch.org Family Tree that illustrate an persistent and unresolved issue with the program.

Here is another screenshot for the same person illustrating another aspect of the same problem.

Although FamilySearch has made significant progress in overcoming many of the problems of the Family Tree and added a number of very useful features, the core problem with the program still persists. Almost without fail, every time I teach a class and mention using the Family Tree, I get a question about merging. 

The two screenshots illustrate two different aspects of the same problem: the inability to merge certain obviously duplicative records and the overwhelming number of duplicates for some individual ancestors. Unfortunately, if I go back in my ancestral line to this particular ancestor, there is nothing in the pedigree that indicates that this multiple duplicate situation exists. Here is a screenshot of the pedigree and the individual entry for this ancestor that comes from clicking back in my family line.

You might also note the proliferation of green Temple icons. In fact, it is very likely that every one of the people shown in this view have the multiple duplicate problem. Going out one more generation finally gets us to the "real" Nathaniel Potter as opposed to his son Nathanial Potter Jr. and the other line's Nathanial Potter Sr. If you know how to read a pedigree chart, you will note that Martha Hazard is apparently the daughter of Nathaniel Potter, Jr. and Elizabeth Stokes. To say that this line is messed up would be a gross understatement.

The basic issue is can this situation be "fixed?" If it can be fixed, when will that happen? Obviously, I have no incentive to dive into this mess until the "Cannot be merged at this time" issue is resolved. It is my perception that the issue is being resolved slowly, but any estimates on the time it might take to wade through this issue are highly speculative. It is interesting to go out one more generation on these lines. Here is another screenshot.

According to my own pedigree as opposed to the one illustrated here, I am a descendant of Nathaniel Potter, but none of my records agree with what is shown on the Family Tree. In fact, in looking at this particular family in my own records, I note that I have several obvious problems. This is back so many generations, I may never get all the corrections made that far into the past.

As far as users of the Family Tree program, getting into this type of situation is similar to trying to negotiate a fire swamp. As far as the green Temple icons go, if you click on any of them you get the same response, "Possible Duplicates Exist" and you are not allowed to proceed with reserving any names until this is resolved.

Is there hope that this type of situation can ultimately be resolved? Yes. The Family Tree is the solution not the problem. This is only an extreme example of the data problems created over the last 100 years or so. Meanwhile, we are still out here waiting for a resolution of the merge issue so we can start untangling this mess.

As a final note, I might mention that Portsmouth, Rhode Island was settled 1638 and wasn't named Portsmouth until 1639. Just in case you are tempted to try and sort out all this stuff.

Do you need a desktop family history program?

The recent Ancestry.com announcement about the retirement of Family Tree Maker has once again raised an interest in the issue of whether or not family historians or genealogists need a local, desktop-based genealogical database program at all. There are many different opinions and reasons for being either in favor of using a desktop program or abandoning the desktop for online family trees. This controversy is particularly evident among members of The Church of Jesus Christ of Latter-day Saints who see the evolving FamilySearch.org Family Tree program as a "permanent" storage option.

When I refer to a "desktop" genealogy database program, I mean a program that resides entirely on your local computer and is designed to record genealogical information and store that information on your computer's hard drive or a connected drive. These programs may or may not have a network connection to an online family tree and/or resource database. A desktop-based program will function independently without an active connection to the Internet. This fact alone has been one of the major considerations in depending on a local program. It is only recently that genealogists became overwhelmingly online.

If you have limited access to the Internet, then the need for a local program is obvious. But for those of us (likely included are those reading a blog post) who have a consistent and active involvement online, the decision becomes less obvious and far less easily decided. The first and most serious consideration is the issue of maintaining separate databases and keeping them in synchronization. If you choose to have a local program as your primary database, how do you maintain an online family tree? Programs such as Ancestry.com's Family Tree Maker and Family Tree Builder from MyHeritage.com in the most recent versions, are designed to synchronize with the user's online family tree. In these programs the entire tree is synchronized at one time. Programs such as RootsMagic.com, Legacy Family Tree and Ancestral Quest are designed to synchronize, one data field at a time, with the FamilySearch.org Family Tree.

The challenge is the time it takes to coordinate or synchronize data between two programs and if a person has more than one online family tree the challenge is magnified. For those with an LDS Account on Ancestry.com, there is a limited ability to synchronize individual data and sources directly between Ancestry.com and FamilySearch.org's Family Tree. Otherwise, the user has to rely on copy and paste to move information between the other programs. Having a desktop-based program adds a significant amount of maintenance overhead to maintaining a family tree, if it is used only for backup purposes.

One of the major reasons for keeping a desktop program is that most of the current supported programs are significantly more full-featured that their online counterparts. Here is where the level of sophistication of the user becomes of paramount importance. Entry level genealogists or family historians are mostly overwhelmed with the complexity of doing the research and recording the information, the thought of duplicating the work by adding in additional family trees does not even become a consideration. On the other hand, as a researcher's level of sophistication increases, the need to have your own "controlled" environment becomes more important and the features present in the desktop programs become interesting and useful.

One factor that arises in this context is the existence of the GEDCOM program. Originally, this was designed to assist in the transfer of genealogical data from one program to another. Because the GEDCOM standard has not been updated for many years now, transferring information, other than basic data, by using a GEDCOM files has become less and less useful. For my part, except in very limited situations, I no longer consider a GEDCOM transfer to be a viable option.

For users of the FamilySearch.org Family Tree program, a major consideration has been the apparent lack of stability of the data. The wiki format is unfamiliar to all but very experienced users and the idea that changes can be made to the data is unsettling to newer users. For this reason, there is a perceived need to have an "archive" copy of the individual's data to restore an incorrect change. This is a very persuasive argument but assumes that the data held by the user in his or her individual file is accurate and current. It is also a fact that the Family Tree contains far more information than any one individual is accustomed to maintain even for those who have extraordinarily large family files. Considering the degree of accuracy shown by many of those making changes to the Family Tree, having your own copy of your data only makes sense if you have extensive documentation.

In summary, the need for a desktop, local program for genealogy depends on your level of expertise. My guess is that most entry level family historians are going to find that the online family tree programs are sufficient. If and when you begin to appreciate the added features of a desktop program, there are plenty of them available.

Resolving the Standardized Place or Date Warnings on FamilySearch.org

If you have been working on the FamilySearch.org Family Tree lately, you will have seen these large, red warning markers advising you that the dates and places are not standardized. This is another housekeeping measure designed to allow the program to make the data searchable and to find record hints. In the example above, the dates and places are all non-standard but the number of warnings seems to stop at three. Unless the user "corrects" the problems with the dates and places, the warnings will keep appearing. Recently, FamilySearch.org added marriage dates and places to the warning list.
See "What’s New on FamilySearch—December, 2015."

I have written about standardized places and dates in a number of blog posts in the past. But these recent additions warrant additional comments. As I have written previously (search for "Genealogy's Star standard date place"), standardizing a place can lose valuable and correct information about the name of the place at the time an event occurred. This occurs when the actual place name is not in the Standardized list of options. Dates do not usually have that problem but they can when referring to certain time periods in the past, particularly those involving the calendar change from the Julian Calendar to the Gregorian Calendar.

Fortunately, there is an extensive Help Center article on the process of adding a non-standard date or place. For help, see "Entering standardized dates and places." The key here is when entering a nonstandard date or place, you do not click on the standard, but you click somewhere else on the screen. The system then leaves the place as you type it but connects it with the standardized place.

It is really important to standardize (including the exceptions) all of the dates and places. In addition, it is a clear message to the user that we should all be more consistent, more careful and more accurate in entering dates and places.

Two New Videos on the BYU Family History Library YouTube Channel

We are averaging more than one new video a week uploaded to the Brigham Young University Family History Family History Library YouTube.com Channel. There are now more than 100 videos on a variety of topics. The two newest uploads are as follows;

The is on RootsSearch, a valuable Chrome Browser app that aids in searching a huge variety genealogical resources. You can find it in the Chrome Web Store.

The next video is an hour long class on how best to utilize the record hints from FamilySearch.org and the Partner programs.

These videos are soon to be followed by more. You might want to subscribe to the BYU Family History Library YouTube.com Channel to receive email notices of newest training videos. We are planning some additional video offerings right after the first of the year, so we will have even more to offer.

A Major Breakthrough in Research on FamilySearch.org

FamilySearch.org's new Thumbnail Gallery for historical record images is a huge improvement to the research capabilities of the entire website. The new features added to the FamilySearch Catalog entries now provide an expedited way to view microfilm and combined with other existing features raise the overall utility of the program significantly. These new features are explained in the "What's New on FamilySearch -- December, 2015" blog post from FamilySearch. I suggest you carefully read this entire blog post.

Unless you are involved in doing serious genealogical and family history research, these new features will probably be either unnoticed or ignored. Even my explanation will be meaningless to those people who do not do in depth research involving microfilm. These new features start out by highlighting the importance of the FamilySearch Catalog. I taught a class this week at the Brigham Young University Family History Library that will shortly be uploaded to the BYU Family History Library YouTube.com Channel. This class outlines some of my current research strategies. The conclusion that I come to regarding online searches is that it is still necessary, in many cases, to go to the FamilySearch.org microfilm collections after you exhaust online resources. I presently have a series of over fifty microfilms that I will need to search. The new features implemented by FamilySearch.org will immeasurably improve this process.

Let's assume that I want to research English parish records. In my case, I would like to find my ancestors in Huntingdonshire. My current records show that my ancestor, Thomas Parkinson, (b. 1830, d. 1906) was born in Farcet, Huntingdonshire, England. This information comes from records in the United States after he immigrated in 1854.

After extensive online searches, I have yet to locate a birth record. I do find an 1841 England, Wales and Scotland Census record showing his family, including his parents, James and Elizabeth Parkinson. He is shown as 10 years old in 1841. The family is living on Glassmoor Street in the parish of Whittlesey St. Mary & St. Andrew in Cambridgeshire. One of my options, of course, is to examine the parish register for his reported birthplace. As far as I can determine, the most available copy of the Farcet parish registers are in the FamilySearch.org microfilm collections.

Here is a screenshot of the FamilySearch.org Catalog entry for Farcet's Church Records. Note that "Farcet" is spelled as "Farcett." Apparently both spellings are used.

I can check to see if perhaps some or all of these microfilms are already available at the Brigham Young University Family History Library and if not, I can order them into the Library. (Note: there is no charge for ordering microfilm into the BYU Family History Library from the Family History Library in Salt Lake City, Utah and once the films arrive in about two weeks, they stay the copies of the film are permanently kept by BYU).

I can then look at the individual entries for each of the four microfilm records listed. The first is the Bishop's transcripts for Farcett, 1660-1857. This covers the time period I am searching so this one goes on my list. But, here is the new feature from FamilySearch. The microfilm listing has two icons. Here is a screenshot.

These are the icons for the new features. The round icon indicates that the records are available on microfilm. The magnifying glass lets you search for similar records already digitized. I must note that this collection of Farcett records is NOT in the digitized records in the Historical Record Collections on FamilySearch.org. Here is a screenshot of the assisted search.

There are 292 entries on this particular search and there are 31 people named Thomas. However, none of these is my Thomas, but I do have a second search that helps to establish that I do need to order the microfilm. I can now order the two microfilms that pertain to the time period I am searching.

Now, what happens if the record has been digitized? Unfortunately, none of the records I am interested in at the moment seems to have been digitized. So, I have to go to some other records for an example of what happens if they are digitized.

This is a screenshot of the microfilm list for Utah death certificates, 1904-1951.

The little camera icon allows you to see the entire roll of microfilm in thumbnails.

You have to have worked through hundreds of microfilm rolls to fully appreciate what this view does for research. You can instantly see the headers and breaks in the different microfilm records on the roll and you can jump to any record and search the roll completely on any computer. This is a great, great benefit.

There are some more important features announced in the post I linked to above. I will be highlighting how those affect research in subsequent posts.

The New FamilySearch Place Research Tool

FamilySearch has been introducing several new features to the FamilySearch.org website recently. I have already written about some of them but there are still more to review. In the next few days, I will be highlighting and writing about more of these helpful and very interesting developments for the website. You can see a list of all of the newest developments on a blog post entitled, "What's New on FamilySearch -- December, 2015" by Steve Anderson.

Even with my intensive use of the FamilySearch.org website, many of these new features would go entirely unnoticed by me without these blog posts or someone pointing out the changes. Some of the changes are obvious, such as the new personalized dashboard, but others can only be discovered if you know what has been changed.

Illustrated above in a screenshot is the Place Research Tool. I could have used this tool earlier this week when I was teaching a series of classes. From the description, it is unclear where this tool is located in reference to the website. There does not yet seem to be a link from any page on FamilySearch.org. There is a link from the Standard Finder page but where is the Standard Finder? Both of these features depend on you knowing that they exist and then finding them with a direct link. I added links to both these tools to the Miscellaneous Research Tools page of the FamilySearch.org Research Wiki so links will show up with a Google search for the tools' names. The Standard Finder is also on the Labs.FamilySearch.org website.

For English research, I should have remembered the interactive map of the English & Wales Jurisdictions of 1851. It would have also helped me this week. It is also located on the Labs.FamilySearch.org website. I guess I need check lists for my check lists.

Back to the Place Research Tool, here is the description from the blog post.

The Place Research Tool will provide jurisdictional information to help you know where to look for records for the place results. In many cases, historical information is provided which will help you know how the place has changed over time. More information about the places will be made available as we expand the capabilities of the tool.

Here is the further description.

The Place Research Tool allows you to look for places that are important to your research. Go to this URL to give the tool a try: https://familysearch.org/int-std-ui-research/. If the place name you enter is found, you will see some information about the place, such as its official name and any variant names, such as The Big Apple for New York City, or the Big Easy for New Orleans. It will also pin your place on a map and provide the latitude and longitude. You can look by name, by jurisdiction, or by location.

Tools like this need to be prominently displayed with links.