DNA-testing for genealogy has three types of tests, and only two of these can find exact connections: Y-DNA and mtDNA which tests direct paternal and maternal lines. What is tested then is relation back to a common ancestor, not where these ancestors were actually from.
Autosomal tests (Family Finder) can find matches from all possible lines in the family tree, but the test is not exact, and the degree of matching varies greatly and has less confidence for each generation back. More than 6 generations back will only give random matches of all the possible relatives one has at that level.
Some people also wonder if DNA-testing can reveal each ancestors ethnic background, but the answer here is that so far this is an almost impossible task, with a few exceptions like Jewish ancestry.
The principle is:
It is the direct paternal and maternal lines one can test for control and verification. Lines through both male and female ancestors can not be tested accurately.
Complex lines through both men and women are harder to verify.
How can we use DNA-testing to explore and verify lines where we are not descendants through a direct paternal or maternal line?
1. Trace direct descendants
If the person in the tree that we wish to explore is a man, then start with mapping all his direct male descendants. Ideally find two or three different lines from different sons of the person in question, with living descendants in the line. Test these descendants through Y-DNA.
If the tested men who all descend from the person in question all match, with more or less identical Y-DNA markers (preferably Y 67 STR), then the line is controlled and verified. Testing three lines will be strong evidence, testing two will be a good indication if other information supports the same common ancestor at that point and not closer.
To check that the line tested is also related to yourself, you can do a Family Finder test of yourself and the Y-tester closest to yourself, provided the relationship is within 6 generations. This can support the probability to the ancestor in question.
If the person in the tree that we wish to explore is a woman, the method is parallel using mtDNA-tests instead: at least two, preferably three direct descendants in different, direct mother-daughter lines which should match to verify the common direct ancestor.
We will try to give a good example later. Read the Norwegian articles: mtDNA-testing av linjer i Sør-Østerdal and hvordan finne og spore en tippoldefar.
2. Family Finder-test from several descendants
If your ancestor you are interested in has several living desendants max 5-6 generations back (second great grandparents to the living), you can test Family Finder for all of them, and look for common gene segments. Preferably the tested people should not share other ancestors within 5-8 generations.
A challenge here is that people might have inherited common gene segments from further back, so it is impossible to assign a common segment to a specific ancestor without having complete ancestry trees for all ancestors far back (8-12 generations).
Not all descendants of the same second great grandparents will have inherited the same gene segments from that ancestor. This method can thus prove to be quite time consuming to use while also never giving a final answer.
Conclusion
It is as shown not easy to use DNA-testing to verify relationships and explore ancestors when the lines go through both males and females, and the generations back to the common ancestor is more than 5 generations. The best option is to trace several direct descendants in direct paternal or maternal lines to the ancestor in question.