Possibly you mean the James Anderson b. Scotland ca. 1808, a tailor in Chatham Parish, Northumberland Co., NB in 1871. He was a widower with three children: Mary (16), Alberta (14) and James (12). Possibly the younger James was the Robert James you mention.
This appears to be the same as, in Chatham Parish in 1881, James Anderson (72) is still a widower and children are 'Albertha' (23) and Robert J. (21).
James (82) is still living in Chatham Parish in 1891 and is still a widower. He live with the family of Joseph (40) and Mary (36) Dickens. James and his parents were b. Scotland. Joseph and his parents were b. England. Mary was b. NB and her parents b. Scotland.
In Chatham Parish, still with the Dickens, James Anderson was b. 2NOV and is age 91 years, so likely b. 1809. James appears to have died by 1911.
In spite of the discrepancy in age, James appears to be the tailor, Jas Anderson, in Chatham Parish in 1851, age 35 years, who arrived in NB in 1838. He was the nephew of James Nesbit (62) and his wife, Mary (71), both b. Scotland. James Nesbit arrived in 1849 and Mary in 1820. What this suggests to me is that Mary, with a previous husband, arrived in 1820 her (or her husband's) nephew, James Anderson, joined them in 1838 and Mary's 1st husband dying, she married the recently arrived James Nesbit. That is, assuming the information derived from the 1851 census is correct. It is entirely possible that the arrival date for James Nesbit is incorrect. In the "Old Saint Andrew's Cemetery - Anglican" in Chatham, are headstones transcribed as:
NISBET - James d. 13 Apr 1862 in his 74th yr
Mary w/o James d. 25 Mar 1859 ae 80 yrs
th/s George d. 15 Jun 1820 ae 14 yrs
th/d Margaret d. 15 Oct 1830 ae 13 yrs
Suggesting that James Nesbit/Nisbit was in NB as early as 1820 himself. In a newspaper announcement, his daughter, Margaret seems to have been age 18 when she died but the reading of one source may have been in error. Mary Nisbet (d. 18MAR1859 age 80 years) was from Dumfriesshire, Scotland and arrived in NB in 1819 with a husband and family.
All of this leads me to James Laurance Anderson b. Mid and South Yell, Midlothian, Shetland Islands, Scotland 21NOV1809, son of Robt. and Mary (Nisbat) Anderson.
I am not sure of the place as given. A Mary Nisbet (31MAY1773-22NOV1862) married Robert Anderson (b. ca. 1774) and had children:
She was a daughter of Hosea and Margaret (Mathewson) Nisbet. She died in a place called Gardiestaing. Her son, James, is given as b. North Sandwick 9NOV1809 which, in the circumstances, is close and may have been a baptismal date. Hosea and Margaret had two sons named James Nisbet, the younger of which was b. Aywick 13JAN1780 and who m. Mary MacVitie (b. ca. 1787). See:http://family.hoseasons.co.nz/getperson.php?personID=I05109&...
which has bits like:
Mary Nisbet b abt 1773 died/bur 22 Nov 1862 age: 89 at Yell (Mid And South) Shetland Father: Hosea Nisbet Mother: Margaret Matthewson Spouse: Robert Anderson
Comments: born ca 1773 Age: 89 Widowed Spouse Robert Anderson Informant: Charles Anderson, son
Bap: 9 Jan 1780 James Nisbet Parents: Hosea Nisbet Yell (Mid And South) Shetland
Note that here 9JAN1780 is a baptismal date, not a birth-date; a common error.
It appears that the information on James Nisbet (?) in 1851 is all incorrect and that on his headstone not much better. In 1851 he was b. ca. 1789 and on his headstone he was b. ca. 1789 but 1780 is more likely. For example, the George Nisbet b. ca. 1806 buried with his parents seems to have bee George Hosea Nisbet b. Aywick 6APR1807, s/o James and Mary (MacVitie) Nisbet.
Bap: 6 Apr 1807 George Hosea Nisbet Parents: James Nisbet Yell (Mid and South) Shetland
the 'birth-date' appears to have actually been a baptismal date.
JAMES LAWRENCE ANDERSON b. Shetlands, Scotland 21NOV1809 (2NOV1809, ca. 1808-9), m. St. John's Church, Chatham, Chatham Parish, Northumberland Co., NB 25MAY1854 ANNIE JANE RUSSELL (d. 27JAN1862).
1. MARY ANDERSON b. 21MAR1855 (ca. 1855), m. JOSEPH DICKENS (b. England 18JUL1851 (ca. 1851)).
2. ALBERTA ANDERSON b. ca. 1857.
3. ROBERT JAMES ANDERSON b. ca. 1859-60, m. BRIDGET FLIEGER.
At a wild guess, Annie may have been Ann Russel (b. ca. 1837), d/o John (54) and Margaret (57) Russell in (Whitneyville?) Northesk Parish in 1851. This Ann, however, was b. in NB (her parents were from Scotland) where her daughter, Mary, in 1891 said that her mother was b. Scotland. This seems the right, general area as Red Bank is in Southesk Parish near Sillikers. Southesk was part of Northesk until 1879. There were two Russel families in Northesk Parish in 1851, and the Ann in the other one was the wife and mother.