Does this seem to be the family in 1880?https://familysearch.org/pal:/MM9.1.1/MN47-RY8
You can find other census records and vital records for free at the www.familysearch.org
In 1870 the family is in Milwaukee, Wisconsin
in 1900 James, Elizabeth and Charles are in Green Bay, Brown County.
I would suggest you post a message on the Brown county board