That's a lot of info, Steve! I don't know how to thank you for the photos & other stuff. It is very kind of you to help!
I don't the Joe Chivers you found is a close relative, although I could be wrong. My ancestor was a collier transported to Australia for manslaughter. "Big Joe" Chivers was born in 1826 on Wrights Lane, Cradley Heath, Rowley Regis, in the West Midlands of England -- about 120 miles from your Joseph the cloth manufacturer? I am able to track him pretty accurately up to the time of his crime, and he was never in fact a cloth manufacturer - but, as you say, I suppose he could have lied. He had good reason to lie, given his criminal record, I suppose?
Big Joe had a daughter born in 1847, but she died the year of his conviction - as did her mother (1856). They had a son, also name Joseph, born in 1850 in Dudley, Staffordshire, England. All of this is clear from birth and death records. After "Big Joe" was convicted & transported on the convict ship Lord Raglan, his son went to live with his maternal grandparents, who moved back to Wales. When son Joe was a bit older, he joined his father in Australia, married while there, and then at some point (not sure when) left for San Fransisco, and eventually ended up in the mining town of Grundy, Illinois. Big Joe married Anne Riley (1825-1908) in Australia in 1865, almost as soon as his sentence would allow him to get married. "Chivers" is a common enough name that I suspect that the other Chivers in NZ are not close relations at all. But I would love to be proven wrong! :D (I've never tried to track down their history.)
I messaged you with my email; when I get your email address I will send along the long summary of all the info I have on "Big Joe" - it's pretty substantial. :)
Again, many, many thanks!