Steve,
I know a little about the unit you are looking for. My great grandfather served in it.
What I can tell you off the top of my head is that Issac W. Avery was the commander and the 4th was the command formed based on Avery's Georgia Mountain Dragoons. The Dragoons were at Chancellorsvile and the unit became the 4th Ga Cavalry while encamped in Dalton Ga prior to October of 1862. The 4th Ga saw action in the Middle Tennessee campaigns, was at Chickamauge, Chattanooga and Knoxville. My great grandfather mustered out in 1864 as a LT. and was pressed into Union service for the last year of the war as a private in the 6th TN Mounted Infantry, USA.
I have a little more, I just have to find it. I too am looking for muster rolls, addtional combat reports and a copy of the book written by Issac W. Avery.
Hope this helps,
Joe