When he lived at Grav farm, he was Anton Andreassen Grav, - and Grav was a sort of address-surname. When he moved to town, he might call himself Anton Andreassen, Anton Grav, or Anton Andreassen Grav. All alternatives were possible, and he might change between these. In 1923 they had to deside one surname to use as a family surname. Some families who had used a very common surname (of patronymic origin) then preferred to rather use the family farm name as family surname.
The point is that the use of surnames, the definition of what a surname is, has changed a lot (and is still changing). You might say that before 1923 most rural Norwegians did not have a family surname. They used individual facts in addition to their given name.
It looks like your family used Andreassen as a family surname in town in 1900 and 1910 census. But they might also have used Grav as a surname, if they wanted to.