As I suspected.
The quick way to implement it is to simply expose the confidence number in the class tag of the line 'div' and map that to a color and maybe a on-hover element showing the confidence number. The only noticeable difference in load on the server side is another attribute showing up in the data. It would be the server side equivalent of adding it as extra text to the Name field.
So the "server load" objection is incorrect...