The details of Byrge's past he would like to keep secret, but unfortunately for him it is more-or-less publicly known in Thorin's Hall.

Byrge's father was Bysir, in turn a descendant of the Dourhands of Sarnúr, though he and his father Thrasir were reconciled to the Firebeards by time of Thráin II and indeed helped the Longbeards to build the mansion later known as Thorin's Hall. In later days however Bysir became a follower of Gormr Doursmith, and it is rumoured both he and Byrge personally participated in his rebellion. But mere days later, Byrge defected back to the Longbeards in order to protect his mother and sister, who had become captives back at the Hall.

Since then Byrge tried to live quietly as an obedient Longbeard subject, taking whatever work he could to support his family. Unsurprisingly, there was not very much offered him in Thorin's Gate, so he moved to Bree-land where he met and married Tiarvi. He now leads a small company of semi-outcast masons that deliver dwarvish quality for cheaper than the usual dwarvish prices, building wells, cellars, and the like.