The first part of the name of the village comes from the river Isle, which separates the parish from Isle Abbots. The second part comes from the family of William Brewer (Brewer) who were the lords of the manor in the early 13th Century.