The "doctor's office" represents the place where Benjy is taken to be castrated. Since we experience that event from his extremely partial perspective - all he remembers of that event is that "when I breathed in, I couldn't breathe out again" (53), which presumably refers to being rendered unconscious by inhaling ether before the operation - we can't say whether the surgery took place in a hospital or a doctor's office, and if the latter, which doctor might have performed it.
Mrs. Grier gives no hint of where she grew up when she tells her son about the way her mother reacted to her brother's determination to enlist during the first World War. Our assumptions that it was on a farm and somewhere in the Frenchman's Bend area of Yoknapatawpha are entirely speculative.