The problem is not self-reference, or observed details, or intimacy, but rather the agenda and purpose of focusing on the self. Of course, agendas will differ, but failing to recognize one’s own agenda does not mean that an agenda does not exist.
Flawed agendas, in my estimation, are those that disrupt the integrity of the poem, interrupt the poem-reader dialogue, or exaggerate the self. I won’t speculate as to the reasons, realized or not, behind these agendas; I’d rather focus on their effect. Here, I take for granted the purpose of poetry (to communicate, entertain, and examine), which is, of course, open to debate.
Poems that rely too much on autobiographical context often frustrate the integrity of the poem. Enjoyment of such a poem relies on a reader's ability and willingness to consider it in the context of the poet’s life. Explanations are required. Interest in the poet is required, and often an examination of the poet’s body of work is required. The reader arrives nowhere near the poem.
Dialogue in poetry allows the reader to enter and leave the poem in a unique way, and allows the poem to be new every time the reader returns, entering and leaving the poem in a new way. This dialogue exists partly because the reader changes, but also because the poem embraces the many-personed reader. For dialogue to occur, the poem must leave room for the reader, must address a broader experience, must have something of the world to show us. The promotional poem, the poem which makes an epic of the poet’s life, is no more a dialogue than a can of Coke.
The greatest bore is the poet who has looked into the world and found it all a mirror. Poets who aggrandize their life, exaggerating their injuries and heartaches, are chasing after celebrity, not poetry. The bore becomes a beast when the poet begins to make a caricature of human suffering to add gravity to their own plights. Poets who make lyrical references to genocide, plagues, famine, natural disasters, and wars in an attempt to add ornament to their sadness are engaging in bald exploitation. If I sound vitriolic, it's because I think that turning human suffering into a personal commodity is about as reprehensible as it gets.
The point is, when a poem relies on the poet talking to and about themself, the poet needs to be both terribly interesting and responsible. (People who, after thirty, still believe that they are interesting, often haven’t consulted their acquaintances on the matter. Family and friends may lie to spare your feelings, but an acquaintance will set you straight. Believe me it is better for a writer to be astute, analytical, imaginative, or empathetic. God help the poet who is an interesting character.)
My poems are often ruined until I go back and take out the talking-to-myself. Redacting the lines is often a delicate negotiation. I have to pet and apologize to myself, because I'm a sensitive flower. I mean to say, I have to talk myself out of talking to myself. My reason for talking to/about myself generally relates to laziness, and always to vanity.
And, my god, when I write a me-me poem, the whole thing stinks like a corpse.