learn_russian | извиниться/to excuse oneself

Once in my journal I said "извиняюсь" (literally "I excuse myself") to mean "excuse me," and all the native Russian speakers replied in unison that this is a bad form that should never be used, because it doesn't make logical sense. (Not making logical sense is a poor reason for not using an expression, but that's irrelevant, I guess - the point is, "извиняюсь," much like "звОнит," is improper Russian, even though people use it in regular speech.)

Question, then: are all forms of the verb "извиниться" equally improper? Or just the first-person singular present tense is wrong? e.g. is it just as bad to say "он извинился", or is that ok?

Thanks, guys! And с праздником прошедшим! (Happy belated Thanksgiving!)