There is, I think, a fairly sharp line between you deciding you don't want to interact with someone in some particular forum, and someone else deciding you can't interact with someone.
That line is not the line between "good" and "bad", necessarily. I'm just saying it's a relevant difference that shouldn't be blurred away.
I'm sorry but I don't see any dichotomy in the two situations you've described. Cancel culture is literally people getting ditched because some people didn't like what some people did. That is, sticks and stones may break my bones but words are terrorism.
It’s also an emblem for “people doing things other people don’t like and getting ditched for it” but which emblem you choose to see is up to you.
That’s kind of the power and the weakness of emblems.