You should read God, Freedom, and Evil by Alvin Plantinga and The Problem of Pain by C.S. Lewis if you haven’t already. I would like to give a little push on your points though.
Yes God is all powerful but he can’t make a square circle. In other words, he created the universe to be able to choose between good and evil, meaning how he can interact with the elements in our universe is limited, not because he lacks power, but because we are limited. Furthermore, good and evil can exist simultaneously because evil is not a separate entity, it is a corruption of a good thing. To give an analogy, if you have a faulty computer part, simply removing the part without any sort of replacement does not solve the issue, but simply stops the computer from working. The idea is nonsense remains nonsense even when talking about God. The reason for this is because just because two words have meaning, doesn’t mean that if you combine them into a sentence it does. Things that don’t make logical sense are not describable or even imaginable because if it was possible to do such a thing, logic would be pointless. Conclusion: taking out evil is not possible because God can’t do illogical things when interacting with the universe he created because the universe itself is limited, not God. If you were to create a different system, we would not exist.
P.S. I would also like to point out that specifically in Christianity, God doesn’t just watch us suffer and do nothing about it, he became a part of it as Jesus. He suffered more than imaginable. If what is recorded is true, he experienced the pain of every murder, rape, torture, hurricane, cancer, disaster, manipulation, etc. over all time. He did not spare himself from suffering but did overcome it. He was the example of how if we trust in him, we can do the same through him.
P.P.S. Heaven is pure joy and bliss, we don’t have a choice to do evil because we made the choice here on Earth. We can live in pure joy without suffering because we suffered on Earth. Keep in mind that if there was no evil, no suffering, no pain, no danger, no risk, there would also lack the ability to grow as a human.