By the work of Delzant, we know that all Hamiltonian toric $4$-manifolds are obtained by equivariant blow-ups of either $CP^2$ or an Hirzebruch surface. On the other end, given such a manifold, the classification of all distinct toric structures one can put on it is still unknown. In this talk, we will explain how one can use $J$-holomorphic curves techniques to prove a) that there is only finitely many such structures on a given $4$-manifold; b) that many symplectic blow-ups of $CP^2$ don't admit any Hamiltonian toric action. The ideas naturaly extend to Hamiltonian $S^1$-actions under mild hypothesis and give rise to some interesting questions.