The results we shall present grew out of the need to better understand the singular locus structure of a reduced projective hypersurface $X=V(f)\subset \mathbb{P}^r$. The first questions split right at the outset into two different geometric problems. One is when is $X\subset\mathbb{P}^r$ homaloidal, meaning that the partial derivatives of $f$ define a Cremona transformation of $\mathbb{P}^r$. An obvious necessary condition for this is the non-vanishing of the Hessian determinant $h(f)$ of $f$. The other is to understand the full impact of a vanishing Hessian. We shall present Hesse' s wrong claim that hypersurfaces with vanishing hessian are cones, the problems it originated, a simple new geometric proof of Hesse's claim for $\mathbb{P}^3$ (and $\mathbb{P}^2$) via the contributions of Gordan and Noether, and the classification of hypersurfaces in $\mathbb{P}^4$ with vanishing Hessian. Finally we shall illustrate the geometric nature of infinite families of irreducible homaloidal hypersurfaces of arbitrary large degree or of polynomials with vanishing Hessian of diverse nature considered in a recent joint paper with Ciliberto and Simis, relating them to the present knowledge in these areas. The problems and results presented have played an interesting role in various other areas, such as differential geometry, algebraic solutions of partial differential equations and approximation theory.