Godeaux surfaces are surfaces of general type with the lowest numerical examples possible. Since the first example was given by Lucien Godeaux in 1930 there is an extensive search for more such surfaces and for a classification. In this talk we consider Godeaux surfaces over fields of positive characteristic with non-vanishing $h^1$, i.e., non-reduced Picard scheme. The existence of these surfaces was already observed by Rick Miranda in 1984. We prove that such surfaces can exist over fields of characteristic at most $5$ and give a complete classification in characteristic $5$.