The sentence that is correct depends on what the word “sure” or “surely” is supposed to describe. Depending on the part of speech that it describes, it will need to be a different part of speech.
The correct sentence here is the second choice: That was surely a mistake. The reason is that surely modifies was. We are talking about how sure someone is that the thing was a mistake. Because the word that is being modified here is was, a linking verb, it needs an adverb to be grammatically correct.
There is another possibility if you want to express your surprise at hearing something. Instead of using surely to describe was, you can use sure as an adjective. That would make a correct sentence: That was sure to be a mistake!
