The symmetric second difference

1. By Taylor–Young at order $2$,
$$f(a\pm h)=f(a)\pm hf'(a)+\frac{h^2}2f''(a)+o(h^2).$$
Adding the two relations, the terms in $h$ cancel:
$$f(a+h)+f(a-h)-2f(a)=h^2f''(a)+o(h^2),$$
and dividing by $h^2$ gives the limit.

2. Take $f(x)=x|x|$. It is odd and $f(0)=0$, so
$$\frac{f(h)-2f(0)+f(-h)}{h^2}=0\ \longrightarrow\ 0.$$
But $f'(x)=2|x|$ for all $x$ (including $x=0$ where $f'(0)=0$), and $\frac{f'(x)-f'(0)}{x}=\frac{2|x|}x$ has different one-sided limits $\pm2$ at $0$. Hence $f''(0)$ does not exist.

Remark. The symmetric second difference is the standard numerical approximation of $f''$; the converse failing means one must not read its limit as a proof that $f''$ exists.