Correct option is C. $$\dfrac{4\sqrt 2 Gm^2}{a^2}$$
$$\angle i = \angle r$$
$$\angle i + \angle r = 90^\circ$$
$$\angle i + \theta = 90^\circ$$
$$\therefore 2\angle i = 90^\circ$$
$$\Rightarrow \angle i = 45^\circ$$
$$\Rightarrow \theta_1 = 45^\circ$$
$$F_1 = \dfrac{Gm^2}{r^2}$$
$$F_2 = \dfrac{2Gm^2}{r^2}$$
$$F_3 = \dfrac{3Gm^3}{r^2}$$
$$F_4 = \dfrac{4Gm^2}{r^2}$$
$$\therefore F_{net} = \sqrt{ (\vec{F_2} - \vec{F_4})^2 + (\vec{F_1 } - \vec{F_3} )^2}$$
$$= \sqrt{ \left ( \dfrac{Gm^2}{r^2} \right ) + \left ( \dfrac{4Gm^2}{r^2} \right )^2} = \sqrt{8} \dfrac{Gm^2}{r^2}$$
Now, $$r = \dfrac{a}{\sqrt 2}$$
$$\therefore \dfrac{\sqrt 8 Gm^2}{r^2} = \dfrac{2\sqrt 2 Gm^2}{\dfrac{a^2}{2}}$$
$$= \dfrac{4\sqrt 2 Gm^2}{a^2}$$