Displacement is defined as the final distance-vector minus the initial distance vector.
Let's say you're driving to work in the morning. You first drive north for 5 miles, and then drive east for another 5 miles.
Now let's say that when you get there, you take out a street map. If you were to draw a straight line starting at your house and ending at your workplace, that line would be your displacement vector. That straight line would be 50 miles long, (use the Pythagorean theorem) and it would be pointing northeast. Since the line has a direction, it needs to be a vector quantity. If it weren't a vector quantity, you wouldn't be able to draw it on a map at all, because it would just be a number.
Another reason why displacement needs to be a vector is that it's defined as the subtraction of two vectors, and a vector minus another vector is always a vector.