The sentence is grammatically correct. It expresses a wish or desire that the speaker did not live so far away from the town centre. The use of "didn't" instead of "wouldn't" implies that the speaker believes it is unlikely that they will move closer to the town centre in the future.