I've never paid special attention to them. As far as I can tell, there aren't any special "rewards" for completing an achievement, except for a select few that give "titles".

What statistic do they raise? Or is it just a completionist thing?

Some achievements offer prizes, like items or titles. But mostly it's just something for users to strive for, be it for personal goals or just something to do/pass time/etc.

If you go to the achievement page you can look for achievements that have little yellow stars. If you click on the star it will tell you what you get.

For example if you look at the New Wardrobe, New Me!, click on the star and it says: [item2=1 Week Gold Account Medal]