Is it really that hard to reach level 10 or am I just being too stupid? It's really frustrating because I usually love games like this.

It is very hard, but as you can see from the highscores, it is also doable.
I was frustrated like you because I'm also usually good at these kind of games. I finally made it to level 10 this week on endless mode with a lucky board. Just play every day and you will get it eventually :)