I don't really want to train speed, is there a minimum I should leave it at just in case?

I don't know about a minimum, but speed is good for critical hits. If you have a higher speed than your opponent you have more of a chance of scoring one, which is useful.

Speed does help better the chances of the secondary effects on weapons (ex: blessing, cursing, freezing, stealing, etc), but it is only at certain intervals. The paintbrush blades' blessing will not activate at all at less than 2000 speed. However, there does not appear to be a weapon whose secondary effects scale past 3000 speed.

Not yet, anyway. ;)

4k speed :p