We consider geometric multigrid solvers for linear systems stemming from the finite element discretisation of partial differential equations on unstructured grids. Our implementation technique reduces the complete solver to sequences of sparse matrix-vector multiplications and is thus well-suited for both GPUs and multicore CPUs. In particular, our implementation can handle several low- and high-order finite element spaces in 2D and 3D, while only the sparse matrix-vector kernel needs to receive significant tuning. For several benchmark problems, we achieve close to an order of magnitude speedup of a single GPU over multithreaded CPU code.