We describe a GPU- and multicore-oriented implementation technique for a key component of finite element based simulation toolkits for partial differential equations on unstructured grids: Geometric Multigrid solvers. We use efficient sparse matrix-vector multiplications throughout the solver pipeline: within the coarse-grid solver, smoothers and even grid transfers. Our implementation can handle several low- and high-order finite element spaces in 2D and 3D, and for representative benchmark problems, we achieve close to an order of magnitude speedup on a single GPU over a multithreaded CPU code. In addition we present preliminary results for experiments with strong smoothers for unstructured problems on the GPU, aiming at augmenting numerical and computational efficiency simultaneously.