The RG503 is still my favourite pocket device
Newer devices are faster, but the RG503 is the one that ends up in my bag. The OLED screen makes 16-bit games look fantastic and the size is just right for a jacket pocket.
With a community firmware it runs everything up to the PlayStation era without fuss. Anything heavier is better played elsewhere anyway.
The only changes I made were a bigger microSD card and a matte screen protector. Five years from now I suspect it will still be in use.